of 38 /38
1 Ircam Artistic Testbed Jerome Barthelemy, Ircam

1 Ircam Artistic Testbed Jerome Barthelemy, Ircam

Embed Size (px)

Text of 1 Ircam Artistic Testbed Jerome Barthelemy, Ircam

  • Slide 1

1 Ircam Artistic Testbed Jerome Barthelemy, Ircam Slide 2 2 Summary Artistic production context in Ircam Music production with digital components since the 70ies An example Structure of our objects Risks and Good practices The preservation issue Definition of Representation Information PDI Assessing authenticity Authenticity in the CASPAR framework Some examples of authenticity protocols Work done and future work MustiCASPAR server Scenario and implementation demonstration and RepInfo validation Repository current status Future work Slide 3 IRCAM context IRCAM : musical production since the 70es Development of audio/music digital processors : 4A, 4X. (Hardware) Max/MSP (Software) Audiosculpt, Modaly, OpenMusic (software) Musical creation using audio digital processing 450 works created since 77 Problem of preservation recognized since the middle of the 80s, but no formal approach Production of documentation (from 80es to 2000 - in paper) From 2002 : digital storage of documentation and digital objects : Mustica 3 Slide 4 IRCAM context GOAL of preservation in this context : To be able to REPERFORM the works Not simply record audio files ! To make possible interactions between human performers and digital processes : Preserve the processes themselves, not the results What processes? Digital instruments (audio effects, like reverberation, harmonizers) Encoded in the form of software 4 Slide 5 Example Diademes by M.A. Dalbavie Creation in 1986, Written for solo viola, acoustic orchestra and live electronic Live-electronic part : 1 YAMAHA TX816 : FM-Synthesis for 2 Yamaha KX88 keyboards 2 YAMAHA SPX1000 : Effects/Transformation of the viola: Harmonizer & Panning 2 YAMAHA REV5 : Effects/Transformation of the viola: Reverberation & Echo Several new performances between 1986 and 1995 No new performance since 1995 (but several attempts), due to issues related to FM synthesis component A new performance on 2008 (December New York) 5 Slide 6 Example Diademes by M.A. Dalbavie 6 Live-electronic part : 1 YAMAHA TX816 : FM- Synthesis for 2 Yamaha KX88 keyboards 2 YAMAHA SPX1000 : Effects/Transformation of the viola: Harmonizer & Panning 2 YAMAHA REV5 : Effects/Transformation of the viola: Reverberation & Echo Slide 7 Example Diademes by M.A. Dalbavie 7 FM Synthesis : Modulation of a waveform by another waveform (both in the audio range) Original implementation by Yamaha (under patent) Results in complex sounds, not produceable by another means Slide 8 The general case 8 Preservation data Data are composed of: - Data files (mainly audio) - Processes (hardware or software based) Plus some information files: - Sketch plans - Instruction files The whole set defines a work (in addition to the musical score) Slide 9 The general case 9 MIDI Max Patc h Max Patc h MIDI Max Patc h Max Patc h instructions equipments information A more specific case : Slide 10 The general case - We can consider the process as a kind of musical digital instrument that is to be preserved (as for acoustic instruments) - Obsolescence is very fast (for example, comparing to audio files) - New performances imply generally a migration (or a porting, or emulation) of the process - Most important question : authenticity Migration Slide 11 The general case 11 Good practices on Authenticity : - Record samples in input - Record audio output In order to keep track of the intended use of the process. CASPAR cope with current good practices ! Slide 12 The preservation issue Preservation Preserve the process itself (not the recording) Preserve the data files (audio files) Preserve all documents that are related to the work Preserve the logic (the relation ship between all the elements) 12 Slide 13 The preservation issue About the process (the patch ) At the very core of the work Encodes the logic of the relationship of the different elements (HW, like microphons or speakers, and data files) Preservation : Very difficult, since the process can be considered a software But fortunately, a specific kind of software, most of the time encoded with specific software like Max/MSP, PureData, or Reaktor, all based on graphical programming In Ircam, generally based on Max/MSP The major risk 13 Slide 14 RepInfo for real-time process 14 Representation Information : Structure: Block-diagram flow Semantics of elements : already existing in documentation Methodology : Reduce the block-diagram flow to an algebra Choice of existing FAUST language, concise and sufficiently expressive (developed by Grame, Lyon, France) Work in progress, to be developed in future project Store the semantics of the elements Extract them from existing documentation Slide 15 RepInfo for real-time process (and PDI) 15 DOC tool : extracts from existing documentation the semantics of elements FUNC tool : parses the code of each single process in order to identify elements, verify existence of RepInfo (if RepInfo missing, a warning is generated), and provides PDI for the process according to patch ontology FILE tool : analyze the global structure of all provided files, encodes PDI of work according to provided ontology LANG tool : reencodes the syntax of the original process Slide 16 Preservation Description Information Based onto CIDOC-CRM (ISO 21127:2006) (cidoc.ics.forth.gr) Ontologies templates expressed in RDF for each element we have to preserve : Work Real-time process : Patch Library Function Documents : Booklet (documentation), Hall program, Biography, Interview, audio sample, Video sample, Score, Recording 16 Slide 17 Preservation Description Information Ontology template for work: 17 Slide 18 Preservation Description Information Ontology template for patch: 18 Slide 19 Authenticity Authenticity is a process (Mariella Guercio, University Urbino) CASPAR defines Authenticity Protocols (AP) Each one composed of several steps (AS) Must be executed at different steps according to the lifecycle of the object (access, migration) Each execution is composed of different execution steps (ASE) The overall result should lead to an evaluation of the authenticity Defined accordingly to a Designated Community 19 Slide 20 Authenticity Need to attach different AP to different steps of the lifecycle. Example: AP1 Migration of component AP2 Maintenance Different AP for context Dependent on the work in which is used (example of FMSynthesis) Dependent from the point of view of community Developers Musical assistant Curators 20 Slide 21 Authenticity protocol #1 Case : maintenance (audio file) Compute fixity Verify provenance 21 Slide 22 Authenticity protocol #2 22 Case : migration of audio file, from format f1 to format f2 AS1 - before migration : verify semantics (is it an audio file) ? Yes : continue No : failure - apply another protocol suitable to the right semantics AS2 - verify file not compressed Yes : continue No : uncompress (or return to analog!) AS3 - before migration : verify availability of source sampling rate and sampling size in target format f2 Yes : continue No : failure - apply another protocol AS4 - after migration : compute fixity of sample size and sample rate Yes : continue No : failure - apply another protocol AS5 - after migration : compute fixity of Pulse Code Modulation from audio files Yes : continue No : failure - apply another protocol Failure - apply another protocol Compression? Sample rate? Sample size? Migration Fixity sample rate & sample size Fixity PCM Decompress Failure - apply another protocol Audio file? Slide 23 Authenticity protocol #3 Case : migration of an audio effect (AFX1 -> AFX2) Ingest phase (AP1): AS1 : Define a list of audio samples in input (using predefined audio samples 1khz sinusoid) AS2 : Apply to them the audio effect AFX1, and store the output audio samples AS3 : Define the comparison features (with range) that are to be validated when executing AP2 Migration phase (AP2) : AS1 : Apply to input audio files the migrated audio effect AFX2 and store the results AS2 : Compare the audio samples resulting from AFX2 to those resulting from AFX1, according to features defined in AP1 23 Slide 24 24 Ircam testbed scenario Start point : Max/MSP no longer available Slide 25 25 SUB-SCENARIO #1 Ingest of a new WORK and components Slide 26 26 SUB-SCENARIO #2 Notification of loss of availability for a COMPONENT Slide 27 27 SUB-SCENARIO #3 Search for equivalent COMPONENTs Slide 28 28 SUB-SCENARIO #4 Ingest of a new version of a COMPONENT Slide 29 29 Demo movie Based onto the MustiCASPAR server Slide 30 RepInfo validation Done in 3 steps : 1 Completeness of information : reconstruction of original process from extracted Representation Information 2 Usefulness : construction of an equivalent process, from extracted RepInfo, but executed from PureData (equivalent to a migration) 3 Authenticity : comparison of audio outputs, according to defined Authenticity protocol 30 Slide 31 Reconstruction of original process 31 RepInfo Verification of completeness of information Slide 32 Migration Usefulness of Representation Information : Ability to reconstruct a new process under a different environment (that is to say, PureData instead of Max/MSP) Some possible loss of information and inconsistencies : manual adjustments to be made 32 RepInfo Slide 33 Authenticity Authenticity protocol #3 At Ingest phase, an authenticity protocol was defined, with 3 steps : Choose an input audio file (inputFile1) Apply audio effect on it Record output audio file (outputFile1) 33 Slide 34 Authenticity Authenticity protocol #3 At Ingest phase, an authenticity protocol was defined, with 3 steps : Choose an input audio file (inputFile1) Apply audio effect on it Record output audio file (outputFile1) Migration Authenticity protocol : After migration, apply new audio effect on inputFile1 Record output audio file (outputFile2) Compare outputFile1 and outputFile2 (by hearing audio engineer, or any other method of comparison, for example comparing spectrograms) 34 ? migration Slide 35 Authenticity Migration based on Representation Information was not sufficient. Adjustments have to be made, particularly on the sliders in order to achieve authenticity Semantic RepInfo is not sufficiently defined on some parameters (the reason why there are sliders that have to be manually adjusted!) 35 Slide 36 36 Repository status From current repository Mustica to MustiCASPAR : Preliminary analysis of the whole content of the repository Several common file formats have been found inside, including zip, dmg, rar and ISO CD. Reconstruction of PDI possible for works (but not for parts of works, as explained below) Migration of Jupiter by Philippe Manoury from Mustica to MustiCASPAR Purpose of the test : to evaluate the amount of work of a complete migration for the whole content Most important Issues : Missing informations about provenance and context Need to identify persons able to provide this information Unexpected files Representation Information missing PDI missing Slide 37 Achievements MustiCASPAR server Methodology for extraction of Representation Information CIDOC-CRM based ontologies for expression of PDI Scenario definition and implementation Evaluation of current state of repository Validation : Ability to build on Authenticity Protocol and Representation Information for migration adequation of CASPAR and OAIS model to current practices and current expression of documentation efficiency : Information about the current state of the repository (RepInfo, PDI missing) 37 Slide 38 38 NEXT DEVELOPMENTS Middle term (until end of 2012): Two research projects (funded by National Research Agency) First one focused on RepInfo production Second one more focused on PDI (tracking provenance) with INA and UTC (CASPAR Partners), and EMI Music Development of MustiCASPAR server Specialized towards specific User Communities ASTREE : Max/MSP developers GAMELAN : audio production Integration MustiCASPAR with current repository Production of adequate (missing) Representation Information Production of adequate (missing) PDI