of 38/38
Nov 30, Fall 2006 IAT 410 1 Audio Sound Audio synthesis

Nov 30, Fall 2006IAT 4101 Audio Sound Audio synthesis

  • View
    213

  • Download
    0

Embed Size (px)

Text of Nov 30, Fall 2006IAT 4101 Audio Sound Audio synthesis

  • Slide 1
  • Slide 2
  • Nov 30, Fall 2006IAT 4101 Audio Sound Audio synthesis
  • Slide 3
  • Nov 30, Fall 2006IAT 4102 Audio Perception Sound: Pressure waves in frequencies between 50Herz - 22,000Herz Lower frequencies more felt by the whole body than heard Sounds can be perceived as coming from a location Not terribly accurate Cone of confusion
  • Slide 4
  • Nov 30, Fall 2006IAT 4103 Cone of confusion Cone-shaped zones in front of and behind head 3D Audio cues: Interaural Time Difference Interaural Intensity Difference Pinnae filtering Body filtering 3D Audio Perception
  • Slide 5
  • Nov 30, Fall 2006IAT 4104 3D Audio Perception Goal for 3D sound is Spatialization The sense that the Sound originates outside your head Sound has a direction Interaural Time Difference The more extremely left or right, the greater the difference Time difference < 5ms
  • Slide 6
  • Nov 30, Fall 2006IAT 4105 3D Audio Perception Interaural Intensity Difference Head absorbs and reflects sound energy The first ear to get sound gets loudest sound Head Shadow Cone of confusion: Time difference too small to detect Intensity is similar in both ears
  • Slide 7
  • Nov 30, Fall 2006IAT 4106 Pinnae Filtering Outer ear (Pinna) shape filters sound based on its direction Childhood learning trains brain to associate filtering effects with direction Unique per person Record directional white noise Microphone in ear canal Sounds from speakers located about head
  • Slide 8
  • Nov 30, Fall 2006IAT 4107 Pinnae Filtering A Generic Pinna can be simulated Record directional white noise received by dummy head Body filtering Reflection and absorption Included in Pinna model
  • Slide 9
  • Nov 30, Fall 2006IAT 4108 Head-Related Transfer Function HRTF is the general term Transformation of real sound to spatialized sound Best delivered by earphones
  • Slide 10
  • Nov 30, Fall 2006IAT 4109 Environmental Effects Sound exists in an environment Bounces off objects Is absorbed by objects Simple effects Reverb: Simulate the environmental echo Echo is the attenuated signal Gives a richer room-like feeling Larger room has longer time delay
  • Slide 11
  • Nov 30, Fall 2006IAT 41010 Audio signals Nyquist limit: Must sample signal at least twice as frequently as highest reproducible frequency Audio: 44.1KHz (CD) 22KHz 11KHz (Analog AM Radio) 8KHz (Telephone)
  • Slide 12
  • Nov 30, Fall 2006IAT 41011 Audio - Digital Implications 44,100 Hz 44,100 Samples/sec 16-bit samples Stereo 172KBytes/sec Specialized hardware - Sound card
  • Slide 13
  • Nov 30, Fall 2006IAT 41012 Reproduction Sampling Record sounds by whatever means Synthesis Analog Synthesis FM Synthesis Wavetable Synthesis
  • Slide 14
  • Nov 30, Fall 2006IAT 41013 Control MIDI - Musical Instrument Digital Interface Developed to control music synthesizers Details of synthesis are controlled by synthesizer MIDI data Sets synthesis parameters Sets music sequence
  • Slide 15
  • Nov 30, Fall 2006IAT 41014 Synthesis Analog Synthesis Simple sum of frequencies Select from a palette of source frequencies Sum of frequencies is filtered FM One frequency is controlled by another Wavetable Digitize audio signals
  • Slide 16
  • Nov 30, Fall 2006IAT 41015 Analog Synthesis Fouriers observation Any signal can be created as the sum of sine waves Square wave: Infinite sum f + 2f + 4f Synthesizer: Collection of oscillators
  • Slide 17
  • Nov 30, Fall 2006IAT 41016 Frequency Domain A perfectly periodic signal plotted in the frequency domain (Time domain plot)
  • Slide 18
  • Nov 30, Fall 2006IAT 41017 Spectrum Spectrum represents the set of frequencies present in the signal
  • Slide 19
  • Nov 30, Fall 2006IAT 41018 Filters Eliminate part of the signal by removing certain frequencies Analog filters dont have these square response shapes Band pass Bandwidth
  • Slide 20
  • Nov 30, Fall 2006IAT 41019 FM Synthesis Modulate the frequency of a wave Carrier frequency is modulated by Modulator signal
  • Slide 21
  • Nov 30, Fall 2006IAT 41020 FM Synthesis for Synthesizers The greater the Modulator amplitude, the greater the Carrier frequency variation Higher Carrier bandwidth DX: Carrier and Modulator are musically- tuned frequency Depends on the note you are playing Controls the harmonic content of a note
  • Slide 22
  • Nov 30, Fall 2006IAT 41021 Wavetable Synthesis Collect a sample of the real sound Issues: Reduce memory load by looping sample Shift pitch instead of sampling each individual note Apply interpolation techniques to make pitch shifting work right
  • Slide 23
  • Nov 30, Fall 2006IAT 41022 Audio Path Raw Sound (Sample, FM, etc) Tuned MIDI Note Sequence Resampling Envelope Loudness Control Mixing/ Combination Reverb, Environment. Spatialization
  • Slide 24
  • Nov 30, Fall 2006IAT 41023 Wavetable Synthesis Example Leyanda (Guitar) Leyanda (CDShaw)
  • Slide 25
  • Nov 30, Fall 2006IAT 41024 Interactive Sound Goal Want to enhance the interactive experience Give the user a sense of presence Add to the emotional content of the game Make it more fun
  • Slide 26
  • Nov 30, Fall 2006IAT 41025 Interactive Sound Music Sound effects Noises Commentary - Sports Narrative Conversations
  • Slide 27
  • Nov 30, Fall 2006IAT 41026 Interactive Problems Regular music composition has Beginning Middle End Interactive user control makes this difficult Some genres have this structure
  • Slide 28
  • Nov 30, Fall 2006IAT 41027 Interactive Music Game genres with order Sports Racing Fighting Semi-Ordered Puzzle Adventure
  • Slide 29
  • Nov 30, Fall 2006IAT 41028 Music Genres The Infinite Loop Theme and variation plays forever Pomp & Circumstance Diablo Problems: 30 second piece repeated over 6 hours! 720 repetitions! Diablo example: 12 Repetitions/hour
  • Slide 30
  • Nov 30, Fall 2006IAT 41029 Repetition Solutions Make the Dominant theme hard to find No catchy theme! Create a variety of textures Make only transitions stand out Where repetition is small Dont repeat musical phrases
  • Slide 31
  • Nov 30, Fall 2006IAT 41030 Music Strategies Play Win or Lose music Music must be long enough to be meaningful Music may be so long that the game situation changes before completion Very short music makes little sense Interrupt current music Sounds jarring
  • Slide 32
  • Nov 30, Fall 2006IAT 41031 Modules Modular chunks Each segment of the game plays independently of others Some thematic relation Disjointed
  • Slide 33
  • Nov 30, Fall 2006IAT 41032 Music Strategies Compose many themes in parallel Switch between themes Connect modular components together
  • Slide 34
  • Nov 30, Fall 2006IAT 41033 Analogy: Parallel Trains N trains of music running in parallel Each train serves an emotional purpose Train A: Calm Train B: Rising Excitement Train C: Climactic moments Train D: Falling Excitement Generally, Train A would be most commonly played
  • Slide 35
  • Nov 30, Fall 2006IAT 41034 Within Each Musical Train Each Car contains a few bars of music Switch between trains when a Car is complete Dont switch in the middle of a Car Simple version: Each musical phrase ends on last bar of Car Complex: Notes at end are carried over to next
  • Slide 36
  • Nov 30, Fall 2006IAT 41035 Parallel Trains Bars 1-8Bars 9-16Bars 17-24Bars 25-32Bars 1-8Bars 9-16Bars 17-24Bars 25-32Bars 1-8Bars 9-16Bars 17-24Bars 25-32Bars 1-8Bars 9-16Bars 17-24Bars 25-32
  • Slide 37
  • Nov 30, Fall 2006IAT 41036 Parallel Trains: Shuffle Cars Shuffle cars Instead of playing cars in order Problem: Random cars sound like random radio tuning Must determine Appropriate car pairings Reasonable paths
  • Slide 38
  • Nov 30, Fall 2006IAT 41037 Repetition In Trains Use repeated phrases carefully Maybe use a statistical tool to analyze paths Bayesian nets Endings: Use transitions as an opportunity to End Use next Car to Begin new series
  • Slide 39
  • Nov 30, Fall 2006IAT 41038 Composing a Train Create a piece with all layers Piece can probably survive a layer or two removed Variation = piece with layer removed Be careful with prominent instruments Fallback: Use instruments with similar acoustical properties Piano, Organ, Woodwinds No Trumpets, Drums or Screaming guitar!