Physically Based Sound

  • View

  • Download

Embed Size (px)


Physically Based Sound. COMP259Nikunj Raghuvanshi. Overview. Background FEM Simulation Modal Synthesis (FoleyAutomatic) Comparison/Conclusions. Motivation. Sounds could in-principle be produced automatically, just like graphics: Sound Rendering - PowerPoint PPT Presentation

Text of Physically Based Sound

  • Physically Based Sound

    COMP259Nikunj Raghuvanshi

  • OverviewBackground

    FEM Simulation

    Modal Synthesis (FoleyAutomatic)


  • MotivationSounds could in-principle be produced automatically, just like graphics: Sound RenderingSound Rendering has not received much research effortMain Goal: Automatic generation of non-music, non-dialogue sound

  • Sound Production TodayMovies: Foley Artists

    Games: Anyone noticed the huge sound directory in Unreal Tournament?

  • PBS: Sound Production in NatureCollisions/Other interactions lead to surface vibrationsVibrations create pressure waves in airPressure waves sensed by earSurface VibrationPressure WaveEarVibrationPropagationPerception

  • Main Aims of PBSPhysics simulator gives contact/collision informationAssign material properties for sound, Wood, concrete, metal etc.Sound simulator generates sound using this data (in real time?)

  • ChallengesSound must be produced at a minimum of ~44,000 HzExtremely High Temporal Resolution (timesteps in the range of 10-6-10-8 s)Stiffness of underlying systems (eg. Metallic sounds. K/m~=108)Stability may require even smaller timesteps

  • Two ApproachesFEM deformable simulation O'Brien, J. F. et. al., Synthesizing Sounds from Physically Based Motion. SIGGRAPH 2001.

    FoleyAutomatic (Modal Synthesis) Kees van den Doel et. Al., FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation. SIGGRAPH 2001.

  • Main ideasDeformable Simulation (arguably) much more physically basedFoley Automatic: Additive SynthesisComponent SinusoidsSound Signal

  • OverviewBackground

    FEM Simulation

    Modal Synthesis (FoleyAutomatic)


  • Simulation RequirementsTemporal ResolutionSimulate Vibration as well as PropagationVibration Modeling: Deformable Model for ObjectsPropagation Modeling: Explicit Surface RepresentationPhysical/Perceptual Realism

  • System Structure

  • Vibration ModellingFEM with Tetrahedral ElementsLinear Basis Functions, greens strainExplicit Time IntegrationTypically #nodes = 500, #elements = 1500, dt = 10-6-10-7 s

  • Sound Propagation ModellingFluid Dynamic FEM simulation of surrounding air? Very expensive. InsteadEmploy Huygens Principle: Pressure Wave may be seen as sum of pressure waveletsReceiverReceiverPressure WavePressure Wavelets

  • Acoustic Impedance of AirSurface Vibrations and SoundPressure contribution of a patch,VelocityDensity of AirSound Propagation Speed in AirUnit Normal

  • Surface Vibrations and SoundApproximate differential elements with surface trianglesApply band pass filters:Low pass: windowed sinc filterHigh pass: DC blocking filterResult: Pressure known for all surface triangles

  • Putting it all togetherPressure/Signal at ReceiverFiltered Average PressureArea of TriangleVisibility TermApproximation of Beam PatternDistance FalloffReceiverVibration

  • Propagation DelayAccumulation BufferReceiverd1d2Sourcet=0t1= d1/ct2= d2/c12Receiver Distance from SourceSound Propagation Speed

  • Results: CapabilitiesGeneral models

    Generated sounds are accurate

    Stereo Sound

    Dopplers Effect

  • Demo

  • Results: Accuracy

  • Results: SpeedScene TimeStep(s) Nodes/Elems Time/Audio TimeBowl 10-6 387/1081 91.3/4.01 minsClamped Bar 10-7 125/265 240.4/1.26 minsVibraphone 10-7 539/1484 1309.7/5.31 mins (~1 day)

    Timings on a 350MHz SGI Origin MIPS R12K processor

  • OverviewBackground

    FEM Simulation

    Modal Synthesis (FoleyAutomatic)


  • FeaturesModal resonance model of solidsLocation dependent soundsImpact, slide, roll excitation modelsReal-time, low latencyEasy integration with simulation/animationPracticalDo not model propagation of sound from source to receiver

  • Synthesis MethodForceVibrationEmissionPropagationListenerSpeakersSound SamplesUser

  • VibrationSurface u(x,t) of body responds to external contact force F(x,t)u(x,t)F(x,t)Strain Functional Speed of SoundUnder suitable boundary conditions, the solution to the PDE is a sum of sinusoids

  • EmissionSound pressure s(t) linear functional L of surface vibration u(x,t)u(x,t)Ls(t)Note that propagation is not modeled in above

  • The Modal Synthesis Modelu(x,t)F(p,t)Ls(t)Impulse response/modal modelThe response u(x,t) of an arbitrary solid object to an external force can be described as a weighted sum of damped sinusoidsSince L is linear, it implies at s(t) must be a sum of damped sinusoids too

  • Example: A 1D string 1st Mode2nd Mode Frequency = f0Higher modes Frequency = f1= 2*f0 Frequency = fk= k*f0Main Idea: Sum contributions of all the modesThe point of impact decides the proportions in which the modes are to be mixed: ak. Therefore, ak is a function of p, the point of impactThe frequencies and damping parameters are a property of the object, and independent of how the object is hit+ +...+a0a1ak

  • The Modal Synthesis Modelu(x,t)F(p,t)Ls(t)Impulse response,modal modelParameters measured experimentallyKth mode: Gain Factor Point Damping Vibration of impact Term Frequency

  • Force ModelingImpactSlidingRollingWavetableStochasticAt runtime: Find gain parameters given the location, strength and kind of force. Synthesize sound from previous equation.

  • Impact Forces

    Duration: hardness (T)Magnitude: energy transfer (w)Multiple micro-collisions Example:

  • Sliding/ScrapingMicro-collisions lead to noisy audio-force

  • Sliding/ScrapingWavetable approachStore force parametersModulate amplitude with energy transferModulate rate with contact speedSynthesis ApproachFractal noise represents roughnessFilter through reson filterResonance ~ contact speedWidth ~ randomness of surface

  • RollingNo relative surface motionDifferences with sliding:Smoother: Use low passMore dampingHarder to createLess understoodEssential coupling?

  • Rolling: Smooth SurfacesPolyhedral objects do not lead to smooth rolling forcesInstead use smooth surfaces directly

  • Rolling: Contact EvolutionEvolve the contact in Reduced coordinatesq = (u,v,s,t, )c(u,v)d(s,t)

  • Rolling: Contact EvolutionPiecewise parametric surfaces, loop subdivision surfacesExplicit integration, no stabilizationMultiple contacts and conforming contacts are not handledUsed only when multiple contacts in close spatio-temporal proximity

  • Demo

  • Dynamic ForcesContact forceRolling speedSlipping speedImpulsesand locationsPebble-in-Wok Demo

  • Results 0.1% CPU time per mode Graceful degradation of quality The bell demo is interactive Uses a PHANToM for interaction Authors do not report any real timings State that sound quality is perception-based and has no metric as of now

  • OverviewBackground

    FEM Simulation

    Modal Synthesis (FoleyAutomatic)


  • DiscussionFEM: Physically Rigorous and GeneralToo slow for interactive applicationsDoesnt scale wellInappropriate to apply a 30fps technique to 44000fps?Maybe too general for the problem domain?

  • DiscussionModal model exploits the vibrational natureHigher EfficiencyBut, not rigorously physically basedFinding the parameters requires experimentation and earballingNo rigorous correlation between physical and perceptual parameters

  • DiscussionFor Realtime: Need for a technique to cover the middle groundExtracting modal parameters in general requires solving PDEsNot possible to do in an automated mannerApproximate modal parameters and then use modal synthesis?

  • ConclusionPBS involves orders of magnitude smaller temporal and spatial scalesResearch is sparse, problems are denseMain contributions of the two papers besides vibration modeling:FEM: Efficient modeling of sound propagationFoleyAutomatic: Efficient, Approximate models to handle surface properties and contact forces

  • ReferencesO'Brien, J. F., Cook, P. R., Essl G., "Synthesizing Sounds from Physically Based Motion." The proceedings of ACM SIGGRAPH2001, Los Angeles, California, August 11-17, pp. 529-536.Kees van den Doel, Paul G. Kry and Dinesh K. Pai, FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation Computer Graphics (ACM SIGGRAPH 01 Conference Proceedings), pp. 537-544, 2001.

  • AcknowledgementsSome images were taken from the referred papers and the corresponding SIGGRAPH slides

    #22/23 - it may be good to mention Pentland's work on modal analysis for simulating deformation #25 - what are c & g? speed of sound and gravity? Surface Properties are modelled using a wavetable which gives the relationship between excitation point, force and the associated gain coefficients. The force model which tries to model the surface properties is stochasticl.Series of microcollisions instead of one single impact for 15 ms.~lower 3 modes taken into account.Surface properties for sliding: Superimpose High frequency noise over low frequency audio force at animation speeds. Essential Coupling: Somehow ball always gives exciation force at frequencies matching the modes of the object: It kn ows about the modal frequencies.