Dynamic Sound for iOS Apps and Games

  • Published on
    02-Jun-2015

  • View
    2.266

  • Download
    0

Embed Size (px)

DESCRIPTION

A talk given at http://360idev.com/ topic, covering sound design, Pure Data, libpd and examples integrating Pure Data into iOS.

Transcript

<ul><li> 1. Dynamic Audio For Apps and Games </li></ul> <p> 2. 360 iDev tackmobile.com About Me @thillerson, +thillerson (if youre into that) Developer at Tack Mobile (tackmobile.com) iOS, Android, and Mobile Web Rails, Node, maybe Elixir (one day) 3. 360 iDev tackmobile.com Rawk Star About Me 4. 360 iDev tackmobile.com About Me Bass Electronic Music Some day I hope to play this thing ---------&gt; 5. 360 iDev tackmobile.com Sound Design Work: Start Alarm App Alarm Sounds App Store: Start by Tack 6. 360 iDev tackmobile.com Sound Design Work: Adrift Puzzle Game - Adrift by Tack in App Store Background Music Sound Effects 7. 360 iDev tackmobile.com Programming Sound In Beta Soon* (*this week) http://pragprog.com/ What were talking about today... ... and much more 8. 360 iDev tackmobile.com What Were Going To Do Today Answer: What Is Sound Design? Discuss UX Principles of Sound Design Introduce Pure Data Introduce libpd Tour Two iOS Apps Embedding Pd 9. 360 iDev tackmobile.com When We Are Done, You Will Understand Sound Design Understand Procedural Audio Know what Pd is, and how libpd makes it awesome Be excited about Dynamic Audio in your apps and games 10. 360 iDev tackmobile.com What is Sound Design? 11. 360 iDev tackmobile.com A Sound Designer Ben Burtt 12. 360 iDev tackmobile.com A Sound Designer... Conceptualizes Captures or Designs Records Engineers Produces ... the sound for an experience 13. 360 iDev tackmobile.com Sound Designer Skills Understand Sound Intuitive vs. Theoretical Music vs. Sound Audio Engineering 14. 360 iDev tackmobile.com Sound Design in Mobile Apps Developers/Designers wear lots of hats Control Efficiency Asset size 15. 360 iDev tackmobile.com Sound Design Principles 16. 360 iDev tackmobile.com A Sound Designers Goals Audible Feedback Fulfill Expectations Immersion Emotional Response 17. 360 iDev tackmobile.com Sound Design and Good UX Your app doesnt need sound Expect to be turned off Dont make me listen to you Listen on lots of devices/environments Engineer for small speakers Make your sound mean something! 18. 360 iDev tackmobile.com Always Test with phone calls Test while playing music Have fallbacks - e.g. No music, just effects Provide control to the user 19. 360 iDev tackmobile.com A Good Example: Clear Realmac Software Josh Mobley Meaningful Sound demo 20. 360 iDev tackmobile.com Sound 21. 360 iDev tackmobile.com Longitudinal Waves 22. 360 iDev tackmobile.com Of Pressure 23. 360 iDev tackmobile.com Through A Medium 24. 360 iDev tackmobile.com Transverse Wave Longitudinal Wave 25. 360 iDev tackmobile.com Longitudinal Waves Of Pressure Through a Medium Sound 26. 360 iDev tackmobile.com Properties of Waves A - Wavelength (Distance), Period (Time) B - Amplitude C - Frequency - 1/Period (measured in Hertz) A B B C 1 2 3 27. 360 iDev tackmobile.com Fourier Series Based on his work -------&gt; Periodic signals composed of simple oscillating functions Sine/Cosine waves are fundamental 28. 360 iDev tackmobile.com Oscillators Something that moves according to a periodic function A physical body An electronic device 29. 360 iDev tackmobile.com Basic Electronic Sound oscillator voltage magnet pressure waves speaker 30. 360 iDev tackmobile.com Digital Audio Stream of Numbers Signal Frequency and Amplitude DAC - Digital to Analog Converter 31. 360 iDev tackmobile.com Dynamic Audio 32. 360 iDev tackmobile.com Procedural Audio Functions Objects Controls and Components Produce a Signal Building Sound from First Principles 33. 360 iDev tackmobile.com Pure Data Procedural Audio Visual Programming Environment Free and Open Source pure-data.info 34. 360 iDev tackmobile.com Pure Data 101 35. 360 iDev tackmobile.com Signals and Values Signals are streams of numbers, usually audio data Values are numbers or symbols 36. 360 iDev tackmobile.com Pds class or, kind of like a command line tool Objects osc~ 440 signal inlet inlet signal outlet class name arguments 37. 360 iDev tackmobile.com Hot vs Cold Inlets Hot inlets make objects immediately act Cold inlets usually store a value Execution is right to left 38. 360 iDev tackmobile.com Signal connections are thick Others are thin Connections osc~ 440 *~ dac~ 1 39. 360 iDev tackmobile.com Messages Used for sending values without a connection [send] and [receive] objects Message boxes 40. 360 iDev tackmobile.com libpd http://libpd.cc Peter Brinkmann - Google Engineer Enables you to embed an instance of Pd in your app iOS and Android 41. 360 iDev tackmobile.com Examples 42. 360 iDev tackmobile.com Source Code github.com/thillerson/dynamic-sound-ios Examples of capabilities, not necessarily the best architecture Goal: understand capabilities, have a place to start 43. 360 iDev tackmobile.com Example 1: A Simple Keyboard Sine wave oscillator Single voice Note on/off 44. 360 iDev tackmobile.com Example 1: A Simple Keyboard Patch osc~ dac~ receive midinote mtof receive gate line~ *~ ; midinote 60; gate 1; ; gate 0$1 1 print 45. 360 iDev tackmobile.com Example 2: A Game Top down perspective space shooter Code by Lucas Jordan from iOS 5 Game Dev -&gt; Now with sound! 46. 360 iDev tackmobile.com Example 2: A Game Patch load a square wave at startup Thruster noise~ *~ samphold~ dac~ hip~ 200 phasor~ 800 line~ $1 200 *~ 0.1 receive thruster BulletShip_HitAsteroid_Hit samphold~ tabosc4~ square dac~ line~ phasor~ 8000 tabosc4~ square line~ unpack f f 0 $1 tabosc4~ square line~ unpack f f 0 $1 hip~ 1000 unpack f f 0 $1 *~ 0.5 receive asteroid receive ship receive bullet 8-bit-ify these sounds cut low frequencies comment comment Test_Messages ; bullet 25 200; ; thruster 0 ; thruster 1 pd wavesetup ; ship 1000 50; ; asteroid 200 100; 47. 360 iDev tackmobile.com Further Reading puredata.info &amp;&amp; libpd.cc youtube.com/user/cheetomoskeeto http://designingsound.org mitpress.mit.edu/books/designing-sound 48. 360 iDev tackmobile.com Questions 49. Thank you! Dynamic Audio Tony Hillerson Were Hiring! careers@tackmobile.com Excellent Team Awesome Projects Great Office </p>