Sound and interaction · Kirsten Rassmus-Gröhn, Avd. för Rehabiliteringsteknik, Inst. för Designvetenskaper Sound and interaction

Kirsten Rassmus-Gröhn, Avd. för Rehabiliteringsteknik, Inst. för Designvetenskaper

Sound and interaction


Sound and hearing


What is sound?

• Sound is the combination of the wave pattern in the medium (eg. air) and what the ear hears.


The cochlea and the basilar membrane

http://www.hearingaidscentral.com/howtheearworks.asp



Equal-loudness contours

http://www.phys.unsw.edu.au/~jw/hearing.html


ITD – Interaural Time Difference

Discontinuous sounds Continuous sounds (<1500Hz)


IID – Interaural Intensity Difference

> 1500 Hz


Additional directional clues

• Pinnae • Ear canal • Torso and head form:

HRTF • Reflections

• Head movements


Reflection


Interaction sounds


Operation sounds


Feedback sounds


Signal sounds


Auditory icons

• Methaphors or resemblances of real actions – Compare to visual icons (like the trash can

for deleting files) – Examples (from a drawing application) –

ex1, ex2, ex3, ex4


Earcons

• Musical symbols, synthesized tones or sound patterns for feedback – Ex1 – Ex2 – Ex3


Sonification • Use of non-speech audio to

convey information or perceptualize data

• Examples: – Maths example – Geiger counter sound – Auditory Towers of Hanoi

http://www.seeingwithsound.com/prgraph.htm�

http://www.tpb.se/barnens_tpb/spel/ljud/flash/tornspel.html�


TTS – Text-to-speech

• Stationary: Erik, Emma och Elin • Android: SWOX: Klara (2 week trial)

http://www.acapela-group.com/text-to-speech-interactive-demo.html�


Artificial 3D-sound

• IID & ITD • HRTF – Head Related

Transfer Function • Includes some amount of

”cheating” (simplification of models)


The listener

• The listener-object (”listener”) – is the placement of the virtual ”ears”

• It has: – Position – Orientation – Roll-off factor (how the volume decreases with

distance) – Dopplerfactor- when sound and listener are

moving in relation to each other


Sound sources

• Soundmanager • Sound source objects

– Sound generator (sound file or other) – Position – Min and max distance – (Sound cones )



Reverberation and effects

• Simple: reverberation, affects the entire sound

• Reflection models (computationally heavy)

• Damping models – Occlusion – Obstruction – Exclusion


Do-it-yourself

• Record your own sounds. Ingredients: – Recording equipment – Sound editing software (eg. Audacity) – Imagination (substitute ”real” sound sources

for fake ones) • And/or:

– Google ”sound effects”, or ”sound FX” – www.freesound.org and other sites

http://www.freesound.org/�


Audacity


Consider this:

• Design for diversity – Let users choose their own volume (note the

difference in Media volume and Ringtone volume) – Choose sounds that are appropriate for the (sound)

environment they should be used in – Perceptualize important information in several

modalities (audio, visual, tactile)


Limitations

• Loudspeaker output on mobile phones • Use of headphones (or not!)

– One : no stereo – Two: less surrounding sounds

• Context, context, context =o)


3D sound on Android

• Android 2.2 and below: http://pielot.org/2010/12/14/openal-on-android/

• Supposedly built-in for Android 2.3 and above

http://pielot.org/2010/12/14/openal-on-android/�

Documents

Sound and interaction · Kirsten Rassmus-Gröhn, Avd. för Rehabiliteringsteknik, Inst. för Designvetenskaper Sound and interaction