Sound follows function

wew

Sound follows functionSound communication and the relevance of timbre.

Lecturer: Rainer Hirt & Kai Bronner (audio-branding.de)

wew

01 Acoustic signs and its functions02 Overview ongoing Functional-Timbre-Study (FTS)

wew

01 Acoustic signs and its functions02 Overview ongoing Functional-Timbre-Study (FTS)

wew

01 Acoustic signs and its functions

wew

A. Basics

A. Basics B. Study C. Results

Signs?


Signs!A sign is a stimulus pattern that has a meaning.

A sign stands for something else.


Semiotics: Theory of signs

IndexAn „Index“ is defined by some sensory feature(something directly visible, audible, smellable, etc) that is connected to it.



IconAn icon is a pattern that physically resembles what it „stands for“.



SymbolSymbols are arbitrary and unmotivated, reliant on conventional usage to determine meaning.


Acoustic signs?


Acoustic signs!

Index Icon SymbolIndex becomes Icon

Symbol becomes Icon


Applications of acoustic signs

Auditory Display Audification Auditory-User-Interface

Corporate SoundCultural


Basic design-approaches of acoustic signs:

Auditory iconsAuditory symbolsAuditory symcons (Mixed)

wew

B. Empirical study


Ambition & research questions

1. Appropriateness of Auditory icons/symbols and symcons for the transfer of information-functions:–> Advice–> Warning–> Emergency

2. Effects of the mixture of icon and symbol („auditory symcon“)

3. Advantages of acoustic signs for „public-user-interfaces“


Analysis of a situation– Talks with manfactures and customers (Höft-Wessel, ICA, DB)

– Secondary literature

– Preliminary study (semantic field-analysis)


Study objectTicket machine


Study designStudy method: Descriptive study

Survey sample size : N=22 Age average: 35 yearsWomen/Men: 10/12


Lab set-up> Lectern (Terminal-simulation)> Table-PC (Touchpad)> Headphone


Stimuli

321

321

symboliciconic

symconic

Advice: „Ticket ready for collection“

Advice: „Money ready for collection “

Warning: „Timeout“

Emergency: Deterrence

Context-Soundscape (Trainstation)

321

321


Evaluation methodsSemantic differential


Evaluation methodsSemantic differentialCategorical scaling


Evaluation methodsSemantic differentialCategorical scalingResponse time measurement


Evaluation methodsSemantic differentialCategorical scalingResponse time measurementParticipant observation


Evaluation methodsSemantic differentialCategorical scalingResponse time measurementParticipant observationAssociation analysis

wew

C. Results



1. Appropriateness of Auditory icons/symbols and symcons for the transfer of information-functions:–> Advice–> Warning–> Emergency

2. Effects of the mixture of icon and symbol („auditory symcon“)



Auditory Symbol Auditory Symcon

Advice

Warning

Emergency

Funtion(intention) partially transferred

Funtion(intention) transferred

Funtion(intention) not transferred or false interpretation

Auditory Icon







The meaning of function of acoustic signs is comprehended intuitively.


Further investigationsThe study reveals amongst others, that further investigation on the acoustical parameter „timbre“ is required.

In the literature there is nearly no advice for selecting timbre according an information-function.

Thus, an ongoing study that addresses the perception of timbre is topic of the further presentation.

wew

02 Timbre and function

A. Timbre: definition & properties B. Sound perception C. Study design

A. Timbre: definition & propertiesB. Sound perceptionC. Study design

wew

A. Timbre: definition & properties


Timbre – what? Definition

Definition (American Standards Association 1960) Timbre is that attribute of auditory sensation in terms of which a listener can judge that two sounds similarly presented and having the same loudness and pitch are dissimilar.

A B


Timbre properties

Verbal descriptionsovertone constellation: bright, dark, mellow, hollow, purenoise content: raspy, breathy, hoarse attack: smooth, abrupt, sharp, gentle, easing

Relationspitch - frequencyloudness - amplitudetimbre - multiple interacting acoustic factors


Timbre properties

H. Fletcher (1934)Experiments show that a simple one-to-one relationship does not exist betweenthe two sets { loudness, pitch, timbre } and { sound intensity, fundamental frequency and overtone structure }

sound intensity

fundamental frequency

overtone structure

loudness

pitch

timbre


Timbre propertiesMore than one dimension: multidimensionality

Erickson (1975)Clearly timbre is a multidimensional stimulus: it cannot be correlated with any single physical dimension.

Plomp (1982)Sounds cannot be ordered on a single scale with respect to timbre. Timbre is a multidimensional attribute of the perception of sounds.


Multi-Dimensionality-Scaling MDSMDS studies reveal perceptual dimensions correlated with acoustic parameters corresponding to spectral, temporal and spectrotemporal properties of the sound events.

– experiments with paired sound stimuli– rating of similarity– axes corresponding to timbre‘s main perceptual dimensions –> timbre space


DimensionsDimension I = brightness Low brightness : french horn and cello

High brightness : oboe, muted trombone

Dimension II = spectral fluxHigh synchronicity and low fluctuation :

clarinet, saxophone

Low synchronicity and high fluctuation :

flute, violoncello

Dimension III = attack quality More transients : strings, flute

Fewer transients : brass, bassoon

Dimension I

Dimension IIIDimension II


Timbre dimensions and related parametersAcross studies, several acoustic/physical parameters corresponding to timbre dimensions were found:

– Spectral centroid

– Spectral deviation

– Spectral density

– Attack time

– Decay time

–Amplitude envelope

– Spectral flux

– Pitch strength

– Attack synchronity

– Attack centroid

– Noisiness

...

wew

B. Sound perception


Timbre and psychoacoustic properties Interview Dr. Gerhard Thoma, chief sounddesigner BMW:(http://www.goethe.de/ges/wrt/dos/aut/sou/de2169434.htm , 10/2007)

(...) There are sets of regression equations that help calculating/predicting a desired sound perception, for example a sporty sound:

a x sharpness + b x roughness + c x fluctuation strength, … = sportiness.!!

About 80 % of people will evaluate the sound in terms of sportiness.But:„acid test“in driving situation (context) = proof of validity!(...)


Psychoacoustic properties and cognitive/affective evaluation – simple psychoacoustic parameters only refer to fixed relations of perception and instantaneous stimuli (Haverkamp 2007)

– psychoacoustics are only capturing parts of human sound perception, largely excluding important affective + cognitive evaluation (Vjästfall/Kleiner 2002)

– any endeavour to evaluate (product) sounds by psychoacoustics solely will fail in the end (Blauert/Jekosch 1997)


Sound perception: target groups/context– individual differences of preferences of particular groups of people for qualitatively different product sounds– no simple „universal“ manipulation that will have the same effect on the sound quality of products (Spence & Zampini 2006)

–> target groups (age, cultural background, milieus/lifestyle) have to be taken into account–> it is important to consider the kind/type of product = context


Emotional evaluation and functional contentModel of affective circumplex (Russell, 1980)

Activation

Deactivation

PleasantUnpleasant

tense

nervous

stressed

upset

alert

excited

elated

happy

sad

depressed

bored

contented

serene

relaxed

Emergency

Warning

AdviceConfirmation


Sound perception:evaluation of sound (timbre)

To be taken in account:

– psychoacoustic properties of sound

– learned associations

– cultural background and personal experiences (target group)

– context: mood of subjects, expectations, situation

– influence from other modalities (vision, touch,...)

wew

C. Study design


Functional Timbre Study („sound-colour the Gestalt“)VARIABLES (TIGA)

Independent:Timbre T Gestalt G [pitch contour, rhythm, tone duration]

Dependent:Information-function I [function, e.g. emergency, warning, advice/confirmation]

Attribution A [emotional/affective evaluation: warm, harsh, sharp,...]

A. Timbre: defi nition & properties B. Sound perception C. Study design

Sound-colour the GestaltApproach:sound-colour T the Gestalt G and examine evaluations of subjects regarding information-function I and attribution A


Example of stimuliSame Gestalt, different timbre

A B


Study DesignStep 1: free association task3 Gestalt x 8 Timbre = 24 sound-stimuli are evaluated by subjects–> pool of attributes

Step 2: rating of stimuliRating of 24 stimuli on Likert scales of attributes obtained from step 1 + assessment of given information function

Step 3: analysis of resultsPrincipal-Component-Analysis, Generalized-Linear-Models.Examination of correlations between acoustic properties, rated attributes, information function.

–> Findings and results serve as a basis for further investigations!


Study Design Timbre selectionTimbre differing in parameter values („timbre dimensions“)

spectral content:

temporal content:

attack/cutoff:

static dynamic

bright

hard soft hard soft

static dynamic

dull/dim

hard soft hard soft


Study Design Gestalt creation Typical forms of sound-gestalt (learned associations)descending major third of a doorbell („ding dong“) (association: welcome) wailing siren, in germany a fourth: a-d a-d (police or ambulance) (association: emergency - alert)

Scherer/Osinsky 1977Tonality: major mode: indicative of pleasantness and happiness minor mode: disgust and anger

Rhythm: rhythmic: more active, fearful, surprised nonrhythmic: boredom

Activity = Fast tempo, high pitch level, many harmonics, large pitch variation, sharp envelope, small amplitude variation


Fields of expertise/collaborationRelevant fields of expertise:music psychology, sound-/acoustic-communication, sounddesign, sound-/music-computing, statistical methods –> interdisciplinary team

We are not the only ones:Product Sound Design Group TU DelftMusic Technology Group BarcelonaMusic, Mind and Machine Group (MIT Media Lab , USA)Music Information Retrieval Community –> exchange of findings, knowledge + possible collaborations


What is the use??

Product Sounddesign

Audio-Branding

AUI (Auditory User Interfaces)

Music Information Retrieval MIR (music catalogues, sound-databases)

Mpeg7-Classification

Electronic Music Distribution

„Perceptual Software-Synthesizers“

...

wew

Thank you!

Documents

Sound follows function