Review of MUSIC PERFORMANCE by Caroline Palmer Ann. Rev. Psychol. 1997, 48:115-38 Professor, Dept of Psychology Canada Research Chair Cognitive Neuropsychology

Review of MUSIC PERFORMANCE

by Caroline PalmerAnn. Rev. Psychol. 1997, 48:115-38

Professor, Dept of PsychologyCanada Research ChairCognitive Neuropsychology of PerformanceMcGill University, Montreal, Quebec, Canada

Presented by Elaine ChewOn January 11, 2006, as part of ISE 599: Topics in Engineering Approaches to Music Cognition- Computational Models of Expressive Performance

2006-01-11 ISE599 (ISE575b / CSCI575b / EE675b pending): Computational Modeling of Expressive Performance 2

AGENDA

INTRODUCTION INTERPRETATION PLANNING MOVEMENT


Forms of Performance

Sight-reading Performing well-learned music from

memory or notation Improvising Playing by ear


Serial Order and Timing Issues

Skilled serial action: • speaking, typing, performing music

Activity must be centrally linked• Little time for feedback for planning• Can be performed w/o kinesthetic feedback

Accurate temporal control: rhythm• Basis for dev models of timing mechanisms• Consensus on requirements for accuracy


Purpose of psychological studies

Develop theories of performance mechanisms (cognitive/motor constraints)

Explain treatment of structural ambiguities Understand relationship between

performance and perception


Components of performance

Interprete piece conceptually Retrieve musical structures and units

from memory Prepare for production Transformed into appropriate

movements


Methodological Issues

Wealth of data• Separating signal from noise• Focus on movement-based information

Judgement of representative piece• Recognized level of performer expertise• Large samples of data hard to find• Rely on converging evidence from both

small and large sample studies


Performance Expression

Variations in timing, intensity (dynamics), timbre, and pitch • form the microstructure of a performance• differentiate it from another of the same pc

Measurements• deviation from fixed or regular values as

notated in score• Relative to performance itself, e.g. pattern of

deviation with repect to a unit s.a. a phrase


AGENDA



System of Communication

Chain of events …


System of Communication

Composers code musical ideas in notation

Performers recode from notation to acoustical signal• Includes performer’s conceptual

interpretation of composition

Listeners recode from acoustical signal to ideas


Interpretation

Performer’s individualistic modeling of a piece according to their own ideas or musical intentions

In western music notation:• Pitch and duration (clear)• Intensity and tone quality (approx)• Group boundaries, metrical levels higher than the

bar, patterns of motion, tension, and relaxation (unspec, implicit)

Could explain inter- and intra-performer performances of the same piece


Role of Analysis

Every performance involves some kind of interpretation or analysis

Analysis offers explanations for the content of a composition as a• Hierarchy of whole/part relations• Linear course following harmonic tension• Series of moods that result in unity of character

Analysis does not indicate how a performer actually produces a desired interpretation


Goal of Interpretation

Convey the meaning of the music• Structure, emotion, and physical movement

Highlight particular structural content Highlight particular emotional content


Highlighting structural content

Nakamura (1987): • Compared musicians’ performances of baroque sonata with

notated interpretations of dynamics• Perceived dynamics matched intended fairly well, even when

underlying acoustic changes were not identifiable Palmer (1989):

• Compared pianists’ notated intepretations of phrase structure and melody with expressive timing patterns

• Melody lead and slowing of tempo at phrase boundaries observed• Expressive timing patterns decr when attempting to play w/o

interpretation, incr in exaggerated interp Palmer (1988):

• Expressive timing patterns incr from novices to experts, during practice of unfamiliar piece, changed in diff interp by same perf


Implication of structural content interp

Palmer (1992):• Pitch deletions tend to occur within

phrases, and pitches tend to persevere at phrase boundaries

• Interpretations strengthen phrase boundaries relative to other locations

Palmer & van de Sande (1993, 1995):• Melodic events are correctly retrieved and

produced relative to nonmelodic events


Goal of Interpretation

Convey the meaning of the music• Structure, emotion, and physical movement

Highlight particular structural content Highlight particular emotional content


Highlighting emotional content

Langner (1953): Music shd sound the way moods feel Gabrielsson (1995), G. & Juslin (1996): • Compared performers’ interp of emotional content with

their use of expression• Happy/angry - faster, larger dynamic range• Soft/sad - slower, smaller dynamic range

Ashkenfelt (1986):• Similar results in tender/aggressive experiments

Schmalfeldt (1985), Shaffer (1995):• Emotional content as part of narrative, dramatic char,

thematic content, conceptions of large-scale structures


Role of experience

Musical experience enhances ability to use and identify interpretations

Nonmusicians can pick up interpretative aspects of performance• Discern general differences among mechanical,

expressive, exaggerated perf• Can hear intended phrase structure• Cannot always find melody interpretation

Sufficiency of expressive features to convey intepretations


AGENDA



Planning and Memory

Related to melodic, harmonic and diatonic structures• Chord errors occur more in homophonic mus• Single note errors more in polyphonic music• Mistakes originate more from key of piece• Mistakes tend to be of same chord type• Child singing pitch errors tend to be harmonically

related to intended events• Pianists’ sight-reading errors in pcs with deliberate

pitch alterations indicate tacit melodic/harmonic knowledge


Subsequence Partitioning

Partitioning into phrases• Errors originate more from same phrase• Interacting errors rarely crossed phrase

boundaries (like in speech)• Errors increased when melodic, metrical,

rhythmic accents unaligned Planning ahead• Eye-hand span 7-8 events, or to phrase end• Range of planning affected by serial & struct


Syntax of Musical Structure

Events at most salient levels are commonly emphasized in performance• Tactus: foot tapping metric level• Phrase: partitioning of melody

More important events are processed at deeper hierarchical (structural) levels• Improvisations tend to retain only

structurally important events from abstract hierarchical levels of reduction


Structure-Expression Link: Phrases

Decrease of tempo/dynamics at end of phrases Amt of slowing at a boundary reflects depth of

phrase embedding More important segments have greater phrase-

final lengthening Greatest corr bet expr timing and intensity

found at interm phrase level Performers’ notated/sounded interpretations

differ most at levels lower than phrase


Structure-Expression Link: Meter

Events on strong beats often lengthened, have delayed onsets

Events on metrical accents louder, longer, more legato

Listeners’ judgements of metric interpretation aligned best with experienced pianists’ intended meter

Articulation most often used as metric cue, loudness not always present

No one set of necessary and sufficient expressive cues to denote meter exist


Structure-Expression Link: Rhythm

Systematic deviations in Vienese Waltz:• Short 1 - long 2 - 3


Expressive Timing Patterns

Structure• Meter, accent pattern, simplicity (dur ratios)

Motion• Rapidity, tempo, forward movement

Emotion• Vitality, excitedness, playfulness


Comments

Melodic/metrical accents sometimes altered by presence of rhythmic accents or each other

Melody lead may serve to separate voices perceptually


Generative model of expr synthesis

Clarke (1993, 1995):• Systematic patterns of expression result from

transformations of the performer’s internal representation of musical structure

Support for view: the abilities to• Replicate same expressive timing profile with little

variation across performances• Change interpretation and produce different

expression with little practice• Sight-read with appropriate expression


Rule-based models

Sundberg et al (1983ab):• Differentiation, grouping, ensemble rules affect

event durations, intensities, pitch tunings, and vibrato

Clynes (1977,1983,1986):• Composer-specific inner pulses applied to different

levels of musical structure

Piece-specific factors contribute as much as piece-transcendent factors captured by rules


Arguments against generative models

Performers can imitate expressive timing patterns with arbitrary relationships to musical structure

Accuracy worse with more disruptive structure-expression relationship, improved with repeated attempts

Suggests expression not generated solely from structural relationships


Perceptual Functions

Communicate particular interpretations and resolve structural ambiguities

Compensate for perceptual constraints of auditory system


Other explanations for expression

Compensatory explanation:• Some notes played louder/longer because they would

be heard softer/shorter otherwise

Musical structure elicits expectations:• Detection of lengthening more difficult where expected• Detection accuracy inversely related to performer’s

natural use of lengthening in same piece

Structure constrains both perception and performance


Music Theories

Narmour (1990,1996):• Model of melodic expectancy

Lerdahl (1996):• Model predicting tonal tension and relaxation

Listeners can apprehend predicted structures Expressive cues emphasize computed structures Interpretations constrained by composition


AGENDA



Movement

Musical rhythm often defined relative to body movement

Different views on relationship:• Motor control - movement generating timing• Timekeeper - internal clock for anticipation

and coordination of gestures


Timekeeper models

Role: regulate and coordinate complex time series, such as those produced between hands or between performers

Constructs beats at abstract level, providing temporal reference for future movements

Evidence: rhythm reproduction better for integer duration ratios


Internal clocks

Single clock model Multiple timekeepers (Jones 1990 review) Attributed to perceptual encoding Attributed to production mechanisms


Clock operation level

Tactus: most salient metrical level Preferred tactus ~ 600ms (spontaneous

clapping period) Typical inter-step interval ~ 540ms Listeners use motion to describe rhythmic

patterns when interbeat intervals ~ 650ms Time periods derived are multiples or

fractions of beat periods


Source of temporal variance

Early models: partitioned temporal variance to lack of precision of timekeeper vs. motor response delay

Extended to hierarchical organizations of timekeepers at multiple metrical levels• Performed durations at metrical level less

variable than durations of residual nested events within that level


Hierarchical clocks

Timekeeping most directly controled at intermediate metrical levels of the sub-beat, the beat, or the bar

Solo piano music: timekeeping controlled at the beat level (hands have independence in coordinating events below beat level)

Separate timekeepers controled timing of individual hands

Duet piano performance: highest precision (least variance) at bar level

(above studies assumed constant global tempo)


Performance timing stability

Not only at tactus / beat / bar level Exists at level of entire piece. Durations of

string quartets over repeat performances highly consistent. • Std dev of piece duration ~1%• Less than variations in movement lengths

Proportional tempos theory: tempos of successive sections of music form simple integer ratios

Phase synchrony, esp at structural boundaries May reflect performer’s memory for tempo


Movement

Musical rhythm often defined relative to body movement

Different views on relationship:• Motor control - movement generating timing• Timekeeper - internal clock for anticipation

and coordination of gestures


Motor Programs

Contains representations of internal actions and processes that translate them into movement sequence

Accounts of motor equivalence across contexts

Possible proof: Relational invariance - tempo changes as parameter change


Relational invariance

Relative durations of notes tend to vary across performances played at different tempi

Hypothesis: structural interpretation does not remain constant across performance tempo• # group boundaries incr at slower tempo

Practicing at different rate than intended performance might be counterproductive

Lesson: do not draw conclusions from average of performances over diff tempi


Tempo changes perceived structure?

Tempo affects perception of duration patterns• Different perceptions may result for same relative

expressive timing pattern at different tempo

Repp (1995b):• Manipulated degree of expressive timing and

global tempo• Listeners preferred reduced expression with fast

tempo and augmented expression w slow


Kinematic models

View: music performance and perception have origins in kinematic/dynamic characteristics of typical motor actions

E.g. walking -> beat Aesthetically satisfying performances

should satisfy kinematic constraints of biological motion


Kinematic models

Final ritards modeled as variable curve followed be linear decrease in tempo

Feldman et al (1992): cubic polynomial models used to minimize jerk/jumpiness in connecting points of tempo changes

Repp (1992b): used quadratic


Models with dynamics

Studies suggest coupling bet expr timing and dynamics

Todd (1992): proposed model where intensity proportional to square of vel. Used constant acceleration

Todd (1995): proposed auditory model of rhythm performance and perception• Temporal segmentation of onsets• Periodicity analysis• Sensory-motor feedback: tactus, body sway


Arguments agains kinematic models

Physical notions of energy cannot be equated with psychological concepts of musical energy

Tempo changes guided by perceptual rather than kinematic properties:• Large tempo changes cannot occur too

quickly (perception to rhythmic categories)


AGENDA



MUSIC PERFORMANCE

Empirical research review Sequence planning research review Motor control Perceptual consequences


MUSIC PERFORMANCE

Empirical research review• Conceptual interpretations• Retrieval from memory of musical structures• Transformation into motor actions

Sequence planning research review Motor control Perceptual consequences


MUSIC PERFORMANCE

Empirical research review Sequence planning research review• Hierarchical and associative retrieval influences• Style-specific syntactic influences• Constraints on range of planning

Motor control Perceptual consequences


MUSIC PERFORMANCE

Empirical research review Sequence planning research review Motor control• Internal timekeeper models• Motor programs• Kinematic models

Perceptual consequences


MUSIC PERFORMANCE

Empirical research review Sequence planning research review Motor control Perceptual consequences• Successful communication of interpretations• Resolution of structural ambiguities• Concordance with listeners’ expectations

Documents

Review of MUSIC PERFORMANCE by Caroline Palmer Ann. Rev. Psychol. 1997, 48:115-38 Professor, Dept of Psychology Canada Research Chair Cognitive Neuropsychology