10/13/10Psyc / Ling / Comm 525 Fall 10 Kim & Osterhout (2005) JML The independence of combinatory semantic processing: Evidence from event-related potentials

10/13/10 Psyc / Ling / Comm 525 Fall 10

Kim & Osterhout (2005) JML

The independence of combinatory semantic processing: Evidence from event-related potentials


Background• Everyone believes

– N400 reflects semantic processing difficulty– P600 reflects form-related processing difficulty

• However, 2 studies have found P600 when they expected to find N400– Kolk et al. (2003) found P600 at joeg

• De vos die op de stropers joeg …

• The fox that the poachers hunted …

• = The fox that hunted the poachers …

– Kuperberg et al. (2003) also found P600 at eat

• For breakfast, the eggs would only eat …

• In both cases, there was a noun that was plausible in some role of the verb’s event, just not the role its position indicated it had


Experiment 1 - Stimuli

• Active Control– The hungry boy was DEVOURING the cookies.

• Passive Control– The hearty meal was DEVOURED by the kids.

• Anomaly– The hearty meal was DEVOURING the kids.


Procedures/Design• Sentences presented word-by-word centrally • SOA = 650 msec (slow!)• End-of-sentence acceptability judgments

• Expt 1– N = 24– 96 sets of 3 sentence versions (32)– 107 distractors, some sem anom, some ungramm

• Numbers varied across lists to make acceptability ~50/50

– 96 + 107 – 203 trials (58% acceptable, 42% unacceptable)

• Expt 2– N = 29– 96 sets of 3 sentence versions (32)– 112 distractors, some sem anom, some ungramm

• Numbers varied across list to make acceptability ~50/50

– 96 + 112 = 208 (50% acceptable, 50% unacceptable)


Experiment 1 - Results

P600 P600


Experiment 1 - Discussion

• Why does “The hearty meal was devouring…” evoke P600 rather than N400?– Because hearty meal can play SOME thematic role

in a devouring event?– Maybe the fact that hearty meal fits so well with

devouring makes the processing system think there’s a grammatical error, like the wrong inflection on devouring, rather than a semantic anomaly


Experiment 2 - Stimuli

• Passive Control– The hearty meal was DEVOURED …

• No-attraction Violation– The dusty tabletops were DEVOURING …

• Attraction Violation– The hearty meal was DEVOURING …


Experiment 2 - Results

N400


Experiment 2 - Discussion

• Argue that these results show that– Semantic processing can “drive” sentence

comprehension– Rather than always having to wait for

structural processing to give the relationships among words before their semantic combination can proceed


The Role of Prosody

• Embedded Clause / Direct Object sentences can be disambiguated with prosodic phrasing

• Acoustic correlates:– Pause– Pre-boundary lengthening– Pitch contour– Pitch reset– …


How does this kind of prosodic boundary marking influence

sentence interpretation?

• Example of DO Prosody– The basketball star accepted the contract…

…because it paid so well.

Example of Clause Prosody- The basketball star accepted …… the contract requires him to play every

game.


Do speakers actually produce different prosody in DO and Clause

structures?

Gahl & Garnsey (2004)


Boundaries marked more strongly when

Structure not consistent with Verb Bias


Using ERPs to Study Prosody

• On-line nature of ERPs especially good for investigating immediate effects of prosody during spoken sentence processing

• Steinhauer, Alter, & Friederici (1999)– Discovered a positive ERP component at

prosodic boundaries– “Closure Positive Shift” (CPS)


Closure Positive Shift (CPS)

Steinhauer, Alter & Friederici (1999)


CPS

• Demonstrated CPS with– Delexicalized speech (Steinhauer & Friederici 2001)

– Jabberwocky sentences (Pannekamp et al. 2005)

– Pseudosentences (Pannekamp et al. 2005)

– Hummed sentences (Pannekamp et al. 2005)

– Musical phrases (Knösche et al. 2005)

– Commas in orthographic stimuli (Steinhauer 2003; described in Frazier, Carlson, & Clifton 2006)

• But only for people with good knowledge of comma rules


CPS Studies from Other Labs

• Kerkhofs et al. (2007)– Embedded stimuli in discourses– Found smaller CPS when discourse made a

boundary highly predictable– Casts some doubt on CPS as “pure prosody”

processing


ERP Prosody Study(Jackson, Patel, & Garnsey, 2010)

• Structure of sentence completion– Beginnings ambiguous

The basketball star accepted the contract…

– Direct Object (DO) ending…because it paid so well.

– Embedded Clause (Clause) ending …requires him to play every game.

• All Verbs DO-Bias & Critical Nouns plausible as DO– To maximize garden-pathing, to have the best possible

chance to see whether prosody can prevent it


Stimuli

• Prosodic Phrasing

– Direct Object (DO) phrasing[ The basketball star accepted the contract ]

[ because it paid so well. ]

– Embedded Clause (EC) phrasing [ The basketball star accepted ]

[ the contract requires him to play every game. ]


More Stimuli & Design

• Fully crossed• Matching conditions:

– DO ending + DO prosody[ The basketball star accepted the contract ]

[ because it paid so well. ]

– Clause ending + Clause prosody[ The basketball star accepted ]

[ the contract requires him to play every game. ]


More Stimuli & Design

• Mismatching conditions:

– Clause ending + DO prosody[ The basketball star accepted the contract ]

[ requires him to play every game. ]

– DO ending + Clause prosody[ The basketball star accepted ]

[ the contract because it paid so well. ]


Materials Construction

• “ Natural” recordings made in all four conditions.

• From these, spliced:– One beginning per prosody condition– One ending per structure condition

• Splice locations counterbalanced to ensure equivalent (un)naturalness across conditions


Splicing example

• Beginnings– DO prosody– Clause prosody

• Endings– DO ending– Clause ending

• Results– DO prosody, DO ending– Clause prosody, DO ending– DO prosody, Clause ending– Clause prosody, Clause ending


ERP Data Analysis

Predictions:– CPS when there’s a boundary compared to no

boundary– No P600 when prosody could prevent garden-pathing

Limitation on data analysis:Can’t directly compare across prosodic conditions at the critical disambiguating word (requires or because)– Because in DO Prosody, CPS directly precedes it,

but not in SC Prosody (CPS earlier, after main verb accepted)

– So compare difference waves across Structures within Prosody• ERP starting at end of pre-boundary word minus ERP starting

at end of same word without boundary shows CPS


Structure Matches Boundary Location

(collapsed over Clause & DO-Structure)0 msec = end of“pre-boundary” word

Difference Waves:Boundary minus No-Boundary


Structure MISmatches Boundary Location

(collapsed over Clause & DO-Structure)0 msec = end of“pre-boundary” word

Difference Waves:Boundary minus No-Boundary


When Structure MatchesBoundary Location


When Structure MISmatchesBoundary Location


Next Steps

• Manipulate Verb Bias

• Manipulate Plausibility– Would a Boundary before a Noun that’s

implausible as a DO prevent N400 effects on Noun?• The referees warned // the game would probably

go into overtime.


Summing Up

• Multiple sources of information constrain sentence interpretation– Lexical bias, plausibility, prosody …

• The sources interact– BUT some provide stronger constraints– And/or are available more rapidly– And thus drive the interaction

• And this all happens really fast!

Documents

10/13/10Psyc / Ling / Comm 525 Fall 10 Kim & Osterhout (2005) JML The independence of combinatory semantic processing: Evidence from event-related potentials