Electrophysiology as a Brain Measure of Perceptual Sensitivity and Abstraction

Electrophysiology as a Brain Measure of Perceptual Sensitivity and Abstraction

/kæt/

what?

when?

where?

ElectrophysiologyHow it worksMapping toolDiscriminationCategorizationPredictionTiming

How electrophysiology works

Electrical Activity

Probing Electrical Activity

• Interfere with it

• Record it indirectly

• Record it directly

Zap!

Transcranial Magnetic StimulationK. L. Sakai et al. 2002, Neuron

Verb generationVerb generation Verb generation after 15 min practice

Verb generation after 15 min practice

Raichle & Posner, Images of Mind cover image

Bang!

Functional MagneticResonance Imaging (fMRI)

MRI studies brain anatomy.Functional MRI (fMRI) studies brain function.

MRI vs. fMRI

Source: Jody Culham, fMRI for Newbies

Electroencephalography (EEG)

Event-Related Potentials (ERPs)

s1 s2 s3

John is laughing.

Phillips, Kazanina, & Abada (2005) Cog. Br. Res.

The producers knew that the actress wished that …The producers knew which jokes the actress wished that ..

ERP Studies

• Cost ($$)– Relatively cheap equipment and maintenance

• Time Investment– Materials: e.g., 20 syllables; 128 x 4 target sentences; 256 fillers

– Acquisition: 2-3 hours x 12-24 participants

– Analysis: 1-2 hours/person preprocessing; rest automatizable

• Strengths– Unbeatable temporal resolution

– Easy to combine across participants

– Movement possible; longer studies more tolerable

– Interpretable results

• Disadvantages– Scalp topography ≠ localization

– … and limited sex appeal

Brain Magnetic Fields (MEG)

Brain magnetic fields recorded fully non-invasively by arrays of SQUID* detectors

[*Superconducting QUantum Interference Device]

V

skull

CSF

tissue

MEG

EEGB

- noninvasive measurement- direct measurement.

scalprecordingsurface

currentflow

orientationof magnetic field

Origin of the signal

How small is the signal?10-4

10-5

10-6

10-7

10-8

10-9

10-10

10-11

10-12

10-13

10-14

10-15

Earth field

Urban noise

Contamination at lung

Heart QRS

MuscleFetal heart

Spontaneous signal (-wave)

Signal from retina

Intrinsic noise of SQUID

Inte

nsity

of

mag

netic

si

gnal

(T)

Evoked signal

Biomagnetism

EYE (retina) Steady activity Evoked activity

LUNGS Magnetic contaminants

LIVER Iron stores

FETUS Cardiogram

LIMBS Steady ionic current

BRAIN (neurons) Spontaneous activity Evoked by sensory stimulation

SPINAL COLUMN (neurons) Evoked by sensory stimulation

HEART Cardiogram (muscle) Timing signals (His Purkinje system)

GI TRACK Stimulus response Magnetic contaminations

MUSCLE Under tension

160 SQUIDwhole-headarray

pickup coil & SQUIDassembly

Sensor layout: recording from 160 channelsResponse peak at 98ms after onset of an auditory stimulus, in the left and right temporal lobes.

Magnetic source imaging (MSI): MEG + MRIDipole fit at response peak, 98ms after onset of stimulus

(Halgren et al. 2002)

(Halgren et al. 2002)

MEG Studies

• Cost ($$)– Expensive equipment; little maintenance; liquid helium supply

• Time Investment– Materials: e.g., 20 syllables; 600 x 3 target sentences; 400 fillers

– Acquisition: 1-2 x 1-2.5 hours x 12 participants (+ structural MRI)

– Analysis: similar to EEG, more individual-specific analyses

• Strengths– Unbeatable temporal resolution; easy set-up

– Possibility of localization

• Disadvantages– Combining across individuals more difficult than in ERP studies

– Longer studies, movement limited

– Inverse problem for localization is very hard

– Sensitivity depends on orientation and depth of source

– Horrendously difficult with children, challenging w/ small heads

Electrophysiology in Syntax/Semantics

1. Classification Tool

2. Sensitive Timing Measure

Sue takes her coffee with cream and socks. N400 - semantic anomalyThe plane took we to paradise. P600 - syntactic anomaly

The hungry guests helped himself to food. P600

Prerequisite: response components sensitive to distinct information types

Mary praised Max’s of proof the theorem. ‘ELAN’, 100-200ms

No bills that the senators supported ever became law. controlThe bills that the senators supported ever became law. ~400ms responseThe bills that no senators supported ever became law. ~400ms response

Prerequisite: explicit process models of syntax

Electrophysiology as a Mapping Tool

Sensory Maps

Internal representations of the outside world. Cellular neuroscience has discovered a great deal in this area.

Sensory Maps

Internal representations of the outside world. Cellular neuroscience has discovered a great deal in this area.

Notions of sensory maps may be applicable to some aspects of human phonetic representations…

…but there has been little success in that regard, and we shouldn’t expect this to yield much.

Vowel Space

Obleser

Lahiri

J. Cogn. Neurosci.,16, 31-39 (2004)

M100• Elicited by any well-defined onset

• Varies with tone frequency

• Varies with F1 of vowels

• May vary non-linearly with VOT variation

• Functional value of time-code unclear

• No evidence of higher-level representations

100

110

120

130

140

150

0 1000 2000 3000 4000 5000

Frequency (Hz)

i (male) i (female) a (male) a (female) u (male) u (female)100

105

110

115

120

125

130

(Poeppel & Roberts 1996)

(Poeppel, Phillips et al. 1997)

(Phillips et al. 1995; Sharma & Dorman 1999)

Electrophysiological measures of discrimination

Mismatch Response

Latency: 150-250 msec.

Many-to-one ratio between standards and deviants

X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...

Localization of Mismatch Response

(Phillips, Pellathy, Marantz et al., 2000)

[Radiological view - left is right]

Basic MMN elicitation

©Risto Näätänen

Mismatch Negativity (MMN)

Sams et al. 1985

Tiitinen et al. 1994

How does MMNlatency, amplitudevary with frequencydifference?

1000Hz tone std.

Different Dimensions of Sounds

• Length

• Amplitude

• Pitch

• …you name it …

Amplitude of mismatch response can be used as a measure of perceptual distance

Impetus for Language Studies

• If MMN amplitude is a measure of perceptual distance, then perhaps it can be informative in domains where acoustic and perceptual distance diverge…

Place of Articulation

• Acoustic variation: F2 & F3 transitions



[bæ] [dæ]



[bæ] [dæ]

withincategory

betweencategory



[bæ] [dæ]

withincategory

betweencategory

Categories in InfancyHigh Amplitude Sucking - 2 month olds

Eimas et al. 1971

20 vs. 40 ms. VOT - yes40 vs. 60 ms. VOT - no

Infants show contrast, but this doesn’t entail phonological knowledge


• No effect of category boundary on MMN amplitude (Sharma et al. 1993)

• Similar findings in Sams et al. (1991), Maiste et al. (1995)

but…

Näätänen et al. (1997)

e e/ö ö õ o

Relevance to Learning Models• Place of articulation continuum (Lalonde & Werker, 1988)

b -- d -- D

• 3 contrastsNative b -- dNon-native d -- DNon-phonetic b1 -- b5

• Conflicting results

– Dehaene-Lambertz 1997native contrast only

– Rivera-Gaxiola et al. 2000native + non-native contrasts

Dehaene-Lambertz Rivera-Gaxiola

Phillips (2001, Cognitive Science)

J. Cogn. Neurosci. 16:577-583 (2004)

Contrast andUnderspecification

Phonetic Category Effects

• Measures of uneven discrimination profiles

• Findings are mixed (…and techniques vary)

• Relies on assumption that effects of contrasts at multiple levels are additive,…plus the requirement that the additivity effect be strong enough to yield a statistical interaction

• Logic of next set of studies:

– Eliminate contribution of lower levels by isolating the many-to-one ratio at a more abstract level of representations

– Do this by introducing non-orthogonal variation among standards

Electrophysiological measures of abstraction

J. Cogn. Neuro., 12, 1038-1055 (2000)

/dæ/ /tæ/

Design

20ms 40ms 60ms

Fixed Design - Discrimination

Design

0ms 8ms 16ms 24ms 40ms 48ms 56ms 64ms

20ms 40ms 60ms


Grouped Design - Categorization

Non-orthogonal within-category variation: precludes grouping via acoustic streaming.

Design


20ms 40ms 60ms


Grouped Design - Categorization


Grouped Design - Acoustic Control

/dæ/ standard vs./dæ/ deviant

So what …

• Auditory cortex generator of MMF accesses representations that treat members of the same category as identical

• No indication of what might be the form of these representations, or where they might be stored

• MMF generator accesses multiple levels of representation

Sound Groupings

(Phillips, Pellathy, & Marantz, 2000)

pæ, tæ, tæ, kæ, dæ, pæ, kæ, tæ, pæ, kæ, bæ, tæ...

(Phillips, Pellathy, & Marantz, 2000)

More on features …

• Alternative account of the findings

– No feature-based grouping

– Independent MMF elicited by 3 low-frequency phonemes

/bæ/ /dæ/ /gæ/ /pæ/ /tæ/ /kæ/29% 29% 29% 4% 4% 4%

87.5% 12.5%

(Yeung & Phillips, 2004)


• Next study distinguishes

– Phoneme-level frequency

– Feature-level status

/gæ//bæ/ /dæ/ /tæ/

37.5% 37.5% 12.5% 12.5%







37.5% 37.5% 12.5% 12.5%

Phoneme-based classification







37.5% 37.5% 12.5% 12.5%

Feature-based grouping



• Design– N = 10

– Multiple exemplars, individually selected boundaries

– 2 versions recorded for all participants, reversing [±voice] value

– Acoustic control, with all VOT values in [-voice] range


37.5% 37.5% 12.5% 12.5%

Feature-based grouping



Left-anteriorchannels


Kazanina et al., 2006

Proceedings of the NationalAcademy of Sciences, 103, 11381-6

Nina Kazanina, U. of Bristol

Russian vs. Korean

• Three series of stops in Korean:– plain (lenis) pa ta ka– glottalized (tense, long) p’a t’a k’a– aspirated ph tha kha

• Intervocalic Plain Stop Voicing:/papo/ [pabo] ‘fool’/ku papo/ [kbabo] ‘the fool’

• Plain stops:– Bimodal distribution of +VOT and –VOT tokens– Word-initially: always a positive VOT– Word-medially intervocalically: a voicing lead (negative VOT)

Identification/Rating

Discrimination

Black: p < .05White: n.s.

Russian vs. Korean

• MEG responses indicate that Russian speakers immediately map sounds from [d-t] continuum onto categories

• Korean speakers do not…… despite the fact that the sounds show bimodal distribution in their language

• Perceptual space reflects the functional status of sounds in encoding word meanings

Dupoux et al. (1999, Percep. Psychophys.)

J. Cogn. Neurosci. 12:635-647 (2000)

EXECTIVE SUITEEXECTIVE SUITE

French: CVC

egma egma

Japanese: *CVC

egma eguma

Early (~200ms)

Late (~600ms)

Dehaene-Lambertz et al. (2000, J. Cogn. Neurosci.)

Egma, egma, egma, egma, eguma, …

Mismatch Studies

• Perceptual experience of sound categories is mapped onto multiple levels of representation, at different degrees of abstraction

• Abstract category representations can be probed using non-orthogonal variation among sounds in an auditory mismatch paradigm more-or-less immediate access during speech perception

• Beware of hasty inferences to normal time course of processing

– Recurring standard sound establishes ‘model’ for parsing upcoming sounds

– Detecting deviance from model, even at abstract level, may be faster than normal analysis processes

– cf. related findings about ELAN (‘Early Left Anterior Negativity’) in syntactic ERP literature (Lau et al., 2006)

Electrophysiology and Prediction

McGurk Effect: auditory [pa] + visual [ka] = perceptual [ta]

Neurosci. Lett. 397:263-268 (2006)

[aba]

[ãba]

Electrophysiology and Timing

syntax > phonology

phonology > syntax

Science, 280: 572-574

Lateralized Readiness Potential (LRP)

Prospects

• Many possibilities for more sophisticated phonetic/phonological studies

– Mapping: what are the questions?

– Early transforms of acoustic space: already an active area

– Discrimination: needs a theory of variable access to levels of detail

– Categorization: can be used to investigate learning and abstraction

– Prediction: phonotactics is barely touched

– Timing: possible LRP studies of computing derived properties

Documents

Electrophysiology as a Brain Measure of Perceptual Sensitivity and Abstraction