Short-Term Reorganization of Auditory Analysis Induced by Phonetic

Experience

Liebenthal et al. (2003). JoCN.

Short-Term Reorganization of Auditory Analysis Induced by Phonetic

Experience

Liebenthal et al. (2003). JoCN.

Audrey Kittredge593: Neuroimaging of Language

Audrey Kittredge593: Neuroimaging of Language

MRI: physicsMRI: physics

Hydrogen nuclei act as magnets (spinning, charged particle)

Hydrogen nuclei act as magnets (spinning, charged particle)

MRI: physicsMRI: physics

In strong magnetic field: spin-axes form vector parallel to field

In strong magnetic field: spin-axes form vector parallel to field

MRI: procedureMRI: procedure

Radio Frequency pulse Changes direction and strength of vector

Eventually, nuclei relax and vector returns to original position

As nuclei relax, give out pulse Pulse type depends on water/fat ratio of tissue --> MRI images!

Radio Frequency pulse Changes direction and strength of vector

Eventually, nuclei relax and vector returns to original position

As nuclei relax, give out pulse Pulse type depends on water/fat ratio of tissue --> MRI images!

Functional MRIFunctional MRI

Hemoglobin shows up better than deoxyhemoglobin on MRI

SO Brain areas with more oxygenated blood will show up better (BOLD)

Hemoglobin shows up better than deoxyhemoglobin on MRI

SO Brain areas with more oxygenated blood will show up better (BOLD)

Connection to neural activity?Connection to neural activity? Increase in net neural activity --> increase in oxygenated blood supply (slow)

Quick succession of images: BOLD signal at various times

Increase in net neural activity --> increase in oxygenated blood supply (slow)

Quick succession of images: BOLD signal at various times

ProsPros

Good spatial resolution

Less risky, faster acquisition than PET

Event-related design

Good spatial resolution

Less risky, faster acquisition than PET

Event-related design

ConsCons

Poor temporal resolution BOLD signal degraded near air/bone boundary

Movement artifacts High speed data acquisition = noisy!

Poor temporal resolution BOLD signal degraded near air/bone boundary

Movement artifacts High speed data acquisition = noisy!

Phonetic perceptionPhonetic perception

How does this occur?

Automatic phonetic analysis module (Liberman & Mattingly, 1989)

Stimulus-independent auditory analysis (Kluender & Greenberg, 1989)

How does this occur?

Automatic phonetic analysis module (Liberman & Mattingly, 1989)

Stimulus-independent auditory analysis (Kluender & Greenberg, 1989)

Past ResearchPast Research

PET, fMRI studies Speech vs nonspeech: superior temporal cortex

PET, fMRI studies Speech vs nonspeech: superior temporal cortex

Problem!Problem!

Confound: perception or stimuli?

Goal: study perception mode independent of stimulus properties

How do we do this?…

Confound: perception or stimuli?

Goal: study perception mode independent of stimulus properties

How do we do this?…

…Sinewave speech!…Sinewave speech!

Sinewave example Sinewave example

Original sentenceOriginal sentence

“The steady drip is worse than a drenching rain”

“The steady drip is worse than a drenching rain”

Sinewave speech: propertiesSinewave speech: properties Sinusoid fit to center frequency and amplitude (over time) of F1-F3 or F4

Result: rapidly changing pure tones Lack fine-grained acoustic properties of speech

Sinusoid fit to center frequency and amplitude (over time) of F1-F3 or F4

Result: rapidly changing pure tones Lack fine-grained acoustic properties of speech

Past studies on sinewave speechPast studies on sinewave speech Remez et al. (1981):

“Describe”: most say non-speech “Transcribe”: most write all/some of sentence correctly

Remez et al. (1981): “Describe”: most say non-speech “Transcribe”: most write all/some of sentence correctly

Tone-matching Task(Remez et al., 2001)Tone-matching Task(Remez et al., 2001) Stimuli

Sinewave word e.g. juice Isolated T2 from T123/4 complex

Task: is tone constituent of complex?

Listeners can do this… When uninformed (not speech) While matching tone complex to printed word

Difficult task!

Stimuli Sinewave word e.g. juice Isolated T2 from T123/4 complex

Task: is tone constituent of complex?

Listeners can do this… When uninformed (not speech) While matching tone complex to printed word

Difficult task!

Creation of stimuliCreation of stimuli

Phonetic stimulus (sinewave word) 3 lowest formants = 1 sinewave each

Tone probe “True”: from word “False”: from other sinewave word

Nonphonetic stimulus T1 and T3 temporally reversed

Phonetic stimulus (sinewave word) 3 lowest formants = 1 sinewave each

Tone probe “True”: from word “False”: from other sinewave word

Nonphonetic stimulus T1 and T3 temporally reversed

Spectrogram of StimuliSpectrogram of Stimuli

Pilot studies Pilot studies

Phonetic transcribed 52.1% accuracy, multiple choice 89.5% accuracy

Rated as “Clearly identifiable word”: 61% phonetic 22% nonphonetic

“Nonspeech”: 58% nonphonetic 20% phonetic

Phonetic transcribed 52.1% accuracy, multiple choice 89.5% accuracy

Rated as “Clearly identifiable word”: 61% phonetic 22% nonphonetic

“Nonspeech”: 58% nonphonetic 20% phonetic

Stimuli: summaryStimuli: summary

288 stimuli total

108 pairs of phonetic, nonphonetic stimuli 1/3 repeated

1/2 trials = false

288 stimuli total

108 pairs of phonetic, nonphonetic stimuli 1/3 repeated

1/2 trials = false

Experimental DesignExperimental Design

Naïve 1 Naïve 2PracticePhonetic Practice

Informed1

Informed2

ProcedureProcedure

Practice Stimuli: arbitrarily composed sinusoids

Sinewaves: same/diff pitch contour?

Tone-matching task (T2-T1234) Naïve condition

“single tone”, “tone complex” 2 blocks

Practice Stimuli: arbitrarily composed sinusoids

Sinewaves: same/diff pitch contour?

Tone-matching task (T2-T1234) Naïve condition

“single tone”, “tone complex” 2 blocks

ProcedureProcedure

Phonetic practice Sinewave stimuli: 8 sentences, 18 words

Chose from 4 transcriptions Feedback given for every 5th sentence

Accuracy data collected Informed condition

“words” 2 blocks

Phonetic practice Sinewave stimuli: 8 sentences, 18 words

Chose from 4 transcriptions Feedback given for every 5th sentence

Accuracy data collected Informed condition

“words” 2 blocks

Results: RTResults: RT

Phonetic: Test Block p < .o4 (N1-N2 p < .02, N2-I1 p < .03, I1-I2 p < .05)

Nonphonetic Test Block p < .001 (N1-N2 p < .01)

In naïve condition, effect of stimulus type p < .04

Phonetic: Test Block p < .o4 (N1-N2 p < .02, N2-I1 p < .03, I1-I2 p < .05)

Nonphonetic Test Block p < .001 (N1-N2 p < .01)

In naïve condition, effect of stimulus type p < .04

Results: AccuracyResults: Accuracy

Phonetic: No significant effect of Test Block p < .11

Nonphonetic No significant effect of Test Block p < .53

In naïve condition, no effect of stimulus type p < .07

Phonetic: No significant effect of Test Block p < .11

Nonphonetic No significant effect of Test Block p < .53

In naïve condition, no effect of stimulus type p < .07

Results: Phonetic Form PracticeResults: Phonetic Form Practice Sentence task: 84 +/- 21% accuracy

Words: 60 +/- 16% accuracy

Chance = 25% in both tasks

Sentence task: 84 +/- 21% accuracy

Words: 60 +/- 16% accuracy

Chance = 25% in both tasks

Results: Subjective ReportsResults: Subjective Reports 29/31 unaware of phonetic quality during naïve blocks

13/31 recognized words during informed blocks

29/31 unaware of phonetic quality during naïve blocks

13/31 recognized words during informed blocks

Conclusions: BehaviorConclusions: Behavior

Phonetic awareness interferes with task

Naïve: subjects perceived only auditory form

Informed: subjects perceived both, focused on auditory

NO explanation for stimulus RT difference in Naïve

Phonetic awareness interferes with task

Naïve: subjects perceived only auditory form

Informed: subjects perceived both, focused on auditory

NO explanation for stimulus RT difference in Naïve

Within each block…Within each block…

2 phonetic trials

2 nonphonetic trials

Baseline (silence)

Clustered image acquisition

9s 9s 9s 9s 9s 9s 9s 9s

Image acquisitionImage acquisition

18 images per trial type per block

36 images per condition/trial type E.g. Naïve, phonetic

18 images per trial type per block

36 images per condition/trial type E.g. Naïve, phonetic

fMRI ImagesfMRI Images

16 slices: Axially oriented (horizontal) Contiguous 3x3x4mm voxels

Slice coverage: Most of temporal lobes Part of frontal and parietal lobes Occipital lobe

Anatomical (MRI) images (1x1x1mm)

16 slices: Axially oriented (horizontal) Contiguous 3x3x4mm voxels

Slice coverage: Most of temporal lobes Part of frontal and parietal lobes Occipital lobe

Anatomical (MRI) images (1x1x1mm)

fMRI analysis: individualsfMRI analysis: individuals AFNI software package Trial - Baseline-->BOLD difference maps

Difference maps: averaged (BOLD vs baseline) Voxel-wise ANOVA (sorted by trial type and condition)

AFNI software package Trial - Baseline-->BOLD difference maps

Difference maps: averaged (BOLD vs baseline) Voxel-wise ANOVA (sorted by trial type and condition)

fMRI analysis: averagingfMRI analysis: averaging Individual statistical maps transformed into standard space Talairach brain Complicated statistics, smoothing…

t values at each voxel averaged across subjects

Individual statistical maps transformed into standard space Talairach brain Complicated statistics, smoothing…

t values at each voxel averaged across subjects

fMRI analysis: significance testingfMRI analysis: significance testing Randomization testing:

t values >/= .37 significant

uncorrected voxel-wise p < .001

Activation foci < 300 microL removed

Randomization testing: t values >/= .37 significant

uncorrected voxel-wise p < .001

Activation foci < 300 microL removed

fMRI Result Summary fMRI Result Summary

fMRI ImagesfMRI Images

Phonetic: Informed-NaivePhonetic: Informed-Naive Left Heschl’s gyrus (HG/BA42)

Left posterior superior temporal gyrus (STG/BA 42/22)

Right HG/BA42

Left Heschl’s gyrus (HG/BA42)

Left posterior superior temporal gyrus (STG/BA 42/22)

Right HG/BA42

Phonetic ExperiencePhonetic Experience

Decreased activation = decreased task execution Underlies reduced performance Interference masks information like noise

STG Primate HG/post STG analogues involved in complex sound analysis, auditory STM

Left-lateralized Specialization for speech

Decreased activation = decreased task execution Underlies reduced performance Interference masks information like noise

STG Primate HG/post STG analogues involved in complex sound analysis, auditory STM

Left-lateralized Specialization for speech

Phonetic Experience cont’dPhonetic Experience cont’d No shift to other areas

No conscious phonetic perception

Phonetic experience induces “short-term functional reorganization of auditory analysis” and is contingent on “dynamic structure”

No shift to other areas No conscious phonetic perception

Phonetic experience induces “short-term functional reorganization of auditory analysis” and is contingent on “dynamic structure”

Phonetic: Informed-NaivePhonetic: Informed-Naive Dorsomedial thalamic nucleus

Superior frontal gyrus (BA8)

Left middle frontal gyrus (MFG/BA10)

Dorsomedial thalamic nucleus

Superior frontal gyrus (BA8)

Left middle frontal gyrus (MFG/BA10)

Unexplained ResultsUnexplained Results

Dorsomedial thalamic nucleus, medial prefrontal cortex: Areas with reciprocal connections to each other and ST area

Connected neural system… Engaged in task Sensitive to interference

Dorsomedial thalamic nucleus, medial prefrontal cortex: Areas with reciprocal connections to each other and ST area

Connected neural system… Engaged in task Sensitive to interference

Nonphonetic: Informed-NaiveNonphonetic: Informed-Naive Left posterior STG (BA 42/22)

Left posterior STG (BA 42/22)

Phonetic: Blocks2-Blocks1Phonetic: Blocks2-Blocks1 Left middle frontal gyrus (BA9)

Left middle frontal gyrus (BA9)

Nonphonetic: Blocks1-Blocks2Nonphonetic: Blocks1-Blocks2 Left inferior frontal gyrus (IFG/BA44)

Left inferior frontal gyrus (IFG/BA44)

Proficiency EffectsProficiency Effects

Left IFG, MFG: Initial difficulty in verbal production task (Raichle et al., 1994)

Not cause of Informed-Naïve difference (no anatomical overlap)

Left IFG, MFG: Initial difficulty in verbal production task (Raichle et al., 1994)

Not cause of Informed-Naïve difference (no anatomical overlap)

What do YOU think?What do YOU think?

Conclusions…?Conclusions…?

“Centrality” of this function Naïve: Phonetic vs nonphonetic RT Reorganization contingent on speech?

Decreased activation: underlies reduced performance?

Proficiency/Informed: frontal overlap?

“Centrality” of this function Naïve: Phonetic vs nonphonetic RT Reorganization contingent on speech?

Decreased activation: underlies reduced performance?

Proficiency/Informed: frontal overlap?

Methodology…?Methodology…?

Response/accuracy inclusion criteria?

RT/accuracy data not parallel RT: correct, incorrect, true, false trials

Word length? Age variation (18-57)? Naïve: phonetic vs nonphonetic? (fMRI)

Response/accuracy inclusion criteria?

RT/accuracy data not parallel RT: correct, incorrect, true, false trials

Word length? Age variation (18-57)? Naïve: phonetic vs nonphonetic? (fMRI)

Some questions…Some questions…

Role of thalamus/medial frontal areas?

Task difficulty --/--> activation increase

Role of thalamus/medial frontal areas?

Task difficulty --/--> activation increase

Some more questions…Some more questions…

Given phonetic practice, is reorganization entirely stimulus-driven?

How generalizable to normal speech-nonspeech analysis?

Original question: automatic phonetic module or auditory analysis?

Given phonetic practice, is reorganization entirely stimulus-driven?

How generalizable to normal speech-nonspeech analysis?

Original question: automatic phonetic module or auditory analysis?

AcknowledgementsAcknowledgements

You--thanks for listening!

Steve Higgins explanation of fMRI procedure, analysis

Gary Oppenheim practice presenting

You--thanks for listening!

Steve Higgins explanation of fMRI procedure, analysis

Gary Oppenheim practice presenting

References References

Sinewave speech information and samples obtained from: http://macserver.haskins.yale.edu/haskins/MISC/SWS/SWS.html

fMRI physics information obtained from publicly accessible websites and “Handbook of Functional Neuroimaging of Cognition”, Cabeza & Kingstone (Ed), 2001.

Sinewave speech information and samples obtained from: http://macserver.haskins.yale.edu/haskins/MISC/SWS/SWS.html

fMRI physics information obtained from publicly accessible websites and “Handbook of Functional Neuroimaging of Cognition”, Cabeza & Kingstone (Ed), 2001.

References: cont’dReferences: cont’d

Images of atoms courtesy of Duke-UNC Brain Imaging and Analysis Center website

Schematic and anatomical brain images (as well as Uncle Sam image) obtained from various publicly accessible websites and MNI brain images

Images of atoms courtesy of Duke-UNC Brain Imaging and Analysis Center website

Schematic and anatomical brain images (as well as Uncle Sam image) obtained from various publicly accessible websites and MNI brain images