Linguistics 001, Spring 2009

Neuroimaging: Syntax

Syntax in the brain

• As we saw in the last lecture and in the readings, it has been proposed that ‘syntax’ resides in Broca’s area

• This finding connects in some ways with what is found in the aphasia literature, although the situation is quite complicated

• The reading moves beyond this basic picture in many ways by looking at imaging of syntax

• Today: Introduce functional Magnetic Resonance Imaging (fMRI), and look at a few experiments that involve syntax

Spatial and Temporal Dimensions

Technique Spatial Temporal

ERP 10-20mm 1msec

MEG 5-10mm 1msec

PET <5mm 30sec

fMRI <2mm ~1sec

Imaging techniques differ in spatial and temporalsensitivity.

Basics of fMRI • Uses extremely strong magnetic field

• Measures changes in local blood flow/blood oxygenation

• Excellent spatial resolution

• Hemodynamic (= blood-related) response is slow in comparison with neuronal activation.

Procedure

• Subjects lie within the bore of the magnet, with heads fixed

• Experimenters acquire structural images of the subjects’ brains

• Functional images, showing changes in blood flow/oxygenation, show “activation” in different areas of the subject’s brain

Further Details

• During changes in neuronal activity, there are local changes in the amount of oxygen in the tissue

• The changes in blood oxygenation are detected by the scanner

• The resulting changes are an indirect measure of neuronal activity (indirect because the hemodynamics are related to the firing)

Cortical Activity and Blood-Flow

Cortical activation results in a local increase in oxy-genated blood, and blood flow, without an increase in oxygen consumption.

Illustration

Areas in which neuronal activity occurs show patterns of activation which the scanner detects.

The image is paired with a graph showing the blood flow in that region by experimental condition (moving vs. stationary dots in this case).

Plan

• We’ll look at two early studies that use fMRI to study aspects of syntax

• While these studies have been followed by many others, they give an idea of the kinds of questions that are addressed in the functional neuroimaging of language

Study 1

• Just, Carpenter, Keller, Eddy, and Thulborn (1996), Science 274

• Cortical activation and sentential complexity

Basics

• Question posed: Do computationally more demanding sentences result in more brain activation than less complex sentences?

• Focusing on areas known to be involved in language, i.e. Wernicke’s and Broca’s

Stimuli

• Three types of sentences, with the same number of words:

– Active: The reporter attacked the senator and admitted the error.

– Subject relative: The reporter that t attacked the senator admitted the error.

– Object Relative: The reporter that the senator attacked t admitted the error.

Control

• Something to control for some of the visual components of the sentence-reading part

• Consonant strings:– Pws ntkgqrfm zjkjrng kwtdc sbfght swn

mrjbxq kgt mxbtq

Idea

• The three different types of sentences have been studied extensively in the processing literature– They increase in difficulty – The reason for this appears to have to do

with the position of the trace, when there is one

Design and Task

• Sentences were arranged in sets of four to five of the same type

• Following each sentence, subjects answered ‘True’ or ‘False’ to a comprehension probe:– “The reporter attacked the senator, True or

False?”

Subject Performance

• Reaction time and error rate data were collected for most of the subjects

• Analysis of the behavioral data showed these indices of complexity to increase monotonically in the expected way with the increasingly complex sentence-types

Images

The analysis looked at the number of activated voxels by Condition; more of these are present from left to right.

NOTE: Left is Right and Right is Left in these images

Cortical Activation

Interpretation

• It is suggested that Wernicke’s is involved in something other than purely lexical matters (since these are constant across conditions)

• Broca’s area is involved in something as well (perhaps syntax; it’s hard to tell)

• Right homologues are implicated in processing as well

• Overall: The brain’s response to increased processing demands is to recruit more tissue in each area in a network of cortical areas

Study 2

• Dapretto and Bookheimer (1999), Neuron 24

• fMRI study, attempting to dissociate syntax and semantics in sentence comprehension

Motivation

• A number of prior studies showing activation in Broca’s area during tasks of different syntactic complexity

• The idea that this does not establish anything about exclusivity– perhaps Broca’s area would also be modulated by increasing demands along another, non-syntactic dimension

Design Basics

• Auditory presentation of pairs of sentences

• Subjects had to decide whether or not the meaning of the two sentences differed

• Sentence pairs were classified as semantic or syntactic; in each case, there were ‘same’ and ‘different’ types

Conditions

• SEMANTIC: Pairs of identical sentences, in which one word was replaced either by a synonym or a different word.

• SYNTACTIC: Sentences in the pair were in different voices (active vs. passive) or had word-order differences

Examples: Semantics

• Same– The lawyer questioned the witness.– The attorney questioned the witness.

• Different– The man was attacked by the doberman.– The man was attacked by the pitbull.

Examples: Syntax

• Same– The policeman arrested the thief.– The thief was arrested by the policeman.

• Different– The teacher was outsmarted by the

student.– The teacher outsmarted the student.

Justification of Stimuli

• In the semantic condition: Judgment is supposed to rely on single word meanings

• In the syntactic condition: Judgment requires computation and comparison of two syntactic structures

• Authors note that both kinds of processing are present in each task, and expect the two conditions to differ in relative ‘weight’ of syntax and semantics

Images

Syntax vs.Rest

Semantics vs.Rest

Results (General)

• Many areas associated with language are found to be active in this experiment

• What about the syntax vs. semantics comparison?

Syntax vs. SemanticsSyntax (A): Task-specific activity centered in the (lower) pars opercularis (BA 44).

Semantics (B): Task-specific activity in the pars orbitalis (BA 47).

Brodmann Areas

Interpretation

• Authors– Syntax: BA 44 is particularly involved in syntactic

processing– Semantics: BA 47 is selectively involved in the

processing of lexico-semantic information• Spend some time thinking about how strong this kind

of conclusion could be, given what happened in the experiment etc.

• I.e., the conditions both involve syntax and semantics; moreover, the tasks are not thoroughly understood; moreover, the behavioral data are not reliable, etc.

Wrapping up

• Broca’s area (and other language areas) can be modulated in different tasks

• How this connects with the idea of specialization is complicated, as discussed in the reading

• Functional neuroimaging is impressive technologically, but the connections with cognitive hypotheses are only starting to become clear

Space/Time/Brain

• Prospects for future theories of linguistic computation in the brain– Integration of spatial and temporal

information– Correlation between linguistic ontology and

neurological ontology– Bringing the neurological theories “up to

speed” with the categories from the representational analyses of language

Linguistics/Neurolinguistics

• If you want to know about the analysis of sentences/sounds/words, etc., you consult linguistic theory

• If you want to know about structures in the brain, cells relevant to brain activity, etc., you consult neurology.

• What role is there for Neurolinguistics of the type that we have been studying? What are this area’s results, and prospects?

Results

• Clinically, results showing that (parts of) language in the broad sense are correlated with certain brain areas are useful

• The next step is to ask two related further questions, and see what has to be done to answer them:– What can looking at the brain tell us about the

complex internal structure of language?– What can looking at language tell us about the

complex internal structure of the brain?

Specialization Question

• Specialization Question: What does it mean for a specific brain region to be specialized for some (linguistic) computation?

• I.e., we have seen many descriptions of the different areas of cortex in the brain

• Brodmann areas:

Specialization, Cont.

• To answer the specialization question, we need to know– Why some areas compute some things, and not

others– What this means neurobiologically– How ‘flexible’ the correlations are

• Then we can ask if there is any explanatory relationship between claims like “syntax happens in region X because this region has certain properties etc..”

Granularity Question

• Linguistic theory and Neurolinguistic theory operate at different levels of granularity

• I.e. we had lots of detailed analyses in the discussion of phonology, morphology, etc.

• For a variety of reasons, many neuroimaging studies ask questions about “syntax”, or “semantics”; extremely coarse by comparison (remember that our intro lectures subdivided “syntax” and “semantics”)

Granularity, Cont.

• When we talk about e.g. syntax, we know computationally that many things are going on; this is not a single “task”:– Creation of tree structures– Linearization of tree structures– Movement of constituents– Etc.

• In order to bring neurolinguistics into line with linguistic theory, the brain has to be investigated in terms of the linguistic categories that seem to be important

Conclusion

• The future of neurolinguistics depends on the development of unifiying hypotheses that crosses these domains

• The way we’ve been looking at this involves integrating linguistics and neurolinguistics by using linguistic categories to explore the brain

• There’s not much “back and forth” at present