III. Brain Function I: Evidence from Linguistics

The Neurological Basis of Language and Thought

Brain, Mind, and Belief: The Quest for Truth

Language, for all its seeming complexity, is more amenable to analysis than other cognitive structures, so that investigation of language is one of the best ways to proceed to an understanding of human mind and nature.

Tim Pulju, 1990

Agenda for TodayThe Problem: How does the brain work?

History of the study of brain function Common errors to be avoided

• Tool-driven inquiry• The misapplied metaphor

Help from the study of linguistic structure• Claim: As language works in the brain, so the

brain works in general• Therefore, If we can understand how language

works, we will know how the brain works• Relational networks

Cortical columns of neurons

2

History of the study of brain function I

Early investigators in the 19th century came up with the idea of locationism: Local areas of the cortex have specific functions• Franz Joseph Gall (1758-1828)

promoted the idea His followers took it too far As a result, the idea of locationism was widely discredited

3

History of the study of brain function II

Locationism: Local areas of the cortex have specific functions• Paul Pierre Broca (1824-1880)

Stroke patient with impaired speech Autopsy after patient’s death Damage in lower left frontal lobe

• Area now known as Broca’s area• Responsible for articulation of speech

4

History of the study of brain function III

The origin of Connectionism Like locationism but more sophisticated:

Local areas of the cortex have highly specific simple functions, and complex functions are carried out by multiple interconnected areas• Carl Wernicke (1848 - 1906)

Stroke patient unable to comprehend speech

Damage in upper left temporal lobe• Area now known as Wernicke’s area• Responsible for speech comprehension

5

Connectionism

Connectionism includes a version of locationism• But is more sophisticated

Each local area performs a very specific simple function

Complex functions require multiple local areas acting together

• They can act together because they are interconnected

» CONNECTIONS RULE!

6

Two basic language areas

Primary Somato-sensory Area

Primary Motor Area

Primary AuditoryArea

PrimaryVisual Area

Mouth

HandFingers

Arm

Trunk

Leg

PhonologicalRecognition

PhonologicalProduction

Broca’s area

Wernicke’s area

7

Arcuate Fasciculus (from langbrain website)

Connects Wernicke’s area to Broca’s area

8

History of the study of brain function IV

The decline and revival of Connectionism• As with Gall, followers of Wernicke were

too speculative and went too far, and the whole idea was discredited for several decades

• Finally revived in the 1960’s by Norman Geschwind (1926-1984)

• Now widely accepted by neurologists (but criticized by some psychologists)

9

History of the study of brain function V

Two major methods of investigation① Lesion studies

E.g., Broca and Wernicke and Geschwind If area A is damaged and function F is

impaired, then A must have function F (or at least contribute to F)

② Functional Brain Imaging• A recent innovation• Made possible by technological advances• Now very widely used• Location-based but sometimes without

the sophistication of connectionism

10

Functional Brain Imaging

Electro-Encephalography (EEG)• Excellent temporal resolution• Very poor spatial resolution

Positron Emission Tomography (PET)• Poor spatial resolution• Very poor temporal resolution

Functional Magnetic Resonance Imaging (fMRI)• Spatial resolution better than PET• Temporal resolution a little better than PET

Magneto-Encephalography (MEG)• Excellent temporal resolution• Spatial resolution not so good

11

Positron emission tomography (PET)

PET shows areas of increased cortical metabolism Spatial resolution: 5-10 mm

• How good is that? Under one sq mm of cortical surface, 130,000 neurons

Temporal resolution: “…on the order of minutes…” (A. Papanicolaou, Fundamentals of Functional Brain Imaging (1998), p. 14)

12

Functional Magnetic Resonance Imaging (fMRI)

Measures the amount of oxygenated blood supplied to different areas of the brain• Common abbreviation: rCBF (regional cerebral blood flow)

When a group of neurons increases its signaling rate, its metabolic rate increases

13

An fMRI example

www.firstscience.com/ SITE/ARTICLES/love.asp

Areas of the brain used in working memory

14

Properties of fMRI

Temporal resolution: Not very good Image reflects the increase in oxygenated blood

that occurs 5 to 8 seconds after the neurons fire Spatial resolution:

• Better than PET• But it is unclear whether the imaged area is

precisely the area involved in the activity The flow of oxygenated blood into the depleted

area may also flow into neighboring vessels in areas where neural firing did not occur

15

Magnetoencephalography (MEG)

MEG (MagnetoEncephaloGraphy) measures the magnetic field around the head

Magnetoencephalography

magnetic brain

pictureproduction of

16

How MEG works

An electric current is always accompanied by a magnetic field perpendicular to its direction

MEG records the magnetic flux or the magnetic fields that arise from electric currents in neurons

Magnetic flux lines are not distorted as they pass through the brain tissue because biological tissues offer practically no resistance to them • Therefore, MEG is more accurate than EEG

17

Magnetic flux from source currents

Source current

Magnetic flux Magnetometer

18

Recording of Magnetic Signals

19

Temporal Resolution of MEG

Excellent – unlike fMRI and PET Therefore, it is possible to discern the temporal order

of activation of cortical areas MEG has potential to detect the activation of several

brain regions as they become active from moment to moment during a complex function such as recognition

20

A major challenge of MEG

The cortex is a parallel processor• Hundreds or thousands of dipoles can be

active simultaneously Multiple dipoles make comprehensive

inverse dipole modeling virtually impossible Hence, compromises are necessary

• Sample larger time spans (up to 500 ms)• Sample larger areas (up to several sq cm)

21

Some MEG results: Speech recognition

Hemispheric Asymmetry Wernicke's Area

22

Wernicke’s area in Spanish-English bilinguals

From MEG lab, UT Houston23

Spatial Resolution

How accurately is location determined?• EEG: Poor• PET: Fair – 4-5 mm• fMRI: Fair – 4-5 mm• MEG: Fairly good – 3-4 mm or less

Under good conditions How good are these figures?

• under 1 sq mm of cortical surface,140,000 neurons

24

Temporal resolution

Temporal resolution• Key neural events can occur within 5 ms• Terrible: PET

40 seconds and up• Pretty bad: fMRI

10 seconds or more• Excellent: MEG and EEG

Instantaneous Theoretically it is possible to do ms by ms tracking,

to follow time course of activation But such tracking is usually difficult or impossible

• The inverse problem• Too many dipoles at each point in time

25

Sensitivity of Imaging Methods

All of the methods have limited sensitivity MEG

• 10,000 dendrites in close proximity have to be active to detect signal

PET and fMRI• Similar limitations

Any activation that involves fewer numbers goes undetected

26

Faulty thinking in neuroscience I:Tool-driven inquiry

Tool-driven inquiry: letting the available tools shape the investigation• Like looking for the lost car keys under the street

light instead of where they got lost The available tools: Brain imaging machines What they are good for: determining locations

of brain activity Therefore, what do they investigate?

• Locations of brain activity The question being asked: Where? The questions not being asked: What?, How?

• What is going on?• How does the brain work?

27

More on the lack of interest in what/how

There are no available machines for investigating the what/how question

Experiments have not been devised for investigating the what/how question

It is necessary to rely on thinking Scientists believe that doing science is conducting

experiments and using high-tech machinery• Thinking is done by theoreticians

Akin to philosophers and poets

28

A mitigating circumstance?

Many have not realized that the what/how question is important

They may assume it is already known:• The brain is assumed to work like a computer

A symbol-manipulating device• This assumption is unwarranted

29

Faulty thinking in neuroscience II:The misapplied metaphor

The brain is assumed to work like a computer This assumption is unwarranted

The computer is a symbol-manipulating device• Not a connectionist system

An example of faulty thinking:• The misapplied metaphor

The brain works by means of connectivity and operations upon its connectivity

30

The Cortex is a NetworkEntirely different structure than that of computers

Connectivity as key property of brain structure Symbol-manipulation is the key property of computers The cortex operates by means of connections

• Transmission of activation along neural pathways• Changes in connection strengths

31

Computers and Brains: Different Structures, Different Skills

Computers• Exact, literal• Rapid calculation• Rapid sorting• Rapid searching• Faultless memory• Do what they are told• Predictable

Brains• Flexible, fault tolerant• Slow processing• Association• Intuition• Adaptability, plasticity• Self-driven activity• Unpredictable • Self-driven learning

32

Things that brains but not computers can do

Acquire information to varying degrees• “Entrenchment” • How does it work?

Variable connection strength Connections get stronger with repeated use

Perform at varying skill levels• Degrees of alertness, attentiveness• Variation in reaction time• Mechanisms:

Global neurotransmitters Variation in blood flow Variation in available nutrients Presence or absence of fatigue Presence or absence of intoxication

33

How to study the what/how question

You have to think harder Also, we can make use of findings from structural

linguistics Language as the key to unlock mysteries of the mind

• If cortical structures for language are like those for other high-level skills

• Then if we figure out language, we also have the answer to how other high-level intellectual processing works

34

Thinking harder

Avoid metaphorical thinking• The brain is not a computer• Not like a human being with paper & pencil & books• In fact it is not like anything else

It is itself: the brain

35

How does your brain tell your fingers what to do?

Question to daughter (age 6):• How does your brain tell your finger to hit that

key on the piano? Sarah:

• Well my brain writes a little note and sends it down to my finger, …

What really happens? Neurons in the motor cortex send activation

(nerve impulses) down (through subcortical structures and the spinal cord) to neurons that activate the muscles that make my finger move

36

Auditory Imagery

Auditory images of words, music, etc.• We can hear things in our heads• What is an auditory image?

What does it consist of?• Sound?

» There is no air inside the head to vibrate• What hears it?

» There are no little ears inside the head

37

Visual Imagery

Visual images of people, buildings, etc. What is a visual image? What does it consist of?

• Is it a little picture? If so, where are the eyes to see it? What is it drawn on? Where is the visual perception system

to interpret it?• If not, what?

38

Vision

When you see something.. A picture on your retina?

• Something in your brain looks at it?• Are there a couple of little eyes inside?

A picture somewhere inside your brain?• Same problems: no eyes inside

And if there were, they would have to be supported by a visual perception system

39

Compare the TV set

Does it have little people inside? Similarly, no pictures inside the brain No sounds inside the brain No words or other symbols inside the brain

40

How vision really works

There is no picture on the retina, just neurons that get activated by light• Some of them get activated by color

The visual perception system learns during early childhood to integrate configurations of these little dots into larger units

Next higher level: larger configurations Many levels up, recognition of objects

• The brain goes through a long process of learning, to build these many levels

• More on this next week!

41

The Nature of Language

Some history• Louis Hjelmslev• Prolegomena to a Theory of Language (1943/60)

Linguistic structure is a system of relationships

42

”The postulation of objects as something different from the terms of relationships is a superfluous axiom and con-sequently a metaphysical hypothesis from which linguistic science will have to be freed.”

Understanding linguistic units as purely relational I

dog

d - o - g

Symbols?Objects?

Understanding linguistic units as purely relational II

dog Seems to be one unit

Three phonemes (or graphemes), in sequence

d o g

Understanding linguistic units as purely relational III

DOG Noun

d o g

dog The object we are considering

The meaning of dog – a concept

Grammatical properties

Understanding linguistic units as purely relational IV

DOG Noun

d o g

dog

Understanding linguistic units as purely relational V

DOG Noun

d o g

We can remove the symbol with no loss of information. Therefore, it is a connection, not an object

Another way of looking at it

DOG Noun

d o g

dog

Another way of looking at it

DOG Noun

d o g

The phonological (or graphemic) segments

DOG Noun

d o g

What about these segments? Are they objects?

The phonological (or graphemic) segments

d o g

What about these segments? Are they objects?

/d/ as a phoneme has components: Articulation: closure of mouth (so also /g/) Position: Tongue tip against alveolar ridge Voiced (compare /t/) (Also auditory components: more complex)

A closer look at the segments

b

boy

y

Phonologicalfeatures

o The phonological segments also are just locations in the network – not objects

(Bob) (toy)

Relations all the way

Perhaps all of linguistic structure is relational It’s not relationships among linguistic items; it

is relations to other relations to other relations, all the way to the top – at one end – and to the bottom – at the other

In that case the linguistic system is a network of interconnected nodes

What is at the bottom?

In the system of the speaker, we have relational network structure all the way down to the points at which muscles of the speech-producing mechanism are activated• At that interface we leave the purely relational

system and send activation to a different kind of physical system

For the hearer, the bottom is the cochlea, which receives activation from the sound waves of the speech hitting the ear

What is at the top?

Somehow at the top there must be meaning

What are meanings?

DOGC

Perceptual

properties

of dogsAll those dogs

out there and

their properties

In the Mind

The World Outside

For example, DOG

What are meanings?

Perceptual

properties

of dogsAll those dogs

out there and

their properties

In the Mind

The World Outside

For example, DOG

Conceptual

properties

of dogs

What are meanings?

Perceptual

properties

of dogsAll those dogs

out there and

their properties

In the Mind

The World Outside

For example, DOG

Conceptual

properties

of dogs

Also relational network structure

The concept DOG

We know what a dog looks like• A visual subnetwork, in occipital lobe

We know what its bark sounds like• An auditory subnetwork, in temporal lobe

We know what its fur feels like• A somatosensory subnetwork, in parietal lobe

All of the above..• constitute perceptual information• are subnetworks with many nodes each• Are interconnected into a larger network

59

The concept of DOG as a network

V

PA

M

T

C

A – Auditory C – ConceptualM – MemoriesP – PhonologicalT – TactileV - Visual

Each node in this diagram connects to a subnetwork of properties

60

Objects in the mind?

When the relationships are fully identified, the objects as such disappear, as they have no existence apart from those relationships

“The postulation of objects as some- thing different from the terms of relationships is a superfluous axiom and consequently a metaphysical hypothesis from which linguistic science will have to be freed.”

Louis Hjelmslev (1943/61)

How the mind operatesExample: language

People are able to use their languages• Speaking• Writing• Comprehension

Such operation takes the form of activation of lines and nodes• Activation travels from node to node• Along connecting lines

Compare the highway system• A network• Operation: vehicles move along the roads

Two different network notations

Narrow notation

ab

a b

b

a b

Abstract notation Bidirectional

ab

a b f

Upward Downward

Narrow relational network notation

Represents network structures in greater detail internal structures of the lines and nodes of the more

abstract notation Closer to neurological structure Each node represents a bundle of neurons Links represent neural fibers (or bundles of fibers)

Abstract and narrow network notation

The lines and nodes of the abstract notation are abbreviations for more complex structures

Compare the representation of a divided highway on a highway map• In a more abstract notation it is shown

as a single line• In a narrow notation it is shown as two

parallel lines of opposite direction

AND vs. OR

AND

OR

twenty seven

27

12

twelve dozen

AND vs. OR: Internal Structure (narrow notation)

AND

OR

2

1

Thresholds in Narrow Notation

1 2 3 4

OR AND

– You can have intermediate degrees, between AND and OR

– The AND/OR distinction was a simplification anyway — doesn’t always work!

Levels of precision: Add another

Abstract relational network notation Narrow relational network notation Neural structures

Narrow RN notation and neural structures

Question: Are relational networks related in any way to neural networks?

We can find out Relational networks were devised as a means

of accounting for linguistic structure• Their properties depend on properties of language• Evidence for them comes from language, not from

the brain Narrow RN notation can be viewed as a set of

hypotheses about brain structure and function• Properties of narrow RN notation can be tested for

neurological plausibility

Some properties of narrow RN notation

Lines have direction (they are one-way)

But they tend to come in pairs of opposite direction (“upward” and “downward”)

Connections are either excitatory or inhibitory

Nerve fibers carry activation in just one direction

Cortico-cortical connections are generally reciprocal

Connections are either excitatory or inhibitory (from different types of neurons, with two different neurotransmitters)

More properties as hypotheses

Nodes have differing thresholds of activation

Inhibitory connections are of two kinds

Additional properties – (too technical for this presentation)

Neurons have different thresholds of activation

Inhibitory connections are of two kinds • (Type 2: “axo-axonal”)

All are verified

Type 1

Type 2

The node of narrow RN notationvis-à-vis neural structures

The node (of narrow RN notation) corresponds (not to a single neuron but) to a bundle of neurons• The cortical column• A column consists of 70-100 neurons stacked

on top of one another• More on this next week!

T h a t ‘ s i t f o r t o d a y !

74