ReadingReading in the brain

NeuronalNeuronal mechanisms of a lt l i ticultural invention

Stanislas DehaeneCollège de France

andand INSERM‐CEA 

Cognitive Neuroimaging UnitNeuroSpin Center, Saclay, France

www.unicog.org Jean‐Honoré Fragonard, La liseuse (1772)National Gallery of Art, Washington 

What is reading?“A conversation with the deceased

(…) [a way to] listen to the dead with my eyes”with my eyes

Francisco de Quevedo

“The art of communicating thoughts to the mind, through the eye ‐‐the great invention of the world ”the great invention of the world.

Abraham Lincoln

“We are absurdly accustomed to the miracle of a few written signs being able to contain immortalbeing able to contain immortal imagery, involutions of thought, new worlds with live people, speaking weeping laughing ”speaking, weeping, laughing.

Vladimir Nabokov, Pale Fire

Writing systems: cultural diversity and biological unityAt first sight, writing systems seemto vary enormously in their forms.

Yet in this talk I will defend the viewthat

- they are all fundamentallyrelated

th ll ll th b i- they all call upon the same braincircuits

this cross cultural universality- this cross-cultural universalitycan be explained:

During reading acquisition we learnDuring reading acquisition, we learnto recycle evolutionarily oldercircuits for visual and gesturerecognition (Neuronal recyclingrecognition (Neuronal recyclingmodel)

The time course of reading

K. Marinkovic, A. Dale, E. Halgren

The time course of reading

K. Marinkovic, A. Dale, E. Halgren

i d i i

The brain architecture for readingLearning to read consists in:

‐ creating an invariant visual representation of  written words‐ connecting it to brain areas coding for speech sounds and meaning

Access to pronunciationand articulation

Visual inputs

Visual word form areaAccess to meaning

Perisylvian language areas are alreadyPerisylvian language areas are already activated by spoken language in two‐

month‐old babiesmonth old babies

L1

Adult (for reference)

•Left perisylvian temporal and inferior frontal regions are activated by speech

Two‐month old infant

•Left perisylvian temporal and inferior frontal regions are activated by speech

•Activation unfolds according to a temporal hierarchy.

•A left‐hemispheric asymmetry is present in Planum Temporale for speech but not for music

Dehaene‐Lambertz et al, Science, 2002; PNAS, 2006; Brain & Language, 2009

A left hemispheric asymmetry is present in Planum Temporale, for speech but not for music.

The brain architecture for readingBefore the child learns to read, the major systems for vision and

speech recognition are already in place.An interface must be created between vision and language

Access to pronunciationand articulation

Visual inputs

Visual word form areaAccess to meaning

The visual word form area (VWFA)(VWFA)

• A reproducible site of activation duringreading in all cultures (e.g. Bolger, Perfetti & Schneider, g2005)

• Always located at the same coordinatesin the left lateral occipito-temporal sulcus

• Whose lesion can cause pure alexia, an acquired selective disability in reading (e.g. Déjerine, 1892; Gaillard et al., 2006)

• Which activates to known scripts more than to other categories of visual stimuli (e.g. Baker et al., 2007)

• A high-level visual area, invariant for location and case in word identification (e.g. Dehaene et al., 2001; Cohen et al., 2002)

• An automated system, capable of activating even to subliminal stimuli (e.g. Dehaene et al., 2001, 2004)

The reading paradox:Why do we seem to have a brainWhy do we seem to have a brain 

“organ” for reading, given that they was no pressure for it to evolve? p

The visual wordThe visual wordform area(VWFA)

WORDSWORDS

The neuronal recycling hypothesis

The architecture of the human brain is tightly constrained by its past evolution.constrained by its past evolution. 

New cultural acquisitions are only possible if they fit within this pre‐existing architecture. 

Each cultural objectmust find its neuronal niche ‐‐ a set of circuits that are sufficiently close to the required function and sufficiently q yplastic to be partially “recycled”.

We inherit specialized circuits for vision, auditory processing and spoken languageprocessing, and spoken language.

The invention of writing consisted in recycling these regions so that they could process written signs.

The brain never evolved “for reading”, but writing systems evolved “for the brain”.writing systems evolved  for the brain .

Recycling the brain for readingThe ventral visual pathway of all primates contains 

evolved circuitry for invariant visual recognition.

Human brain Macaque monkey

After size normalization

Visual recognition  Visual recognitionof objects, faces;and written words

of objects and faces

Neurons in this region respond to an alphabet of shape g p p pdescriptors.

We adopted many of these shapes as the foundation of our writing systems.

Jean‐Baptiste Siméon Chardin, Le singe antiquaire

our writing systems.

The properties of this area can explain puzzling features of reading acquisition 

Organization of the primate inferotemporal visual pathwayinferotemporal visual pathway

Gross Tanaka Logothetis Poggio Perrett Orban Rolls etcGross, Tanaka, Logothetis, Poggio, Perrett, Orban, Rolls, etc.

From Tamura H & Tanaka K (2001) Cerebral CortexFrom Rolls, Neuron 2000 From Tamura, H., & Tanaka, K. (2001). Cerebral Cortex.,

A putative precursor of the visual word form:The « alphabet » of object features in macaqueThe « alphabet » of object features in macaque 

infero‐temporal cortex

A putative precursor of the visual word form:The « alphabet » of object features in macaqueThe « alphabet » of object features in macaque 

infero‐temporal cortex

K. Tanaka, 1996

Brincat & Connor, 2004

Origins of the universal « alphabet » of shapes :The importance of non‐accidental properties in visionp p p

Different types of line junctions (T, L, Y…) provide essential information for visual shaperecognition.g

TT

YE

Biederman, Psychological Review, 1987 

The topology of strokes in written symbolsobeys a universal statistical distributionobeys a universal statistical distribution

Changizi’s universal distribution

Symboles

Changizi’s 9 most frequent configurations

Changizi & Shimojo (2005)Changizi et al (2006)

A hypothesis for the reproducible localization of the VWFASzwed, Dehaene, et al., Neuroimage 2011

• Fusiform, but not lateral occipital object areas are sensitive to the presence of line junctions rather than line segments.

• This factor together with other factors (excentricity connections to

junctions > segmentsin objects

• This factor, together with other factors (excentricity, connections to language cortex) may explain the reproducible location of the VFWA.

object areas

reading areasz=-16

Lateral Occipital (LO)object areas >

F if

vertex>midsegment in objects

Fusiformobject areas* >

peak of line junctions > segments in words*

Another source of constraints:The left‐hemispheric lateralization of spoken languageThe left hemispheric lateralization of spoken language

- The « letter box » visual word form area varies in its lateralization across individuals. In individuals with unusual- Spoken language lateralization predicts itslateralization.

In individuals with unusualright-hemisphere language, the VWFA is also right-lateralized

Pinel & Dehaene, J. Cognitive Neuroscience (2009)

lateralization map for sentence reading

Colateralization

Cai et al., Cerebral Cortex (2009):

Colateralization--- within reading--- within speech listening

f li t i t di--- from listening to readingtypical

lateralizationatypical

lateralization

Involvement of the VWFA and early visual cortices in readingSzwed, Dehaene, et al., Neuroimage 2011

words - controls

- Once controlling for low-level visual features, early visual areas are more active for words than pictures

Interaction(words – controls)

> VP/V4

2 2

Left hemisphere Right hemisphere

z=-16

(pictures – controls)

0

1

2

0

1

2

pictures - controlsV1/V2

2

-1

2

-1

-

-1

0

1

2

-1

0

1

2

-2 -2

pictures – scrambled pictures

words – scrambled wordsExpertise and retinotopic

t l l i i l

z=-16

pictures scrambled pictures

word gestalts – scrambled words

perceptual learning in early visual cortex?

Directly visualizing the brain changes due to literacyDehaene Pegado Braga Ventura Nunes Filho Jobert Dehaene Lambertz Kolinsky Morais and CohenDehaene, Pegado, Braga, Ventura, Nunes Filho, Jobert, Dehaene‐Lambertz, Kolinsky, Morais and Cohen

How learning to read changes the cortical networks for vision and language. Science, 2010

What is changed in the visual system whenwe learn to read?

• In the VWFA?

• At a lower visual level?• At a lower visual level?

• Elsewhere?

What stimuli activate the VWFA before welearn to read?

Do we lose or do we gain when learning to read? Is there cortical competition?

Juan de Roelas, Saint  Ann teaching Virgin Mary to read (1615, Sevilla)

read? Is there cortical competition?

Can the same changes be induced by learning to read in adulthood?

fMRI and high‐density ERP recordings in six groups of subjects

– Illiterates (Brasil only, n=10)

– Ex‐illiterates (Brasil, n=10, Portugal, n=11)Ex illiterates (Brasil, n 10, Portugal, n 11)

– Literates (Brasil: high SES, n=10; low SES, n=10; Portugal, n=12)

Six populations of subjects and their performances

Reading speed is highly variable.Given enough time, all groups (except the illiterates) can read.

The illiterates scanning protocol• Three fMRI experiments: 

– Localizer with checkers, spoken and written sentences (Pinel et al. 2007)

10 short sentences 10 subtractions 10 motor instructions 20 Flashing

the river

across

fifteen

from

button

left

Visual Stimuli 

10 short sentences 10 subtractions 10 motor instructions 20 Flashingcheckerboards

many bridges

There are

seven

Subtract

on  the

Press

time

10 short sentences 10 subtractions 10 motor instructions

– Visual categories (with a minimal task of detecting occasional stars)

Audio. Stimuli  “we easily find ataxi in Paris”

“press theleft button”

“subtract nineto eleven”

10 short sentences 10 subtractions 10 motor instructions

– Visual categories (with a minimal task of detecting occasional stars)Faces  Houses Tools Strings False fonts      Checkers

– Spoken words and pseudowords during lexical decisionp p g• with consistent or inconsistent spelling, testing for the orthographic consistency effect 

(Ziegler and Ferrand, 1998)

Increases in activation to written sentences withliteracy

LB1LP

4

Response at [-44, -50, -14]

VWFA

6Response at [-46, -2, 52]

Left frontalz =‐14 The left ventral 

visual pathway i itLP

LB2EXB

ILBEXP

1

2

3

0

2

4increases its responses to written words

Checkers Spoken Written Checkers Spoken Written

x =‐48

0

Response at [-54, 26, -6]Response at [24, -84, -10]

Occipital

L R

In language areas,

0

1

2

4

In language areas, written wordsattain the samelevel of activation

Checkers Spoken Written

Response at [-50, 12, -24]

-1

Response at [-50, -44, 6] Response at [-54, -12, -12]Left superior temporal sulcus

0Checkers Spoken Written

level of activation as spoken words. 

1

2

4

6

1

2Literates (LB1)Literates (LP)

Ex‐illiterates (EXB)Literates (LB2)

Checkers Spoken Written Checkers Spoken WrittenCheckers Spoken Written

00

2

C S

0Ex‐illiterates (EXP)Ex illiterates (EXB)

Illiterates (ILB)

Directly visualizing the brain changes due to literacy

Literacy enhances the brain activation to written sentences.

Brainactivation

2

3

4

Ex‐illiterates

Literates(schooled in childhood)

0

1

2 (learned to read in adulthood)

Illiterates

0 50 1000

Words read per minute

A massive enhancement in the visual word form area in response to  letter strings.

This effect exists even in ex‐illiterates who learned to read in adulthoodThis effect exists even in ex‐illiterates who learned to read in adulthood.No critical period.

What does the visual word form area do prior to reading?Literates (LB1)

li i f h li ff i h l i It responds mostly to faces

(and somewhat less to objects and checkers)2

Literates (LP)

Ex‐illiterates (EXP)Ex‐illiterates (EXB)

Illiterates (ILB)

Literates (LB2)

Replication of the literacy effect with letter strings

z =‐14

objects and checkers)

Responses to faces 1

1.5

2 Illiterates (ILB)

pand checkersdecrease with

literacy.F H T S FF Ck

0

0.5

L R F H T S FF Ck

VWFA activation to :

L R

2 2 2 2 2 2

Faces Houses Tools Letter strings False fonts CheckersVWFA activation to :

0.5

1

1.5

0.5

1

1.5

0.5

1

1.5

0.5

1

1.5

0.5

1

1.5

0.5

1

1.5

0 50 100 1500

0 50 100 1500

0 50 100 1500

0 50 100 1500

0 50 100 1500

0 50 100 1500

Words read per minute

What does the visual word form area do prior to reading?Literates (LB1)

li i f h li ff i h l i It responds mostly to faces

(and somewhat less to objects and checkers)2

Literates (LP)

Ex‐illiterates (EXP)Ex‐illiterates (EXB)

Illiterates (ILB)

Literates (LB2)

Replication of the literacy effect with letter strings

z =‐14

objects and checkers)

Responses to faces 1

1.5

2 Illiterates (ILB)

pand checkersdecrease with

literacy.F H T S FF Ck

0

0.5

L R

In the right anteriorfusiform gyrus, 4

F H T S FF CkL R

responses to faces increasewith literacy.

faces are 2

3

« pushed » towardsthe right hemisphere1

2

F H T S FF Ck

A competition between written words and faces

The peak response to faces is not changed by literacy.

The competition between faces and words occurs at the boundary: as greater space is gained by words, less space is occupied by faces.

Shells centered on peak response to faces – houses

Subject specific peakf f h faces  houses

p=0.015

Mary Cassatt, Young girl reading (Smithsonian)

e.g. for faces ‐ house

p=0.037

nsnsns

C t i h ll

shell radius in voxels

peak 2 4 6 8Concentric « shells » around the peak

Literacy mostly has a positive impact on the visual system 10 Response at [24, -84, -10]z =‐10

6

Response at [24, 84, 10]Literates (LB1)Literates (LP)

Ex‐illiterates (EXB)Literates (LB2)

2

4

Responses to 

Ex‐illiterates (EXP)Ex illiterates (EXB)

Illiterates (ILB)

L R

pall categories of 

stimuli increase in the 

Response at [16, -88, 2]Response at [-12, -88, 2] z = 2

occipital cortex.

4

64Even the primaryvisual cortex selectivelyi i

0

2

4

0

2

L R

increases itsresponse to 

horizontal shapes( h k )

Horiz Verti0

Horiz Verti(checkers)

Literacy also changes spoken language processing

Activation of Planum Temporale to spoken language is doubled in literates

x =‐38

6Response at [-38, -28, 18]

4

6 LB1LP

2

4z =18 LB2

EXB

0

2

ILBEXP

0Checkers Spoken Written

L R

The main brain changes induced by reading

Regionsinvolved in spokenl

Phonemicrepresentation

Planum temporalelanguage Planum temporale

Visual cortex

l fVisual word form area:The brain’s « letterbox »

How do single neurons tune to reading?

Model of object or face recognitionSize and structureof receptive fields Sample preferred stimuli

Model of word recognition

E

vraimentCOMMENT

En

EE

+- -

Shimon Ullman

Testing the predicted hierarchical organization of the visual word form areathe visual word form area

False fonts Infrequent letters Frequent letters Bigrams Quadrigrams Words

100%

0%

Percent activation relative to words in the occipitotemporal cortexAverage of non-word stimuliVinckier, Cohen , Dehaene et al, Neuron 2007

Bigram coding can explain some puzzles of reading(Grainger & Whitney, 2004)

Aoccdrnig to rseaecrh at Cmabrigde Uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are the olny iprmoatnt tihng iswaht oredr the ltteers in a wrod are, the olny iprmoatnt tihng istaht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs isy pbcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. 

Matt Davis, http://www.mrc‐cbu.cam.ac.uk/~mattd/Cmabrigde/, p // / / g /

• If words are coded by bigrams (ordered pairs of letters)

BADGE =BADGE = 

BA + BD + BG + BE + AD + AG + AE + DG + DE + GE

then « BAGDE » and « BADGE » have very similar codes(Perea & Lupker, 2003; Schoonbaert & Grainger, 2004)

• The proportion of shared bigrams predicts similarity (Grainger et al, 2006)

There is no such thing as « whole‐word reading »

A classical debate in education: « whole‐words » or « phonics » ?

In adults, reading is so fast that it seems instantaneous.The time to read a word is independent of the number of letters.These findings give an illusion of whole‐word reading.

ds)

millisecon

d

In reality:The adult brain relies on letters

ng time (m ‐ The adult brain relies on letters, 

but they are processed in parallel‐ In young children, reading relies on 

i l l tt b l tt d di

Read

in serial, letter‐by‐letter decoding‐ Learning with the whole‐wordmethod ismuch slower and trains 

Number of letters in the wordthe wrong brain area in the right hemisphere.

Explaining another puzzle of reading acquisition:mirror reading and writing

Mirror writing in children The reinvention of « boustrophedon »

g g

All children show a « mirror stage » in learning to read

Mirror reading and writing are neverexplicitly taught.

% hild

How can youngchildren be more competent than

100

normalmirror

% children able to write their name

competent thanadults?

50

mirror

0<5 y 5-6 y 6-7 y 7-8 y > 8 y

(Data from Cornell, 1985)

Children’s age

Mirror generalization : the Panda’s thumb of reading?

• We have evolved a symmetrymechanism that helps to recognizefaces and objects regardless of theirfaces and objects regardless of theirorientation

•This « symmetry generalization » isun‐learned when we learn to read.

Infero‐temporal neurons generalize across mirror images

Infero‐temporal neurons of macaque monkeys respond more similarly tomonkeys respond more similarly to horizontal mirror‐image pairs than to vertical mirror‐image pairs (Rollenhagen & Olson, Science, 2000).(Rollenhagen & Olson, Science, 2000).

Following training with a specific random wireframe shape infero temporal neurons in

Preferred view Mirror generalization

‐72°‐108°‐144° ‐36° 0° +36° +72° +108° +144° +180°

Following training with a specific, random wireframe shape, infero‐temporal neurons in the macaque spontaneously generalize to its mirror image (Logothetis, 1995).

Mirror invariance in a face patch in macaquesFreiwald & Tsao Science 2010Freiwald & Tsao, Science 2010

View‐point selectiveselective

Mirror invariance

View‐invariant responseto specific individuals

Mirror invariance ispresent in the majority of neuronsin patch AL. 

Mirror invariance in the human brainDehaene, Nakamura et al., NeuroImage, 2009

30

Normal pairs

Mirror

Same

Different

SameX

Priming for words Priming for pictures25

30 Mirror pairs

Same

Different

BUT NOT their mirror images AND their mirror images

15

20

5

10

0

Words Pictures

Replication with simpler images, matched to lettersPegado, Nakamura, Cohen and Dehaene, NeuroImage, 2011

Surprisingly, we learn to readprecisely with the brain area thatbest recognizes mirror pictures.g No wonder that we have trouble learning the letters p, q, b or d

No relation to dyslexia

g 

30

Normal pairs

Mirror

Same

Different

SameX

No relation to dyslexia

Priming for words Priming for pictures25

30 Mirror pairs

Same

Different

BUT NOT their mirror images AND their mirror images

15

20

5

10

0

Words Pictures

illiterates perform relatively better in mirror‐image judgements(Pegado et al., in revision)

Letter stringsPictures False fontsAre these images « the same »?

pictures false fonts stringspictures false‐fonts strings

RT Mirror Cost Index

The cost for decidingthat two mirrorimages are the sameincreases with literacy

r2=7.7%;p<0.05

r2=18.1%;p<0.001

r2=23.6%;p<0.0001

Reading score (words and pseudowords read per minute)

The flip side: illiterates have problems in making left‐right distinctionsilliterates have problems in making left right distinctions

Kolinsky et al., Journal of Experimental Psychology: General, 2011

Conclusion: Understanding the « miracle of reading »We can acquire literacy because:We can acquire literacy because:

‐We inherit a structured and efficient visual recognition system, attuned to line junctions and other invariants

‐ This system is sufficient plastic to bepartially reoriented towards thepartially reoriented towards the particular shapes of written words

‐ Generations of scribes selected writtenh h fi b i hishapes that fit our brain architecture.

We now understand that the changes induced by literacy go much beyond the y y g yvisual word form area.

A neural understanding of this system canexplain many of the quirks of this

Pierre‐Auguste Renoir, Portrait de Claude Monet

explain many of the quirks of thiscultural invention (e.g. mirror reading).

It also clarifies how reading should betaught (letters and graphemes, not whole words).

A final word on culture and brain

Can the neuronal recycling model be extended to other domains of  human culture? Mathematics, tool use, art, religion, language…, , , g , g g

Brain research leads to new educational toolsBrem et al., PNAS 2010

Lyytinen and colleagues have developed the graphogame, free software that teaches the knowledge of letter-sound correspondencesg p

Kindergarten pre-reader children were trained for less than four hours (spread over 8 weeks) with the graphograme or with a number control gamegraphograme or with a number control game.

fMRI and ERPs revealed the emergence of the visual word form and other brain responses to printp p(words versus false-fonts)