Cognitive Maps: How the Brain Organizes Knowledge Ling 411 – 18

Cognitive Maps:How the Brain Organizes Knowledge

Ling 411 – 18

The Cognitive Map Hypothesis

Hypothesis: Knowledge is organized in the cortex as maps Established (hence not hypothetical):

• The cognitive map of the body Primary motor and somatosensory areas

• The map of pitch frequency In primary auditory area

Hypothesized:• Conceptual• Phonological

Properties of cognitive maps

Established for somatic and frequency maps• Local specificity

Every cortical location has a specific function• Adjacency

Adjacent locations for adjacent functions Nearby locations for related functions Comes in degrees

Hypothesis: these properties apply to • all homotypical cortical areas• all types of knowledge represented in the cortex

First step in exploring the hypothesis:Categories

Understanding phonology• Phonological structure is organized around

phonological categories E.g., vowels and consonants, voiceless stops

Understanding semantics• Semantic structure is largely a matter of conceptual

categories• Understanding how categories work is the key to

unlock the mysteries of semantics• To understand how categories work we need to

understand how the brain manages categorial information

What is a concept?Concepts vs. percepts

Percept: one sensory modality• Locations are known

• Auditory: temporal lobe• Visual: occipital lobe• Somatosensory: parietal lobe

Concept: more than one sensory modality• Higher level (more abstract)• Locations, for nominal concepts:

Angular gyrus (?)MTG

Types of Conceptual Categories

Discrete• Even integers• Counties in Texas

Radial• Birds• Vehicles

Family resemblance• Games• Furniture

Ill-defined• Thought• Mind

Phenomena associated with conceptual categories

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members4. Categories are in the mind, not in the real world5. Categories and their memberships vary from one

language/culture system to another6. Categories influence thinking, in both appropriate and

inappropriate ways7. Subcategories, and sub-subcategories, in hierarchical

chains

Phenomena associated with categories: 1

1. No small set of defining features (with rare exceptions)

• The feature-attribute model fails Works for some mathematical objects, but

doesn’t apply to the way people’s cognitive systems apprehend most things

Example: CUP

Phenomena associated with categories: 2

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries

• Example: VEHICLE Car, truck, bus Airplane? Boat? Toy car, model airplane? Raft? Roller skate? Snowboard?

Fuzzy Categories

No fixed boundaries Membership comes in degrees

• Prototypical • Less prototypical• Peripheral• Metaphorical

The property of fuzziness relates closely to the phenomenon of prototypicality

Phenomena associated with categories: 3

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members

• Prototypical CAR, TRUCK, BUS

• Peripheral: AIRPLANE, TOY CAR, RAFT, ROLLER SKATE, etc.

• Varying degrees of peripherality

Prototypicality phenomena

The category BIRD

• Some members are prototypical ROBIN, SPARROW

• Others are peripheral EMU, PENGUIN

The category VEHICLE• Prototypical: CAR, TRUCK, BUS

• Peripheral: ROLLER SKATE, HANG GLIDER

Phenomena associated with categories: 4

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members

4. Categories are in the mind, not in the real world

• In the world, everything is unique lacks clear boundaries changes from day to day (even moment to

moment)• Whorf: “kaleidoscopic flux”

Phenomena associated with categories: 51. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members4. Categories are in the mind, not in the real world

5. Categories and their memberships vary from one language/culture system to another

cloche (of a church)clochette (on a cow)sonnette (of a door)grelot (of a sleigh)timbre (on a desk)glas (to announce a death)

English: French:

bell

Phenomena associated with categories - 6

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members4. Categories are in the mind, not in the real world5. Categories and their memberships vary from one language/culture

system to another

6. Categories influence thinking, in both appropriate and inappropriate ways

• B.L. Whorf• Example: Racial profiling

Phenomena associated with categories - 7

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members4. Categories are in the mind, not in the real world5. Categories and their memberships vary from one language/culture

system to another6. Categories influence thinking, in both appropriate and

inappropriate ways

7. Subcategories, and sub-subcategories, in hierarchical chains

• ANIMAL – MAMMAL – CARNIVORE – CANINE – DOG –TERRIER – JACK RUSSELL TERRIER – EDDIE

• Each subcategory has the properties of the category plus additional properties

• Smallest subcategory has the most properties

Beyond description to explanation

How can we explain these phenomena? The answer this question depends on how our

information about categories is represented in the brain The brain is where our linguistic and cultural knowledge

is represented

Facts and a hypothesis that we can build on

Fact: The brain is a network• Composed, ultimately, of neurons• Cortical neurons are clustered in columns

Columns come in different sizes Each minicolumn acts as a unit

Therefore a person’s linguistic and conceptual system is a network

Hypothesis: Every word and every concept is represented as a sub-network• Term: functional web (Pulvermüller 2002)

Properties of functional webs

I: Functional Webs• A concept is represented as a functional web

II: Columnar Nodes• Nodes are implemented as cortical columns

III: Nodal Specificity • Every node in a functional web has a specific function

III(a): Adjacency• Nodes of related function are in adjacent locations

More closely related function, more closely adjacent

Property III(a): Adjacency

Nodes of related function are in adjacent locations• More closely related function, more closely adjacent

Examples:• Adjacent locations on cat’s paw represented by

adjacent cortical locations• Similar line orientations represented by adjacent

cortical locations

Hypotheses concerning conceptual webs

Hypothesis I: Extrapolation to Humans• The findings about cortical structure and function

from experiments on cats, monkeys, and rats can be extrapolated to humans

• Hypothesis I(a): The extrapolation can be extended to linguistic and conceptual structures and functions

Hypothesis II: Hierarchy • A functional web is hierarchically organized

Hypothesis III: Cardinal nodes• Every functional web has a cardinal node • Hypothesis III(a):

Each subweb likewise has a cardinal node

(Part of) the functional web for CAT

V

P

A

M

C

The cardinal node for the entire functional web

T

Cardinal nodes of the subwebs

Phenomena associated with categories

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members4. Subcategories, and sub-subcategories, in hierarchical

chains5. Categories are in the mind, not in the real world6. Categories and their memberships vary from one

language/culture system to another7. Categories influence thinking, in both appropriate and

inappropriate ways

REVIEW

How to explain?

Description is fine, but its only a start Next step: Explanation How to explain?

• By answering the question of how categories are represented in the brain

REVIEW

Phenomena associated with categories: 1-3

1. No small set of defining features (with rare exceptions) • Example: CUP

• More realistic alternative: radial categories2. Fuzzy boundaries

• Example: VEHICLE

3. Prototypical members and peripheral members• VEHICLE

Prototypical:• CAR, TRUCK, BUS

Peripheral: • AIRPLANE, TOY CAR, RAFT, ROLLER SKATE, etc.• Varying degrees of peripherality

These three phenomena are interdependent

How do radial categories work?

Different connections have different strengths (weights) More important properties have greater strengths For CUP,

• Important (but not necessary!) properties: Short (as compared with a glass) Ceramic Having a handle

Cups with these properties are more prototypical

The properties of a category have different weights

T

CUP

MADE OF GLASS

CERAMIC

SHORT

HAS HANDLE

The properties are represented by nodes which are connected to lower-level nodes

The cardinal node

The threshold

More important properties have greater weights, represented by greater thicknesses of lines

Activation of a category node

The node will be activated by any of many different combinations of properties

The key word is enough – it takes enough activation from enough properties to satisfy the threshold

The node will be activated to different degrees by different combinations of properties• When strongly activated, it transmits stronger

activation to its downstream nodes.

Prototypical exemplars provide stronger and more rapid activation

T

CUP

MADE OF GLASS

CERAMIC

SHORT

HAS HANDLE

Stronger connections carry more activation

Activation threshold (can be satisfied to varying degrees)

Inhibitory connection

Explaining Prototypicality

Cardinal category nodes get more activation from the prototypical exemplars • More heavily weighted property nodes

E.g., FLYING is strongly connected to BIRD • Property nodes more strongly activated

Peripheral items (e.g. EMU) provide only weak activation, weakly satisfying the threshold (emus can’t fly)

Borderline items may or may not produce enough activation to satisfy threshold

Activation of different sets of properties produces greater or lesser satisfaction of the activation threshold of the cardinal node

CUP

MADE OF GLASS

CERAMICSHORT

HAS HANDLE

Explaining prototypicality: Summary

Variation in strength of connections Many connecting properties of varying strength Varying degrees of activation Prototypical members receive stronger activation from

more associated properties BIRD is strongly connected to the property FLYING

• Emus and ostriches don’t fly• But they have some properties connected with BIRD• Sparrows and robins do fly

And as commonly occurring birds they have been experienced often, leading to entrenchment – stronger connections

Phenomena associated with categories: 4

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members

4. Categories are in the mind, not in the real world• In the world, everything

is unique lacks clear boundaries changes from day to day (even moment to

moment)• Whorf: “kaleidoscopic flux”

Phenomena associated with categories: 51. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members4. Categories are in the mind, not in the real world

5. Categories and their memberships vary from one language/culture system to another

cloche (of a church)clochette (on a cow)sonnette (of a door)grelot (of a sleigh)timbre (on a desk)glas (to announce a death)

English: French:

bell

REVIEW

Phenomena associated with categories - 6

1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries3. Prototypical members and peripheral members4. Categories are in the mind, not in the real world5. Categories and their memberships vary from one language/culture

system to another

6. Categories influence thinking, in both appropriate and inappropriate ways

• B.L. Whorf• Example: Racial profiling

These phenomena (4-6) are interrelated

4. Categories are in the mind, not in the real world5. Categories and their memberships vary from one

language/culture system to another6. Categories influence thinking, in both appropriate and

inappropriate ways• B.L. Whorf• Example: Racial profiling

Bidirectional processing and inference

T

CUP

MADE OF GLASS

CERAMICSHORT

HANDLE

These connections are bidirectional

Separate fibers for the two directions; shown as one line in the notation

Bidirectional processing and inference

T

CUP

SHORT

HANDLE

Thought process: 1. The cardinal concept node is activated by a subset of its property nodes 2. Feed-backward processing activates other property nodes

Consequence: We “apprehend” properties that are not actually perceived

Another hypothesis of Whorf

Grammatical categories of a language influence the thinking of people who speak the language

Can we explain this too in terms of brain structure?

Example: Grammatical gender

Does talking about inanimate objects as if they were masculine or feminine actually lead people to think of inanimate objects as having a gender?

Could the grammatical genders assigned to objects by a language influence people’s mental representation of objects?

Boroditsky (2003)

Experiment: Gender and Associations(Boroditsky et al. 2002)

Subjects: speakers of Spanish or German• All were fluent also in English• English used as language of experiment

Task: Write down the 1st 3 adjectives that come to mind to describe each object• All the (24) objects have opposite gender

in German and Spanish Raters of adjectives: Native English speakers

Examples:

Key (masc in German, fem in Spanish)• Adjectives used by German speakers:

Hard, heavy, jagged, metal, serrated, useful• Adjectives used by Spanish speakers:

Golden, intricate, little, lovely, shiny, tiny Bridge (fem in German, masc in spanish)

• Adjectives used by German speakers: Beautiful, elegant, fragile, peaceful, pretty

• Adjectives used by Spanish speakers: Big, dangerous, long, strong, sturdy, towering

Results of the Experiment(Boroditsky et al. 2002)

Raters of adjectives were native English speakers Result: Adjectives were rated as masculine or feminine

in agreement with the gender in subject’s native language

Categories and the brain

All of these phenomena associated with categories can be explained as inevitable consequences of the structure and function of the human brain

Phenomena associated with categories: 7

7. Subcategories, and sub-subcategories, in hierarchical chains

• ANIMAL – MAMMAL – CARNIVORE – CANINE – DOG –TERRIER – JACK RUSSELL TERRIER – EDDIE

• Each subcategory has the properties of the category plus additional properties

• Smallest subcategory has the most properties

How to explain? Perceptual Neuroscience

Hypothesis I: Extrapolation• The findings described by Mountcastle can be

extrapolated to humans Hypothesis I(a): Extrapolation can be extended to

linguistic and conceptual structures Why? Cortical structure, viewed locally, is

• Uniform across mammalian species • Uniform across different cortical regions

Cortical structure and function, locally, are essentially the same in humans as in cats and monkeys and rats• Moreover, in humans, the regions that support language have

the same structure locally as other cortical regions

Support for the extrapolation hypothesis

Conceptual systems in humans evidently use the same structures as perceptual systems

Therefore it is not too great a stretch to suppose that experimental findings on the structure of perceptual systems in monkeys can be applied to an understanding of the structure of conceptual systems of human beings

In particular to the structures of conceptual categories

REVIEW

Columns of different sizes

Minicolumn• Basic anatomically described unit• 70-110 neurons (avg 75-80)• Diameter barely more than that of pyramidal cell body (30-50 μ)

Maxicolumn (term used by Mountcastle)• Diameter 300-500 μ• Bundle of about 100 continuous minicolumns

Hypercolumn – up to 1 mm diameter• Can be long and narrow rather than cylindrical

Functional column• Intermediate between minicolumn and maxicolumn• A contiguous group of minicolumns

Functional Columns

Intermediate in size between minicolumn and maxicolumn

Hypothesized functional unit whose size is determined by experience/learning

A maxicolumn consists of multiple functional columns A functional column consists of multiple minicolumns Functional column may be further subdivided with

learning of finer distinctions

Columns of different sizes In order according to size

Minicolumn• The smallest unit• 70-110 neurons

Functional column• Variable size – depends on experience• Intermediate between minicolumn and maxicolumn

Maxicolumn (a.k.a. column)• 100 to a few hundred minicolumns

Hypercolumn• Several contiguous maxicolumns

Hypercolums: Modules of maxicolumns

A visual area in temporal lobe of a macaque monkey

Perceptual subcategories andcolumnar subdivisions of larger columns

Nodal specificity applies for maxicolumns as well as for minicolumns

The adjacency hypothesis likewise applies to larger categories and columns• Adjacency applies for adjacent maxicolumns

Subcategories of a category have similar function• Therefore their cardinal nodes should be in adjacent

locations

Functional columns

The minicolumns within a maxicolumn respond to a common set of features

Functional columns are intermediate in size between minicolumns and maxicolumns

Different functional columns within a maxicolumn are distinct because of non-shared additional features • Shared within the functional column• Not shared with the rest of the maxicolumn

Mountcastle: “The neurons of a [maxi]column have certain sets of static and dynamic properties in common, upon which others that may differ are superimposed.”

Similarly..

Neurons of a hypercolumn may have similar response features, upon which others that differ may be superimposed

Result is maxicolumns in the hypercolumn sharing certain basic features while differing with respect to others

Such maxicolumns may be further subdivided into functional columns on the basis of additional features

That is, columnar structure directly maps categories and subcategories

Hypercolums: Modules of maxicolumns

A visual area in the temporal lobe of a macaque monkey

Category (hypercolumn)

Subcategory(can be further subdivided)

Category representations in the cortex

Hypercolumn

Maxicolumn

Functional column

Sub-functional column

Supercategory

Category

Subcategory

Sub-subcategory

Hypothesis applied to conceptual categories

A whole maxicolumn gets activated for a category• Example: BEAR

Different functional columns within the maxicolumn for subcategories

• BROWN BEAR, GRIZZLY, POLAR BEAR, etc.

Adjacent maxicolumns for categories related to BEAR (sharing various features)• I.e. , other carnivores

Similarly, CUP has a column surrounded by columns for other drinking vessels

Perceptual subcategories andcolumnar subdivisions of larger columns

Nodal specificity applies for maxicolumns as well as for minicolumns

The adjacency hypothesis likewise applies to larger categories and columns• Adjacency applies for adjacent maxicolumns

Subcategories of a category have similar function• Therefore their cardinal nodes should be in adjacent

locations

Support from patients with brain damage(from Rapp & Caramazza 1995)

J.B.R. and S.B.Y. (905b-906a)

Herpes simplex encephalitis Both temporal lobes affected Could not define animate objects

• ostrich, snail, wasp, duck, holly Much better at defining inanimate objects

tent, briefcase, compass, wheelbarrow, submarine, umbrella

Conclusion: cortical areas for conceptual categories

Additional support from cases of brain damage

J.J. and P.S. (Hillis & Caramazza 1991) (906-7)• J.J. – left temporal, basal ganglia (CVA)

Selective preservation of animal concepts• P.S. – mostly left temporal (injury)

Selective impairment of animate category

P.S

J.J.

Two different patients with anomia

Deficit in retrieval of animal names(Damage from stroke)

Inability to retrieve words for unique entities(Left temporal lobectomy)

Two more patients with anomia

Deficit in retrieval of words for man-made manipulable objects(Damage from stroke)

Severe deficit in retrieval of words for concrete entities(Herpes simplex encephalitis)

What is it that determines location?

Logical categories like ANIMALS vs. TOOLS/UTENSILS?• If so, why?

Abstract categories based on cognitively salient properties?

Animals vs. Tools/Utensils?

These two categories have been studied most extensively in the literature

What is it that determines location? Observations:

• Most animals are known mostly in the visual modality• Many tools and utensils are known largely in the

somatosensory and motor modalities

Proximity principle and nominal concepts

Supramarginal gyrus, angular gyrus, and middle temporal gyrus are all close to Wernicke’s area

Angular gyrus occupies intermediate location between the major perceptual modalities

Supramarginal gyrus especially close to somatosensory perception

Middle temporal gyrus especially close to visual perception

Functional columns in phonological recognitionA hypothesis

Demisyllable (e.g. /de-/) activates a maxicolumn Different functional columns within the maxicolumn

for syllables with this demisyllable• /ded/, /deb/, /det/, /dek/, /den/, /del/

Demisyllables [di, de, da, du]

F1 and F2For [de]

It is unlikely that [d] is represented as a unit in perception

Functional columns in phonological recognitionA hypothesis

[de-]

A maxicolumn (ca. 100 minicolumns)

Divided into functional columns

(Note that all respond to /de-/)

deb

ded den de- det del

dek

Phonological hypercolumns (a hypothesis)

Maybe we have • Hypercolumn of contiguous maxicolumns for /e/• With maxicolumns for /de-/, /be-/, etc.• Each such maxicolumn subdivided into functional

columns for different finals /det/, /ded/, /den/, /deb/, /dem/. /dek/

N.B.: This is a hypothesis, not proven• But there is indirect evidence• Maybe someday soon we’ll be able to test with

sensitive brain imaging

Adjacent maxicolumns in phonological cortex?

ge- ke-

be- pe-

te- de-A module of six

contiguous maxicolumns

Each of these maxicolumns is

divided into functional columns

Note that the entire module responds to [-e-]

Hypercolum

Adjacent maxicolumns in phonological cortex?

ge- ke-

be- pe-

te- de-A module of six

contiguous maxicolumns

The entire module responds to [-e-]

deb

ded den de- det del

dek

The entire maxicolumn responds to [de-]

Learning phonological distinctions:A hypothesis

ge- ke-

be- pe-

te- de-1. In learning,

this hypercolumn

gets established

first, responding to

[-e-]2. It gets subdivided into maxicolumns for demisyllables

deb

ded den de- det del

dek

3. The maxicolumn gets divided into functional columns

Indirect evidence for the hypothesis

Fits the structural organization demonstrated in monkey vision

Cortical structure and function have a high degree of uniformity

MEG is able to pick up different locations in Wernicke’s area for different vowels• MEG can only detect activity of at least 10,000

contiguous apical dendrites (Papanicolaou) Requires perhaps at least 250 adjacent minicolumns The size of a maxicolumn or hypercolumn

Remaining question: The process of learning distinctions

When a hypercolumn is first recruited, no lateral inhibition among its internal subdivisions

Later, when finer distinctions are learned, they get reinforced by lateral inhibition

Question: How does this work?

Inhibitory connections Based on Mountcastle (1998)

Columnar specificity is maintained by pericolumnar inhibition (190)

• Activity in one column can suppress that in its immediate neighbors (191)

Inhibitory cells can also inhibit other inhibitory cells (193)

Inhibitory cells can connect to axons of other cells (“axoaxonal connections”)

Large basket cells send myelinated projections as far as 1-2 mm horizontally (193)

Neural processes for learning

Basic principle: when a connection is successfully used, it becomes stronger• Successfully used if another connection to same

node is simultaneously active Mechanisms of strengthening

• Biochemical changes at synapses• Growth of dendritic spines• Formation of new synapses

Weakening: when neurons fire independently of each other their mutual connections (if any) weaken

Neural processes for learning

A

B

C

If connections AC and BC are active at the same time, and if their joint activation is strong enough to activate C, they both get strengthened

(adapted from Hebb)

Synapses here get strengthened

end