Unified Cognitive Science

•Neurobiology•Psychology•Computer Science•Linguistics•Philosophy•Social Sciences•Experience

Take all the Findings and Constraints Seriously

What are schemas?

• Regularities in our perceptual, motor and cognitive systems

• Structure our experiences and interactions with the world.

• May be grounded in a specific cognitive system, but are not situation-specific in their application (can apply to many domains of experience)

Basis of Image schemas

•Perceptual systems

•Motor routines

•Social Cognition

•Image Schema properties depend on

•Neural circuits

•Interactions with the world

Image schemas

•Trajector / Landmark (asymmetric)

•The bike is near the house •? The house is near the bike

•Boundary / Bounded Region

•a bounded region has a closed boundary•Topological Relations

•Separation, Contact, Overlap, Inclusion, Surround•Orientation

•Vertical (up/down), Horizontal (left/right, front/back)

•Absolute (E, S, W, N)

LMTR

bounded region

boundary

Similarity:

•Perceptual and motor systems

•Basic functional interactions with the world

•Environment

Variation:

Cross-linguistic variation in how schemas are used.

Cross-linguistic Variations

English

Japanese

English

ON

AROUND

OVER IN

Bowerman & Pederson

Dutch

Bowerman & Pederson

AAN

OM

BOVEN IN

OP

Chinese

Bowerman & Pederson

SHANG

ZHOU

LI

Spatial schemas•TR/LM relation

•Boundaries, bounded region

•Topological relations

•Orientational Axes

•Proximal/Distal

Trajector/Landmark Schema•Roles:

Trajector (TR) – object being located

Landmark (LM) – reference object

TR and LM may share a location (at)

TR/LM -- asymmetry

•The cup is on the table

•?The table is under the cup.

•The skateboard is next to the post.

•?The post is next to the skateboard.

Boundary Schema

Region ARegion B

Boundary

Roles:BoundaryRegion ARegion B

Topological Relations

•Separation

Topological Relations

•Separation

•Contact

Topological Relations

•Separation

•Contact

•Coincidence:

Topological Relations

•Separation

•Contact

•Coincidence:

- Overlap

Topological Relations

•Separation

•Contact

•Coincidence:

-Overlap

-Inclusion

Topological Relations

•Separation

•Contact

•Coincidence:

-Overlap

-Inclusion

-Encircle/surround

Orientation•Vertical axis -- up/down

up

down

above

belowupright

OrientationHorizontal plane – Two axes:

Language and Frames of Reference

•There seem to be three prototypical frames of reference in language (Levinson)

•Intrinsic

•Relative

•Absolute

Intrinsic frame of reference

frontback

right

left

Relative frame of reference

frontback

left??

right??

Absolute frame of reference

north

west

south

east

TR/LM and Verticality Schemas

•The book is under the table.

up

down

under

Proximal/Distal Schema

.

Simple vs. Complex Schemas

Container Schema•Roles:

•Interior: bounded region

•Exterior

•Boundary

C

C C

TR

TR

out in

TR/LM + Container

Container Schema Elaborated

•Complexities –more roles/specifications:

•Boundary properties•Strength•Porosity•Portals

Source-Path-Goal

Constraints:

initial = TR at Source

central = TR on Path

final = TR at Goal

Source Path Goal

SPG -- simple example

She drove from the store to the gas station.

TR = she

Source = the store

Goal = the gas station

Source Path Goal

SPG and ContainerShe ran into the room.

SPG. Source ↔ Container.Exterior

SPG.Path ↔ Container.Portal

SPG. Goal ↔ Container.Interior

PATH landmarks

past across along

LM

LM

LM

Part-Whole Schema

Part

Whole

semantic schema Containerroles:

interiorexteriorportalboundary

Representing image schemas

Interior

Exterior

Boundary

PortalSource

Path

GoalTrajector

These are abstractions over sensorimotor experiences.

semantic schema Source-Path-Goalroles:

sourcepathgoaltrajector

Language and Spatial Schemas• People say that they look up to some people, but

look down on others because those we deem worthy of respect are somehow “above” us, and those we deem unworthy are somehow “beneath” us.

• But why does respect run along a vertical axis (or any spatial axis, for that matter)? Much of our language is rich with such spatial talk.

• Concrete actions such as a push or a lift clearly imply a vertical or horizontal motion, but so too can more abstract concepts.

• Metaphors: Arguments can go “back and forth,” and hopes can get “too high.”

Regier Model Lecture

Jerome A. FeldmanFebruary 27, 2007

With help from Matt Gedigian

Neural Theory of Language

Language Development in Children

•0-3 mo: prefers sounds in native language

•3-6 mo: imitation of vowel sounds only

•6-8 mo: babbling in consonant-vowel segments

•8-10 mo: word comprehension, starts to lose sensitivity to consonants outside native language

•12-13 mo: word production (naming)

•16-20 mo: word combinations, relational words (verbs, adj.)

•24-36 mo: grammaticization, inflectional morphology

•3 years – adulthood: vocab. growth, sentence-level grammar for discourse purposes

Trajector/Landmark Schema

•Roles:

Trajector (TR) – object being located

Landmark (LM) – reference object

TR and LM may share a location (at)

TR/LM -- asymmetry

•The cup is on the table

•?The table is under the cup.

•The skateboard is next to the post.

•?The post is next to the skateboard.

Language and Frames of Reference

•There seem to be three prototypical frames of reference in language (Levinson)

•Intrinsic

•Relative

•Absolute

Basis of Image Schemas

•Perceptual systems

•Motor routines

•Social Cognition

•Image Schema properties depend on

•Neural circuits

•Interactions with the world

Image schemas

•Trajector / Landmark (asymmetric)

•The bike is near the house •? The house is near the bike

•Boundary / Bounded Region

• bounded region has a closed boundary•Topological Relations

•Separation, Contact, Overlap, Inclusion, Surround•Orientation

•Vertical (up/down), Horizontal •Absolute (E, S, W, N)

LMTR

bounded region

boundary

Spatial schemas•TR/LM relation

•Boundaries, bounded region

•Topological relations

•Orientational Axes

•Proximal/Distal

Regier’s Model

•Training input: configuration of TR/LM and the correct spatial relation term

•Learned behavior: input TR/LM, output spatial relation

Learning System

above below left right in out on off

Input:TR

LMabove

Issue #1: Implicit Negatives

• Children usually do not get explicit negatives

• But we won’t know when to stop generalizing if we don’t have negative evidence

• Yet spatial relation terms aren’t entirely mutually exclusive

• The same scene can often be described with two or more spatial relation terms (e.g. above and outside)

• How can we make the learning problem realistic yet learnable?

Dealing with Implicit Negatives

• Explicit positive for above

• Implicit negatives for below, left, right, etc

• in Regier:

E = ½ ∑i,p (( ti,p – oi,p) * βi,p )2,

where i is the node, p is the pattern,

βi,p = 1 if explicit positive,

βi,p < 1 if implicit negative

Learning

Systemdynamic relations(e.g. into)

structured connectionistnetwork (based on visual system)

Topological Relations

•Separation

•Contact

•Coincidence:

-Overlap

-Inclusion

-Encircle/surround

Issue #2: Shift Invariance

• Backprop cannot handle shift invariance (it cannot generalize from 0011, 0110 to 1100)

• But the cup is on the table whether you see it right in the center or from the corner of your eyes (i.e. in different areas of the retina map)

• What structure can we utilize to make the input shift-invariant?

Limitations

•Scale

•Uniqueness/Plausibility

•Grammar

•Abstract Concepts

•Inference

•Representation

Language and Thought

• We know thought (our cognitive processes) constrains the way we learn and use language

• Does language also influence thought?

• Benjamin Whorf argues yes

• Psycholinguistics experiments have shown that linguistics categories influence thinking even in non-linguistics task

Language

Thought

cognitive processes