Intro to CogSci: Embodiment 2

Amodal versus perceptual symbolsCategorization

Cognitive Semantics(some) Empirical Evidence

Cognitive science paradigms: Embodiment 2grounding language in embodiment

Kristína Rebrová

Kristína Rebrová Cognitive science paradigms: Embodiment 2



Contents

1 Amodal versus perceptual symbols

2 Categorization

3 Cognitive Semantics

4 (some) Empirical Evidence




Amodal Symbols ... again

internal (cognitive) structure does not resemble the perceptualstates from which it originates




The mind IS embodied

nature of the (human) mind is largely determined by the formof the (human) bodycognition arises from bodily interactions with the worldcognition shares (neural) mechanisms with perception, actionand introspectioncognition is embedded in its environmentamodal symbols represent knowledge (descriptive knowledge)




Perceptual Symbol Systems (Barsalou,1999)

Perceptual symbols resemble (are a subset of) perceptualstates from which they originateSimulation

a core form of computation in the brainreenactment of perceptual, motor and introspective statesacquired during experienceas experience occurs, the brain captures the states acrossmodalities and integrates them with a multimodalrepresentation stored in memory




Effect of changing the reference

Transforming a word or an amodal symbol fails to produce ananalogous transformation in reference, whereas transforming aperceptual simulation does.




Categorization

Lakoff’s “Woman, Fire and Dangerous Things: Whatcategories reveal about the mind.”Categorization is one of the most basic ability of living beings.Even amoeba categorizes the things into food and nonfood.Animals categorize food predators, possible mates, members oftheir own species, etc.Why do we need categorization?

reducting complex rich sensory inputgeneralization, prediction

How do we form concepts and categories?




Categories are no closed containers

classic story:defining features (individually necessary and jointly sufficientfeatures)Boolean membership - clear-cut boundaries

prototype theory:family resemblances (Wittgenstein, 1953)Fuzzy sets (Zadeh,1965): a degree of membershipprototypicality: some members are more typical of a categorythan othersRosh(1973): Prototypes of a category are the clearest cases of categorymembership defined operationally by people’s judgment of goodness ofmembership in the category




Color categorization

Eleanor Rosch: research in New GuineaDani language, only 2 color terms

Mili = dark& cool (black, green, blue)Mola = light& warm (white, red, yellow)

primary colors (basic color terms of English) arepsychologically real even if they can’t name themfocal colors = prototypes, i.e. best examples

are learned more readilymutual agreement among speakers

basic level categories




Color categorization

Berlin and Kay (1969): basic color terms in a language namethe universal set of basic color categoriesthere exists a smallest final set of simple words with which thespeaker can name any colorproperties: monolexemic, general, psychologically salient,common and generally knowntheory supported appr. 20 years later by the results from theWorld Color Surveywhy are color categories universal?

fyziology of the eye - the shape of perceptual color space(s)(Jameson a D’Andrade, 1997)

properties of daylight (Yendrikhovskij, 2001; Shepard, 1992)

social negotiation (Steels & Belpaeme, 2005)




Basic level categories




Basic level objects (Rosch, 1976a)

categories within taxonomies of concrete objects are structured suchthat there is generally one level of abstraction at which the mostbasic category cuts can be madeI saw a [mammal | dog | greyhound] sitting on the porchshort words, most commonly used labelsthe first terms to be learned by children and first to appear in alanguage (sooner than sub- and superordinates)gestalt perception: the highest level at which category membershave similarly perceived overall shapesmotor programs: the highest level at which a person uses similarmotor actions for interacting with category members.

knowledge structure: the level at which most of our knowledge isorganized




Cognitive Semantics

e.g. Lakoff & Johnson(1980), Lakoff (1987), Langacker(1987), Lakoff and Johnson (1999), Talmy (2000), ...semantic structure is conceptual structureconceptual structure is embodiedmeaning representation is encyclopaedicmeaning-construction is conceptualisation




Language comprehension is embodied

a sentence can evoke an imagined scene and resulting inferences




Embodied inferences

syntax is not independent of semantics

The scientist walked into the wall.

The hobo drifted into the house.

The smoke drifted into the house.




Image schemas

Johnson (1987), Lakoff (1987)recurring structures within our cognitive processes whichestablish patterns of understanding and reasoningformed from our bodily interactions, from linguistic experience,and from historical context

Boundary

Contact

Container




Conceptual metaphor theory (Lakoff and Johnson, 1980, 1999)

Classical theories viewed metaphors as novel or poetic linguisticexpressions outside the realm of ordinary everyday language.Metaphor has is in many cases central to understanding themeaning of many abstract concepts.Many concepts that are important to us are either abstract ornot well-defined in our experience (emotions, thoughts, time,etc.)We need to mediate access to them through the concepts thatwe understand more clearly (spatial orientation, objects, etc.)




Affection is Warmth

Subjective Judgment: AffectionSensory-Motor Domain: TemperatureExample: They greeted me warmly.Primary Experience: Feeling warm while being heldaffectionately.




Important is Big

Subjective Judgment: ImportanceSensory-Motor Domain: SizeExample: Tomorrow is a big day.Primary experience: As a child, important things in yourenvironment are often big, e.g., parents, but also large thingsthat exert a force on you




Knowing is Seeing

Subjective Judgment: KnowledgeSensory-Motor Domain: VisionExample: I see what you mean.Primary Experience: Getting information through vision




Time is Motion

Subjective Judgment: The passage of timeSensory-Motor Domain: MotionExample: Time flies.Primary Experience: Experiencing the passage of time as onemoves or observes motion




Embodiment and Cultural variance

Núñez & Sweetser (2006)speakers of Aymara face the past andhave their backs to the futureNayra = past (eye, sight, or front)

Q”ipa = future (behind, back)Q”ipüru = tomorrow = q”ipa + uru(some day behind one’s back)

every language has a system of markerswhich forces the speaker to pay attentionto some aspects of the information beingconveyed and not others




Grounded cognition

grounded = anchored in the physical world (embodied +embedded)

embodied: agent has a body that provides direct sensationsand allows actionsembedded: situated in an environment that provides stimuli

language is grounded in perception and action as well !




Mental imagery

functional-equivalence hypothesis, supported bymany cognitive psychologists (Farah, Finke,Kosslyn, Shepard, Rumelhart,...)

visual imagery as functionally equivalent tovisual perceptionshared neural substrates

mental rotations (e.g. Shepard, Metzler, 1971)image scaling (Kosslyn, 1975, 1976)image scanning (Kosslyn, Ball, Reiser, 1978;Pinker, 1980)




Mirror Neurons

motor neurons with perceptual properties (visual, auditory)facilitate (mediate) understanding

understanding of the actions “from the inside”(e.g. Rizzolatti and Sinigaglia, 2010)empathy, mind-reading (Gallese et al., 2004)

originally discovered in monkeys, recently confirmed in humans




Motor Resonance

partial activation of motor circuits without producing motoractivitytriggered by various modalities: visual, auditory, linguisticmight provide us with a simulation mechanism -understanding, prediction, empathyEEG mu-rhytm

dominant frequencies in the 8–13 and 15–25 Hz bands (alphalike)typical for motor restdesynchronizes/supresses not only when subject produces, butalso observes action

first indirect evidence of mirror neurons in humans(Cohen-Seat et al.,1954; Gastaut and Bert,1954)




Motor Resonance

the more closely the observed action maps onto the observer’sown motor repertoire, the more accurate will be the observer’sprediction of the course and the result of the actionconclusions (see van der Wel et al., 2011)

motor preparation enhances the performance in perceptualtasksstimulus-response compatibility (facilitation of reaction on thebasis of congruence with the stimulus)ideomotor action: involuntary movement that tends to arisewhen observing another’s performanceinfluence of familiarity: self-actions vs. actios of othersinfluence of proficiency: more skilled individuals - betterjudgement (but only for moving percepts)influence of praxis: Triton effect example




Grounding language in action: neural evidence

Pulvermueler et al. (2001): hearing/reading action verbsproduces somatotopic activation in the primary motor cortexEEG study, movement vs. lexical decision taskkick (leg), pick (arm), lick (face)




Grounding language in action: behavioral evidence

Glenberg and Kashak (2002): comprehending a sentence thatimplies action in one direction interferes with real action in theopposite directionJohn gave you a pizza. You gave a pizza to John.Also with abstract transfer sentences (Glenberg et al., 2008)Mary told you a story. You told a story to Mary.Works also with rotation movement: Zwaan and Taylor (2006)John increased the speakers volume. Mary opened a jar ofpickles.




Embodiment effects

Activating elderly stereotype causes people to walk slowly andto perform lexical decision slowly (Dijksterhuis and Bargh2001)Engaging the smiling musculature produces positive affect(Strack et al. 1988)Standing upward and stretching arms helps to gainself-confidence, watch Amy Cuddy’s Ted talk:http://www.youtube.com/watch?v=Y4386jSnFEU


http://www.youtube.com/watch?v=Y4386jSnFEU



The end

Thank you for your attention

[email protected]


Education

Intro to CogSci: Embodiment 2