Deictic Centers& the Cognitive Structure

of Narrative Comprehension

Gail A. Bruder, Judith F. Duchan, David M. Mark, William J. Rapaport, Erwin M. Segal,

Stuart C. Shapiro, Leonard Talmy, David A. Zubin, et al.

Center for Cognitive Sciencerapaport@buffalo.edu

http://www.cse.buffalo.edu/~rapaport/dc.html

Interdisciplinary Cognitive-ScienceResearch Project

(~1982 - ~1995)

• AI• cognitive psychology• education• geography• communicative disorders• linguistics• literary theory & practice• philosophy

Deictic Center

• “mental model” – of spatial, temporal, & character info

– contributed by reader

– in order to understand narrative• by tracking:

– WHERE events occur

– WHEN events occur

– to WHOM events occur

– from WHOSE perspective

• DC ::= <WHERE, WHEN, WHO>

Understanding a Narrative

1. Bottom-up approach: Local Textual Cohesion– “John was ill, but he sang anyway.”– …can be understood in 2 ways:

a) Syntactic (closed-text, interpretive) view: :-(– links between text elements:

‘he’ linked to ‘John’ 2 clauses linked by ‘but’

Understanding a Narrative (cont’d)

b) Semantic (open-text, constructive) view: :-)

• Reader constructsmental-model meaning (= theory)from narrative text (= data)• cf. Kamp’s Discourse Representation Theory

– ‘John’ picks out item in (R’s MM of) story-world

– ‘he’ picks out same one

– ‘he’/‘John’ link inferred

– ‘but’ links 2 conceptual representations

• Problem: Can’t see the forest for the trees.

Understanding a Narrative (cont’d)

• Top-down approach: Global Textual Coherence

– Text-structure theories:• Rumelhart’s “story grammars”

• Schank’s “scripts”

– Logical coherence of story world

– Problem: Looks only at the forest, not at the

trees

Role of the Deictic Center

• story world: coherence theories– articulated by text-structure theories (too general)

• text: cohesion theories (too specific)

(gap)

filled by DC (inter alia)• conceptual domain where coherence & cohesion can meet

Role of the Deictic Center (cont’d)

• Goals

– To determine how DC can be computed by reader from:• cohesive devices

• world knowledge

• inference

– & how reader can use DC to construct coherence

Linguistic Clues for Tracking DC(David Zubin)

• Deictic operations(mental operations of the reader on the DC):– introducing– shifting– maintaining– voiding (= shift to null)

• actors (WHO)• places (WHERE)• times (WHEN)

Linguistic Clues for Tracking DC (cont’d)

• Sample deictic operations on WHERE:

– Hemingway:

• “The door to the café opened, and two men…

– came in.” WHERE = inside– went in.” WHERE = outside– entered.” WHERE = indeterminate

Linguistic Clues for Tracking DC (cont’d)

• Sample deictic operations on WHERE– Introducing/shifting (preposed adverbials):

• “Kino awakened in the near dark. The stars still shone and the day had drawn only a pale wash of light….The roosters had been crowing…, and the… pigs were…turning…twigs and bits of wood to see whether anything to eat had been overlooked. Outside the brush house in the tuna clump,…birds chittered and flurried with their wings.”

– Steinbeck, The Pearl

– Maintaining WHERE (& shifting WHO)(deictic verbs come, go, bring, take):

• “Kino squatted by the fire…and ate. When Kino had finished, Juana came back to the fire and ate.”

• WHO = Kino, WHERE = location of WHOJuana enters the WHERE(come maintains the WHERE)

Psychological Support(Gail Bruder, Erwin M. Segal)

• ‘come’ maintains DC’s WHERE;‘go’ shifts it

• takes more time to respond to question about previous DC

Psychological Support (cont’d)

1. John and Mary were eating dinner when there was a knock at the door.

2. (g) John got up and went to answer the door.(c) John looked up to see his partner come in.

3. Kevin greeted John with a bottle of champagne and a big hug.

4. They had just won a large advertising account.

• Is Mary in the dining room?

Psychological Support (cont’d)

• “Juana was preparing a fire. She broke little pieces of brush. Kino got up and wrapped his blanket around him. He came / went outside to watch the dawn. Kino squatted down.”

• Is Juana inside the brush house?

• Initial WHERE = location of WHO; indeterminateKino’s destination = outside

• came:– if female reader

then WHO = Juana so WHERE = outside so Kino moved to WHERE

• went:– if female reader

then WHO = Juana so WHERE = inside so Kino moved from WHERE

Communicative Disorders(Judith Felson Duchan)

• Goal:– To facilitate comprehension & production of fiction in

language- & learning-disabled children

• Unique property of fiction:– How characters think & feel is transparent to reader

(D. Cohn)

• Children can manipulate mental states in fiction– 4 yrs: can understand subjective fictional experience– 6 yrs: refer to internal response of characters to events

in story– Can children produce narratives using subjectivity markers?

Communicative Disorders (cont’d)

• Results:– Subject: 5-yr-old, pre-literate girl– 44 stories generated by subject from picture books– psycho-narration: 48% of stories

• descriptions of thoughts of character• “she felt terrible”

– internal monologues: 16% of stories• exact language of character’s thoughts• “ ‘I feel terrible’, she thought”

– represented thought: 11% of stories• representation of character’s subjective experience

in deictically shifted language• “She winced as she heard them crash to the platform.

The lovely little mirror that she had brought for Ellen, and the gifts for the baby!”

Communicative Disorders (cont’d)

• Individual differences:– Comprehenders understand narrative from different

perspectives– DC of some narratives is not obvious

• child’s narrative; adult comprehender:

1. “Once there was a little boy who lived in the forest with his mother and father and his pet water beetle.”

2. * “He waited in the rain for his water beetle to come inside.”3. “He said, ‘Come inside, water beetle!’ ”4. “ ‘No, I’m supposed to like the rain.’ ”5. “ ‘That’s why they call me the water beetle.’ ”

• Where are the boy, mother, father, beetle in (1)?• Where are the boy, beetle in (2)? In (3)?

Computational Implementation

• SNePS– fully intensional,

propositional,semantic-networkknowledge-representation & reasoning system

• rule-based inference• generalized inheritance• belief revision

– can “understand” and “generate” natural language

Computational Implementation (cont’d)

• WHEN (Stuart C. Shapiro, Michael J. Almeida):– Temporal structure of narratives

• WHERE (Shapiro, Albert H. Yuhan):– Reference-frame problem in narrative understanding

• WHO– Belief representation (William J. Rapaport)– Recognizing subjective sentences (Janyce Wiebe)

if pressed for time, can omit next 4 slides

Computational Implementation (cont’d)

• “Cassie” = name of our computational model of a reader– Cognitive Agent of the SNePS System—an Intelligent Entity :-)

• SNePS nodes represent objects of Cassie’s thoughts:– “intentional” objects– individuals, properties, relations, etc.– propositions

• Cassie’s “mind” grows (changes)

• Cassie believes what we tell her,as if it were fictional narrative

Computational Implementation (cont’d)

• IntenSional knowledge representation:– To model a mind,

a KRR system must model only intensional entities– I.e., entities that can be:

• distinct, even if logically or numerically equivalent• non-existent

– Argument from fine-grained representation:• intenTional entities (i.e., objects of thought) are intenSional• can have 2 objects of thought, but only 1 extensional object

– morning star & evening star– President of the US, Commander-in-Chief of the US armed forces– 2+3 & 5

– Argument from displacement:• can think and talk about non-existent objects

– fictional (Sherlock Holmes, Santa Claus)– impossible (mermaids, round squares)

Computational Implementation (cont’d)

• WHEN: Temporal Structure of Narratives (Almeida) (Are we there now?)

Linear narrative:“John arrived at the house. The sun was setting. He rang the bell; a minute later, Mary opened the door.”

– Initially, NOW is some time before John arrives at the house.– An instant later, NOW has moved to during the time that the

sun is setting.– A short time later, John rings the bell.– An instant later, NOW has moved to a minute before Mary

opens the door.– An instant later, at the end of the narrative, NOW has moved

to after Mary opens the door.

Computational Implementation (cont’d)

• WHEN: Non-linear narrative:

“John was walking to the office. He entered the office at 3:00 in the afternoon. The secretary was busy. She was typing a letter. John waited for ten minutes. He left the office. On Thursday, he returned in the morning. The secretary gave him a check. On the following Tuesday, John returned to the office. He had lost the check on the previous afternoon.”

Computational Implementation (cont’d)

• WHERE: The Reference-Frame Problem (Yuhan)

“Mary, tom, and Bob went to a theater together in order to see Bob’s uncle’s show. They walked to the front of the hall. Bob sat two rows in front of Mary. Tom sat just behind her. They had a few minutes before the show would start. Mary was turned around in her seat talking with Tom. Then she saw a person who looked like Bob walking down the aisle toward her with a tall girl on his left. Recognizing Mary, he stopped in front of her to say hello. Mary glanced back and saw that Bob was still there in his seat. The person standing in front of Mary was Jim, who was Bob’s twin brother. She had met him once before. Jim and the tall girl found seats a little distance away to Mary's left. Then the lights in the hall dimmed. They saw Bob’s uncle standing behind a lectern to the left of a microphone.”

Computational Implementation (cont’d)

• WHERE: The Reference-Frame Problem (cont’d)

– Input:• Mary, Tom and Bob went to a theater together in order to see Bob’s uncle’s

show.

– Original narrative has no explicit statement that they went in to the theater• must be inferred in order to interpret ‘front’ correctly

– Output (after constructing SNePS “mental model”):• I understand that a group of individuals namely, Bob, Tom and Mary went to

a theater and that they moved to a place which has a spatial relation of ideal-point to a member of the class theater. Furthermore, I infer that, presumably, the group were located at a place in a theater at a time after the time of the going and before the time of the seeing. WHEN is the time of being in the theater. WHERE is the place in the theater. WHO is the group.

if pressed for time, can omit next 4 slides

Computational Implementation (cont’d)

• WHO: Belief Reports

• “Columbus believed that Castro’s island was India”– True? De re report.– False? De dicto report.

• “Mary believes that she* (herself) is rich”– De dicto, de se report– ‘she*’ = quasi-indicator

• “logophoric” pronoun• a 3rd-person indexical that refers to the believer in a 1st-person way

Computational Implementation (cont’d): AI + Literary Theory (Mary Galbraith, SDSU)

• Subjectivity in narrative (Wiebe, U/Pitt)– References in narrative must often be understood

w.r.t. character’s beliefs:• in subjective sentences

– portraying character’s thoughts, perceptions

• not in objective sentences– presenting story directly

– Subjective context =def

• sequence of subjective sentencesportraying thoughts or perceptions of 1 character

– Problem:• how to recognize subjective sentences & their character

Computational Implementation (cont’d): AI + Literary Theory

1. (a) “She winced as she heard them crash to the platform. (b) The lovely little mirror that she had brought for Ellen, and the gifts for the baby!”

a) describes emotional reactionb) 1st mention of mirror, Ellen, gifts, baby

2. (a) “Suddenly Zoe gasped. (b) She had touched somebody!”b) exclamation / represented thought

‘somebody’: Zoe’s reference

3. (a) “But what Muhammed had seen in those few moments made him catch his breath in amazement. (b) On the floor of the cave,…there were several large cylindrical objects standing in a row.”

a) push to M’s belief space at ‘seen’ or ‘amazement’b) superordinate description: shows what M believes

(that he doesn’t know what the objects are,which isn’t stated elsewhere)

DC Projectas Interdisciplinary Cognitive Science

UnderstandingNarrative

Understanding

AI & PHI

LIN

CDS

GEO

ENG

PSY

DC Projectas Interdisciplinary Cognitive Science

UnderstandingNarrative

Understanding

AI & PHI

LIN

CDS

GEO

ENG

PSY

References

• Duchan, Bruder, & Hewitt (eds.) (1995), Deixis in Narrative: A Cognitive Science Perspective (Hillsdale, NJ: Erlbaum).

• http://www.cse.buffalo.edu/~rapaport/dc.html