Discourse theories and technologies

Borovets, sept. 2003

Discourse theories and technologies

Dan Cristea

“Al. I. Cuza” University of Iasi,

Faculty of Computer Science

and

Romanian Academy,

Institute of Theoretical Computer Science

[email protected]


ContentIntroduction

– What is discourse? Text and discourse. Coherence and cohesion.

Theories– attentional state theory– rhetorical structure theory– centering theory– veins theory

Technologies– segmentation of discourse– Marcu’s parser – VT parser

Related issues on discourse – anaphora resolution, summarisation, information extraction


What is discourse?

Longman: 1. a serious speech or piece or writing on a particular subject: Professor Grant delivered a long discourse on aspects of moral theology. 2. serious conversation between people: You can’t expect meaningful discourse when you two disagree so violently. 3. the language used in particular kinds of speech or writing: scientific discourse.


Text and discourse

Syntactically – a discourse is more than a single sentence.

A text is not a discourse!

But it becomes a discourse the very moment it is read or listen by a human... or a machine.


Time and discourse

Discourse has a dynamic nature

Time axesreal time

discourse time

story time

1 2

2 11000 1030800 920

1 2


Cohesion and coherence

A text has cohesion when its parts closely correlate.

A text is coherent when it makes sense, with respect to an accepted setting, real or virtual.


Interpretation of discourse

discourse interpretation

text knowledge base

knowledge about the language

knowledge about the world

knowledge about the situation

knowledge about the author


Discourse phenomena: interruptions and flash-backs

E: Now attach the pull rope to the top of the engine.

By the way, did you buy gasoline today?

A: Yes. I got some when I bought the new lawnmower wheel.

I forgot to take the gas with me, so I bought a new one.

E: Did it cost much?

A: No, and we could use another anyway to keep with the tractor.

E: OK, how far have you got?

Did you get it attached?

from [Allen, 1987]


Discourse phenomena: pop-overs

E: Now attach the pull rope to the top of the engine.








from [Allen, 1987]


Discourse phenomena: inference load and pronoun use

Why is it that some discourses seem more difficult to understand than others?

Why do we use the pronouns as we do?


Discourse theories?

Sub-domain of Computational Linguistics: searching for the laws that govern the discourse and the models making possible an automated analysis, representation and generation of the discourse.


Discourse theories

• atentional state theory• rhetorical state theory• centering theory• veins theory


Attentional state theory (AST)Barbara Grosz & Candence Sidner, 1987

Models the linguistic structure of discourse

Gives an account on intentions and how are they combined

Explains the shift of attention during discourse interpretation and referentiality in terms of discourse structure

3 components


AST: 1st component

• a linguistic structure: – more sentences are aggregated in the same

segment– segments display a recursive structure


AST: 2nd component• an intentional structure:

– a segment communicates an intention, it has a goal to accomplish in the reader;

– the goals of the component segments contribute to the realisation of the goal of the overall segment;

– two type of relations between segment goals: dominance and satisfaction-precedence


AST: 2nd componentRelations: dominance

DSP A dominates DSP AA: the intention associated with DSP AA contributes to the satisfaction of the intention associated with DSP A

A

AA AB AC

AAA AAB ABA ABB


AST: 2nd componentRelations: satisfaction-precedenceDSP AA satisfaction-precedes DSP AB: DSP AA must be satisfied before

DSP AB

A

AA AB AC

AAA AAB ABA ABB


AST: 3rd component• an attentional state

– to each segment corresponds a space of entities under focus

– these spaces have the dynamics of a stack

A

AA AB AC

AAA AAB ABA ABB

A

SA





A

AB AC

AAA AAB ABA ABB

A

SA

AASAA





A

AB AC

AAB ABA ABB

A

SA

AASAA

AAA

SAAA





A

AB AC

ABA ABB

A

SA

AASAA

AAA AAB

SAAB





A

AB AC

ABA ABB

A

SA

AASAB

AAA AAB





A

AB AC

ABA ABB

A

SA

AASAB

AAA AAB

SABA





A

AB AC

ABA ABB

A

SA

AASAB

AAA AAB

SABB





A

AB AC

ABA ABB

A

SA

AASAC

AAA AAB



– accessibility modeled by the top-down access in the stack

A

AA AB AC

AAA AAB ABA ABB SA

SAB

SABB


AST: pluses

• Discourse structure: – a proposal for discourse structure (an example)– stack behavior models hierarchical relationships

among text segments

• Reference: accounted for by accessibility in stack

• Interruptions

• Flash-backs


AST: interruptionsE: Now attach the pull rope to the top of the engine.








from [Allen, 1987]

An interruption is a discourse segment whose DSP is not dominated nor satisfaction-preceded by the DSP of the immediately proceeding segment.


AST: interruptionsE: Now attach the pull rope to the top of the engine.

…

…

…

…


A: Yes. I got some when I bought the new lawnmower wheel.I forgot to take the gas with me, so I bought a new one.






AST: flashbacksSinit …

SABC …

SBill OK. Now how do I say that Bill is...Whoops I forgot about ABC.I need an individual concept for the company ABC. …

SBill Now back to Bill.How do I say that Bill is an employee of ABC?

From [Grosz & Sidner, 1987]

A flashback is a particular kind of interruption whose DSP satisfaction-precedes the interrupted segment or a segment that dominates the interrupted segment.

SBill

Sinit

SFB

Sinit

SABC

SBill

SFB

SBill

SFB

Sinit

SABC SBill

Sinit

SFB


AST: minuses

Stack mechanism fails for certain dominant/dominated segment configurations when granularity is sufficiently fine

Does not accommodate left satellites


AST doesn‘t accommodate left satellites

a. Jack and Sue went to buy a new lawn mowerb. since their old one was stolen.c. Sue had seen the men who took it andd. had chased them down the street,e. but they'd driven away in a truck. f. After looking in the store g. they realized they couldn't afford a new one.h. By the way, Jack lost his job last monthi. so he's been short of cash recently.j. He has been looking for a new one,k. but so far hasn't had any luck. l. Anyway, they finally found a used one at a garage sale.

Allen, 1993


AST doesn‘t accommodate left satellites

a. Jack and Sue went to buy a new lawn mower b. since their old one was stolen.

f. After looking in the store

g. they realized they couldn't afford a new one.

l. Anyway, they finally found a used one at a garage sale.

c. Sue had seen the men who took it and

d. had chased them down the street,

e. but they'd driven away in a truck.

h. By the way, Jack lost his job last month

i. so he's been short of cash recently. j. He has been looking for a new one, k. but so far hasn't had any luck.


Attentional state stack

a

a. Jack and Sue went to buy a new lawn mower

a,b


c,d,e






f. After looking in the storeg. they realized they couldn't afford a new one.




a,b,f,g








h,i,j,k

a,b,f,g,l










Problem:a finer granularity

f. After looking in the store

a. Jack and Sue went to buy a new lawn mower

g. they realized they couldn't afford a new one.


b. since their old one was stolen.


Problem

a a a

b

a

b

c,d,e

a, g

f


Rhetorical structure theory

Basics• text span: un uninterrupted linear interval of text• relation: holds between two non-overlapping spans, called

nucleus and satellite– a nucleus is more important than a satellite (deletion and

substitution tests)– relations: hypotactic (nucleus + satellite) and paratactic (2 nuclei)

• scheme: integrates by a relation two or more text spans (like grammar rules)

• RST analysis are trees• they reflect a judge interpretation (therefore could be

subjective)

William Mann & Sandra Thompson, 1987


RST schemes

relation

text span: nucleus

text span: satellite

relation

text span: nucleus

text span: nucleus


RST schemes: equivalences

relation1 relation2

relation1

relation2

relation1

relation2


RST schemes: equivalences

relation relation

relation

relation

relation

relation


EVIDENCE relation

1. The program as published for calendar year 1980 really works.

2. In only a few minutes, I entered all the figures from my 1980 tax return

3. and got a result which agreed with my hand calculations to the penny.

EVIDENCE

1-3

2-31

EVIDENCE

constraint on N: R might not believe N to a degree satisfactory to W

constraint on S: R believes S or finds it credible

effect: R’s belief of N is increased


CONCESSION relationCONCESSION

constraint on N: W has positive regard to the situation presented in N

constraint on S: W is not claiming that the situation presented in S doesn’t hold

constraint on the combination N+S: W acknowledges a potential incompatibility between the situations presented in N and S; W regards the situation presented in N and S as compatible

effect: R’s positive regard for the situation presented in N is increased

1. Although Dioxin is toxic to certain animals,2. evidence is lacking that it has any serious

long-term effects on human beings.

CONCESSION

1-2

21


CIRCUMSTANCE relationCIRCUMSTANCE

constraint on N: none

constraint on S: S presents a situation

constraint on the combination N+S: S sets a framework (spatial or temporal) within which R is intended to interpret the situation presented in N

effect: R recognizes that the situation presented in S provides the framework for interpreting N

1. Probably the most extreme case of Visitors Fever I ever witnessed was a few summers ago

2. when I visited relatives in Midwest.

CIRCUMSTANCE

1-2

21


A more complex example1. Farmington Police had to help control traffic recently

2. when hundreds of people lined up to be among the first applying for jobs at the yet-to-open Marriot Hotel.

3. The hotel’s help-wanted announcement – for 300 openings – was a rare opportunity for many unemployed.

4. The people waiting in line carried a message of claims that the jobless could be employed if only they showed enough moxie.

5. Every rule has exceptions,

6. but the tragic and too-common tableaux of hundreds of people snake-lining up for any task with a paycheck illustrates a lack of jobs,

7 not laziness.

circumstance

32

2-3

volitional result

1-3

4

evidence

5

6

antithesis

7

6-7

concession

5-7

4-7

background

1-7


RST relationsSubject matter

(informational)

ElaborationCircumstanceSolutionhoodVolitional CauseVolitional ResultNon-Volitional CauseNon-Volitional ResultPurposeConditionOtherwiseInterpretationEvaluationRestatementSummarySequenceContrast

Presentational (intentional)

Motivation

Antithesis

Background

Enablement

Evidence

Justify

Concession


Problem: multiple interpretations[Moore & Polack, 1992]

1

2

motivation

3

motivation

Intentional level

3

1

condition

2

condition

Motivational level

1. Come back at 5:00.2. Then we can go to the hardware store before it closes.3. This way we can finish the bookshelves tonight.


Any other complains?

• no indication on referentiality

• how many relations?

• how relations are discovered?

• ...


How distant are AST & RST?

• Mosser&Moore (1996) and Marcu (1997):– granularity: AST - undefined, RST - fine

(clause level)– structure: trees– internal nodes: relations (AST:2, RST: 28,

Hobbs, Knott: hierarchy of relations)


Centering - a theory of local discourse coherence• Joshi,A.K. and Weinstein,S., 1981: “Control of Inference: Role

of Some Aspects of Discourse-Structure Centering“• Grosz,B.; Joshi,A.K. and Weinstein,S., 1986: “Towards a

computational theory of discourse interpretation”• Brennan,S.E.; Friedman,M.W. and Pollard,C.J., 1987: “A

Centering approach to pronouns“ • Grosz,B.; Joshi,A.K. and Weinstein,S, 1995: “Centering: A

framework for modeling the local coherence of discourse”

• Strube,M. and Hahn,U., 1996: “Functional Centering“• Walker,M.A.; Joshi,A.K. and Prince,E.F. (eds.), 1997:

“Centering in Discourse“• Kameyama,M., 1997: “Intrasentential Centering: A Case

Study“


Goals of the theory

• explains why certain texts are more difficult to process than others

• explains why we use the pronouns as we use them

• anchors a practical approach for anaphora resolution


A smooth discourse

from [Walker, Joshi and Prince, 1997]

a. Jeff1 helped Dick2 wash the car.

b. He1 washed the windows as Dick2 waxed the car.

c. He1 soaped a pane.

He in c. is Jeff because soaping can only be related to

the washing event


A ... smooth discourse

a. Jeff1 helped Dick2 wash the car.

b. He1 washed the windows as Dick2 waxed the car.

c. He2 buffed the hood.

He in c. is Dick, because buffing can only be related to the waxing event.

less


Focus helps to disambiguate pronominal anaphora

1. Susan1 is a fine friend.2. She1 gives people the most wonderful

presents.3. She1 just gave Betsy2 a wonderful bottle of

wine.4. She1 told her2 it was quite rare.5. She1 knows a lot about wine.

from Grosz, Joshi&Weinstein, 1995


CT explains the normality or oddness of some utterances

a. Terry really goofs sometimes. b. Yesterday was a beautiful day and he was

excited about trying out his new sailboat. c. He wanted Tony to join on a sailing

expedition. d. He called him at 6 A.M.e. He was sick and furious at being woken up

so early. from Grosz, Joshi&Weinstein, 1995


This text repairs the proceeding, but...a. Terry really goofs sometimes. b. Yesterday was a beautiful day and he was excited

about trying out his new sailboat. c. He wanted Tony to join on a sailing expedition. d. He called him at 6 A.M.e. Tony was sick and furious at being woken up so

early. f. He told Terry to get lost and hung up.g. Of, course he hadn’t intended to upset Tony.


Finally, we are done!

a. Terry really goofs sometimes. b. Yesterday was a beautiful day and he was excited

about trying out his new sailboat. c. He wanted Tony to join on a sailing expedition. d. He called him at 6 A.M.e. Tony was sick and furious at being woken up so

early. f. He told Terry to get lost and hung up.g. Of, course Terry hadn’t intended to upset Tony.


The main lines

• Applies to one segment of discourse– refers to Grosz&Sidner´s Attentional State Theory– Hahn&Strube applied centering for finding

discourse segments

• Sees the segment made of adjacent utterances (sentences)

• Kameyama, proposes intrasentential centering (1997)


Centers

John gave Mary a flower.

person1type = personname = Johngender = masc

person2 type = personname = Marygender = fem

flower1type = flowernumber = sg

the realisation relation


What is a center?

John went to Mary‘s house

person1 type = personname = Marygender = fem

house1type = housenumber = sg

the realisation relation

He met her down the street.

He found it down the street.


For each utterance, compute:

- a list of forward-looking centers

Cf(u) = <e1, e2, ... ek>

- a backward-looking center

Cb(u) = ei

- a prefered center

Cp(u) = e1


Rule 1: pronoun realisation

• If some element of Cf(ui-1) is realised as a pronoun in ui, than so is Cb(ui)– it captures the intuition that pronominalisation is

one way to indicate discourse salience– if there are multiple pronouns in a sentence,

realising discourse entities from the previous utterance, than Cb must be one of them

– if there is just one pronoun, then the pronoun must be the Cb


Rule 1 obeyeda. Terry really goofs sometimes.

Cf = ([Terry])b. Yesterday was a beatiful day and he was excited

about trying out his new sailboat.

Cf = (he=his=[Terry], [the sailboat])

Cb = [Terry]c. He wanted Tony to join on a sailing expedition.

Cf = (he=[Terry], [Tony], [the expedition])

Cb = [Terry]d. He called him at 6 A.M.

Cf = (he=[Terry], him=[Tony])

Cb = [Terry]


Rule 1 still obeyeda. Terry really goofs sometimes.






Cb = [Terry]d. He called Tony at 6 A.M.

Cf = (he=[Terry], [Tony])

Cb = [Terry]


Rule 1 disobeyeda. Terry really goofs sometimes.






Cb = [Terry]d. Terry called him at 6 A.M.

Cf = ([Terry], him=[Tony])

Cb = [Terry]


Rule 2: transitions

Cb(u) = Cb(u-1) Cb(u) Cb(u-1)

Cb(u) = Cp(u)

Cb(u) Cp(u)

CONTINUING SMOOTH SHIFT

RETAINING ABRUPT SHIFT

CON > RET > SSH > ASH


Rule 2 disobeyeda. Terry really goofs sometimes.

Cf = ([Terry])b. Yesterday was a beatiful day and he was excited about trying

out his new sailboat.




Cb = [Terry]d. He called Tony at 6 A.M.


Cb = [Terry]e. He was sick and furious at being woken up so early.

Cf = (he=[Tony])

Cb = [Tony]

CONTINUING

CONTINUING

CONTINUING

SMOOTH SHIFT


Rule 1 disobeyed again...

d. He called Tony at 6 A.M.


Cb = [Terry]

e. Tony was sick and furious at being woken up so early.

Cf = ([Tony])

Cb = [Tony]

f. He told Terry to get lost and hung up.

Cf = (he=[Tony], [Terry])

Cb = [Tony]

g. Of, course he hadn’t intended to upset Tony.


Cb = [Tony]

CONTINUING

SMOOTH SHIFT

CONTINUING

RETAINING


The discourse repaired...d. He called Tony at 6 A.M.


Cb = [Terry]e. Tony was sick and furious at being woken up so early.

Cf = ([Tony])

Cb = [Tony]f. He told Terry to get lost and hung up.

Cf = (he=[Tony], [Terry])

Cb = [Tony]g. Of, course Terry hadn’t intended to upset Tony.

Cf = ([Terry], [Tony])

Cb = [Tony]

CONTINUING

SMOOTH SHIFT

CONTINUING

RETAINING


Centering hints on pronominal anaphoraa. I haven’t seen Jeff for several days.

b. Carl thinks he’s studying for his exams.

c. I think he? went to the Cape with Linda.

from [Grosz, Joshi & Weinstein, 1983]

Cf = (I=[I], [Jeff])

Cb = [I]

Cf = ([Carl], he=[Jeff], [Jeff´s exams])

Cb = [Jeff]


Centering explains why we understand he in unit c as Jeff

b. Carl thinks he’s studying for his exams.

c. I think he? went to the Cape with Linda.

Cf = ([Carl], he=[Jeff], [Jeff´s exams])

Cb = [Jeff]

RETAINING

ABRUPT SHIFT

Cf = (I=[I], he=[Jeff], [the Cape], [Linda])

Cb = [Jeff]Cf = (I=[I], he=[Carl], [the Cape], [Linda])

Cb = [Carl]

Jeff


Centering: any problems?

• applies inside a segment– BUT what a segment is?

• allignment of Cf ellements– on what criteria (surface-order, syntactic role,

functional [Strube&Hahn, 1996]) language dependant

• null pronouns – Italian, Japanese, Romanian

parameterised centering [Poesio et al., 2000]


Veins theory (VT)

• to offer a better understanding of the relationship between discourse structure and referentiality

• to accommodate the left satellite problem• to be independent of granularity• to generalise Centering from local to global• to allow summarization and, especially,

focused summaries• to assist anaphora resolution processes• to guide discourse parsing

Cristea, Ide & Romary, 1998


VT intuitions

A satellite or a nucleus can refer a nuclear sibling on the left: in combinations u1

n R u2s, or u1

n R u2n, u2

can refer u1.

Ex. 11. John left without an umbrella 2. although he watched the TV meteo anouncing raining.


VT intuitions

A nucleus can refer a left satellite: in combinations u1

s R u2n, u2 can refer u1.

Ex. 2

1. Although John saw the TV whether forecast announcing raining

2. he left home without an umbrella.


VT intuitionsA right satellite of a nucleus u is not accessible

from another right sibling of u, nuclear or satellite: in combinations (u1

n R1 u2s)n R2 u3

n or (u1n R1 u2

s)n R2 u3s, u3

can refer u1 but not u2.

Ex. 31. John told Mary that he loves her.2. He was never married 3. and lived until 40 with his mother. 4. She, on the contrary, was married twice.


VT intuitionsA nucleus blocks the accessibility from a right

satellite towards a left satellite of it: in combinations (u1

s R1 u2n)n R2 u3

s, u3 can refer u2

but not u1.

Ex. 41. With one year before finishing his mandate of president of the company2. Mr. W. Ross has begun to manipulate its bankrupt. *3. Gossips claimed that he has obtained it by fraud.


VT intuitionsPossible repairs of Ex. 4: Ex. 5

1. Mr. W. Ross has begun to manipulate the bankrupt of his company2. with one year before finishing his mandate of president.3. Gossips claimed that he has obtained it by fraud. Ex. 61. With one year before finishing his mandate of president of the company2. Mr. W. Ross has begun to manipulate its bankrupt. 3. Gossips claimed that he has been elected by fraud.


VT´s basics

• a discourse structure can be represented by a (binary) tree

• a terminal node (unit) – a span of text, syntactically a sentence or clause

• an intermediate level node (relation) – a span of text that has a structure of its own

• nodes are polarised: nuclei (most important) and satellites (less important)

• relation names are ignored


1 2 3 4

5

6 7 8

9

10

11 12

13-??

??-??

H = 1 9 *V = 1 9 *

H = 1V = 1 9 *

H = 9V = 1 9 *

H = 1V = 1 9 *

H = 5V = 1 5 9 *

H = 1V = 1 9 *

H = 3V = 1 3 5 9 *

H = 6 7V = 1 5 6 7 9 *

H = 9V = 1 9 *

H = 9V = 1 9 *

H = 9V = 1 (8) 9 *

H = 10V = 1 9 10 *

H = 11V = 1 9 10 11 *H = 3

V = 1 3 5 9DRA = 1 3 H = 9

V = 1 (8) 9DRA = 1 8 9

Trees as in RST

1

4

2 3

relations

units

nuclear


The definitions of VT: heads

• Head expression of a node: the sequence of the most important units within the corresponding span of text:– the head of a terminal node: its label– the head of a non-terminal node: the

concatenation of the head expressions of the nuclear children

• the important units are projected up to the level where the corresponding span is seen as a satellite


Head expressions are computed bottom-up

ba

c

H=aH=b

H=a H=c

H=cH=a c


Veins

to understand a piece of text in the context of the whole discourse one needs the significant units within the span together with other surrounding units

Vein expression of a node: the sequence of units that are required to understand the span of text covered by the node, in the context of the whole discourse


Veins

• Vein expressions are computed top-down starting with the root – the vein expression of the root is its head

expression


Vein expressions

• A nuclear node with no satellites to the left

V=v

V=v

V=v


Vein expressions

• A satellite node on the right (simplified)

V=seq(h, v)

V=v

H=h


Consider the text:

1. John sold his bicycle

2. although Bill would have wanted it.

3. He obtained a good price for it,

4. which Bill could not have afforded.

5. Therefore he decided to use the money to go on a trip.


Computing the veins ofJohn and Bill

H=1 3

H=1

H=3

H=1

H=2H=3

H=4

H=5

H=1 3 5

1 2 3 4

5V=1 3 5

V=1 3 5

V=1 3 5

V=1 3 5

V=1 3 5

V=1 2 3 5

V=1 3 5

V=1 3 5

V=1 3 4 5


Computing the veins ofJohn and Bill

1 2 3 4

5V=1 3 5

V=1 3 5

V=1 2 3 5 V=1 3 5

V=1 3 4 5


The account of VT on cohession• Grounds

– a three layered approach to referential representation

– no need for right reference (cataphora)!– vein expressions are contexts in which to

look for referents


A three layered approach to anaphora

the text layer ………………..……………………….…………….

the semantic layer …………………………..…centera

a b

the projections layer ………………………………………………fsa

RE a projects fsa

fsa proposes centerafsb evokes centera

fsb

RE b projects fsb

Cristea&Dima, 2001; Cristea et al., 2002


A three layered approach to anaphora



the text layer ………………………………………………………

the semantic layer ……………………...………

it his bicycle

the projections layer ………………………………………………

evokesproposes

no = sgsem=bicycledet = yes

projects


projects

no = sgsem=¬human


No need for right references!

1. Although Bill would have wanted it,

2. John sold his bicycle to somebody else.

the text layer ………………………………………………………

the semantic layer ……………………………...

it his bicycle

the projections layer ………………………………………………

projects

no = sgsem=¬human

projects


evokesproposes

no = sgsem=¬humanno = sgsem=bicycledet = yes


Domain of evocative accessibility (DEA)

DEA(u) = pref(u, vein(u))

Reminder! The vein expression of a terminal node (discourse unit): the sequence of units that are required to understand just that unit, in the context of the whole discourse.

(simplified)


Direct referencesIf A and B are units in this textual order, A belongs to the DRA of B

and aA is linearly the most recent (to B) RE that realizes the same center as bB, we say that b directly co-refers the center evoked by a.

text layer ………………………………………………………………………

semantic layer ………..………………………………………………….

A B

centera

a b

a. direct co-reference b. functional direct co-reference

A B

centera

a b

centerb

role


Indirect referencesIf A, B and C are units in this order, bB is linearly the most recent

(to C) RE that realises the same center as cC, B is not on the DRA of C, A is linearly the most recent (to B) unit that is both on the DRA of B and of C, and it contains a RE aA such that bB realises the same center as a, we say that c indirectly co-refers the center realised by a.

A similar definition applies for indirect functional references.

units and DRAs …………………….. A B

text layer…..………………….. …………………

semantic layer……...……………….centera

b

C

c a


Inferential references

We call any reference that is not direct or indirect – a inferential reference: there is no intersection between the backward looking chain of co-references’ units and the DRA of the anaphor's unit.

A particular category of inference references - pragmatic references, or pseudo references: REs that practically do not refer back in the discourse although an entity identically realised was already introduced.


units and their DRAs REs projection restrictions evocative and situations processes post-evocative processes

bicycle1type = bicycleowner = [John]gender = humannum = sg

his bicycle

he

Johntype = personname = Johngender = mascnum = sg

John

Bill

which

pos = def nountype = bicycleowner = [John]gender = humannum = sg

pos = proper nountype = personname = Johngender = mascnum = sg

pos = pron type = humangender = mascnum = sg

it

price

pos = proper nountype = personname = Billgender = mascnum = sg

Billtype = personname = Billgender = mascnum = sg

pos = pron gender = humannum = sg

pos = undef noun type = pricenum = sg

itpos = pron gender = humannum = sg

Bill

pos = proper nountype = personname = Billgender = mascnum = sg

pos = pron

he

pos = pron type = humangender = mascnum = sg

the moneypos = def noun type = moneynum = pl

a trippos = undef noun num = sg

2

1

price1type = pricegender = humannum = sg

money1type = moneynum = pl

trip1type = tripnum = sg

direct references

indirect reference

inferential reference

3

4

5


The first conjecture (of cohesion)

Antecedents of anaphora are found mainly in the DEA given by the vein expression of the unit the anaphor belongs to.

Only if they are not found there, the rest of the units must be searched (Extended-DEA) – inferential:

Unit Vein DEA E-DEA

4 1 3 4 5 1 3 4 2 1 3 4


direct references >

indirect references >

inferential references

“>” = “more frequent”, “easier to process”

The first conjecture (of cohesion)


Experimental results

Source No. of units

Total no. of refs

Direct Indirect Inferential

English 62 97 77.3% 14.4% 8.3%

French 48 110 89.1% 10.0% 0.9%

Romanian 66 111 93.7% 1.8% 4.5%

Total 176 318 87.1% 8.5% 4.4%


Two classes of models

• Linear-k models– the antecedent of an RE can be found in the k

discourse units that immediately precede it.• Linear-0: intra-unit model.• Linear-k: most anaphora and co-reference resolution systems.

• Discourse-k models– the antecedent of an RE can be found in the k

discourse units that hierarchically precede it.• Use Veins Theory to compute the units that hierarchically

precede a unit.


Experiments

• Took 30 texts from the MUC7 corpus (news stories about changes in executive management personnel)

• The texts were annotated for co-reference relations of identity [Hirschman and Chinchor,97]

• The texts were annotated for rhetorical structure [Marcu, Amorrortu, and Romera,99]

• Fused the annotations; computed the veins; computed the DEAs and E-DEAs

Cristea et al., 2000


Experiment 1

• Estimated the potential of Linear-k and Discourse-VT-k models to establish correct co-referential links.– For each k, each re, and each model M

(Linear or VT)• p(M-k,re,EDEAk) =

• p(M-k,Corpus) = re Corpus p(M-k,re,EDEAk)

1, re can be resolved to antecedents in EDEAk

0, otherwise.{


Potentials (2)

70,00%

75,00%

80,00%

85,00%

90,00%

95,00%

0 1 2 3 4 5 6 7 8 9EDRA size

VT-k Linear-k


Direct comparison: VT-k/Linear-k

0,96

0,98

1

1,02

1,04

1,06

1,08

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

EDRA sizeVT-k/Lin-k


Experiment 2

• Estimated the effort required to establish correct co-referential links when using Linear-k and Discourse-VT-k models.– For each k, each re, and each model M

(Linear or VT)• e(M-k,re,EDEAk) =

• e(M-k,Corpus) = re Corpus e(M-k,re,EDEAk)

d, the distance between re and the closestantecedent in E-DEAk

k, if no such antecedent exists.{


Example1. Michael D. Casey, a top Johnson&Johnson manager, moved to Genetic

Therapy Inc., a small biotechnology concern here,

2. to become its president and chief operating officer.

3. Mr. Casey, 46 years old, was president of J&J's McNeil

Pharmaceutical subsidiary,

4. which was merged with another J&J unit, Ortho Pharmaceutical Corp.,

this year in a cost-cutting move.

5. Mr. Casey succeeds M. James Barrett, 50, as president of Genetic

Therapy.

6. Mr. Barrett remains chief executive officer

7. and becomes chairman.

8. Mr. Casey said

9. he made the move to the smaller company

VT: DEA(9) = <1, 8, 9>

Linear VT-EDEA

K=0 9 9 K=1 9,8 9,8 K=2 9,8,7 9,8,1 K=3 9,8,7,6 9,8,1,7 K=4 9,8,7,6,5 9,8,1,7,6


Example

Michael D. Casey

Genetic Therapy Inc.

10 11 122 3 4

Mr. Casey

Genetic Therapy Inc.

5 6 7

Mr. Casey

81

the smaller company

9


Results

0

1000

2000

3000

4000

5000

6000

7000

8000

1 3 5 7 9 11 13 15 17 19 25 35 45 55 65 75 85 95

EDEA sizeVT Process Lin Process


The account of VT on coherence

• Veins give a natural way to generalize Centering from local to global


1 2 3 4

5 V=1 3 5

V=1 3 5

V=1 2 3 5 V=1 3 5V=1 3 4 5

Vein expressions give „lines of argumentation“






1 2 3 4

5 V=1 3 5

V=1 3 5

V=1 2 3 5 V=1 3 5V=1 3 4 5

Lines of argumentation






1 2 3 4

5 V=1 3 5

V=1 3 5

V=1 2 3 5 V=1 3 5V=1 3 4 5

Lines of argumentation






Evaluating the coherence of a discourse• A smoothness score:

– CONTINUING = 4– RETAINING = 3– SMOOTH SHIFT =2– ABRUPT SHIFT = 1– NO Cb = 0

• A global smoothness score: summing up the score of all units


The second conjecture (on coherence)• The global smoothness score of a

discourse when computed following VT is at least as high as the score computed following CT.

• That is, we claim that long-distance transitions computed using VT are systematically smoother than accidental transitions at segment boundaries.


Transitions and scores on a linear adjacency metricJ = [John], b = [John's bicycle], B = [Bill], p = [price], m = [the money], t = [a trip])

1 2 3 4 5

Cf J, b B, b J, p, b p, B J, m, t

Cb J b b p -

Trans ASH RET SSH No Cb

Score 1 3 2 0

Global 6/4 = 1.5


Transitions and scores on a hierarchical adjacency metric

1 2

Cf J, b B, b

Cb J B

Trans ASH

Score 1

Global

1 3 4

J, b J, p, b p, B

J J p

CON SSH

4 2

1 3 5

J, b J, p, b p, m, t

J J J

CON

4

11/4=2.75


Verifying the second conjecture

Source No. of transitions

CT score Average CT score

per transition

VT score Average VT score

per transition

English 59 76 1.25 84 1.38

French 47 109 2.35 116 2.47

Romanian 65 142 2.18 152 2.34

Total 173 327 1.89 352 2.03


VT vs. Linear and Stack-based

Length of DEA

Effort

stack-based

VT linear


Exceptions

• Cases where neither VT nor the stack-based model finds an antecedent

• Four types:– pragmatic

• the Senate, the key in lock them up and throw away the key, our in our streets

– proper nouns• Mr. Gerstner, Senator Biden

– common nouns• the steelmaker, the proceeds, the top job

– pronouns


Exceptions

decreasing evoking

power

Type of RE VT

Stack-based

pragmatic 56.3% 0.0%

proper nouns 22.7% 26.1%

common nouns 16.0% 39.1%

pronouns 5.0% 34.8%


Interpretation

• alignment between evoking power and percentage of exceptions– VT predictions wrt DEA are fundamentally

correct

• pronoun exceptions linked to the RST attribution relation – integrate X said with the attributed

quotation?


Fix points of most discourse theories• structure seen as trees (AST, RST, Hobbs,

Polanyi, Marcu, etc.)• sub-trees of the whole tree make up

discourse segments• sequentiality (with some exceptions in Hobbs)• compositionality• a view on the relationship between

referentiality and structure


Parsing discourse

• Marcu’s parser

• expectations-driven incremental parsing

• VT parsing


Elements of Marcu’s constructions• a set of rhetorical relations between elementary

discourse units• consider a guesser (oracle) giving all RR between

elementary or extended text spans• each span:

– status (nuclearity)– type (RR)– promotion set

• Strong compositionality criteria: if a relation R holds between two textual spans, than R also holds between the most important units of the constituent spans.


Example[Animals heal 1][but trees compartmentalize. 2][They

endure a lifetime of injury and infection by setting boundaries that resist the spread of the invading microorganisms. 3]

1-2

21

3

1-3

S=satT= leafPS=[3]

S=nuc-satT= ELABORATION

PS=[1, 2]

S=nucT= leafPS=[2]

S=nucT= leafPS=[1]

S=nucT= CONTRAST

PS=[1, 2]


Constraint-satisfaction parsing

• Two ways to derive valid discourse structures:– a model-theoretic approach: apply axioms that

characterize the general constraints of a text structure to the text under scrutiny declarative specification of the constraints that characterize a valid structure

– a theorem-proving approach: specify rewriting rules that map a sequence of textual units into a a valid text structure control the process of derivation

Daniel Marcu, 1997, 2000


Example of constraints

• text spans do not overlap• at most one rhetorical relation can connect

two adjacent discourse spans• there exists a discourse segment (the root)

that spans over the entire text• the status, type and promotion set that are

associated with a discourse segment reflect the strong compositionality criteria


Marcu’s constraint propagation algorithm

INPUT – N textual units each characterized by 3 variables

(S, T, PS)– the set of RR that hold between units and spans

OUTPUT– 3*N*(N+1)/2 variables– all potential structures

Performance: exponential, for texts longer than 4 units does not finish in less than 3 hours (on a Sparc Ultra 2-2170)


Conversion into a propositional logic proverINPUT

– a sequence of N textual units– a set of RR that hold between units and spans

OUTPUT– all valid discourse structures

Performance (using Davis-Putnam and greedy derivation methods): still exponential, but much better (for a text with 19 units - 250 trees computed in 50 sec.)


The proof-theoretic approach

INPUT – a sequence of N textual units– a set of RR that hold between units and

spans

OUTPUT– all valid discourse structures

Performance: 19 units aprox. 3 hours and > 24000 trees


No surprise for the poor performanceFinding solutions of constraint-satisfaction

problems and finding models of theories of propositional formulas are NP-complete problems

Garey&Johnson, 1979

Parsing phrase-structure trees in the presence of functional constraints can be exponential in the worst case

Barton et al., 1985


Compiling grammars in Chomsky normal formSet of constraints applied to the input set RR to

compile a CNF grammarThen apply a top-down derivation of the grammar

rulesThe compiled grammars shown to be sound and

completePerformance: the algorithm generates O(N6)

grammar rules in O(N6) steps. Then the CKY algorithm derives the text structures in O(N3) steps (the 19 units text processed in 5.2 sec.)


Example of all possible derivations[Animals heal 1][but trees compartmentalize. 2][They

endure a lifetime of injury and infection by setting boundaries that resist the spread of the invading microorganisms. 3]

1-2

21

3

1-3

S=satT= leafPS=[3]

S=nuc-satT= ELABORATION

PS=[1, 2]

S=nucT= leafPS=[2]

S=nucT= leafPS=[1]

S=nucT= CONTRAST

PS=[1, 2]

2-3

32

1

1-3

S=satT= leafPS=[3]

S=satT= ELABORATION

PS=[2]

S=nucT= leafPS=[2]

S=nucT= leafPS=[1]

S=nuc-satT= CONTRAST

PS=[1, 2]


Incremental discourse parsing

The principle of sequenciality

• A left to right reading of the terminal frontier of the tree associated with a discourse must correspond to the span of text it analyses in the same left-to-right order.

6


Incremental discourse parsing - a TAG inspired approach

Adjoining to the right frontier

a1

0

0

1

a

a*

’a

a1

7


Substitution

k+1

k. Although Bill would have wanted it,

k

k+1. John sold his bicycle to somebody else.

k

’

k+1


Expectations-driven incremental parsing

a. Clinton is bound to win the elections.

b. He is a natural born campaigner.

c. If you hold some position on an issue,

d. then if Clinton wants to get your vote,

e. he will assure you with great sincerity that he holds that position too.

a

b

EVIDENCE

c

d e

EVIDENCE

ANT-CONS

ANT-CONS

8

Cristea&Webber, 1997



a

b

EVIDENCE

*


9



a. Clinton is bound to win the elections.b. He is a natural born campaigner.

a b

EVIDENCE


c

EVIDENCE

*

10





a

b

EVIDENCE

c

EVIDENCE

11



a. Clinton is bound to win the elections.b. He is a natural born campaigner.c. If you hold some position on an issue,

a

b

EVIDENCE

EVIDENCE

c

ANT-CONS

?

12




a b

EVIDENCE


EVIDENCE

*c

ANT-CONS

?

13



a. Clinton is bound to win the elections.b. He is a natural born campaigner.c. If you hold some position on an issue,

a

b

EVIDENCE

EVIDENCE

c

ANT-CONS

?

d. he will assure you with great sincerity that he holds that position too.

14






d. he will assure you with great sincerity that he holds that position too.

a

b

EVIDENCE

EVIDENCE

c

ANT-CONS

d

15





c. If you hold some position on an issue,d. then if Clinton wants to get your vote,

d

ANT-CONS

?

a

b

EVIDENCE

EVIDENCE

c

ANT-CONS

?

16





c. If you hold some position on an issue,d. then if Clinton wants to get your vote,e. he will assure you with great sincerity that he holds that position too.

d

ANT-CONS

a

b

EVIDENCE

EVIDENCE

c

ANT-CONS

?

17





c. If you hold some position on an issue,d. then if Clinton wants to get your vote,e. he will assure you with great sincerity that he holds that position too.

d

ANT-CONS

a

b

EVIDENCE

EVIDENCE

c

ANT-CONS

e

18



The non-monotonicity of incremental parsing[Because John is such a generous man 1] [– whenever he is asked

for money, 2] [he will give whatever he has, for example 3] [– he deserves the “Citizen of the year” award. 4]

From [Cristea&Webber,1998]

1

EXAMPLE

2 3

CIRCUMSTANCE

4

EVIDENCE

1

-, - for example

2 3

whenever -, -

4

because -, -


Marcu’s shallow approach

• use markers to identify: – discourse segments– rhetorical relations– discourse structure

• markers are three-ways ambiguous:– sentential versus discourse usage– rhetorical relation they signal– the size of the textual span they connect

• corpus investigation of cue phrases


What can cue-phrases tell us about structure?

[Because John is such a generous man 1] [– whenever he is asked for money, 2] [he will give whatever he has, for example 3] [– he deserves the “Citizen of the year” award. 4]

1 2 3 4

because -, -1 2 3 4

1 2 3

1 2

1 3 42

1 32

1 42 3

because <something>, <something>

Marcu, 1997, 2000; Cristea et al., 2003


What cue-phrases can tell us about structure?[Because John is such a generous man 1] [– whenever he is asked


2 3

whenever -, -2 3 4

2 3

2 43

whenever <something>, <something>




1 2 3

-, - for example1 2 3

2 3

1 32

<something>, <something> for example




1

-, - for example

2 3

whenever -, -

4

because -, -

1

2 3

4

because -, -

-, - for example

whenever -, -

because < >, < >1 2 3 4

whenever < >, < >2 3

< >, < > for example1 2 3

because < >, < >1 2 3 4

whenever < >, < >2 3

< >, < > for example2 3

4


Marcu’s algorithm for rhetorical parsing of free texts• determine the set D of all cue phrases in T with a

discourse function• segment T in sections, paragraphs, sentences and

clauses using the corpus (C)• for each section, paragraph and sentence:

– use C to hypothesize rhetorical relations signaled by markers in D

– use cohesion to hypothesize rhetorical relations not signaled by markers

– find all the valid text trees that correspond to that level– select the best tree based on assigned weights

• merge the resulted trees


How can references help in discovering the structure?

a. Because Mary was upset,

b. even if John agreed,

c. they didn’t speak to one another for several days.

c

a b V=(a)cDRA=ac

a

b c

V=(a)(b)cDRA=abcrightwrong


VT guides an incremental discourse parsing

The tree resulted after the parsing process is the one which manifests:– the more natural overall references over

the discourse structure – the smoothest overall CT transitions on

veins

Cristea, 2000; Cristea et al., 2003


The architecture

blackboard

shallow parser

POS tagger

reference resolution expert

discourse parser

transitions expert

cue-words expert


reference expert

transitions expert

The discourse parser implements a „beam search“

N

N


Processing the text:first 2 steps


[John‘s bike]

[John]

1 2

concession


2

concession

*

E-DEA=12

[Bill]

it

REF score

1 direct 2

2


[Bill]

[John]


1 2

concession


ABRUPT SHIFT

CT score

1

1

[bike]

[bike]

Processing the text:first 2 steps


3

unknown

*

Processing the text:step 3


1 2

concession



1

2

concession

3

unknown

He

[price]

it

[Bill]

[bike]

REF score

1 direct 2

2

42 direct

E-DEA=123




1 2

concession



1

2

concession

3

unknown

[price]

CT score

1.5

[bike]

[Bill]

ABRUPT SHIFT 1

[Bill]

[bike]

[John]

[bike]

2SMOOTH SHIFT


4

elaboration

*






1

2

concession

3

unknown1

2

concession

3

unknown

4

elaboration

E-DEA=1234which

[Bill]

[Bill]

[price]

[bike]

REF score

1 direct 2

1.4

42 direct

21 direct

-11 direct: prop-n/pron







1

2

concession

3

unknown1

2

concession

3

unknown

4

elaboration

CT score

2

3RETAINING

ABRUPT SHIFT 1

2SMOOTH SHIFT

[bike]

[John]

[Bill]

[bike] [price]

[Bill]

[bike]

[bike]

[Bill]


Processing the text:step 3, variant




3

unknown

*

REF score

1 direct 2

2

42 direct

1 2

concession

1 2

concession 3

unknown

E-DEA=213

He

[price]

it

[bike]

[John]






1 2

concession 3

unknown

CT score

2.5

CONTINUING 4

ABRUPT SHIFT 1

[bike]

[John]

[price]

[John]

[bike]

[Bill]

[bike]



4

elaboration

*





1 2

concession 3

unknown

1 2

concession

4

elaboration

3

unknown

[price]

[bike]

[John]

[Bill]

[bike]

REF score

1 direct 2

1.6

42 direct

21 direct

01 inferential

which

[Bill]

E-DEA=2134







1 2

concession 3

unknown

1 2

concession

4

elaboration

3

unknown

[bike]

[John]

[price]

[John]

[bike]

[Bill]

[bike]

[bike]

[Bill]

CT score

2.3

CONTINUING 4

ABRUPT SHIFT 1

2SMOOTH SHIFT


4

conclusion

*






1 2

concession 3

unknown

1 2

concession

4

elaboration

3

unknown

5. Therefore he decided to use the money for going in a trip.

1 2

concession

4 5

conclusion3

elaboration

unknown

he

[money]

[trip]

REF score

1 direct 2

1.7

42 direct

21 direct

01 inferential

21 direct

REF score

[bike]

[Bill]E-DEA=21345







1 2

concession 3

unknown

1 2

concession

4

elaboration

3

unknown

REF score5. Therefore he decided to use the money for going in a trip.

1 2

concession

4 5

conclusion3

elaboration

unknown

[bike]

[John]

[price]

[John]

[bike]

[Bill]

[bike]

[bike]

[Bill][money]

[trip]

[Bill]

CT score

2.25

CONTINUING 4

ABRUPT SHIFT 1

2SMOOTH SHIFT

2SMOOTH SHIFT


4

conclusion

*

Processing the text:step 5, variant 2





1 2

concession

4

elaboration

3

unknown


REF score

1 direct 2

1.7

42 direct

21 direct

01 inferential

21 direct

REF score

1 2

concession

4

5

conclusion

3

elaboration

unknown

[price]

[bike]

[John] E-DEA=24135

he

[money]

[trip]







1 2

concession 3

unknown

1 2

concession

4

elaboration

3

unknown


1 2

concession

4

5

conclusion

3

elaboration

unknown

[bike]

[John]

[price]

[John]

[bike]

[Bill]

[bike]

[bike]

[Bill][money]

[trip]

[John]

CT score

2.75

CONTINUING 4

ABRUPT SHIFT 1

2SMOOTH SHIFT

CONTINUING 4


4

conclusion

*






1 2

concession

4

elaboration

3

unknown


1 2

concession

4

5

conclusion

3

elaboration

unknown

[bike]

[John] E-DEA=23415

he

[money]

[trip]

REF score

1 direct 2

1.7

42 direct

21 direct

01 inferential

21 direct

REF score







1 2

concession 3

unknown

1 2

concession

4

elaboration

3

unknown


1 2

concession

4

5

conclusion

3

elaboration

unknown

CT score

2.75

CONTINUING 4

ABRUPT SHIFT 1

2SMOOTH SHIFT

CONTINUING 4

[bike]

[John]

[price]

[John]

[bike]

[Bill]

[bike]

[bike]

[Bill]

[money]

[trip]

[John]


Processing the text:results






1 2

concession

4 5

conclusion3

elaboration

unknown

1 2

concession

4

5

conclusion

3

elaboration

unknown

1 2

concession

4

5

conclusion

3

elaboration

unknown

Ref=1.7 HCT=2.25

Ref=1.7 HCT=2.75 Ref=1.7 HCT=2.75


Conclusions

• a versatile architecture that allows for a „detached“ behaviour

• the more experts you have and the more accurate knowledge they have the more finer results you get

• adequate for an on-line processing of discourse

• real-time abstracting of typed text, IE, real-time translation, deep understanding...


Related work

• towards an integrated discourse parsing – anaphora resolution algorithm

• we need corpora resources annotated to fit discourse needs

• anaphora resolution guided by discourse structure• “reverse” anaphora: correct discourse structure• focused summarization • in search for an underspecified representation

Documents

Discourse theories and technologies