The Learning Non-Isomorphic Tree Mappings for Machine Translation Jason Eisner - Johns Hopkins Univ....

the

Learning Non-Isomorphic Tree Mappings for Machine Translation

Jason Eisner - Johns Hopkins Univ. a

b

A

B

events of

misinform

wrongly

report

to-John

events

him

“wrongly report events to-John” “him misinform of the events”

2 words become 1

reorder dependents

0 words become 1

0 words become 1

Syntax-Based Machine Translation

• Previous work assumes essentially isomorphic trees– Wu 1995, Alshawi et al. 2000, Yamada & Knight 2000

• But trees are not isomorphic! – Discrepancies between the languages

– Free translation in the training data

the

a

b

A

B

events of

misinform

wrongly

report

to-John

events

him

Two training trees, showing a free translation from French to English.

Synchronous Tree Substitution Grammar

enfants(“kids”)

d’(“of”)

beaucoup(“lots”)

Sam

donnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

kids

Sam

kiss

quite

often

“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”

enfants(“kids”)

kids

NPd’

(“of”)

beaucoup(“lots”)

NP

NP

SamSam

NP

Synchronous Tree Substitution Grammar

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

quitenullAdv

oftennullAdv

nullAdv

“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”

Two training trees, showing a free translation from French to English.A possible alignment is shown in orange.

enfants(“kids”)

kids

Adv

d’(“of”)

beaucoup(“lots”)

NP

SamSam

NP

Synchronous Tree Substitution Grammar

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NPquite

often

“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”

Two training trees, showing a free translation from French to English.A possible alignment is shown in orange.A much worse alignment ...

enfants(“kids”)

kids

NPd’

(“of”)

beaucoup(“lots”)

NP

NP

SamSam

NP

Synchronous Tree Substitution Grammar

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

quitenullAdv

oftennullAdv

nullAdv

“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”

Two training trees, showing a free translation from French to English.A possible alignment is shown in orange.

enfants(“kids”)

kids

NPd’

(“of”)

beaucoup(“lots”)

NP

NPquitenull

Adv

oftennullAdv

nullAdv

SamSam

NP

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

Synchronous Tree Substitution Grammar

“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”

Start

Two training trees, showing a free translation from French to English.A possible alignment is shown in orange. Alignment shows how trees are generated synchronously from “little trees” ...

SamSamNP

Grammar = Set of Elementary Trees

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

idiomatictranslation

enfants(“kids”)

kids

NP

enfants(“kids”)

kids

NP

SamSamNP

Grammar = Set of Elementary Trees

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

idiomatictranslation

SamSamNP

enfants(“kids”)

kids

NP

Grammar = Set of Elementary Trees

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

SamSamNP

enfants(“kids”)

kids

NP

d’(“of”)

beaucoup(“lots”)

NP

NP

“beaucoup d’” deletes inside the tree

d’(“of”)

beaucoup(“lots”)

NP

NP

“beaucoup d’” deletes inside the tree

Grammar = Set of Elementary Trees

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

SamSamNP

enfants(“kids”)

kids

NP

enfants(“kids”)

kids

NPd’

(“of”)

beaucoup(“lots”)

NP

NP

“beaucoup d’” matches nothing in English

Grammar = Set of Elementary Trees

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

SamSamNP

enfants(“kids”)

kids

NP

SamSamNP

enfants(“kids”)

kids

NPquitenull

Adv

Grammar = Set of Elementary Trees

oftennullAdv

nullAdv

d’(“of”)

beaucoup(“lots”)

NP

NP

kissdonnent (“give”)

baiser(“kiss”)

un(“a”)

à (“to”)

Start

NP

NP

nullAdv

adverbial subtree matches nothing in French

Probability model similar to PCFG

Probability of generating training trees T1, T2 with alignment A

P(T1, T2, A) = p(t1,t2,a | n)probabilities of the “little”

trees that are used

p(is given by a maximum entropy model

wrongly

misinform

NP

NP

reportVP | )VP

FEATURES• report+wrongly misinform?

(use dictionary)

• report misinform? (at root)

• wrongly misinform?

Maxent model of little tree pairs

• verb incorporates adverb child?

• verb incorporates child 1 of 3?

• children 2, 3 switch positions?

• common tree sizes & shapes?

• ... etc. ....

p(wrongly

misinform

NP

NP

reportVP | )VP

Inside Probabilities

the

a

b

A

B

events of

misinform

wrongly

report

to-Johnevents

him

VP

( ) = ...misinformreport VP

* ( ) * ( ) + ...

p( | )VP

Inside Probabilities

the

a

b

A

B

events of

misinform

wrongly

report

to-Johnevents

him

VP

NP

NP

( ) = ...misinformreport VP

events ofNP to-John himNP* ( ) * ( ) + ...

p( | )VP

NP

misinform

wrongly

report VP

NP

only O(n2)

An MT ArchitectureViterbi alignment yields output T2

dynamic programming engine

Probability Model p(t1,t2,a) of Little Trees

score little tree pair

propose translations t2of little tree t1

each possible (t1,t2,a)

inside-outsideestimated counts

update parameters

for each possible t1, various (t1,t2,a)

each proposed (t1,t2,a)

DecoderTrainerscores all alignmentsof two big trees T1,T2

scores all alignmentsbetween a big tree T1

& a forest of big trees T2