Forest-based Semantic Role Labeling

INS

TIT

UTE O

F C

OM

PU

TIN

G

TEC

HN

OLO

GY

Forest-based Semantic Role

Labeling

Hao Xiong, Haitao Mi, Yang Liu and Qun Liu

Institute of Computing TechnologyAcademy of Chinese SciencesAAAI 2010, Atlanta7/15/10 1

INSTITUTE OF COMPUTING

TECHNOLOGY

Semantic Role Labeling

Given a sentence and its verbs Identify the arguments of the verbs Assign semantic labels (the roles they play)

This company

last year

1000 cars

in the U.S.

sold

AgentAgent PatientPatientArgMod-

TeMPoral

ArgMod-

TeMPoral

ArgMod-LOCationArgMod

-LOCation

PropBank (Kingsbury and Palmer 2002)

7/15/10 2


TECHNOLOGY

One Conventional Approach

the role of Celimene is played by Kim Cattrall

Patient Agent

AAAI 2010, Atlanta7/15/10 3


TECHNOLOGY



Patient Agent

S

NP PP

VP

AUX VBN PP

VPNP



TECHNOLOGY



Patient Agent

S

NP PP

VP

AUX VBN PP

VP?

more than 15%



TECHNOLOGY

…

1 2 3 k…

Solution

k-best parses: limited scope: k too much redundancy

25<50<26

S

NP PP

VP

AUXVBN PP

VPNP

S

NP PP

VP

AUXVBN PP

VP …AAAI 2010, Atlanta7/15/10 6


TECHNOLOGY

Our Solution Forest

A compact representation of many parses By sharing common sub-derivations Polynomial-space encoding of exponentially large

set S

NP PP

VP

AUX VBN PP

NPVP

S

NP PP

VP

AUXVBN PP

VPNP

S

NP PP

VP

AUXVBN PP

VP …Unpack



TECHNOLOGY

Our Solution Forest

A compact representation of many parses By sharing common sub-derivations Polynomial-space encoding of exponentially large

set S

NP PP

VP

AUX VBN PP

NPVP



TECHNOLOGY

Outline

Tree-based Semantic Role Labeling Parsing Selecting candidates Extracting features Classifying

Forest-based Semantic Role Labeling Experiments Conclusion



TECHNOLOGY

Parsing

S

NP NP VP

DT NN JJ NN VBD NP PP

CD NNS IN NP

NNPDT

This company last year sold

1000 cars in

the U.S.AAAI 2010, Atlanta7/15/10 10


TECHNOLOGY

Selecting Candidates

S

NP NP VP


CD NNS IN NP

NNPDT

soldThis company last year

1000 cars in

the U.S.AAAI 2010, Atlanta7/15/10 11


TECHNOLOGY

Extracting Features

S

NP NP VP


CD NNS IN NP

NNPDT


1000 cars in

the U.S.

Path to the predicate

This company last year

1000 cars in

the U.S.

NNS

NPSVPVBN

AAAI 2010, Atlanta7/15/10 12


TECHNOLOGY

Extracting Features

S

NP NP VP


CD IN NP

NNPDT


1000 cars in

the U.S.

Position: left


1000 cars in

the U.S.

NNS

NPSVPVBN

left

AAAI 2010, Atlanta7/15/10 13


TECHNOLOGY

Extracting Features

S

NP NP VP


CD IN NP

NNPDT


1000 cars in

the U.S.

Head word: company


1000 cars in

the U.S.

NNS

NPSVPVBN

leftcompany

AAAI 2010, Atlanta7/15/10 14


TECHNOLOGY

Extracting Features

S

NP NP VP


CD IN NP

NNPDT


1000 cars in

the U.S.

Head POS tag: NN


1000 cars in

the U.S.

NNS

NPSVPVBN

leftcompany

NN…

AAAI 2010, Atlanta7/15/10 15


TECHNOLOGY

Classifying

S

NP NP VP


CD IN NP

NNPDT


1000 cars in

the U.S.S(Agent)=0.1S(Patient)=0.1S(None)=0.5

…

S(AM-TMP)=0.9S(Patient)=0.1S(None)=0.1

…

S(Agent)=0.2S(Patient)=0.8S(None)=0.1

…


…

S(AM-LOC)=0.9S(Agent)=0.1S(None)=0.1

…

Computing Score using a trained classifier


1000 cars in

the U.S.

NNS

16


TECHNOLOGY

Classifying

S

NP NP VP


CD IN NP

NNPDT


1000 cars in

the U.S.

S(Agent)=0.8

…

S(AM-LOC)=0.9

…


1000 cars in

the U.S.

NNS

S(None)=0.5

…

S(AM-TMP)=0.9

…

S(Patient)=0.8

…

Best score for each constituentSimply sort themChoose the best label sequence

NP

17


TECHNOLOGY

Classifying

S

NP NP VP


CD IN NP

NNPDT


1000 cars in

the U.S.

Agent AM-TMP V

Patient

AM-LOC


1000 cars in

the U.S.

NNS

18


TECHNOLOGY

Outline

Tree-based Semantic Role Labeling Forest-based Semantic Role Labeling

Parsing into a forest Selecting candidates Extracting features on forest Classifying

Experiments Conclusion

AAAI 2010, Atlanta7/15/10 19


TECHNOLOGY

Forest


S

NP PP

VP

AUX VBN PP

NP

VP

Hyper-graph Hyper-edge

Node

AAAI 2010, Atlanta7/15/10 20


TECHNOLOGY

Selecting Candidates


S

NP PP

VP

AUX VBN PP

NP

VP

AAAI 2010, Atlanta7/15/10 21


TECHNOLOGY

Exacting features



S

NP PP

VP

AUX VBN PP

NP

VP

NPNPSVPVPVBN

AAAI 2010, Atlanta7/15/10 22


TECHNOLOGY

Exacting features



S

NP PP

VP

AUX VBN PP

NP

VP

NPSVPVPVBN

NPNPSVPVPVBNshortes

t

AAAI 2010, Atlanta7/15/10 23


TECHNOLOGY

Exacting features

Parent LabelNPSVPVPVBN


S

NP PP

VP

AUX VBN PP

NP

VP

AAAI 2010, Atlanta7/15/10 24


TECHNOLOGY

Exacting features

Parent Label


NP PP

VP

AUX VBN PP

VP

NPSVPVPVBN

in the shortest path

AAAI 2010, Atlanta7/15/10 25


TECHNOLOGY

New Features Parsing score (Fractional value (Mi et al., 2008))

Inside-outside Marginal prob.


S

NP PP

VP

AUX VBN PP

NP

VP

NPSVPVPVBN

f(NP3)

AAAI 2010, Atlanta7/15/10 26


TECHNOLOGY

Classifying

S(Patient)=0.8S(Agent)=0.1S(None)=0.2

…

S(Patient)=0.5S(Agent)=0.1S(None)=0.3

…


S

NP PP

VP

AUX VBN PP

NP

VP


…

AAAI 2010, Atlanta7/15/10 27


TECHNOLOGY

Classifying

S(Patient)=0.8

…


S

NP PP

VP

AUX VBN PP

NP

VP

S(Agent)=0.8

…

Patient AgentAAAI 2010, Atlanta7/15/10 28


TECHNOLOGY

Outline

Tree-based Semantic Role Labeling Forest-based Semantic Role Labeling Experiments Conclusion

AAAI 2010, Atlanta7/15/10 29


TECHNOLOGY

Experiments

Corpus: CoNLL-2005 shared task Sections 02-21 of PropBank for

training Section 24 for development set Section 23 for test set Total

43,594 sentences 262,281 arguments

AAAI 2010, Atlanta7/15/10 30


TECHNOLOGY

Experiments

Training sentences Parse into 1-best and forest Prune forest using inside-outside algorithm Train classifiers

Decoding sentences Parse into 1-best and forest Prune forest using inside-outside algorithm Use classifiers

AAAI 2010, Atlanta7/15/10 31


TECHNOLOGY

Features

Predicate lemma Path to predicate Path length Partial path Position Voice Head word/POS tag …

AAAI 2010, Atlanta7/15/10 32


TECHNOLOGY

Results on Dev Set

precision

recall

F

1-best 50-best forest(p3)9.63×105

forest(p5)5.78×106

33


TECHNOLOGY

Results on Tst Set

AAAI 2010, Atlanta7/15/10 34


TECHNOLOGY

Outline

Tree-based Semantic Role Labeling Forest-based Semantic Role Labeling Experiments Conclusion

AAAI 2010, Atlanta7/15/10 35


TECHNOLOGY

Conclusion

Forest Exponentially encode many parses Enlarge the candidate space Explore more rich features

Improve the quality significantly Not necessary using very large forest Can NOT use k-best to simulate

Future works Features on forest

AAAI 2010, Atlanta7/15/10 36


TECHNOLOGY

Thank you!Patient

AAAI 2010, Atlanta7/15/10 37

Documents

Forest-based Semantic Role Labeling