View
97
Download
0
Category
Tags:
Preview:
DESCRIPTION
In Proceedings of the ACL 2014 Student Research Workshop A notably challenging problem related to event processing is recognizing the relations holding between events in a text, in particular temporal and causal relations. While there has been some research on temporal relations, the aspect of causality between events from a Natural Language Processing (NLP) perspective has hardly been touched. We propose an annotation scheme to cover different types of causality between events, techniques for extracting such relations and an investigation into the connection between temporal and causal relations. In this thesis work we aim to focus especially on the latter, because causality is presumed to have a temporal constraint. We conjecture that injecting this presumption may be beneficial for the recognition of both temporal and causal relations.
Citation preview
Extracting Temporal and Causal Relations between Events
Paramita Mirza
Under the supervision of Sara Tonelli
ACL SRW 2014
http://www.newsreader-project.eu/
What if computers could read the NEWS?
Recording events as stories
Event Extraction
On September 2008, Porsche increased its shares by another 4.98%, in effect taking control of the company.
6 Jan 2009 – Porsche has been on a quest to takeover VW for more than two years.
NAMED ENTITY TIME EXPRESSION EVENT
PhD topic
timeanchor
e1: increased- Porsche- shares- 4.98%
e2: taking control- Porsche- the company
e3: quest to takeover- Porsche- VW
t1: September 2008 t2: 6 Jan 2009 t3: two years
BEFORE
CAUSE
DURINGON ON
EVENT PARTICIPANT EVENT RELATION EVENT FACTUALITY
TEMPORAL RELATION CAUSAL RELATION
CAUSE
BEFORE
The Relationship between Events
IS_INCLUDED
Temporal Relations
Causal Relations
Typhoon Haiyan struck the eastern Philippines on Friday,
killing thousands of people.
Temporal Constraint of Causalitycause BEFORE effect
creating event timelines, multi-document summarization
predicting future events, risk analysis,decision making support
Research Questions
“Given a text annotated with events and time expressions, how to automatically extract temporal relations and causal relations between them?”
“Given the temporal constraint of causality, how to utilize the interaction between temporal relations and causal relations for building an integrated extraction system for both types of relations?”
Research Methodology
Temporal Relation
Extraction
1st
• Finding ways to improve the current state-of-the-art performance on temporal relation extraction
Causal Relation
Extraction
2nd
• Creating a standard benchmarking corpus for evaluating causal relation extraction
• Building an automatic extraction system for event causality
PhD!
Integrated System for Temporal
and Causal Relations
3rd
• Utilizing the interaction between temporal and causal to build an integrated system for temporal and causal relations
Research Methodology
Temporal Relation
Extraction
1st
• Finding ways to improve the current state-of-the-art performance on temporal relation extraction
Temporal Relation Extraction
• Common approach dividing the task:– Identifying the pairs of entities having a temporal link
• Often simplified, rule-based approach:– Main events of consecutive sentences– Pairs of events in the same sentence– An event and a time expression in the same sentence– An event and the document creation time
– Determining the relation types• Often regarded as a classification problem, supervised learning
approach: – Given an ordered pair of entities (e1, e2), the classifier has to
assign a certain label (temporal relation type)
TempEval-3 (2013)
• Shared task on temporal and event processing• Automatic identification of temporal expressions, events, and
temporal relations within a text annotated with TimeML Task F1 Precision Recall
Task A –Temporal Expression 90.30% 93.09% 87.68%Task B – Event Extraction 81.05% 81.44% 80.67%Task ABC – Temporal Awareness 30.98% 34.08% 28.40%Task C1 – Temporal Relations (identification + classification)
36.26% 37.32% 35.25%
Task C2 – Temporal Relations (only classification)
56.45% 55.58% 57.35%
Low performances on temporal relation extraction!
Classifying Temporal Relation Type
• Supervised classification approach• Support Vector Machines (SVMs) algorithm• TempEval-3 dataset for training & evaluation• Feature engineering: event attributes,
temporal signals, event duration, temporal connectives (disambiguation), etc.
• Bootstrapping the training data: inverse relations and closure
“Given a pair of entities (e1, e2), which could either be event-event, event-timex or timex-timex1, the classifier has to assign a certain label (temporal relation type).”
1so few number of pairs, so they are not considered
IS_INCLUDEDBEFORE
EVENT EVENTTIMEX
a BEFORE bb AFTER a
a IBEFORE bb IAFTER a
a BEGINS b b BEGUN_BY a
a ENDS b b ENDED_BY a
a DURING b b DURING_INV a
a INCLUDES bb IS_INCLUDED a
a SIMULTANEOUS b
a IDENTITY b
Microsoft Corp. broke sales records in 2010 when it released its Kinect.
a b
ab
ab
a
a
a
a
a
b
b
b
b
b
Relation event-event event-timextp fp fn tp fp fn
BEFOREAFTERIBEFOREIAFTERBEGINSBEGUN_BYENDSENDED_BYDURINGDURING_INVINCLUDESIS_INCLUDEDSIMULTANEOUSIDENTITY
18663
0000010012
209
18640
0000010024
3335
40104
12001010
394561
6
8214
00000000
27114
00
17700000020
1340
01
1415
56112211
1511
60
Classifying Temporal Relation Type (result)
Can be improved by including causality as a feature?
System F1 Precision RecallTRelProUTTime-1,4UTTime-3,5UTTime-2NavyTime-1NavyTime-2JU-SCE
58.48%56.45%54.70%54.26%46.83%43.92%34.77%
58.80%55.58%53.85%53.20%46.59%43.65%35.07%
58.17%57.35%55.58%55.36%47.07%44.20%34.48%
Best performing system!
Paramita Mirza and Sara Tonelli. 2014. Classifying Temporal Relations with Simple Features. In Proceedings of EACL 2014.
TempEval-3 Task
Research Methodology
Causal Relation
Extraction
2nd
• Creating a standard benchmarking corpus for evaluating causal relation extraction• Annotation guidelines for adding causal information in TimeML
• Building an automatic extraction system for event causality
C-SIGNAL and CLINK TimeML annotation
- EVENT- TIMEX3- SIGNAL- TLINK
+ Causality
- C-SIGNAL- CLINK
• C-SIGNAL → textual elements indicating the presence of causal relations• Prepositions • Conjunctions• Adverbial connectors• Clause-integrated expressions
because of, as a result of, due to, …because, since, so that, … as a result, so, therefore, … the result is, that’s why, …
• CLINK → a directional one-to-one relation where source = causing event and target = caused event(optional) c-signalID = ID of related C-SIGNAL
Causal ConceptsDynamics Model based on Force Dynamics Theory (Talmy, 1988)
• Captures the concept of causality, along with its related concepts, in terms of three dimensions:– the patient tendency for the result– the presence of concordance between the affector and the patient– the occurrence of the result
• Able to distinguish the concept of CAUSE from ENABLE, which is not available in the counterfactual model
• Was tested by linking it with natural language• The causality concepts can be lexicalized as verbs (Wolff and
Song, 2003):– CAUSE-type cause, influence, persuade, prompt, …– ENABLE-type aid, allow, enable, let, …– PREVENT-type block, constrain, prevent, restrain, …
CLINK: explicit causal constructions linking two events (source to target)
• Basic construction– The purchaseS caused the creationT of the current building
– The purchaseS enabled the diversificationT of their business
– The purchaseS prevented a future transferT
• Expressions with affect verbs affect, influence, determine, change
– Ogun CAN crisisS affects the launchT of the All Progressives Congress
• Expressions with link verbs link, lead, depend (on)
– An earthquakeT in North America was linked to a tsunamiS in Japan
• Periphrastic causatives– The blastS prompts the boat to heelT violently
– The oxygenS lets the fire getsT bigger
– The poleS restrains the tent from collapsingT
• Expressions with C-SIGNALs– Iraq said it invadedT Kuwait because of disputesS over oil and money
Manual Annotation
• TimeBank corpus from TempEval-3– with gold events, time expressions and temporal relations
• Inter-annotator agreement (on 5 documents):– 0.844 Dice’s coefficient on C-SIGNAL– 0.73 Dice’s coefficient on CLINK
Annotation EVENT C-SIGNAL CLINKManual 3933 78 144Manual-w/o new events 3872 78 95Automatic (rule-based) 3872 59 52
Paramita Mirza, Rachele Sprugnoli, Sara Tonelli and Manuela Speranza. 2014. Annotating causality in the TempEval-3 corpus. In Proceedings of CAtoCL 2014.
Causal Relation Extraction System
• To be presented at COLING 2014!
Paramita Mirza and Sara Tonelli. 2014. An Analysis of Causality between Events and its Relation to Temporal Information. (to appear) In Proceedings of COLING 2014.
Research Methodology
PhD!
Integrated System for Temporal
and Causal Relations
3rd
• Utilizing the interaction between temporal and causal to build an integrated system for temporal and causal relations
Temporal and Causal: the Interaction
• Temporal constraint of causality:“The cause happened BEFORE the effect”
• Bethard et al. (2008) on corpus analysis:– 32% of CAUSAL relations in the corpus did not have an
underlying BEFORE relation– “The walls were shaking because of the earthquake."
• Rink et al. (2010) makes use of temporal relations as a feature for classification model of causal relations– Causal relation extraction evaluation: F-score 57.9%
Integrated System for Temporal and Causal Relations
Temporal Expressions
Event Extraction
Temporal Relation
Extraction
Temporal & Causal Relation Extraction
Causal Relation Extraction
Cascading? Order?One pass for all? e.g. CRF?Online algorithm?
How?
Thank you!
CAUSE
BEFORE
Paramita closes the presentation so the question-answering session may start.
Recommended