View
216
Download
2
Category
Tags:
Preview:
Citation preview
Event Ordering using TERSEO Event Ordering using TERSEO systemsystem
Research Group on Research Group on Language Processing and Information SystemsLanguage Processing and Information Systems g g
PLSIPLSI
Estela Saquete Boró, Rafael Muñoz, Patricio Martinez-Barco
Departamento de Lenguajes y Sistemas Informáticos
NLDB 2004 2
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
1. Introduction
2. Previous work
3. Description of the Event Ordering System
4. Application of Event Ordering in NLP tasks
5. System evaluation
6. Conclusions
Index
NLDB 2004 3
Introduction
•Automatic processes to extract relevant
information
•Event ordering using dates and time
–Identification of temporal expressions
–Resolution of temporal expression
–Chronological order
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 4
Introduction
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
•Example:“Today is July the 3rd (2003).
Tomorrow is my birthday”
–Anaphoric expression: “Tomorrow”
–Antecedent: July the 3rd (2003)
–Referent: 07/04/2003
NLDB 2004 5
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
1. Introduction
2. Previous work
3. Description of the Event Ordering System
4. Application of Event Ordering in NLP tasks
5. System evaluation
6. Conclusions
Index
NLDB 2004 6
Previous work•Types of systems:
–Based on Machine Learning: A supervised
annotated corpus needed to automatically generate
the system rules (percentage of appearance).–High precision results with concrete domains
–Not very flexible, large annotated corpus
–Based on knowledge: Previous knowledge base
with rules to solve temporal expressions.–Greater flexibility
•Our system based on Spanish knowledge,
but this knowledge is automatically extended
using automatic acquisition of rules for new
languages
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 7
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
1. Introduction
2. Previous work
3. Description of the Event Ordering System
4. Application of Event Ordering in NLP tasks
5. System evaluation
6. Conclusions
Index
NLDB 2004 8
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
Graphic representation
TEMPORAL INFORMATION
DETECTION
EVENT ORDERING
DATEESTIMATION
Dictionary
TEMPORALEXPRESSION
COREFERENCERESOLUTION
T.E. TAGS
ORDEREDTEXT
Document
Temporal Expression Detection
Temporal Signal Detection
ORDERING KEY
OBTAINING
ORDERING KEYS
TEMPORAL EXPRESSIONS
TEMPORAL SIGNALS
NLDB 2004 9
•Detection of temporal information:–Temporal Expression Detection Unit
–Temporal Signal Detection Unit
•Temporal expressions are resolved by the
Temporal Expression Coreference Resolution
unit that generates the XML tags.
•Ordering key is obtained by the Ordering
Key unit
•With all this information, the Event
Ordering Unit orders the text.
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I Description of the Event Ordering system
NLDB 2004 10
•Detection of temporal information:–Temporal Expression Detection Unit
–Temporal Signal Detection Unit
•Both share a common pre-processing of
texts. Text are tagged with lexical and
morphological information by a PosTagger and
this information is the input of a temporal parser.
•The temporal parser is implemented using
and ascending technique and it is based on a
temporal grammar.
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I Description of the Event Ordering system
NLDB 2004 11
•One of the main tasks involved in trying to
recognize and resolve temporal expressions is to
classify them. A taxonomy with two different
classification of the temporal expressions has
been established:–Classification of the expression based on the kind
of reference
–Classification by the representation of the
temporal value of the expression
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
Temporal Expression Detection
NLDB 2004 12
•Classification of the expression based on the
kind of reference:–Explicit Temporal Expressions:
–Complete dates with or without time
exp:01/01/2003
–Dates of events: Christmas
–Implicit Temporal Expressions:
–Exp. that refer to the Document Date: yesterday
–Exp. that refer to another Date: a month later
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
Taxonomy of TE´s
NLDB 2004 13
Taxonomy of TE´s
•Classification by the representation of the
temporary value of the expression:–Concrete. Give back a concrete day or/and time
–Period. Give back a time interval.
–Fuzzy. Give back approximate time interval.
–Fuzzy concrete: a day of the last week
–Fuzzy period: some months before
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 14
Temporal Signal Detection
•Temporal signals:–Relate the different events in texts
–Establish a chronological order between these
events.
•Some examples of Temporal signals:–After
–Before
–During
–When
–Previously
–While
–At the time of
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 15
•Temporal Expression Coreference Resolution:–Anaphoric relation resolution based on a temporal
model
–Tagging of Temporal Expressions
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I Description of the Event Ordering system
NLDB 2004 16
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
• Looking for antecedents. – Two main candidates:
• Newspaper´s date (DateP),
• Date named before in the text (DateAnt).
– Proccess:
• By default, the newspaper´s date is used as a
base referent if it exists.
• If a non-anaphoric TE is found, this is stored as
DateAnt.
Anaphoric Relation Resolution
NLDB 2004 17
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
Anaphoric Relation Resolution
REFERENCE DICTIONARY ENTRY
‘ayer’ (yesterday)‘mañana’ (tomorrow)
DateP – 1DateP – 1
DateP +1DateP +1
‘durante el mes siguiente’ (during the following month)
[DayI/Month(DateAnt) +1/Year(DateAnt) -- DayF/Month(DateAnt)+1/Year(DateAnt)]
‘un día antes’ (a day before)
DateAnt-1
‘días después’ (some days later)
>>>>>DateAnt
NLDB 2004 18
Tagging of TEs
• Set of XML tags (eXtensible Markup Language).
Targets:
– Showing the results of our system
– Standarise the date-time formats of Internet texts.
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 19
Tagging of TEs
•Set of XML tags (eXtensible Markup Language).
•Explicit Dates< DATE_TIME ID =”value”
TYPE=”value”
VALDATE1=”value”
VALTIME1=”value”
VALDATE2=”value”
VALTIME2=”value” >
Expression
</DATE_TIME>
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 20
Tagging of TEs
•Implicit dates< DATE_TIME_REF ID =”value”
TYPE=”value”
VALDATE1=”value”
VALTIME1=”value”
VALDATE2=”value”
VALTIME2=”value” >
Expression
</DATE_TIME>
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 21
Ordering Keys Obtaining
• The study of the corpus revealed a set of
temporal signals.
• Each temporal signal denotes a relationship
between the dates of the events that it is
relating.
• Example: in EV1 S EV2, the signal S denotes a
relationship between EV1 and EV2. Assuming
that F1 is the date of EV1 and F2 the date of
EV2, S establish an order between EV1 and EV2.Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 22
Ordering Keys Obtaining
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
SIGNAL ORDERING KEY
After F1 > F2
When F1 = F2
Before F1 < F2
During F2i <= F1 <= F2f
Previously F1 > F2
On/in F1 = F2
While F2i <= F1 <= F2f
For F2i <= F1 <= F2f
NLDB 2004 23
Event ordering method• Building of a table with the complete information
from the XML tags
– This table includes the columns ID, VALDATE1,
VALTIME1, VALDATE2, VALTIME2 and VALORDER.
• Ordering rules:
– EV1 is previous to EV2, if the range associated with
TE1 is prior to and not overlapping the range
associated with TE2 or the ordering key is EV1<EV2
– EV1 is concurrent to EV2, if the range associated with
TE1 overlaps the range associated with TE2 or the
ordering key is EV1=EV2Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 24
System example
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
In December 1, the French bathyscaphe Nautilus arrives at the
Galician coast, previously there were some cracks.
Text
TEMPORAL INFORMATION
DETECTION
TEMPORALEXPRESSION
COREFERENCERESOLUTION
T.E. TAG: <DATE_TIME_REF
VALDATE1=“12/01/2002”>in December 1
</DATE_TIME_REF>
ORDERING KEY
OBTAINING
ORDERING KEY: event 1 > event 2
TEMPORAL EXPRESSION: In December 1
TEMPORAL SIGNAL: previously
EVENT ORDERING
Order Event Date
1 There were some cracks <<< 12/01/2002
2 The French bathyscaphe Nautilus arrives at the Galician Coast
12/01/2002
NLDB 2004 25
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
1. Introduction
2. Previous work
3. Description of the Event Ordering System
4. Application of Event Ordering in NLP tasks
5. System evaluation
6. Conclusions
Index
NLDB 2004 26
Application of Event Ordering in NLP tasks
• Applied in different tasks:
• Summarization
• Question Answering
• Etc.
• Temporal Question Answering can help current
QA system to answer complex questions.
Complex questions consist of two or more
events related with a temporal signal, which
establish the order between them.Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 27
Application in Question Answering
• Possible questions:
• When did Iraq invade Kuwait?
• When is the next New Hampshire Democratic
primary?
• Which US ship was attacked by Israeli forces during
the Six Day war in the sixties?
• Where did Bill Clinton study before going to Oxford
University?
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
NLDB 2004 28
Application in Question Answering
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
GENERAL PURPOSE QUESTION ANSWERING SYSTEMGENERAL PURPOSE QUESTION ANSWERING SYSTEM
TEMPORAL TEMPORAL Q. A. Q. A.
PROCESSINGPROCESSING
SCRIPT SCRIPT Q. A. Q. A.
PROCESSINGPROCESSING
TEMPLATE TEMPLATE Q. A. Q. A.
PROCESSINGPROCESSING . . . .
Complex Question
Simple Questions Simple Answers
Complex Answer
INTERFACEINTERFACE
Multilayered Question Answering Architecture
NLDB 2004 29
Example of Application in Question Answering
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
•Question: “Where did Bill Clinton study before
going to Oxford University?
•First of all, the unit recognizes the temporal
signal, which in this case is “before”
•Secondly, the complex question is divided:
• Q1: Where did Bill Clinton study?
• Q2: When did Bill Clinton go to Oxford University?
NLDB 2004 30
Example of Application in Question Answering
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
•Answers Q1:
• Georgetown University (1964-1968)
•Oxford University (1968-1970)
•Yale Law School (1970-1973)
•Answers Q2:
•1968
•Only Georgetown University fulfill the temporal
constrainst, so that is the answer to the complex
question.
NLDB 2004 31
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
1. Introduction
2. Previous work
3. Description of the Event Ordering System
4. Application of Event Ordering in NLP tasks
5. System evaluation
6. Conclusions
Index
NLDB 2004 32
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
• Corpus Spanish: Training (50 articles) and Test (50
articles)
• Kappa factor: measures the affinity in agreement
between a set of annotators when they make
categories judgments k=0.953
• Two measures– Precision: Num Successes / Num Treated Ref
– Recall: Num Successes / Num Real Ref
System evaluation
NLDB 2004 33
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
• The establishment of a correct order between the
events implies that the resolution is correct and
the events are placed on a timeline. For this
reason, an evaluation of the resolution of Temporal
Expressions has been made.
System evaluation
-EVENT 1: Jan. 1, 1967
-EVENT 2: a year later
-EVENT 3: two months before
EVENTS AND ITS TEMPORAL EXPRESSIONSEVENTS AND ITS TEMPORAL EXPRESSIONS
EV1EV1 EV3EV3 EV2EV2
01/01/196701/01/1967 01/01/196801/01/196810/01/196710/01/1967
NLDB 2004 34
System evaluation
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
SPANISH
TRAINING TEST
Num. Art. 50 50
Real Ref. 238 199
Treated Ref. 201 156
Successes 170 138
Precision 84.58% 88.46%
Recall 71.43% 69.35%
NLDB 2004 35
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
• Expressions like “el sábado hubo cinco accidentes”
(Saturday there were five accidents) need context
information of the sentence where the reference is,
in this case, the time of the sentence´s verb. Our
system does not use this information.
• There is not a world knowledge database, for
instance: “two days before the Iraqi war”. We don
´t have this information nowadays.
System evaluation
NLDB 2004 36
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
1. Introduction
2. Previous work
3. Description of the Event Ordering System
4. Application of Event Ordering in NLP tasks
5. System evaluation
6. Conclusions
Index
NLDB 2004 37
• Obtaining facts related to an event from a
Documental Database Chronology.
• System:1. Title of the news linked to the date of the
documents
2. Recognition of temporal expressions. Events
sentences with TE
3. Module for treating TE is applied
4. The ordering module tags the order of the events
in the text
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
Conclusions
NLDB 2004 38
• Application in Temporal Question Answering:
Decomposition of complex temporal
questions in simple ones.
• Future work:
• Cope with context information and world
knowledge
• Multilingual evaluation of the system
Rese
arc
h G
roup o
n L
anguage P
roce
ssin
g a
nd Info
rmati
on
Syst
em
sg
g P
LS
IP
LS
I
Conclusions
Event Ordering using TERSEO Event Ordering using TERSEO systemsystem
Research Group on Research Group on Language Processing and Information SystemsLanguage Processing and Information Systems g g
PLSIPLSI
Estela Saquete Boró, Rafael Muñoz, Patricio Martinez-Barco
Departamento de Lenguajes y Sistemas Informáticos
Recommended