Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
Chains of Reasoning over Entities, Relations, and Text using
Recurrent Neural Networks
Rajarshi Das, Arvind Neelakantan, David Belanger, Andrew McCallum
Presentation in Deep Learning Summer School,
Montreal Aug 4 2016
2
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
2
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Steve Balmer
Microsoft
Bill Gates
MicrosoftSteve Balmer
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve Balmer
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friendtake over as
CEO
4
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friend
take over as
CEO
Open IE textual relation types (Banko et al, 2007)
5
Universal Schema for Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friend
take over as
CEO
Open IE textual relation types (Banko et al, 2007)
Riedel, Yao, McCallum, Marlin, 2013
Union of all relation types from all structured sources and text
5
Universal Schema for Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friend
take over as
CEO
Open IE textual relation types (Banko et al, 2007)
Union of all relation types from all structured sources and text
5
Universal Schema for Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friend
take over as
CEO
Open IE textual relation types (Banko et al, 2007)
Union of all relation types from all structured sources and text
Predict relation between entities based on mentions in text
Chains of Reasoning for Relation Extraction
6
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
headquartered
Chains of Reasoning for Relation Extraction
7
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
headquarteredBill
MicrosoftSeattle
Chains of Reasoning for Relation Extraction
7
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.
Bill
MicrosoftSeattle
Chains of Reasoning for Relation Extraction
7
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.
Bill
MicrosoftSeattle
Melinda
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Symbolic; Doesn't generalize
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Spouse(A,B) & CEO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)Symbolic;
Doesn't generalize
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Spouse(A,B) & CEO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)Spouse(A,B) & COO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
Symbolic; Doesn't generalize
Chains of Reasoning for Relation Extraction
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Spouse(A,B) & CEO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)Spouse(A,B) & COO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
Symbolic; Doesn't generalize
Chains of Reasoning for Relation Extraction
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
Vector representation of the path
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metriclives in??
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metriclives in??
0.94
RNNs for Chains of Reasoning
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metriclives in??
0.94
RNNs for Chains of ReasoningLogical Inference
inVector space!
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metriclives in??
0.94
Neelakantan et al’ 15
RNNs for Chains of ReasoningLogical Inference
inVector space!
10
Contributions
10
Contributions
1. Single RNN for chains of reasoning.
10
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
10
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
10
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
%MAP 64.43% 73.26%
10
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
%MAP 64.43% 73.26%13.7%
11
1. Single RNN for chains of reasoning.
11
1. Single RNN for chains of reasoning.
lives in??
11
1. Single RNN for chains of reasoning.
lives in?? works in??
11
1. Single RNN for chains of reasoning.
lives in?? works in?? shops in??
12
/ /lives in?? works in?? shops in?? /….
1. Single RNN for chains of reasoning.
12
1. Single RNN for chains of reasoning.
12
1. Single RNN for chains of reasoning.
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metric
0.94
12
1. Single RNN for chains of reasoning.
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metric
0.94
Shared representations for relations
{
Single Composition Matrix for all query
relations
12
1. Single RNN for chains of reasoning.
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metric
0.94
Shared representations for relations
{
Single Composition Matrix for all query
relations
{All query relation vectors
lives in??
shops in??
works in??
Results
13
Results
Model %MAP
PRA 64.43
PRA + Bigram 64.93
13
Results
Model %MAP
PRA 64.43
PRA + Bigram 64.93
RNN-Path(Neelakantan’ 15) 65.23
13
Results
Model %MAP
PRA 64.43
PRA + Bigram 64.93
RNN-Path(Neelakantan’ 15) 65.23
Single Model 68.77
13
Results
Model %MAP
PRA 64.43
PRA + Bigram 64.93
5%
p < 5e-6
RNN-Path(Neelakantan’ 15) 65.23
Single Model 68.77
13
14
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
15
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
15
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
15
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
15
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
16
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
16
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
16
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
16
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
16
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
lives in??
16
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
similarity metric
0.96 0.94 0.18
lives in??
17
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
0.96
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
0.69
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
3. Top(k): 1
k
kX
i=1
si
!
Top(2)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
3. Top(k): 1
k
kX
i=1
si
!
Top(2)
0.95
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
LSE
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
LSE
1.85
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
LSE
1.85
@LSE
@↵i=
exp(↵i)Pi exp(↵i)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
LSE
1.85
@LSE
@↵i=
exp(↵i)Pi exp(↵i)
Gradient Steps #1040 0.5 1 1.5 2
Trai
ning
Los
s
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2LogSumExpMaxMeanTop-k
Results
20
Results
Model %MAP Pooling
RNN-Path(Neelakantan ’15) 65.23 Max
20
Results
Model %MAP Pooling
RNN-Path(Neelakantan ’15) 65.23 Max
RNN-Path 68.43 LogSumExp
20
Results
Model %MAP Pooling
RNN-Path(Neelakantan ’15) 65.23 Max
RNN-Path 68.43 LogSumExp
Single Model 68.77 Max
Single Model 55.8 Avg.
Single Model 68.20 Top(K)
20
Results
Model %MAP Pooling
RNN-Path(Neelakantan ’15) 65.23 Max
RNN-Path 68.43 LogSumExp
Single Model 68.77 Max
Single Model 55.8 Avg.
Single Model 68.20 Top(K)
Single Model 70.11 LogSumExp20
21
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
JFK NYC NY
located in located in
airport_serves_location??
JFK NYC NY
located in located in
airport_serves_location??
0.9276~
JFK NYC NY
located in located in
airport_serves_location??
0.9276~
Yankee Stadium NYC NY
located in located in
airport_serves_location??
JFK NYC NY
located in located in
airport_serves_location??
0.9276~
Yankee Stadium NYC NY
located in located in
airport_serves_location??
~ 0.9276
JFK NYC NY
located in located in
airport_serves_location??
0.9276~
Yankee Stadium NYC NY
located in located in
airport_serves_location??
~ 0.9276
SeattleMelinda Bill Microsoft headquarteredchairmanspouse
Entity Representation
SeattleMelinda Bill Microsoft headquarteredchairmanspouse
Entity Representation
SeattleMelinda Bill Microsoft
Entity Representation
SeattleMelinda Bill Microsoft
Entity Representation1. Learn separate representation for each entity.
SeattleMelinda Bill Microsoft
Entity Representation1. Learn separate representation for each entity.
Jane Doe
SeattleMelinda Bill Microsoft
Entity Representation1. Learn separate representation for each entity.
Jane Doe
rare occurrence; hard to learn good representations.
SeattleMelinda Bill Microsoft
Entity Representation
2. Represent entities by their annotated types 1. Learn separate representation for each entity.
Jane Doe
rare occurrence; hard to learn good representations.
SeattleMelinda Bill Microsoft
Entity Representation
2. Represent entities by their annotated types 1. Learn separate representation for each entity.
Jane Doe
Annotated types in KB
1. CEO2. Philanthropist3. Duke University alumni4. American Citizen
1. American Citizen2. Small business owner
Jane DoeMelinda
rare occurrence; hard to learn good representations.
Melinda
Entity Representation
Jane Doe
1. CEO 2. Philanthropist 3. Duke University alumni 4. American Citizen
1. American Citizen 2. Small business owner
+ + +
+
=
=
Melinda
Entity Representation
Jane Doe
1. CEO 2. Philanthropist 3. Duke University alumni 4. American Citizen
1. American Citizen 2. Small business owner
+ + +
+
=
=
Entity is represented as the sum of their annotated types
Entity Aware RNNs for Chains of Reasoning
RNN RNN RNN RNN
similarity metriccountryofHQ??
0.94
SeattleMelinda Bill Microsoftchairmanspouse headquartered dummy_rel
Results
Results
Model %MAP
Single Model 70.11
Results
Model %MAP
Single Model 70.11
Single Model + Entity 71.74
Results
Model %MAP
Single Model 70.11
Single Model + Entity 71.74
Single Model + Types 73.26
Results
4.4%
p < 5e-6
Model %MAP
Single Model 70.11
Single Model + Entity 71.74
Single Model + Types 73.26
Results
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
Single Model 68.77 Max
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
Single Model 68.77 Max
Single Model 70.11 LogSumExp
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
Single Model 68.77 Max
Single Model 70.11 LogSumExp
Single Model + Types 73.26 LogSumExp
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
13.7% Single Model 68.77 Max
Single Model 70.11 LogSumExp
Single Model + Types 73.26 LogSumExp
Predictive Clauses
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
Universal Schema
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Entity Aware Model
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Sandy_Lake_Airport Sandy_Lake_First_Nation
/aviation/airport/serves?? Entity Aware Model
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Sandy_Lake_Airport/location/contains
Ontario‘(in northwestern)’-1
Sandy_Lake_First_Nation
Sandy_Lake_Airport Sandy_Lake_First_Nation
/aviation/airport/serves?? Entity Aware Model
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Sandy_Lake_Airport Sandy_Lake_First_Nation
/aviation/airport/serves?? Entity Aware Model
29
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Sandy_Lake_Airport/location/contains
Ontario‘(and at the)’
Toronto_Royal_Music_Hall
’including the’
Canada
/geography/lake/basin_countries
Big Trout Lakeand toSandy_La
ke_First_Nation
Sandy_Lake_Airport Sandy_Lake_First_Nation
/aviation/airport/serves?? Entity Aware Model
29
Conclusion
30
Conclusion
30
• Introduced a high-capacity single RNN model for relation extraction
Conclusion
30
• Introduced a high-capacity single RNN model for relation extraction
• Combines evidences among multiple paths and sources of evidence
Conclusion
30
• Introduced a high-capacity single RNN model for relation extraction
• Combines evidences among multiple paths and sources of evidence
• Entity Aware!
Conclusion
30
• Introduced a high-capacity single RNN model for relation extraction
• Combines evidences among multiple paths and sources of evidence
• Entity Aware!
Thanks!