Embed Size (px)
Citation preview
Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from
Sparse Dyadic Event Counts
KDD 2015
Aaron Schein UMass Amherst
Joint work with: John Paisley, Dave Blei & Hanna Wallach Columbia University Microsoft Research
International relations are multilateral
From Schrodt (1993) Event Data in Foreign Policy Analysis:
From Schrodt (1993) Event Data in Foreign Policy Analysis:
From Schrodt (1993) Event Data in Foreign Policy Analysis:
From Schrodt (1993) Event Data in Foreign Policy Analysis:
From Schrodt (1993) Event Data in Foreign Policy Analysis:
From Schrodt (1993) Event Data in Foreign Policy Analysis:
From Schrodt (1993) Event Data in Foreign Policy Analysis:
From Schrodt (1993) Event Data in Foreign Policy Analysis:
From Schrodt (1993) Event Data in Foreign Policy Analysis:
A coherent thread of international events.
Multilateral relation
A coherent thread of international events.
Multilateral relation
Characterized by:
1. sender countries 2. receiver countries 3. action types 4. time steps
A coherent thread of international events.
Multilateral relation
Characterized by:
1. sender countries 2. receiver countries 3. action types 4. time steps
A coherent thread of international events.
Multilateral relation
Characterized by:
1. sender countries 2. receiver countries 3. action types 4. time steps
Who is doing what to whom, when?
Tease apart the coherent threads of:
Goal: Infer multilateral relations
Tease apart the coherent threads of:
Goal: Infer multilateral relations
(millions of unstructured dyadic events)
Sample result:
time steps
senders receivers action types
Sample result: Afghanistan War
time steps
senders receivers action types
Uni
ted
Sta
tes
Rus
sia
Isra
elIraq
Chi
naU
nite
dK
ingd
om Iran
Pal
estine
Fran
ceJa
pan
Eur
ope
Pak
ista
nGer
man
yTur
key
Afg
hani
stan
Egy
ptAfric
aSyr
iaLe
bano
nN
orth
Kor
eaSou
thK
orea
Aus
tral
iaJo
rdan
Ital
ySau
diAra
bia
Can
ada
Indo
nesia
Nig
eria
Sud
anSpa
inPhi
lippi
nes
Ukr
aine
Ken
yaN
AT
OPol
and
PolandNATOKenya
UkrainePhilippines
SpainSudan
NigeriaIndonesia
CanadaSaudi Arabia
ItalyJordan
AustraliaSouth KoreaNorth Korea
LebanonSyria
AfricaEgypt
AfghanistanTurkey
GermanyPakistanEuropeJapanFrance
PalestineIran
United KingdomChina
IraqIsrael
RussiaUnited States
Uni
ted
Sta
tes
Rus
sia
Isra
elIraq
Chi
naU
nite
dK
ingd
om Iran
Pal
estine
Fran
ceJa
pan
Eur
ope
Pak
ista
nGer
man
yTur
key
Afg
hani
stan
Egy
ptAfric
aSyr
iaLe
bano
nN
orth
Kor
eaSou
thK
orea
Aus
tral
iaJo
rdan
Ital
ySau
diAra
bia
Can
ada
Indo
nesia
Nig
eria
Sud
anSpa
inPhi
lippi
nes
Ukr
aine
Ken
yaN
AT
OPol
and
July 2002
Express intent to cooperate Threaten
Sparse event counts
Uni
ted
Sta
tes
Rus
sia
Isra
elIraq
Chi
naU
nite
dK
ingd
om Iran
Pal
estine
Fran
ceJa
pan
Eur
ope
Pak
ista
nGer
man
yTur
key
Afg
hani
stan
Egy
ptAfric
aSyr
iaLe
bano
nN
orth
Kor
eaSou
thK
orea
Aus
tral
iaJo
rdan
Ital
ySau
diAra
bia
Can
ada
Indo
nesia
Nig
eria
Sud
anSpa
inPhi
lippi
nes
Ukr
aine
Ken
yaN
AT
OPol
and
PolandNATOKenya
UkrainePhilippines
SpainSudan
NigeriaIndonesia
CanadaSaudi Arabia
ItalyJordan
AustraliaSouth KoreaNorth Korea
LebanonSyria
AfricaEgypt
AfghanistanTurkey
GermanyPakistanEuropeJapanFrance
PalestineIran
United KingdomChina
IraqIsrael
RussiaUnited States
Uni
ted
Sta
tes
Rus
sia
Isra
elIraq
Chi
naU
nite
dK
ingd
om Iran
Pal
estine
Fran
ceJa
pan
Eur
ope
Pak
ista
nGer
man
yTur
key
Afg
hani
stan
Egy
ptAfric
aSyr
iaLe
bano
nN
orth
Kor
eaSou
thK
orea
Aus
tral
iaJo
rdan
Ital
ySau
diAra
bia
Can
ada
Indo
nesia
Nig
eria
Sud
anSpa
inPhi
lippi
nes
Ukr
aine
Ken
yaN
AT
OPol
and
July 2002
Express intent to cooperate Threaten
Sparse event counts
Poisson matrix factorization
Australia
New Zealand
China
Vietnam
Cambodia
Laos
Aus
tral
ia
New
Zea
land
Chin
a
Vie
tnam
Cam
bodi
a
Laos
Poisson matrix factorization
Australia
New Zealand
China
Vietnam
Cambodia
Laos
Aus
tral
ia
New
Zea
land
Chin
a
Vie
tnam
Cam
bodi
a
Laos
Poisson matrix factorization
Australia
New Zealand
China
Vietnam
Cambodia
Laos
Aus
tral
ia
New
Zea
land
Chin
a
Vie
tnam
Cam
bodi
a
Laos
Poisson matrix factorization
Australia
New Zealand
China
Vietnam
Cambodia
Laos
Aus
tral
ia
New
Zea
land
Chin
a
Vie
tnam
Cam
bodi
a
Laos
Poisson matrix factorization
Australia
New Zealand
China
Vietnam
Cambodia
Laos
Aus
tral
ia
New
Zea
land
Chin
a
Vie
tnam
Cam
bodi
a
Laos
Poisson matrix factorization
N
N
Y=
Poisson matrix factorization
N
N
(1) (2)yij ⇠ Poisson
� �X
k
✓ik ✓jk
Y=
Poisson matrix factorization
N
N
how active country is as a sender in community
i
k
(1) (2)yij ⇠ Poisson
� �X
k
✓ik ✓jk
Y=
Poisson matrix factorization
N
N
how active country is as a sender in community
i
k
(1) (2)yij ⇠ Poisson
� �X
k
✓ik ✓jk
jhow active country is as a receiver in community k
Y=
Poisson matrix factorization� �Y ⇠ Pois
⇥(1)
⇥(2)
Poisson matrix factorization� �Y ⇠ Pois
⇥(1)
⇥(2)
Y(1)
⇥ 2 R+⇥KN
(2)⇥ 2 R+
N⇥K
Fitting this model is a form of nonnegative matrix factorization:
Poisson matrix factorization
Australia
New Zealand
China
Vietnam
Cambodia
Laos
Aus
tral
ia
New
Zea
land
Chin
a
Vie
tnam
Cam
bodi
a
Laos
component 1
✓(2)1
✓(1)1
Poisson tensor factorization
(1) (2)Poisson
X
k
✓ik ✓jk ✓ak ✓tk
!yijat ⇠
(3) (4)
Poisson tensor factorization
(1) (2)Poisson
X
k
✓ik ✓jk ✓ak ✓tk
!yijat ⇠
(3) (4)
how active country is as a sender in
multilateral relation
i
k
jhow active country is as a receiver in
multilateral relation k
Poisson tensor factorization
(1) (2)Poisson
X
k
✓ik ✓jk ✓ak ✓tk
!yijat ⇠
(3) (4)
how active country is as a sender in
multilateral relation
i
k
jhow active country is as a receiver in
multilateral relation k
a
k
how relevant is action type to multilateral relation
Poisson tensor factorization
(1) (2)Poisson
X
k
✓ik ✓jk ✓ak ✓tk
!yijat ⇠
(3) (4)
how active country is as a sender in
multilateral relation
i
k
jhow active country is as a receiver in
multilateral relation k
a
k
how relevant is action type to multilateral relation
how active at time step is multilateral relation k
t
Poisson tensor factorization
(1) (2)Poisson
X
k
✓ik ✓jk ✓ak ✓tk
!yijat ⇠
(3) (4)
Poisson tensor factorization
(1) (2)Poisson
X
k
✓ik ✓jk ✓ak ✓tk
!yijat ⇠
(3) (4)
Fitting this model is a form of nonnegative tensor factorization:
Poisson tensor factorization
(1) (2)Poisson
X
k
✓ik ✓jk ✓ak ✓tk
!yijat ⇠
(3) (4)
Fitting this model is a form of nonnegative tensor factorization:
Y
(1)⇥ 2 R+
⇥KN sender factors
(2)⇥ 2 R+
N⇥Kreceiver factors
(3)⇥ 2 R+
A⇥Kaction type factors
(4)⇥ 2 R+T ⇥K
time step factors
Sample component
time-steps
senders receivers action types
Sample component: Yugoslav Wars
Components correspond to multilateral relations
yijat ⇠ Poisson
X
k
✓(1)ik ✓(2)jk ✓(3)ak ✓(4)tk
!
Parameter estimation: Bayesian inference
yijat ⇠ Poisson
X
k
✓(1)ik ✓(2)jk ✓(3)ak ✓(4)tk
!
✓(1)ik ⇠ Gamma⇣↵, ↵�(1)
⌘
✓(2)jk ⇠ Gamma⇣↵, ↵�(2)
⌘
✓(3)ak ⇠ Gamma⇣↵, ↵�(3)
⌘
✓(4)tk ⇠ Gamma⇣↵, ↵�(4)
⌘
Parameter estimation: Bayesian inference
yijat ⇠ Poisson
X
k
✓(1)ik ✓(2)jk ✓(3)ak ✓(4)tk
!
✓(1)ik ⇠ Gamma⇣↵, ↵�(1)
⌘
✓(2)jk ⇠ Gamma⇣↵, ↵�(2)
⌘
✓(3)ak ⇠ Gamma⇣↵, ↵�(3)
⌘
✓(4)tk ⇠ Gamma⇣↵, ↵�(4)
⌘
Parameter estimation: Bayesian inference
Sparsity-inducing Gamma priors} ↵ < 1
P⇣⇥(1),⇥(2),⇥(3),⇥(4) | Y
⌘
Parameter estimation: Bayesian inference
P⇣⇥(1),⇥(2),⇥(3),⇥(4) | Y
⌘cannot compute this analytically
Parameter estimation: Bayesian inference
P⇣⇥(1),⇥(2),⇥(3),⇥(4) | Y
⌘
Parameter estimation: Variational inference
P⇣⇥(1),⇥(2),⇥(3),⇥(4) | Y
⌘cannot compute this analytically
Parameter estimation: Variational inference
P⇣⇥(1),⇥(2),⇥(3),⇥(4) | Y
⌘cannot compute this analytically
Q⇣⇥(1),⇥(2),⇥(3),⇥(4)
⌘Define a convenient family of distributions:
Parameter estimation: Variational inference
P⇣⇥(1),⇥(2),⇥(3),⇥(4) | Y
⌘cannot compute this analytically
Q⇣⇥(1),⇥(2),⇥(3),⇥(4)
⌘Define a convenient family of distributions:
Parameter estimation: Variational inference
Optimize the parameters of to minimize:Q
KL(Q||P )
Parameter estimation: Variational inference
⇥(m)
✓̂(m)ik
Parameter estimation: Variational inference
Q(✓(m)ik )
Parameter estimation: Variational inference
Q(✓(m)ik ) = Gamma
⇣✓(m)ik ; �(m)
ik , �(m)ik
⌘
Parameter estimation: Variational inference
Q(✓(m)ik ) = Gamma
⇣✓(m)ik ; �(m)
ik , �(m)ik
⌘
variational parameters
{
Parameter estimation: Variational inference
Q(✓(m)ik ) = Gamma
⇣✓(m)ik ; �(m)
ik , �(m)ik
⌘
After inference, if we need a point estimate:
EQ[✓(m)ik ] =
�(m)ik
�(m)ik
✓̂(m)ik :=
Parameter estimation: Variational inference
Q(✓(m)ik ) = Gamma
⇣✓(m)ik ; �(m)
ik , �(m)ik
⌘
After inference, if we need a point estimate:
✓̂(m)ik := GQ[✓
(m)ik ] =
exp( (�(m)ik ))
�(m)ik
Parameter estimation: Variational inference
Parameter estimation: Variational inference
Parameter estimation: Variational inference
Bayesian PTF generalizes much better than maximum likelihood PTF when the data is very sparse!
Parameter estimation: Variational inference
Bayesian PTF generalizes much better than maximum likelihood PTF when the data is very sparse!
No sacrifice in efficiency!
Code and more sample results available: https://github.com/aschein/bptf
Jan 2011 Mar 2011 May 2011 Jun 2011 Aug 2011 Sep 2011 Nov 2011 Jan 2012 Feb 2012 Apr 2012 May 2012 Jul 2012 Sep 2012 Oct 2012 Dec 20120.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
Ecu
ador
Uni
ted
Kin
gdom
Uni
ted
Stat
es
Swed
en
Aus
tral
ia
Spai
n
Rus
sia
Japa
n
Belar
us
Cub
a
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
Ecu
ador
Uni
ted
Kin
gdom
Uni
ted
Stat
es
Swed
en
Aus
tral
ia
Rus
sia
Belar
us
Spai
n
Par
agua
y
Eur
ope
Con
sult
Aid
App
eal
Mak
eSt
atem
ent
Coe
rce
Disap
prov
eIn
tend
toCoo
pera
teCoo
pera
te(D
iplo
mat
ic)
Rejec
tCoo
pera
te(M
ater
ial)
time steps
senders
receivers
action types
Our favorite component
Jan 2011 Mar 2011 May 2011 Jun 2011 Aug 2011 Sep 2011 Nov 2011 Jan 2012 Feb 2012 Apr 2012 May 2012 Jul 2012 Sep 2012 Oct 2012 Dec 20120.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
Ecu
ador
Uni
ted
Kin
gdom
Uni
ted
Stat
es
Swed
en
Aus
tral
ia
Spai
n
Rus
sia
Japa
n
Belar
us
Cub
a
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
Ecu
ador
Uni
ted
Kin
gdom
Uni
ted
Stat
es
Swed
en
Aus
tral
ia
Rus
sia
Belar
us
Spai
n
Par
agua
y
Eur
ope
Con
sult
Aid
App
eal
Mak
eSt
atem
ent
Coe
rce
Disap
prov
eIn
tend
toCoo
pera
teCoo
pera
te(D
iplo
mat
ic)
Rejec
tCoo
pera
te(M
ater
ial)
time steps
senders
receivers
action types
Our favorite component
Jan 2011 Mar 2011 May 2011 Jun 2011 Aug 2011 Sep 2011 Nov 2011 Jan 2012 Feb 2012 Apr 2012 May 2012 Jul 2012 Sep 2012 Oct 2012 Dec 20120.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
Ecu
ador
Uni
ted
Kin
gdom
Uni
ted
Stat
es
Swed
en
Aus
tral
ia
Spai
n
Rus
sia
Japa
n
Belar
us
Cub
a
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
Ecu
ador
Uni
ted
Kin
gdom
Uni
ted
Stat
es
Swed
en
Aus
tral
ia
Rus
sia
Belar
us
Spai
n
Par
agua
y
Eur
ope
Con
sult
Aid
App
eal
Mak
eSt
atem
ent
Coe
rce
Disap
prov
eIn
tend
toCoo
pera
teCoo
pera
te(D
iplo
mat
ic)
Rejec
tCoo
pera
te(M
ater
ial)
time steps
senders
receivers
action types
Our favorite component
Thanks!
Come to poster 897
