Embed Size (px)
Citation preview
Lineariza(onofBeliefPropaga(ononPairwiseMarkovRandomFields
Nodelabelingoverlargegraphsisimportantforapplica5onssuchasfrauddetec5oningraphsone-commercetransac5onsornodeclassifica5oninsocialnetworksBeliefPropaga5on(BP)isanitera5vemessage-passingalgorithmforinferenceinMarkovRandomFields(MRFs)thatisoBenusedfornodelabeling
MOTIVATION&PROBLEMFORMULATION
OurapproachsimplifiestheproblemofapplyingBeliefpropaga(ontonodelabeling
PAIRWISELinBPINANUTSHELL
OURAPPROACH
THEORETICALRESULTS
Q1:Howaccurateisourapproxima(on,andunderwhichcondi(onsisitreasonable?
• Thelineariza5ongivescomparablelabelingaccuracyasBPforgraphswithweakpoten5als.
• Theaccuracydeteriorateswithdensernetworksandstrongpoten5als.
EXPERIMENTS&RESULTS
• LinearizedBPwithconvergenceguaranteesandclosed-formsolu5onforarbitrarypairwiseMRFsthatcanbesolvedwithitera5veupdates.
• Compellingcomputa5onaladvantageandverysimplecode• SourcecodeavailableathRps://github.com/sslh/sslh
CONCLUSIONS&FUTUREWORK
BP FaBP LinBP this work
# node types arbitrary 1 1 arbitrary
# node classes arbitrary 2 const k arbitrary
# edge types arbitrary 1 1 arbitrary
edge symmetry arbitrary required required arbitrary
edge potential arbitrary doubly stoch. doubly stoch. arbitrary
closed form no yes yes yes
Q3:Howfastisthelinearizedapproxima(onascomparedtoBP?
• Thelineariza5onisaround1005mesfasterthanBPperitera5onandoBenneeds105mesfeweritera5onsun5lconvergence.
• Inprac5ce,thiscanleadtoaspeed-upof10005mes.ExperimentsareruninPython.
Applyingthisideatoarbitrarypoten5alsismorecomplicated• Hardnessarisesduetodifferentre-centeringinboththedirec5onsofanedge
• Beliefvectorsdonotremainstochas5canymore.
r(j)
j=1 -0.04 -0.02 -0.06
j=2 -0.01 0.01 0
j=3 0.02 0.04 0.06
c(i) -0.03 0.03 s=0
i=1 i=2 r(j)
j=1 0.96 0.98 1.94
j=2 0.99 1.01 2
j=3 1.02 1.04 2.06
c(i) 2.97 3.03 s=6
i=1 i=2 r(j)0
0.495 0.505 1
0.495 0.505 1
0.495 0.505 1
c(i)0 1.485 1.515 3
i=1 i=2 r(j)00
0.323 0.323 0.646
0.333 0.333 0.666
0.343 0.343 0.686
c(i)00 1 1 2
r(j)00
-0.01 -0.01 -0.02
0 0 0
0.01 0.01 0.02
c(i)00 0 0 0
r(j)0
-0.005 0.005 0
-0.005 0.005 0
-0.005 0.005 0
c(i)0 -0.015 0.015 0
centering(around-1(
row-recentering(
column-recentering(
un2centering(around-½(
un2centering(around-⅓(
0 00
0
00
FaBP:Koutraetal.Unifyingguilt-by-associa5onapproaches:Theoremsandfastalgorithms.ECML/PKDD2011LinBP:GaEerbaueretal.Linearizedandsingle-passbeliefpropaga5on.PVLDB2015
s! t!
st!m!
s! t!ks!
kt!
ks!
kt!
ts!m!
=! ! = ! |ts st
s!y! t!y! s!y! t!y!
• Butconvergenceisnotguaranteedinloopygraphs
• BPneedsalotoffine-tuningtomakeitconvergerge
ThisworkderivesanapproachtoapproximateloopyBPonanypairwiseMRFbylinearizingtheupdateequa(onsandgivingexactconvergenceguarantees
Input:• x_s:priorlabeldistribu5onfornodes• A:(nxn)adjacencymatrixbetweennodes.• :poten5alsonedges-t
h=2,includematrixH(h) d=5and10,f=0.05
Q2:Howpredictableisourformula(onascomparedtoBP?
• Ourapproxima5oncanalwaysbemadetoconvergebyscalingthepoten5alswithaknownscalingfactor.
• Incontrast,loopyBPsome5mesdoesneedsdampingandscalingwithanunknownfactor.
• ABerre-centering,theMaxMarginalsolu5onforapar5allylabeledMRFcanbeapproximatetbythesolu5onoffollowingequa5onsystem
• Thesolu5oncanbefoundquicklywithitera5veupdatesandfollowingconvergencecondi5on:
Importantprac(caltake-away:Theseupdateequa(onscomewithexactconvergenceguaranteesandcanbeimplementedveryefficientlywithexis(nglinearalgebrapackages
Residual
0.50.5 =
+
0.60.4
0.360.16
Σ=0.50
��Center 0.50.5
0.250.25
0.50.5
Value 0.60.4
0.1-0.1
0.2-0.2
0.1-0.1
�0.690.31
0.70.3
Σ=0.52
=
= ⇒Σ=0
�
Σ=1
Σ=1
Σ=1
KeyIdea:Bystar5ngwithmessagesandpoten5alsthatareappropriatelyrecenteredaround1andhavesmallstandarddevia5ons,theresul5ngequa5onsdonotrequirefurthernormaliza5on.
LinearizedsystemthathascomparablelabelingaccuracyasBPforgraphswithweakpoten5als,whilespeeding-upinferencebyordersofmagnitude.• Theformalismcanmodelarbitraryheterogeneousnetworks• Itcomeswithexactconvergenceguaranteesandafastmatriximplementa5on• Itera5veupdatesusinglinearequa5ons• Thecodelibraryisavailableongithubandisveryeasytouse
Output:• y_s:posteriorlabeldistribu5onforeachnodes
Wereplacemul(plica(onwithaddi(onandnormaliza(onisnotnecessaryanymore
Approx.
st
= + + −
321
1
2
3 03⇤
02⇤
01⇤
0|12
0|21
0|23
0|32
b1
b2
b3
1 2
=3
b2 b3b1
= 23 12 y1y2y3
1 2
=3
b2 b3b1
= 23 12
1 2
=3
b2 b3b1
= 23 12
1 2
=3
b2 b3b1
= 23 12
y1 y2 y3
⇢⇣
0�
02⌘ 1 ⇢ : spectral radius of matrix
RowCentering:Centereachelementofarowwiththeaverageofitsrow-sum.Formally,
SimilarlyColumnCentering.
0(j, i) = 1k
⇣ (j, i)� r(j)
k
⌘
where,r(j) =
Pi (j, i)
y(t+1) �x+ c
0⇤�+
�
0�
02�y(t)
ExactEqua5on
Itera5veUpdates
y = x+ 0Tk+
0Ty �
0T 2
y
