THE POWER OF CERTAINTY - Carnegie Mellon …deswaran/talks/sdm17-netconf-talk.pdf · the power of...

A DIRICHLET-MULTINOMIAL APPROACH TO BELIEF PROPAGATION

THE POWER OF CERTAINTY

Dhivya Eswaran* CMU

deswaran@cs.cmu.edu

Stephan Guennemann TUM

guennemann@in.tum.de

Christos Faloutsos CMU

christos@cs.cmu.edu

PROBLEM

MOTIVATION

NETCONF

GUARANTEES

EXPERIMENTSEswaran,  Guennemann  &  Faloutsos

PROBLEM

ADS PLACEMENT

ALICE

BOB

CAROL

SMITH

RECOMMENDATION

ALICE

BOB

CAROL

SMITH

JOHN

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

GRAPH LABELING / NODE CLASSIFICATIONEXPERIMENTS

4

GIVEN

‣ a graph of nodes & edges

‣ labels for a few nodes

‣ label compatibility

FIND

‣ labels of all nodes

ALICE BOB

CAROL

SMITH

JOHN

H0.8 0.2

0.2 0.8

H0.2 0.7 0.1

0.7 0.2 0.1

0.1 0.1 0.8

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

NODE CLASSIFICATION IS HARDEXPERIMENTS

5

SMITH

4 x

1 x

JOHN

30 x

15 x

Who is more likely to buy an Android phone?

“Higher fraction of android friends”

“Higher number of android friends”

PROBLEM

MOTIVATION

NETCONF

GUARANTEES

EXPERIMENTSEswaran,  Guennemann  &  Faloutsos

MOTIVATION

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

CLASSIFICATION BY “PROPAGATION”EXPERIMENTS

7

INITIALIZE:

‣ Set nodes to random/given values.

PROPAGATE:

‣ Update each node’s value based on the values of its neighbors.

CONVERGENCE:

‣ If no value changes, terminate.

‣ Else continue propagation.

SMITH

ALICE

BOB

CAROL

Q1. What are values here?

Q2. How are they updated?

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

“BELIEF” PROPAGATIONEXPERIMENTS

8

SMITH

ALICE

BOB

CAROL

A1. Values : beliefs

A2. Update : 2 stages

(probability vectors)

(i) Each neighbor sends a message

(ii) Node updates its belief based on messages

[0.4, 0.6]

[0.5, 0.5]

[0.8, 0.2]

[0.73, 0.27] bu(i) eu(i)Y

v2N (u)

mvu(i)

mvu(i) kX

j=1

H(i, j)ev(j)Y

v2N (v)\u

mwu(i)

DETAILS!

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

BP LEADS TO COUNTER-INTUITIVE RESULTSEXPERIMENTS

9

Who is more likely to buy an Android phone?

110 x

100 xJOHN

13 x

3 xSMITH

INTUITION

BP

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

MAIN IDEAEXPERIMENTS

10

“belief distributions” as values

10.5

prob

abilit

y

0

belief / leaning(ratio of android : apple neighbors)

certainty / confidence

(absolute count of

neighbors)

PROBLEM

MOTIVATION

NETCONF

GUARANTEES

EXPERIMENTSEswaran,  Guennemann  &  Faloutsos

NETCONF

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

NODE CLASSIFICATION WITH CERTAINTYEXPERIMENTS

12

GIVEN

‣ a graph of nodes & edges

‣ belief distributions for a few nodes

‣ label compatibility

FIND

‣ belief distributions of all nodes

SUBJECT TO

‣ theoretically-grounded algorithm

‣ fast & scalable implementation

H0.8 0.2

0.2 0.8

ALICE BOB

CAROL

SMITH

JOHN

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

DIRICHLET BELIEF DISTRIBUTIONS (1/2)EXPERIMENTS

13

2D Dirichlet distribution (Beta distribution)

p(x;↵+ 1,� + 1) / x

↵(1� x)�

Belief/Leaning : ↵+ 1

↵+ � + 2

BACKGROUND

# #

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

DIRICHLET BELIEF DISTRIBUTIONS (2/2)EXPERIMENTS

14

Certainty:↵+ �

EXTENDS TO ANY NUMBER OF DIMENSIONS!!

Image  source:  UBC  Wiki

BACKGROUND

Example: 3 dimensions (talkative / silent / confused)

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

MULTINOMIAL MESSAGE DISTRIBUTIONSEXPERIMENTS

15

BP belief update rule

parameters of belief distribution

parameters of message distribution

NETCONF belief update rule

MULTINOMIAL DISTRIBUTION!

DIRICHLET DISTRIBUTION

DETAILS!

bu(i) eu(i)Y

v2N (u)

mvu(i)

bu eu +X

v2N (u)

mvu

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

MESSAGES FROM BELIEFSEXPERIMENTS

16

JOHNSMITH

(a, b)

(a, b)

perfect homophily

(b, a)

perfect heterophily

M =k

k � 1

✓H� 1

k

◆+

(0, 0)

no network effects

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

NETCONF BELIEF & MESSAGE UPDATE RULESEXPERIMENTS

17

message update

belief update

M

bu eu +X

v2N (u)

mvu

mvu M

0

@eu +X

v2N (v)\u

mwu

1

A

muvbu

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

NETCONF MATRIX BELIEF UPDATEEXPERIMENTS

18

modulation

B E+ (ABM�DBM2)(I�M2)�1

prior belief distribution

final belief distribution

~~ ~~~~

diagonal degreeadjacency

~~ ~~~~ ~~

~ ~~ ~

~ ~~ ~

PROBLEM

MOTIVATION

NETCONF

GUARANTEES

EXPERIMENTSEswaran,  Guennemann  &  Faloutsos

GUARANTEES

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

KEY THEORETICAL QUESTIONSEXPERIMENTS

20

UNIQUENESS

CONVERGENCE

Is the steady state solution unique?

Can we predict convergence?

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

NETCONF HAS CLOSED-FORM SOLUTIONEXPERIMENTS

21

ITERATIVE UPDATE

CLOSED FORM

Roth’s Column Lemma

DETAILS!

vec(B) =⇣I� (MM)T ⌦A+ (M2M)T ⌦D

⌘�1vec(E)

B E+ (ABM�DBM2)(I�M2)�1

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

PRECISE CONVERGENCE GUARANTEES

22

GRAPH STRUCTURE LABEL COMPATIBILITY

CLOSED FORM

NECESSARY &

SUFFICIENT CONDITION

DETAILS!

⇢⇣(MM)T ⌦A+ (M2M)T ⌦D

⌘< 1

vec(B) =⇣I� (MM)T ⌦A+ (M2M)T ⌦D

⌘�1vec(E)

GUARANTEES

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

KEY THEORETICAL GUARANTEESEXPERIMENTS

23

CLOSED-FORM

CONVERGENCE

Closed-form solution and unique fixed point!

Necessary and sufficient conditions for convergence!

PROBLEM

MOTIVATION

NETCONF

GUARANTEES

EXPERIMENTSEswaran,  Guennemann  &  Faloutsos

EXPERIMENTS

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

KEY QUESTIONS FOR EXPERIMENTSEXPERIMENTS

25

EFFECTIVENESS

SCALABILITY

INTERPRETABILITY

Improves accuracy & precision?

Fast and scalable?

Are final results interpretable?

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

DATAEXPERIMENTS

26

POLBLOGS

2 classes

1.5K nodes, 19K edges

DBLP

4 classes

28K nodes, 67K edges

POKEC

10 classes

1.6M nodes, 30M edges

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

NETCONF IS ACCURATE AND PRECISEEXPERIMENTS

27

HIGHER ACCURACY %

DATASET BP NETCONF

POLBLOGS 91.38 92.40

DBLP 76.26 81.89

POKEC 73.78 75.02

BETTER PRECISION

Ideal

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

NETCONF IS FAST AND SCALABLEEXPERIMENTS

28

30M edges in ~7 seconds!

Linear scaling with graph size

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

NETCONF GIVES INTERPRETABLE RESULTSEXPERIMENTS

29

AUTHOR H-index

Michael J Carey 48

Rakesh Agrawal 96

Jiawei Han 139

Hamid Pirahesh 40

David J Dewitt 81

Serge Abiteboul 77

AUTHOR H-index

Jiawei Han 139

Annie W Shum -

Werner Keibling -

Xiaofang Zhou 36

Bertram Ludascher 45

Amarnath Gupta -

TOP DATABASES AUTHORS IN DBLP

Many papers &high H1 indices!

BPNETCONF

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

KEY EXPERIMENTAL FINDINGSEXPERIMENTS

30

EFFECTIVENESS

SCALABILITY

INTERPRETABILITY

Improves accuracy & precision!

Scales linearly with graph size!

Certainty scores reflect intuition!

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran,  Guennemann  &  Faloutsos

SUMMARY: NETCONFSUMMARY

31

✓Theoretically grounded

✓Closed-form solution

✓Convergence guarantees

✓Improved performance

✓Fast and scalable

✓Interpretable results

Questions? deswaran@cs.cmu.edu