Node similarity and classification - Hasso-Plattner-Institut · Node similarity and classification...

Node similarity and classification

Graph Mining course Winter Semester 2017

DavideMottin,AntonTsitsulinHasso Plattner Institute

Acknowledgements

§ Somepartofthislectureistakenfrom:http://web.eecs.umich.edu/~dkoutra/tut/icdm14.html

§ Otheradaptedcontentisfrom SocialNetworkDataAnalytics(Springer)Ed.Charu Aggarwal,March2011

GRAPH MINING WS 2017 2

3GRAPH MINING WS 2017

FriendsnetworkinanAmericanhighschool:1. Nodesarepeople2. Edgesarefriendship3. Colors=races

"Race, school integration, and friendship segregation in America," American Journal of Sociology 107, 679-716 (2001).

?

Whichcolorwillthishave?

What about politics?

Aretheyrepublicanordemocrats?

?

Source: http://adequatebird.com/2010/05/03/the-political-blogosphere-and-the-2004-u-s-election-divided-they-blog/

5

Give me your own example!

GRAPH MINING WS 2017

Classification with Network Data

§ Givenagraphandfewnodesforwhichweknowthe”label”ora”class”howcanwepredict userattributesorinterests?

Predictthelabelsfornonmarkednodes?

GRAPH MINING WS 2017 6

Why node classification?

§ Isthisafriendoranaquitance?§ Recommendationsystemstosuggestobjects(music,movies,

activities)§ Automaticallyunderstandrolesinanetwork(hubs,activators,

influencingnodes,…)§ Identifyexpertsforquestionansweringsystems§ Targetedadvertising§ Studyofcommunities(keyindividuals,groupstarters...)§ Studyofdiseasesandcures§ Identifyunusualbehaviorsorbehavioralchanges§ Findingsimilarnodesandoutliers

GRAPH MINING WS 2017 7

Why node classification is useful?

§ Notallthenodeshavelabels(usersarenotwillingtoprovideexplanations)

§ Rolesarenotexplicitlydeclared(whoismoreimportantinacompany?Thinkabouttheexchangedemails;))

§ Labelsprovidedbytheuserscanbemisleading§ Labelsaresparse(somecategoriesmightbemissingor

incomplete)

GRAPH MINING WS 2017 8

Node classification problem

§ Given:• Graph𝐺: 𝑉, 𝐸,𝑊 withverticesV,edgesEandweightmatrixW• Labelednodes𝑉'() ⊂ 𝑉,unlabelednodes𝑉+, = 𝑉 ∖ 𝑉'()• 𝒴 thesetofm possiblelabels (e.g.,𝒴={republican,democrat})• 𝑌'() = 𝑦D, … , y' thelabelsonlabelednodesin𝑉'()

§ Problem:Inferlabels𝑌+, forallnodesin𝑉+,

GRAPH MINING WS 2017

G2

1

1

𝑉'()

𝑉+,

? ?

?

?

𝒴={1,2}

9

Node classification problem (2)

§ Canbegeneralizedtomultilabel andmulticlassclassification:• Withmulticlassclassificationassumethateachlabelednodehasaprobabilitydistributiononthelabels.

§ Canworkongeneralizedgraphstructures• hypergraphs,graphswithweighted,labeled,timestampededges,multigraphs,probabilisticgraphsandsoon.

GRAPH MINING WS 2017 10

Influential factors on Networks

§ Individualbehaviorsarecorrelatedinanetworkenvironment

Homophily Influence Confounding

GRAPH MINING WS 2017 11

Beingrepublican→ Friendship

Friendship→ Sameshoes

BorninBerlin→ (individual)Likeelectronicmusic(connection)participatetomarathon

Externalfactor

The importance of the graph structure

§ Thegraphstructureencodesimportantinformationfornodeclassification

§ Soitisreasonabletothinkthatlabelspropagateinthenetworkfollowingthelinks

§ Methodsthatworkwithpointsinthespaceperformpoorlyinagraph

GRAPH MINING WS 2017

Assumption:Thelabelpropagatesonthenetwork

12

Node features

§ Nodefeatures:measurablecharacteristicsofthenodesthathelpdiscriminatinganodefromanotherorstatingthethesimilaritywithothernodes.

§ Examplesoffeatures:• In/outdegreeofthenode• Numberofl-labelededgesfromthatnode• Numberofpathsinthatgoesthroughthenode• Numberoftriangles• Degreeandnumberwithinego-netedges• …

GRAPH MINING WS 2017 13

Node classification approaches

§ Similaritybased• Findnodesthatsharethesamecharacteristicswithothernodes

§ Iterativelearning• Learnasetoflabelsandpropagatetheinformationtosimilarnodes

§ Labelpropagation• Labelednodespropagatetheinformationtotheneighborswithsomeprobability

GRAPH MINING WS 2017 14

Lecture road

Similaritybased

Iterativeclassification

GRAPH MINING WS 2017

Labelpropagation

15

Real-world Applications

GRAPH MINING WS 2017 16

Movies recommendations

GRAPH MINING WS 2017 17

Search Engines (IR)Topical Sessions

“popularmusicvideos”

QueriesURLs

“music”

“yahoo”

similar

GRAPH MINING WS 2017 18

Similarity based approaches

§ Equivalencesintermsofstructure• Structural,Automorphic,andRegular

§ Roleextractionmethods:• RolX

§ Recursivesimilarities• Paths,Max-flow,SimRank

19GRAPH MINING WS 2017

History: Equivalences

20GRAPH MINING WS 2017

Twonodesuandvarestructurally equivalentiftheyhavethesamerelationshipstoallothernodes

(Lorrainandwhite,F.,1971)

Twonodesuandvareautomorphically equivalentifallthenodescanberelabeled toformanisomorphicgraphwiththe

labelsofuandvinterchanged(justchangethenodeid)(Borgatti andEverett,1992)

Twonodesuandvareregularly equivalentiftheyareequallyrelatedtoequivalentothers

(Borgatti andEverett,1992)

Regular equivalence

Borgatti,S.P.andEverett,M.G.,1992.Regularblockmodels ofmultiway,multimodematrices.SocialNetworks.

GRAPH MINING WS 2017

§ Assumesasimilaritybetweensetsofnodes

Billy John

Prof.Einstein Prof.Hilbert

Professors

Students

BillyandJohnaresimilarbecausetheyarebothconnectedtoaprofessor.Sameforprof.EinsteinandHilbert

Regularequivalencedoesn’tcareaboutwhichconnectionsbuttowhichset/groupanodeisconnected

21

Twonodesuandvareregularly equivalentiftheyareequallyrelatedtoequivalentothers

(Borgatti andEverett,1992)

Relation among equivalences

GRAPH MINING WS 2017

Whatistherelationamongthethreeequivalences?

Regular

Automorphic

Structural

22

RolX: Role eXtraction algorithm

Henderson,K.,Gallagher,B.,Eliassi-Rad,T.,Tong,H.,Basu,S.,Akoglu,L.,Koutra,D.,Faloutsos,C.andLi,L.,2012.Rolx:structuralroleextraction&mininginlargegraphs.SIGKDD

GRAPH MINING WS 2017

Adjacencymatrix(𝑛×𝑛)

RecursiveFeatureExtraction

NodexFeaturematrix

RoleExtraction

NodexRolematrix

RolexFeatureMatrix

Input

Output

ndimensionalspace

r dimensionalspace

ddimspace

Non-negativematrixfactorization(NMF)

23

Recursive Feature extraction (ReFex)

Henderson,K.,Gallagher,B.,Li,L.,Akoglu,L.,Eliassi-Rad,T.,Tong,H.andFaloutsos,C.,2011.It'swhoyouknow:graphminingusingrecursivestructuralfeatures.SIGKDD.

GRAPH MINING WS 2017

§ Transformthenetworkconnectivityintorecursivestructuralfeatures.

§ Technically,embedsthegraphintoan|ℱ| dimensionalspace,whereℱisasetoffeatures(degree,self-loops,avg edgeweight,#ofedgesinegonet)

24

ReFeX: mining features

§ Local:• Measuresofthenodedegree

§ Egonet:• Theegonet (orego-network)ofanodeisthenodeitself,theadjecent nodes,andthegraphinducedbythosenodes

• Computedbasedoneachnode’segonetwork:#ofwithin-egonet edges,#ofedgesentering&leavingtheegonet

§ Recursive• Someaggregate(mean,sum,max,min,…)ofanotherfeatureoveranode’sneighbors

• Theaggregationcanbecomputedoveranyreal-valuedfeature,includingotherrecursivefeatures(thisprocessmightnotstopifuncontrolled!!!)

GRAPH MINING WS 2017

Neighborhood

Regional

25

ReFex (2)

§ Numberofpossiblerecursivefeaturesisinfinite§ ReFeX pruning• Featurevaluesaremappedtosmallintegersviaverticallogarithmicbinning• Logbinning:discretizethefeaturestakingnonuniform(butlogarithmic)bins=thefirstp|V|nodeswiththelowestfeaturevalueareassignedtobin0,thandividetheremaining|V|- p|V|takingthefirstp(|V|- p|V|)nodesandsoon.

• Logarithmicbinningincreasethechancestwofeatures

§ Lookpairsoffeatureswhosevaluesneverdisagreeatanynodebymorethanathresholds,andconnectinagraph.Foreachcomponenttakeonefeature.

§ Agraphbasedapproach(motivatedbypowerlawdistribution)§ Thresholdautomaticallyset

GRAPH MINING WS 2017 26

RolX: Role eXtraction algorithm

Henderson,K.,Gallagher,B.,Eliassi-Rad,T.,Tong,H.,Basu,S.,Akoglu,L.,Koutra,D.,Faloutsos,C.andLi,L.,2012.Rolx:structuralroleextraction&mininginlargegraphs.SIGKDD

GRAPH MINING WS 2017

Adjacencymatrix(𝑛×𝑛)

RecursiveFeatureExtraction

NodexFeaturematrix

RoleExtraction

NodexRolematrix

RolexFeatureMatrix

Input

Output

ndimensionalspace

r dimensionalspace

ddimspace

Non-negativematrixfactorization(NMF)

27

Role extraction: Feature grouping

§ Findr overlappingclustersinthefeaturespace• Eachnodecanhavemultiplerolesatthesametime

§ GeneratearankrapproximationofthenodexfeaturematrixV§ Usenon-negativematrixfactorization:𝑉 ≈ 𝐺𝐹

§ TheGmatrixassignsnodestoroles§ TheFmatrixrepresentshowthefeaturesexplaintheroles

GRAPH MINING WS 2017 28

A (very brief) glimpse to matrix factorization

§ IfVisamatrix,itispossibletofindanapproximationofthismatrixmultiplyingtwo(lowerrank)matrices

§ Inparticular,wewanttofindtwomatricesG,F,suchthatV ≈ 𝐺𝐹

Example: 3 46 8 = 1

2 34

§ However, theexactfactorizationisnotalwayspossible!!!§ Idea:letusfindG, 𝐹, withG ≥ 0, 𝐹 ≥ 0 suchthat

argminZ,[

𝑉 − 𝐺𝐹 [

where ⋅ [ istheFrobenius norm§ Intuitively:youwanttominimizethedifferencebetweenthe

singleelementsofthematrixV andtheproductGF

GRAPH MINING WS 2017 29

?

§ Rolessummarizethebehaviororalternatively,theycompressthefeaturematrixV(lowerdimensiondescription)

§ Whatisthebestmodel?

§ Idea:usetheMinimumDescriptionLength(MDL)paradigmtoselectthenumberofrolesthatresultsinthebestcompression• L:descriptionlength• M:#ofbitstodescribethemodel• E:costofdescribingtheerrorsin𝑉 − 𝐺𝐹• Findrsuchthatitminimizes𝐿 = 𝑀 + 𝐸

§ Minimize𝑀 + 𝐸• Assuminganyvaluerequiresbbits(therolevalues),thanthenumberofbitsforMis𝑀 = 𝑏𝑟(𝑛 + 𝑓),why?(thinkaboutthedimensionofthematrices)

• WhataboutE?Eistheamountoferrors.However,since𝑉 − 𝐺𝐹 isnotnormallydistributed,RolX usestheKLdivergence𝐸 = ∑ 𝑉e,f log

gh,i(Z[)h,i

− 𝑉e,f + (𝐺𝐹)e,f�e,f

Thebestmodelistheonethathasfewererrors andrequireslessspace

Selecting the right number of roles

GRAPH MINING WS 2017 30