Leonid E. Zhukov - Leonid Zhukov · Evaluation metrics Precision and Recall, F-measure Precision =...

Link prediction

Leonid E. Zhukov

School of Data Analysis and Artificial IntelligenceDepartment of Computer Science

National Research University Higher School of Economics

Structural Analysis and Visualization of Networks

Leonid E. Zhukov (HSE) Lecture 16 21.05.2016 1 / 23

Lecture outline

1 Link prediction problem

2 Proximity measures

3 Prediction by supervised learning

4 Other methods

Link prediction

Link prediction. A network is changing over time. Given a snapshotof a network at time t, predict edges added in the interval (t, t ′)

Link completion (missing links identification). Given a network, inferlinks that are consistent with the structure, but missing (findunobserved edges)

Link reliability. Estimate the reliability of given links in the graph.

Predictions: link existence, link weight, link type

Link prediction

Graph G(V,E)

Number of ”missing edges”: |V |(|V | − 1)/2− |E |In sparse graphs |E | � |V |2, Prob. of correct random guess O( 1

|V |2 )

Scoring algorithm

Link prediction by proximity scoring

1 For each pair of nodes compute proximity (similarity) score c(v1, v2)

2 Sort all pairs by the decreasing score

3 Select top n pairs (or above some threshold) as new links

Scoring functions

Local neighborhood of vi and vj

Number of common neighbors:

|N (vi ) ∩N (vj)|

Jaccard’s coefficient:|N (vi ) ∩N (vj)||N (vi ) ∪N (vj)|

Adamic/Adar: ∑v∈N (vi )∩N (vj )

log |N (v)|

Liben-Nowell and Kleinberg, 2003

Scoring functions

Paths and ensembles of paths between vi and vj

Shortest path:−min

s{pathsij > 0}

Katz score:

∞∑l=1

βl |paths(l)(vi , vj)| =∞∑l=1

(βA)lij = (I − βA)−1 − I

Personalized (rooted) PageRank:

PR = α(D−1A)TPR + (1− α)|

Scoring functions

Expected number of random walk steps:– hitting time: −Hij

– commute time −(Hij + Hji )– normalized hitting/commute time (Hijπj + Hjiπi )

SimRank:

SimRank(vi , vj) =C

|N (vi )| · |N (vj)|∑

m∈N (vi )

∑n∈N (vj )

SimRank(m, n)

Vertex feature aggregations

Preferential attachment:

ki · kj = |N (vi )| · |N (vj)|

orki + kj = |N (vi )|+ |N (vj)|

Clustering coefficient:CC (vi ) · CC (vj)

orCC (vi ) + CC (vj)

Low-rank approximations

Low-rank approximation (truncated SVD)

A ≈∑k

UkSkVTk

Link prediction

Binary classification

Challenging classification problem:

Computational cost of evaluating of very large number of possibleedges (quadratic in number of nodes)

Highly imbalanced class distribution: number of positive examples(existing edges) grows linearly and negative quadratically withnumber on nodes

Prediction difficulty

Actual and possible collaborations between DBLP authors

Extreme class imbalance

from Rattigan and Jensen, 2005

Link prediction with supervised learning

Supervised learning:1 Features generation2 Model training3 Testing (model application)

Features:

Topological proximity features

Aggregated features

Content based node proximity features

Features

Discriminative abilities of features

from Rattigan and Jensen, 2005

Simple evaluation

Simple ”hold out set” evaluation

Training and testing

Evaluation for evolving networks

image from Y. Yang et.al, 2014

Evaluation metrics

Precision and Recall, F-measure

Precision =TP

TP + FP, Recall =

TP + FN

F =2 · Precision · RecallPrecision + Recall

True positive rate (TPR), False positive rate (FPR), ROC curve, AUC

TPR =TP

TP + FN, FPR =

FP + TN

Performance of classification algorithms

BIOBASE database (research publications)

DBLP dataset (research publications)

from M. Al Hasan, 2010

ROC curves

from Lichtenwalter, 2010

Probabilistic models

Local model, Markov random fields [Wang, 2007]

Hierarchical probabilistic model [Clauset, 2008]

Probabilistic relations models:– Bayesian networks [Getoor, 2002]– relational Markov networks [Tasker, 2003, 2007]

References

D. Liben-Nowell and J. Kleinberg. The link prediction problem forsocial networks. Journal of the American Society for InformationScience and Technology, 58(7):1019?1031, 2007

R. Lichtenwalter, J.Lussier, and N. Chawla. New perspectives andmethods in link prediction. KDD 10: Proceedings of the 16th ACMSIGKDD, 2010

M. Al Hasan, V. Chaoji, S. Salem, M. Zaki, Link prediction usingsupervised learning. Proceedings of SDM workshop on link analysis,2006

M. Rattigan, D. Jensen. The case for anomalous link discovery. ACMSIGKDD Explorations Newsletter. v 7, n 2, pp 41-47, 2005

M. Al. Hasan, M. Zaki. A survey of link prediction in social networks.In Social Networks Data Analytics, Eds C. Aggarwal, 2011.

Leonid E. Zhukov - Leonid Zhukov · Evaluation metrics Precision and Recall, F-measure Precision =...

Documents

BUILDING CLASSIFICATION MODELS FROM IMBALANCED FRAUD ... · F-measure = (2 x Recall x Precision) / (Recall + Precision) F-measure is the harmonic mean of precision and recall (Fawcett,

Fishing for compliments: Precision and recall in …Fishing for compliments: Precision and recall in corpus-linguistic compliment research Andreas H. Jucker, Gerold Schneider, Irma

Analysis of Mahatma Gandhi National Rural Employment ... · incorrect classification, True Positive Rate (TP), False Positive Rate (FP), Precision (P), Recall (R) and F-measure (F)

Machine Learning From Imbalanced Data Sets · EVALUATION METRICS Accuracy = (TN + TP)/(TN+FN+FP+TP) Precision = TP/(TP+FP) i.e. completeness Recall = TP/(TP+FN) i.e. exactness F-measure:

Lecture 4: Logistic Regression - GitHub Pages regression.pdf · Precision recall curve •The precision recall curve, or pr curve, is another plot to measure the performance of binary

Top Page | Center for Advanced Intelligence Project2018/10/18 · attribute precision recall f1-score precision recall f1-score precision recall f1-score precision recall f1-score

PRECISION AND RELIABILITY - Lincoln Electric...PRECISION AND RELIABILITY TIG DC RANGE INVERTEC® 175TP / V205-TP 2V 220TPX / V270-T / V270-TP V270-TP 2V / 300TPX / 400TPX GTAW PROCESS

Tp 38 000 - Precision Temperature Calibrators

pilsung kang@korea.ac.kr zoon@snu.acdm.snu.ac.kr/static/docs/TR/SNUDM-TR-2015-01.pdfprecision= TP TP+ FP (3) recall= TP TP+ FN (4) F score= 2 precision recall precision+ recall (5)

Evaluation: From Precision, Recall and F-Factor to ROC ... · Powers 3 Evaluation: From Precision, Recall and F-Factor and negative and are predicting the opposite case. Inverse Recall

Precision Vacuum Transmitter - TEMC Inctemc-inc.com/products/Pressure Measurement/Vacuum...Precision Vacuum Transmitter. Variation of SCM2100 SCM2100 SCM2100-TP SCM2100 -H SCM2100-H-TP

Controlling and visualizing the precision-recall tradeo ... · The precision-recall space is close to the ROC space that is de ned by the false and true positive rates. Some relationships

Precision and Recall

Hak Cipta Terpelihara © 2011 IIUM/UiTM/USIMmyrepositori.pnm.gov.my/bitstream/123456789/3093/1/38EvaluatingRetrievalPerformance...Measures include recall, precision and overlap. Recall

Precision & Recall - University of Washingtonprecision-recall curve • given training data, each trained classiﬁer achieves a certain (recall,precision) on the test data • which

Lecture 15: Detecting Objects by Partsvision.stanford.edu/teaching/cs131_fall1718/files/15_notes.pdf · the model. Recall= TP TP+FN Recall can be thought of as the fraction of ground

Precision-Recall-Gain Curves: PR Analysis Done Rightpeople.cs.bris.ac.uk/~flach/PRGcurves/PRcurves_ISL.pdf · A Precision-Recall-Gain curve plots Precision Gain on the y-axis against

Information Retrieval Performance Measurement Using ...users.umiacs.umd.edu/~oard/desi6/papers/dimm-final.pdf · precision-recall curves based on a reasonable model of how a precision-recall

Governo do Estado do Pará - paginas.uepa.br · precision, recall and area under the curve. Keywords: Biomass. Amazon. ... F1 Medida F CA Acurácia Global TP Verdadeiro Positivo FN

Query - NISTC C ! C ED + / F * 7 9 ! AC D 554 . N.5 0.0 0.2 0.4 0.6 0.8 1.0 Recall 0.0 0.2 0.4 0.6 0.8 1.0 Precision Interpolated Precision at recall precision n shots 0.0 0.4180 …