Efficient Identification of Starters and Followers in Social Media

Michael Mathioudakis, Nick Koudas

Goals

• Formalize a definition of “starters” and “followers” in blogs

• Random sampling approaches to achieve significant efficiency while identifying “starters” and “followers”

Starters vs Followers• Starter: a blogger who generates posts that

others link to over a period of time• Follower: a blogger that links to other blog

posts over a period of time

NotationNotation Definition

P The set of all posts in the query result set

B The set of blogs in a query result set P

The set of posts in P coming from blog B

L The set of all links between posts in P

G A graph used as an abstract representation of P and the links L

V The node set of G

E The edge set of G

A query at time T

Calculating Starters and Followers

• In degree of node

• Out degree of node

• Degree of node

Brute Force

• Query the database for all posts• Calculate the degree of every node and sum

• Why not?– Retrieving all posts can be costly– Lots of overhead

Deterministic Early-Stopping Conditions

• = enumerated subset of

• is the set of k starters

• If , then exists a pair ,with and such that

• Use linear equalities to determine feasibility

Linear Inequalities

Linear Inequality Issues

• Result?– Large domains– Easily feasible– Traverse almost all edges before stopping

• Solution?– Relax requirements, use probabilistic guarantees

Probabilistic Early-Stopping Conditions

• Trade efficiency with accuracy

• Still aim to return starters

• Assume edges chosen uniformly at random

Probabilities

• for all pairs of nodes

• Pr < 10% return the result set

• How do you determine the bound for the probability?

Hoeffding’s Inequality

• Provides a lower bound

• Lower bound =

• Uniform sample should capture any skew

• Starters appear after few sampled edges

Random Sampling Techniques• Out-degrees among nodes is known• Maximum out-degree of a node is known• Sampling nodes uniformly at random• Random walk approach

Out-Degrees Known

Out-Degrees Known Issues

• Knowing out-degree = strong assumption• Requirements– Retrieve all posts in query– Extract all links

• Solution?– Weaker assumption on distribution of edges

Maximum Out-Degree Known

Maximum Out-Degree Issues

• Blog graphs typically heavy-tailed

• Probability at one iteration =

• Expected iterations =

Sampling Nodes Uniformly at Random

Sampling Nodes Uniformly at Random Issues

• Not sampled uniformly at random

• Only unbiased estimates of edges from one node to another

• Can’t handle heavy-tailed distributions

• Leads to poor accuracy

Random Walk Approach

• 2 step approach

– Obtain a new graph from the input graph

– Obtain a Markov chain

Step 1 – Obtain New Graph

• Create a new graph H(V, E) from input graph – Remove direction of edges– Add self-loops– Add edges between nodes returned in order

Step 2 – Create Markov Chain

• Markov Chain = MC(K, T)– K = the possible stats (nodes) – T = possible transitions (edges)

The Random Walk

At a step of the walk

Follows a transition to one of its states

(b): Edge of current node = no lookup cost

(c): Edge of new node = random access cost

Stopping the Random Walk

• At each step, for each pair of nodes

• Average the score over all pairs of nodes

• Stop when confScore > threshold

Results

Most in-links doesn’t necessarilymean the best starter

Results (continued)

Real World Application

• BlogScope– Project of University of Toronto– Provides graph and search output

of blog data– How does it work?• Crawler to gather blog data and filter spam• Stored in MySQL (1174.14 million posts)• Build statistics regularly• Provide correlation discovery, popularity curves, and hot

keywords

Related WorkDiscovering Leaders from Community Actions

Amit Goyal, Francesco Bonchi, Laks V. S. Lakshmanan

Users perform actions (bookmark url, rate song, buying gadgets, etc)

Friends see actions and may perform same actions (influence)

Compute influence matrix with a sliding window working backwards

Pass over actions log only once

Uses frequent pattern discovery to determine leaders

Finds tribes where one user influences a group of people over a series of actions

Problem when there is a popular action where influence might not be a factor