Relating Web Characteristics with Link-Based Ranking

Relating Web Characteristics

Ricardo Baeza-Yates

Carlos CastilloUniversidad de Chile


Agenda

• Introduction

• Link-based ranking

• Web structure

• Web characteristics

• Web usage

• Web dynamics

• Conclusions


Introduction: Sample

• Web sample: .CL domain on year 2000• 670,000 pages in 7,500 domains• 15kb average page size• Collection from the TodoCL web search

engine


Introduction: Emphasis

• Broder et al.: Graph Structure on the Web (2000)– Page-based structure based on strongly

connected components

– The Web graph is not a random graph

– Process: cut & paste model

• Our is mostly a site-based analysis– Trying to make Web structure meaningful


Introduction: The Empire


Introduction: One Map


Link ranking: Pagerank

∑=

−+=k

i

irPagerankqN

qpPagerank

1

)()1()(

Pages that pointto page p

Probability of a random jump over number of pages

Currently used byGoogleBrin & Page, 1998


Link ranking: Hubs & Authorities

• HITS algorithm (Kleinberg, 1998)

• A good authority is a page pointed by good hubs, so we assume that it has good content

• A good hub is a page that points to good authorities, so we assume it is a good set of links

• Linear system calculated by numerical iteration


Link ranking: Distribution

9% with relevanthub score 2-3% with relevant

authority score

<2% with relevant Pagerank


Link ranking: Correlation

Hub score,authority scoreand Pagerankdo not seem

to be correlated


Link ranking: Sites

• Which measure to use for sites ?

• Average score– But good sites can have lots of bad pages

• Maximum score– But one good page cannot be all that is

needed to be a good site

• Sum of the scores of all pages– Natural for Pagerank


Link ranking: Sites Graph

90% relevant site-Pagerank

It’s harder to have a good hub than a good authority (site)


Web Structure: Basis

• The Web graph has structure:

INOUT

MAIN

ISLANDS


Web Structure: Basis (cont.)

• The MAIN component has structure:

INOUTMAIN NORM

MAIN IN

MAIN MAIN MAIN OUT


Web Structure: Sketch


Web Structure: Degree


Web Structure: Sizes


Web Structure: Preferences


Web Structure: Preferences

OUT

MAINMAIN

MAINMAIN

OUTMAINOUT

Real ODP TodoCL


Web Structure: Various


Web Structure: Link Scores


Web Dynamics: Ages

• The kernel of the Web comes from the past


Web Dynamics: By Component


Web Dynamics: Pagerank

Pagerank is biased against newer pages


Web Dynamics: Hubs & Authorities

Age (months)

Aut

horit

y S

core

Hub

Sco

re


Conclusions

• Pagerank/HITS do not seem to be correlated– And Pagerank is biased to older pages

• Site ranking can help to make good human-selected directories

• Finding good pages is not so simple

• Characterizing Web structure gives valuable insight– Web Graph Mining is just starting

Technology

Relating Web Characteristics with Link-Based Ranking