Data mining, interactive semantic structuring, and

collaboration: A diversity-aware method for sense-making in search

Mathias Verbeke, Bettina Berendt,

Siegfried Nijssen

Dept. Computer Science, KU Leuven

AgendaAgenda

MotivationMotivationDiversity Diversity Diversity-aware tools Diversity-aware tools (our) (our)

ContextContext

Main partMain partMeasures of diversity Measures of diversity Tool Tool

OutlookOutlook

Motivation (1): Diversity Motivation (1): Diversity is ...is ...

Speaking different Speaking different languages (etc.) languages (etc.) localisation / localisation / internationalisationinternationalisation

Having different Having different abilities abilities accessibilityaccessibility

Liking different Liking different things things collaborative collaborative filteringfiltering

Structuring the Structuring the world in different world in different ways ways ? ?

Motivation (2): Motivation (2): Diversity-aware applications ...Diversity-aware applications ... Must have a (formal) notion of Must have a (formal) notion of

diversitydiversity Can follow aCan follow a

– ““personalization approach“personalization approach“ adapt to the user‘s value on the diversity adapt to the user‘s value on the diversity

variable(s)variable(s)

transparently? Is this paternalistic?transparently? Is this paternalistic?

– ““customization approach“customization approach“ show the space of diversityshow the space of diversity

allow choice / semi-automatic!allow choice / semi-automatic!

(Our) Context(Our) Context

1.1. Diversity and Web usage: language, cultureDiversity and Web usage: language, culture

2.2. Family of tools focussing on interactive Family of tools focussing on interactive sense-making helped by data mining sense-making helped by data mining – PORPOISE: global and local analysis of news and PORPOISE: global and local analysis of news and

blogs + their relations blogs + their relations – STORIES: finding + visualisation of “stories” in STORIES: finding + visualisation of “stories” in

newsnews– CiteseerCluster: literature search + sense-makingCiteseerCluster: literature search + sense-making– Damilicious: CiteseerCluster + re-use/transfer of Damilicious: CiteseerCluster + re-use/transfer of

semantics + diversitysemantics + diversity

Measuring grouping Measuring grouping diversitydiversity

Diversity = 1 – similarity = 1 - Normalized mutual Diversity = 1 – similarity = 1 - Normalized mutual informationinformation

NMI = 0

NMI = 0.35

By colour &

Measuring user diversityMeasuring user diversity

““How similarly do two users group How similarly do two users group documents?“documents?“

For each query For each query qq, consider their groupings , consider their groupings grgr::

““How similarly do two users group How similarly do two users group documents?“documents?“

For each query For each query qq, consider their groupings , consider their groupings grgr::

For various queries: aggregateFor various queries: aggregate

... and now: the application ... and now: the application domaindomain

... that‘s only the 1st step!

WorkflowWorkflow

1. Query2. Automatic clustering3. Manual regrouping 4. Re-use

1. Learn + present way(s) of grouping2. Transfer the constructed concepts

ConceptsConcepts

ExtensionExtension– the instances in a groupthe instances in a group

IntensionIntension– Ideally: “squares vs. Ideally: “squares vs.

circles“circles“– Pragmatically: defined Pragmatically: defined

via a classifiervia a classifier

Step 1: RetrieveStep 1: Retrieve

CiteseerX via OAI Output: set of

– document IDs, – document details– their texts

Step 2: ClusterStep 2: Cluster

“the classic bibliometric solution“ CiteseerCluster:

– Similarity measure: co-citation, bibliometric coupling, word or LSA similarity, combinations

– Clustering algorithm: k-means, hierarchical Damilicious: phrases Lingo How to choose the How to choose the “best“? best“?

– Experiments: Lingo better than k-means at Experiments: Lingo better than k-means at reconstruction and extension-over-timereconstruction and extension-over-time

Step 3 (a): Re-organise Step 3 (a): Re-organise & work on document groups& work on document groups

Step 3 (b): Step 3 (b): Visualising document groupsVisualising document groups

Steps 4+5: Re-useSteps 4+5: Re-use Basic idea: Basic idea:

1.1. learn a classifier from the final grouping (Lingo phrases)learn a classifier from the final grouping (Lingo phrases)2.2. apply the classifier to a new search result apply the classifier to a new search result

“ “re-use semantics“re-use semantics“ Whose grouping?Whose grouping?

– One‘s ownOne‘s own– Somebody else‘sSomebody else‘s

Which search result?Which search result?– “ “ the same“ (same query, structuring by somebody else)the same“ (same query, structuring by somebody else)– “ “ More of the same“ (same query, later time More of the same“ (same query, later time more more

doc.s)doc.s)– “ “ related“ (... Measured how? ...)related“ (... Measured how? ...)– arbitraryarbitrary

Visualising user diversity (1)Visualising user diversity (1)Simulated users with different Simulated users with different

strategiesstrategies U0: did not change anything U0: did not change anything

(“System“)(“System“) U1: U1: tried produce a better fit of the

document groups to the cluster intensions; 5 regroupings

U2: attempted to move everything that did not fit well into the remainder group “Other topics”, & better fit; 10 regroupings

U3: attempted to move everything from „Other topics“ into matching real groups; 5 regroupings

U4: regrouping by author and institution; 5 regroupings

5*5 matrix of diversities gdiv(A,B,q) multidimensional scaling

Visualising user diversity (2)Visualising user diversity (2)

aggregatedaggregatedusing using gdiv(A,B)gdiv(A,B)

Web miningWeb mining Data miningData mining RFIDRFID

Evaluating the applicationEvaluating the application

Clustering only: Does it generate Clustering only: Does it generate meaningful document groups?meaningful document groups?– yes (tradition in bibliometrics) – but: data?yes (tradition in bibliometrics) – but: data?– Small expert evaluation of CiteseerClusterSmall expert evaluation of CiteseerCluster

Clustering & regroupingClustering & regrouping– End-user experiment with CiteseerClusterEnd-user experiment with CiteseerCluster

– 5-person5-person formative user study of formative user study of DamiliciousDamilicious

Summary and Summary and (some) open questions(some) open questions

Damilicious: a tool that helps users in sense-making, Damilicious: a tool that helps users in sense-making, exploring diversity, and re-using semanticsexploring diversity, and re-using semantics

diversity measures when queries and result sets are different?

how to best present of diversity? – How to integrate into an environment supporting user and

community contexts (e.g., Niederée et al. 2005)? Incentives to use the functionalities? how to find the best balance between similarity and

diversity? which measures of grouping diversity are most meaningful?

– Extensional?– Intensional? Structure-based? Hybrid? (cf. ontology matching)

which other sources of user diversity? Thanks!