Upload
corey-jacobs
View
242
Download
1
Tags:
Embed Size (px)
Citation preview
Clustering
Bamshad MobasherDePaul University
Bamshad MobasherDePaul University
2
What is Clustering in Data Mining?
i Cluster:4 a collection of data objects that
are “similar” to one another and thus can be treated collectively as one group
4 but as a collection, they are sufficiently different from other groups
i Clustering4 unsupervised classification4 no predefined classes
Clustering is a process of partitioning a set of data (or objects) in a set of meaningful sub-classes, called clusters
Clustering is a process of partitioning a set of data (or objects) in a set of meaningful sub-classes, called clusters
Helps users understand the natural grouping or structure in a data set
Applications of Cluster Analysis
i Data reduction
4 Summarization: Preprocessing for regression, PCA, classification, and association analysis
4 Compression: Image processing: vector quantizationi Hypothesis generation and testingi Prediction based on groups
4 Cluster & find characteristics/patterns for each group
i Finding K-nearest Neighbors
4 Localizing search to one or a small number of clusters
i Outlier detection: Outliers are often viewed as those “far away” from any cluster
3
Basic Steps to Develop a Clustering Task
i Feature selection / Preprocessing4 Select info concerning the task of interest
4 Minimal information redundancy
4 May need to do normalization/standardization
i Distance/Similarity measure4 Similarity of two feature vectors
i Clustering criterion4 Expressed via a cost function or some rules
i Clustering algorithms4 Choice of algorithms
i Validation of the results
i Interpretation of the results with applications
4
Quality: What Is Good Clustering?
i A good clustering method will produce high quality clusters
4 high intra-class similarity: cohesive within clusters
4 low inter-class similarity: distinctive between clusters
i The quality of a clustering method depends on
4 the similarity measure used by the method
4 its implementation, and
4 Its ability to discover some or all of the hidden patterns
5
Measure the Quality of Clustering
i Distance/Similarity metric4 Similarity is expressed in terms of a distance function, typically metric:
d(i, j)4 The definitions of distance functions are usually rather different for
interval-scaled, Boolean, categorical, ordinal ratio, and vector variables4 Weights should be associated with different variables based on
applications and data semantics
i Quality of clustering:4 There is usually a separate “quality” function that measures the
“goodness” of a cluster.4 It is hard to define “similar enough” or “good enough”
h The answer is typically highly subjective
6
7
Distance or Similarity Measures
i Common Distance Measures:
4 Manhattan distance:
4 Euclidean distance:
4 Cosine similarity:
( , ) 1 ( , )dist X Y sim X Y 2 2
( )( , )
i ii
i ii i
x ysim X Y
x y
8
More Similarity Measures
Simple Matching
Cosine Coefficient
Dice’s Coefficient
Jaccard’s Coefficient
i In vector-space model many similarity measures can be used in clustering
2 2
( )( , )
i ii
i ii i
x ysim X Y
x y
sim X Y x yi ii
( , ) sim X Y
x y
x y
i ii
i iii
( , )
2
2 2
sim X Yx y
x y x y
i ii
i i i iiii
( , )
2 2
9
Distance (Similarity) Matrix
similarity (or distance) of to ij i jd D D
Note that dij = dji (i.e., the matrix is symmetric. So, we only need the lower triangle part of the matrix.
The diagonal is all 1’s (similarity) or all 0’s (distance)
Note that dij = dji (i.e., the matrix is symmetric. So, we only need the lower triangle part of the matrix.
The diagonal is all 1’s (similarity) or all 0’s (distance)
i Similarity (Distance) Matrix4 based on the distance or similarity measure we can construct a symmetric
matrix of distance (or similarity values)4 (i, j) entry in the matrix is the distance (similarity) between items i and j
10
Example: Term Similarities in Documentsi Suppose we want to cluster terms that appear in a collection of
documents with different frequencies
i We need to compute a term-term similarity matrix4 For simplicity we use the dot product as similarity measure (note that this is the non-
normalized version of cosine similarity)
4 Example:
1
( , ) ( )i j jk
N
ikk
sim T T w w
Each term can be viewed as a vector of term frequencies (weights)
Each term can be viewed as a vector of term frequencies (weights)
T1 T2 T3 T4 T5 T6 T7 T8Doc1 0 4 0 0 0 2 1 3Doc2 3 1 4 3 1 2 0 1Doc3 3 0 0 0 3 0 3 0Doc4 0 1 0 3 0 0 2 0Doc5 2 2 2 3 1 4 0 2
N = total number of dimensions (in this case documents)wik = weight of term i in document k.
N = total number of dimensions (in this case documents)wik = weight of term i in document k.
Sim(T1,T2) = <0,3,3,0,2> * <4,1,0,1,2>0x4 + 3x1 + 3x0 + 0x1 + 2x2 = 7
Sim(T1,T2) = <0,3,3,0,2> * <4,1,0,1,2>0x4 + 3x1 + 3x0 + 0x1 + 2x2 = 7
11
Example: Term Similarities in Documents
T1 T2 T3 T4 T5 T6 T7 T8Doc1 0 4 0 0 0 2 1 3Doc2 3 1 4 3 1 2 0 1Doc3 3 0 0 0 3 0 3 0Doc4 0 1 0 3 0 0 2 0Doc5 2 2 2 3 1 4 0 2
sim T T w wi j jkikk
N
( , ) ( )
1
T1 T2 T3 T4 T5 T6 T7T2 7T3 16 8T4 15 12 18T5 14 3 6 6T6 14 18 16 18 6T7 9 6 0 6 9 2T8 7 17 8 9 3 16 3
Term-TermSimilarity Matrix
Term-TermSimilarity Matrix
12
Similarity (Distance) Thresholds4 A similarity (distance) threshold may be used to mark pairs that are
“sufficiently” similar
T1 T2 T3 T4 T5 T6 T7T2 7T3 16 8T4 15 12 18T5 14 3 6 6T6 14 18 16 18 6T7 9 6 0 6 9 2T8 7 17 8 9 3 16 3
T1 T2 T3 T4 T5 T6 T7T2 0T3 1 0T4 1 1 1T5 1 0 0 0T6 1 1 1 1 0T7 0 0 0 0 0 0T8 0 1 0 0 0 1 0
Using a threshold value of 10 in the previous example
13
Graph Representationi The similarity matrix can be visualized as an undirected graph
4 each item is represented by a node, and edges represent the fact that two items are similar (a one in the similarity threshold matrix)
T1 T2 T3 T4 T5 T6 T7T2 0T3 1 0T4 1 1 1T5 1 0 0 0T6 1 1 1 1 0T7 0 0 0 0 0 0T8 0 1 0 0 0 1 0
T1 T3
T4
T6T8
T5
T2
T7If no threshold is used, thenmatrix can be represented asa weighted graph
If no threshold is used, thenmatrix can be represented asa weighted graph
14
Connectivity-Based Clustering Algorithms
i If we are interested only in threshold (and not the degree of similarity or distance), we can use the graph directly for clustering
i Clique Method (complete link)4 all items within a cluster must be within the similarity threshold of all other
items in that cluster4 clusters may overlap4 generally produces small but very tight clusters
i Single Link Method4 any item in a cluster must be within the similarity threshold of at least one
other item in that cluster4 produces larger but weaker clusters
i Other methods4 star method - start with an item and place all related items in that cluster4 string method - start with an item; place one related item in that cluster; then
place anther item related to the last item entered, and so on
15
Simple Clustering Algorithmsi Clique Method
4 a clique is a completely connected subgraph of a graph4 in the clique method, each maximal clique in the graph becomes a cluster
T1 T3
T4
T6T8
T5
T2
T7
Maximal cliques (and therefore the clusters) in the previous example are:
{T1, T3, T4, T6}{T2, T4, T6}{T2, T6, T8}{T1, T5}{T7}
Note that, for example, {T1, T3, T4} is also a clique, but is not maximal.
16
Simple Clustering Algorithmsi Single Link Method
4 selected an item not in a cluster and place it in a new cluster4 place all other similar item in that cluster4 repeat step 2 for each item in the cluster until nothing more can be added4 repeat steps 1-3 for each item that remains unclustered
T1 T3
T4
T6T8
T5
T2
T7
In this case the single link method produces only two clusters:
{T1, T3, T4, T5, T6, T2, T8} {T7}
Note that the single link method does not allow overlapping clusters, thus partitioning the set of items.
Major Clustering Approaches
i Partitioning approach: 4 Construct various partitions and then evaluate them by some criterion,
e.g., minimizing the sum of square errors4 Typical methods: k-means, k-medoids, CLARANS
i Hierarchical approach: 4 Create a hierarchical decomposition of the set of data (or objects) using
some criterion4 Typical methods: Diana, Agnes, BIRCH, CAMELEON
i Density-based approach: 4 Based on connectivity and density functions4 Typical methods: DBSACN, OPTICS, DenClue
i Grid-based approach: 4 based on a multiple-level granularity structure4 Typical methods: STING, WaveCluster, CLIQUE
17
Major Clustering Approaches (Cont.)
i Model-based: 4 A model is hypothesized for each of the clusters and tries to find the best
fit of that model to each other4 Typical methods: EM, SOM, COBWEB
i Frequent pattern-based:4 Based on the analysis of frequent patterns4 Typical methods: p-Cluster
i User-guided or constraint-based: 4 Clustering by considering user-specified or application-specific
constraints4 Typical methods: COD (obstacles), constrained clustering
i Link-based clustering:4 Objects are often linked together in various ways4 Massive links can be used to cluster objects: SimRank, LinkClus
18
19
Partitioning Approachesi The notion of comparing item similarities can be extended to
clusters themselves, by focusing on a representative vector for each cluster4 cluster representatives can be actual items in the cluster or other “virtual”
representatives such as the centroid4 this methodology reduces the number of similarity computations in
clustering4 clusters are revised successively until a stopping condition is satisfied, or
until no more changes to clusters can be made
i Partitioning Methods4 reallocation method - start with an initial assignment of items to clusters
and then move items from cluster to cluster to obtain an improved partitioning
4 Single pass method - simple and efficient, but produces large clusters, and depends on order in which items are processed
The K-Means Clustering Method
i Given k, the k-means algorithm is implemented in four
steps:
4 Partition objects into k nonempty subsets
4 Compute seed points as the centroids of the clusters of
the current partitioning (the centroid is the center, i.e.,
mean point, of the cluster)
4 Assign each object to the cluster with the nearest seed
point
4 Go back to Step 2, stop when the assignment does not
change
20
21
K-Means Algorithmi The basic algorithm (based on reallocation method):
1. Select K initial clusters by (possibly) random assignment of some items to clusters and compute each of the cluster centroids.
2. Compute the similarity of each item xi to each cluster centroid and (re-)assign each item to the cluster whose centroid is most similar to xi.
3. Re-compute the cluster centroids based on the new assignments.4. Repeat steps 2 and 3 until three is no change in clusters from one iteration to the next.
Initial (arbitrary) assignment:C1 = {D1,D2}, C2 = {D3,D4}, C3 = {D5,D6}
Cluster Centroids
Example: Clustering Documents
T1 T2 T3 T4 T5
D1 0 3 3 0 2D2 4 1 0 1 2D3 0 4 0 0 2D4 0 3 0 3 3D5 0 1 3 0 1D6 2 2 0 0 4D7 1 0 3 2 0D8 3 1 0 0 2C1 4/2 4/2 3/2 1/2 4/2C2 0/2 7/2 0/2 3/2 5/2C3 2/2 3/2 3/2 0/2 5/2
22
Example: K-Means
D1 D2 D3 D4 D5 D6 D7 D8C1 29/2 29/2 24/2 27/2 17/2 32/2 15/2 24/2C2 31/2 20/2 38/2 45/2 12/2 34/2 6/2 17/2C3 28/2 21/2 22/2 24/2 17/2 30/2 11/2 19/2
Now compute the similarity (or distance) of each item with each cluster, resulting a cluster-document similarity matrix (here we use dot product as the similarity measure).
For each document, reallocate the document to the cluster to which it has the highest similarity (shown in red in the above table). After the reallocation we have the following new clusters. Note that the previously unassigned D7 and D8 have been assigned, and that D1 and D6 have been reallocated from their original assignment.
C1 = {D2,D7,D8}, C2 = {D1,D3,D4,D6}, C3 = {D5}
This is the end of first iteration (i.e., the first reallocation). Next, we repeat the process for another reallocation…
This is the end of first iteration (i.e., the first reallocation). Next, we repeat the process for another reallocation…
23
Example: K-Means
T1 T2 T3 T4 T5
D1 0 3 3 0 2D2 4 1 0 1 2D3 0 4 0 0 2D4 0 3 0 3 3D5 0 1 3 0 1D6 2 2 0 0 4D7 1 0 3 2 0D8 3 1 0 0 2
C1 8/3 2/3 3/3 3/3 4/3C2 2/4 12/4 3/4 3/4 11/4C3 0/1 1/1 3/1 0/1 1/1
Now compute new cluster centroids using the original document-term matrix
This will lead to a new cluster-doc similarity matrix similar to previous slide. Again, the items are reallocated to clusters with highest similarity.
C1 = {D2,D7,D8}, C2 = {D1,D3,D4,D6}, C3 = {D5}C1 = {D2,D7,D8}, C2 = {D1,D3,D4,D6}, C3 = {D5}
D1 D2 D3 D4 D5 D6 D7 D8C1 7.67 15.01 5.34 9.00 5.00 12.00 7.67 11.34C2 16.75 11.25 17.50 19.50 8.00 6.68 4.25 10.00C3 14.00 3.00 6.00 6.00 11.00 9.34 9.00 3.00
C1 = {D2,D6,D8}, C2 = {D1,D3,D4}, C3 = {D5,D7}C1 = {D2,D6,D8}, C2 = {D1,D3,D4}, C3 = {D5,D7}New assignment
Note: This process is now repeated with new clusters. However, the next iteration in this exampleWill show no change to the clusters, thus terminating the algorithm.
24
K-Means Algorithmi Strength of the k-means:
4 Relatively efficient: O(tkn), where n is # of objects, k is # of clusters, and t is # of iterations. Normally, k, t << n
4 Often terminates at a local optimum
i Weakness of the k-means:4 Applicable only when mean is defined; what about
categorical data?4 Need to specify k, the number of clusters, in advance4 Unable to handle noisy data and outliers
i Variations of K-Means usually differ in:4 Selection of the initial k means4 Dissimilarity calculations4 Strategies to calculate cluster means
25
Single Pass Methodi The basic algorithm:
1. Assign the first item T1 as representative for C1
2. for item Ti calculate similarity S with centroid for each existing cluster
3. If Smax is greater than threshold value, add item to corresponding cluster and recalculate centroid; otherwise use item to initiate new cluster
4. If another item remains unclustered, go to step 2
See: Example of Single Pass Clustering Technique
i This algorithm is simple and efficient, but has some problems4 generally does not produce optimum clusters4 order dependent - using a different order of processing items will result in a
different clustering
Hierarchical Clustering Algorithms• Two main types of hierarchical clustering
– Agglomerative: • Start with the points as individual clusters• At each step, merge the closest pair of clusters until only one cluster (or k clusters) left
– Divisive: • Start with one, all-inclusive cluster • At each step, split a cluster until each cluster contains a point (or there are k clusters)
• Traditional hierarchical algorithms use a similarity or distance matrix
– Merge or split one cluster at a time
27
Hierarchical Clustering Algorithmsi Use dist / sim matrix as clustering criteria
4 does not require the no. of clusters as input, but needs a termination condition
a
b
c
d
e
ab
cd
cde
abcde
Step 0 Step 1 Step 2 Step 3 Step 4
Step 4 Step 3 Step 2 Step 1 Step 0
Agglomerative
Divisive
28
Hierarchical Agglomerative Clustering
i Hierarchical Agglomerative Methods4 starts with individual items as clusters 4 then successively combine smaller clusters to form larger ones4 combining clusters requires a method to determine similarity or distance
between two existing clusters
i Some commonly used HACM methods for combining clusters4 Single Link: at each step join most similar pairs of objects that are not yet in the
same cluster4 Complete Link: use least similar pair between each cluster pair to determine
inter-cluster similarity - all items within one cluster are linked to each other within a similarity threshold
4 Group Average (Mean): use average value of pairwise links within a cluster to determine inter-cluster similarity (i.e., all objects contribute to inter-cluster similarity)
4 Ward’s method: at each step join cluster pair whose merger minimizes the increase in total within-group error sum of squares (based on distance between centroids) - also called the minimum variance method
29
Hierarchical Agglomerative Clustering
i Basic procedure4 1. Place each of N items into a cluster of its own.4 2. Compute all pairwise item-item similarity coefficients
h Total of N(N-1)/2 coefficients
4 3. Form a new cluster by combining the most similar pair of current clusters i and j
h (use one of the methods described in the previous slide, e.g., complete link, group average, Ward’s, etc.);
h update similarity matrix by deleting the rows and columns corresponding to i and j;
h calculate the entries in the row corresponding to the new cluster i+j.
4 4. Repeat step 3 if the number of clusters left is great than 1.
Hierarchical Agglomerative Clustering:: Example
Nested Clusters Dendrogram
3 6 4 1 2 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
1
2
3
4
5
61
2 5
3
4
Input/ Initial settingi Start with clusters of individual points and a distance/similarity
matrix
p1
p3
p5
p4
p2
p1 p2 p3 p4 p5 . . .
.
.
.
Distance/Similarity Matrix
...p1 p2 p3 p4 p9 p10 p11 p12
Intermediate Statei After some merging steps, we have some clusters
C1
C4
C2 C5
C3
C2C1
C1
C3
C5
C4
C2
C3 C4 C5
...p1 p2 p3 p4 p9 p10 p11 p12
Distance/Similarity Matrix
Intermediate Statei Merge the two closest clusters (C2 and C5) and update the
distance matrix
C1
C4
C2 C5
C3
C2C1
C1
C3
C5
C4
C2
C3 C4 C5
...p1 p2 p3 p4 p9 p10 p11 p12
Distance/Similarity Matrix
After Mergingi “How do we update the distance matrix?”
C1
C4
C2 + C5
C3 ? ? ? ?
?
?
?
C2 +
C5C1
C1
C3
C4
C2 + C5
C3 C4
...p1 p2 p3 p4 p9 p10 p11 p12
Distance between two clustersi Single-link distance between clusters Ci and Cj is the minimum
distance between any object in Ci and any object in Cj
4 The distance is defined by the two most similar objects
jiyxjisl CyCxyxdCCD ,),(min, ,
I1 I2 I3 I4 I5I1 1.00 0.90 0.10 0.65 0.20I2 0.90 1.00 0.70 0.60 0.50I3 0.10 0.70 1.00 0.40 0.30I4 0.65 0.60 0.40 1.00 0.80I5 0.20 0.50 0.30 0.80 1.00
1 2 3 4 5
Distance between two clustersi Complete-link distance between clusters Ci and Cj is the
maximum distance between any object in Ci and any object in Cj
4 The distance is defined by the two least similar objects
I1 I2 I3 I4 I5I1 1.00 0.90 0.10 0.65 0.20I2 0.90 1.00 0.70 0.60 0.50I3 0.10 0.70 1.00 0.40 0.30I4 0.65 0.60 0.40 1.00 0.80I5 0.20 0.50 0.30 0.80 1.00
jiyxjicl CyCxyxdCCD ,),(max, ,
1 2 3 4 5
Distance between two clustersi Group average distance between clusters Ci and Cj is the average
distance between objects in Ci and objects in Cj
4 The distance is defined by the average similarities
I1 I2 I3 I4 I5I1 1.00 0.90 0.10 0.65 0.20I2 0.90 1.00 0.70 0.60 0.50I3 0.10 0.70 1.00 0.40 0.30I4 0.65 0.60 0.40 1.00 0.80I5 0.20 0.50 0.30 0.80 1.00
ji CyCxji
jiavg yxdCC
CCD,
),(1
,
1 2 3 4 5
Strengths of single-link clustering
Original Points Two Clusters
• Can handle non-elliptical shapes
Two Clusters
Limitations of single-link clustering
Original Points
• Sensitive to noise and outliers• It produces long, elongated clusters
Strengths of complete-link clustering
Original Points Two Clusters
• More balanced clusters (with equal diameter)• Less susceptible to noise
Limitations of complete-link clustering
Original Points
• Tends to break large clusters• All clusters tend to have the same diameter• Small clusters are merged with larger ones
Two Clusters
Average-link clustering
i Compromise between Single and Complete Link
i Strengths4 Less susceptible to noise and outliers
i Limitations4 Biased towards globular clusters
43
Clustering Application: Collaborative Filtering
i Discovering Aggregate Profiles4 Goal: to capture “user segments” based on their common behavior or interests4 Method: Cluster user transactions to obtain user segments automatically, then
represent each cluster by its centroidh Aggregate profiles are obtained from each centroid after sorting by weight and filtering
out low-weight items in each centroid
4 Profiles are represented as weighted collections of items (pages, products, etc.)h weights represent the significance of item within each cluster
4 profiles are overlapping, so they capture common interests among different groups/types of users (e.g., customer segments)
44
Aggregate Profiles - An ExampleA B C D E F
user0 1 1 0 0 0 1user1 0 0 1 1 0 0user2 1 0 0 1 1 0user3 1 1 0 0 0 1user4 0 0 1 1 0 0user5 1 0 0 1 1 0user6 1 1 0 0 0 1user7 0 0 1 1 0 0user8 1 0 1 1 1 0user9 0 1 1 0 0 1
A B C D E FCluster 0 user 1 0 0 1 1 0 0
user 4 0 0 1 1 0 0user 7 0 0 1 1 0 0
Cluster 1 user 0 1 1 0 0 0 1user 3 1 1 0 0 0 1user 6 1 1 0 0 0 1user 9 0 1 1 0 0 1
Cluster 2 user 2 1 0 0 1 1 0user 5 1 0 0 1 1 0user 8 1 0 1 1 1 0
PROFILE 0 (Cluster Size = 3)--------------------------------------1.00 C1.00 D
PROFILE 1 (Cluster Size = 4)--------------------------------------1.00 B1.00 F0.75 A0.25 C
PROFILE 2 (Cluster Size = 3)--------------------------------------1.00 A1.00 D1.00 E0.33 C
Original Session/user
data
Result ofClustering
Given an active session A B, the best matching profile is Profile 1. This may result in a recommendation for item F since it appears with high weight in that profile.
Given an active session A B, the best matching profile is Profile 1. This may result in a recommendation for item F since it appears with high weight in that profile.
45
Web Usage Mining: clustering example
i Transaction Clusters: 4 Clustering similar user transactions and using centroid of each cluster as an
aggregate usage profile (representative for a user segment)
Support URL Pageview Description
1.00 /courses/syllabus.asp?course=450-96-303&q=3&y=2002&id=290
SE 450 Object-Oriented Development class syllabus
0.97 /people/facultyinfo.asp?id=290 Web page of a lecturer who thought the above course
0.88 /programs/ Current Degree Descriptions 2002
0.85 /programs/courses.asp?depcode=96&deptmne=se&courseid=450
SE 450 course description in SE program
0.82 /programs/2002/gradds2002.asp M.S. in Distributed Systems program description
Sample cluster centroid from dept. Web site (cluster size =330)
46
Clustering Application: Discovery of Content Profiles
i Content Profiles4 Goal: automatically group together documents which partially deal with
similar concepts4 Method:
h identify concepts by clustering features (keywords) based on their common occurrences among documents (can also be done using association discovery or correlation analysis)
h cluster centroids represent docs in which features in the cluster appear frequently
4 Content profiles are derived from centroids after filtering out low-weight docs in each centroid
4 Note that each content profile is represented as a collections of item-weight pairs (similar to usage profiles)
h however, the weight of an item in a profile represents the degree to which features in the corresponding cluster appear in that item.
47
Content Profiles – An Example
PROFILE 0 (Cluster Size = 3)--------------------------------------------------------------------------------------------------------------1.00 C.html (web, data, mining)1.00 D.html (web, data, mining)0.67 B.html (data, mining)
PROFILE 1 (Cluster Size = 4)-------------------------------------------------------------------------------------------------------------1.00 B.html (business, intelligence, marketing, ecommerce)1.00 F.html (business, intelligence, marketing, ecommerce)0.75 A.html (business, intelligence, marketing)0.50 C.html (marketing, ecommerce)0.50 E.html (intelligence, marketing)
PROFILE 2 (Cluster Size = 3)-------------------------------------------------------------------------------------------------------------1.00 A.html (search, information, retrieval)1.00 E.html (search, information, retrieval)0.67 C.html (information, retrieval)0.67 D.html (information, retireval)
Filtering threshold = 0.5
48
User Segments Based on Contenti Essentially combines usage and content profiling techniques
discussed earlier
i Basic Idea:4 for each user/session, extract important features of the selected
documents/items4 based on the global dictionary create a user-feature matrix4 each row is a feature vector representing significant terms associated with
documents/items selected by the user in a given session4 weight can be determined as before (e.g., using tf.idf measure)4 next, cluster users/sessions using features as dimensions
i Profile generation:4 from the user clusters we can now generate overlapping collections of features
based on cluster centroids4 the weights associated with features in each profile represents the significance
of that feature for the corresponding group of users.
49
A.html B.html C.html D.html E.html
user1 1 0 1 0 1
user2 1 1 0 0 1
user3 0 1 1 1 0
user4 1 0 1 1 1
user5 1 1 0 0 1
user6 1 0 1 1 1
A.html B.html C.html D.html E.html
web 0 0 1 1 1
data 0 1 1 1 0
mining 0 1 1 1 0
business 1 1 0 0 0
intelligence 1 1 0 0 1
marketing 1 1 0 0 1
ecommerce 0 1 1 0 0
search 1 0 1 0 0
information 1 0 1 1 1
retrieval 1 0 1 1 1
User transaction matrix UT
Feature-DocumentMatrix FP
50
Content Enhanced Transactions
web data mining business intelligence marketing ecommerce search information retrieval
user1 2 1 1 1 2 2 1 2 3 3
user2 1 1 1 2 3 3 1 1 2 2
user3 2 3 3 1 1 1 2 1 2 2
user4 3 2 2 1 2 2 1 2 4 4
user5 1 1 1 2 3 3 1 1 2 2
user6 3 2 2 1 2 2 1 2 4 4
User-FeatureMatrix UF Note that: UF = UT x FPT
Example: users 4 and 6 are more interested in concepts related to Web information retrieval, while user 3 is more interested in data mining.
51
Clustering and Collaborative Filtering :: clustering based on ratings: movielens
52
Clustering and Collaborative Filtering :: tag clustering example
53
Hierarchical Clustering:: example – clustered search results
Can drill down within clusters to view sub-topics or to view the relevant subset of results