Upload
anna-jones
View
222
Download
0
Embed Size (px)
Citation preview
1
Pattern Recognition:Statistical and Neural
Lonnie C. Ludeman
Lecture 29
Nov 11, 2005
Nanjing University of Science & Technology
2
Lecture 29 Topics
1. Review Fuzzy Sets and Fuzzy Partitions
2. Fuzzy C- Means Clustering Algorithm Preliminaries
3. Fuzzy C-Means Clustering Algorithm Details
4. Fuzzy C-Means Clustering Algorithm Example
5. Comments about Fuzzy Clustering
3
Example of a Fuzzy Set:
Define a Fuzzy set A by the following membership function
Or equivalently
Example
Review
4
S
Function defined on S
Review
5Membership Functions for Fuzzy Clusters
Domain Pattern Vectors
Domain Pattern Vectors
Membership Function for F1
Membership Function for F2
EXAMPLE
Review
6
A Fuzzy Partition F, of a set S, is defined by its membership functions for the fuzzy sets Fk : k =1, 2, ... , K
) ]
Fuzzy Partition
Review
7
where
Each value bounded by 0 and 1
Sum of each columns values =1
Sum of each row is less than n
Review
8
Conversion of Fuzzy Clustering into Crisp Clustering
) ]
1 1 1
2 2 2
C C C
Fuzzy Clusters
Crisp Clustering
All membership values are equal to 1 or 0
Only one 1 in each column where membership value for a Fuzzy cluster is a maximum
9
Fuzzy Clusters
Crisp Clusters from Fuzzy Clusters
Cl1 : [ 0.6 0.7 0.3 0.1 0.4 0.2 0.1 ] Cl2 : [ 0.1 0.1 0.3 0.5 0.1 0.7 0.1 ] Cl3 : [ 0.3 0.2 0.4 0.4 0.5 0.1 0.8 ]
Cl1 : [ 1 1 0 0 0 0 0 ] Cl2 : [ 0 0 0 1 0 1 0 ] Cl3 : [ 0 0 1 0 1 0 1 ]
Example: Given Fuzzy clusters below convert to Crisp Clusters
10
Cl1 : [ 1 1 0 0 0 0 0 ] Cl2 : [ 0 0 0 1 0 1 0 ] Cl3 : [ 0 0 1 0 1 0 1 ]
Cl1 : { x1 , x2 }
Cl2 : { x4 , x6 }
Cl3 : { x3 , x5 , x7 }
Cl1 : { x1 x2 }
Cl2 : { x4 x6 }
Cl3 : { x3 x5 x7 }
Answer: Crisp Clusters
11
Fuzzy C-Means Clustering Preliminary
Given a set S composed of pattern vectors which we wish to cluster
) ]
1 1 1
2 2 2
C C C
S = { x1, x2, ... , xN}
Define C Cluster Membership Functions
... ...
12
Define C Cluster Centroids as follows
Let Vi be the Cluster Centroid for Fuzzy
Cluster Cli , i = 1, 2, …, C
Define a Performance Objective Jm as
where
13
The Fuzzy C-Means Algorithm minimizes Jm
by selecting Vi and i , i =1, 2, … , C by an
alternating iterative procedure as described in the algorithm’s details
m = Fuzziness Index (m >1 ) Higher numbers being more fuzzy
A is a symmetric positive definite matrix
Ns is total number of pattern vectors
Definitions
14
Fuzzy C-Means Clustering Algorithm (a) Flow Diagram
No Yes
15
Fuzzy C-Means Clustering Algorithm
(b) Details of Steps in the Algorithm
16
Step 1: Initialization
Select initial membership functions such that
This is equivalent to specifying fuzzy clusters F1, F2, … , FC
17
One Method to accomplish this selection is
to chose rki randomly from the open
interval (0, 1) and then Normalize
18
Step 2: Computation of Fuzzy Centroids
Compute the Fuzzy Centroids as
19
Step 3: Compute New Fuzzy Membership Functions
Using the Vi, i = 1, 2, … , C from step 2 compute i(k)
20
Alternative method for computing the Membership Functions
where
21
If membership functions do not change convergence has occurred
If algorithm converges then the i
represent the fuzzy clusters and we Stop
If Convergence has not occurred and the number of iterations is less than some preassigned maximum value (MAXIT) then return to step 2
If otherwise then Stop with no solution.
Step 4: Check for Convergence
22
Usually convergences
Answer not unique as depends upon initial conditions.
Can converge to a local minimum
Can be used to produce hard clusters
Properties of Fuzzy C-Means Algorithm
23
Example – Application of Fuzzy Clustering Algorithm
Given the following set of data vectors
(a) Perform a Fuzzy Clustering of the data using Fuzzy C-Means Algorithm to obtain two fuzzy clusters. Try several initial conditions. Is the result unique? (use MAXIT=1000)
(b) Using the results of (a) give a crisp clustering of the data.
(c) Repeat (a) and (b) for three Fuzzy Clusters.
24
Plot of Data for Fuzzy clustering example
25
(a) Solution for two clusters
Fuzzy cluster membership functions randomly selected
Calculation of Fuzzy Centroids
26
Calculation of New Membership Functions
Calculation of Performance
Not the same as preceding iteration Membership function (No convergence)
Number of iterations not greater than 1000 therefore the iterations continue.
27
Results Converge at Iteration 17
Cluster membership Functions
Performance Measure
Cluster Centroids
28
(a) Final Cluster membership Functions
Cl2 :
Cl1 :
29
(b) Solution Crisp Clustering
Cl1 = { x1, x2, x3 } Cl2 = { x4 , x5 }
Crisp Membership functions
Set Assignment
Cl2 :
Cl1 :
Fuzzy Membership functions
30
Jm = 0.9337
(c) Solution for three fuzzy clustersApplying the Fuzzy Clustering Algorithm convergence was obtained in ?? iterations as
Final Cluster membership functions
Cl1 :
Cl2 :
Cl3 :
31
(c) Solution Crisp Clustering
Cl1 = { x1 , x2 } Cl2 = { x4 , x5 } Cl3 = { x3 }
Membership functions
Set Assignment
F1 : [ 1, 1, 0, 0, 0 ] F2 : [ 0, 0, 0, 1, 1 ] F3 : [ 0, 0, 1, 0, 0 ]
“Crisp Clusters”
32
Comments: Fuzzy Clustering Can be used to
Produce Hard Clustering
The larger the value of m the fuzzier the clusters
The Fuzzy algorithm is relatively stable and usually converges in a reasonable number of iterations
The Fuzzy algorithm is relatively insensitve to initial conditions
33
Of the two different fuzzy clusterings given
below, which clustering is the Fuzzier ???
Cl1 : [ 0.52 0.51 0.04 0.47 0.46 ] Cl2 : [ 0.48 0.49 0.96 0.5 0.53 ]
Cl1 : [ 0.89 0.85 0.04 0.26 0.15 ] Cl2 : [ 0.11 0.15 0.96 0.74 0.85 ]
# 1
# 2
or
34
Cl1 : [ 0.52 0.51 0.04 0.47 0.46 ] Cl2 : [ 0.48 0.49 0.96 0.53 0.53 ]
Cl1 : [ 0.89 0.85 0.04 0.26 0.15 ] Cl2 : [ 0.11 0.15 0.96 0.74 0.85 ]
# 1
# 2
ANSWER: #1 is the fuzzier of the two different clusterings
35
Why is #1 the Fuzzier of the two ???
36
Why is #1 the Fuzzier of the two ???
ANSWER: Because the cluster membership functions contain many entries close to 0.5 ( for the two class case) as opposed to values close to 0 and 1.
For the M class case values close to 1/M would indicate most fuzziness..
*
*
37
Lecture 29 Topics
1. Reviewed Fuzzy Sets and Fuzzy Partitions
2. Presented Fuzzy C- Means Clustering Algorithm Preliminaries
3. Gave Fuzzy C-Means Clustering Algorithm Details
4. Showed an Example of the Fuzzy C-Means Clustering Algorithm
5. Made a few comments about Fuzzy Clustering in General.
38
End of Lecture 29