8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 1/26

Thiago Christiano Silva

Liang Zhao

Institute of Mathematics and Computer Science

University of São Paulo, São Carlos, São Paulo, Brazil

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 2/26

Summary      Introduction

      Complex Networks

      Communities

      Competitive Learning

      Proposed Technique

      Description of the Technique

      Mathematical Analysis of the Model

      Time Complexity Analysis of the Model

     Determining the optimal number of particles in the model

      Computer Simulations

      Artificial Data Sets

      Real-world Data Sets

      Conclusions

2

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 3/26

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 4/26

Communitiesy A sub-graph whose nodes are densely connected within

itself, but sparsely connected with the rest of the network

4M. Girvan and M. E. J. Newman. Community structure in social and biological networks. Proceedings of the National Academy of Sciences of the United States of America, 99(12):7821±7826.

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 5/26

Competitive Learningy Observed in nature and in many social systems sharing limited

resources

y Water, food, mates, territory, recognition, etc.

y Important field of Machine Learning

y Widely implemented in neural networks

y Several real-world applications

y Early works include:

y Self-organizing maps (SOM)

y Differential Competitive Learning

y Adaptive Resonance Theory (ART)

5

T. Kohonen, ³The self-organizing map,´ Proceedings of the IEEE, vol. 78, no. 9, pp. 1464 ±1480, 1990.

B. Kosko, ³Stochastic competitive learning,´ IEEE Trans. Neural Networks, vol. 2, no. 5, pp. 522±529, 1991.

S. Grossberg, ³Competitive learning: From interactive activation to adaptive resonance,´ Cognitive Science, vol.

11, pp. 23±63, 1987.

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 6/26

Prior Related Worky Originally proposed by Quiles et Al.

y Several particles walk in the network and compete with each

other to mark their own territory, while attempting to reject

intruder particles

y Each particle can perform: Random Walk or Deterministic

Walk

y Only a procedure of particle competition is introduced

without formal definition

y Only applied to community detection tasks

6

M. G. Quiles, L. Zhao, R. L. Alonso, and R. A. F. Romero, ³Particle competition for complex network

community detection,´ Chaos, vol. 18, no. 3, p. 033107, 2008.

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 7/26

Contributions of the Proposed Techniquey A new type of competitive learning mechanism inspired

by the work in Quiles et Al.

y Here, the particle competition is formally represented by

a stochastic dynamical system

y A mathematical analysis has been carried out to predict

the outcome of the technique

y

We have applied the model not only for communitydetection, but also for data clustering

y A procedure for estimating the number of clusters in a

data set is presented

7

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 8/26

Description of the Techniquey

8

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 9/26

Notationy

9

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 10/26

Particles Movement Policyy

10

RANDOM TERM

y Adventurous Behavior

y Does not take into account

the dominated vertices

DETERMINISTIC TERM

y Defensive Behavior

y Prefers visiting vertices with

high domination levels

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 11/26

Stochastic Dynamical Systemy Essentially, formed by two expressions:

1. Perform the transition from all particles: merely by

random number generation, whose probability transitiondistribution is equal to the transition matrix previously

given

2. Update of the number of visits received by all vertices by

the particles:

11

OBS.: One can see that the proposed dynamical system is Markovian, since it only depends

on the present state to completely define the immediate future state

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 12/26

12

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 13/26

Mathematical Analysis

y

13

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 14/26

14

2

4

3

113

14

176

5

7

8

9

15

16

18

19

20

10

11

12

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 15/26

Time Complexity Analysisy

15

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 16/26

16

Experiments conducted on random clustered networks

L. Danon, A. Díaz-Guilera, J. Duch, and A. Arenas, ³Comparing community structure identification,´ J. Stat. Mech., p.P09008, 2005.

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 17/26

Determination of the number of clustersy The number of clusters is not known a priori

y The proposed dynamical system carries a rich set of 

information

y We are able to use this information to create a new

embedded measure that estimates the number of 

clusters

y Since it is embedded, no extra processing is necessary

y We will verify that the optimal number of particles

happens exactly when it is equal to the number of 

clusters

17

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 18/26

y

18

Determination of the number of clusters

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 19/26

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 20/26

20

Computer Simulations

Particles are randomly

inserted into vertices

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 21/26

2121

Computer Simulations

Particles are purposefully

inserted into the worst

case scenario at the

beginning (all in one

community)

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 22/26

22

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 23/26

2323

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 24/26

Real-world data sets

24

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 25/26

25

8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering

http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 26/26

Conclusionsy We have proposed an unsupervised technique based on

competitive learning

y A rigorous definition has been provided using a nonlinear

stochastic dynamical system

y A mathematical analysis has been carried out

y The proposed method presents low time complexity

y A method for determining the optimal number of particles

and the number of clusters has been discussed

y Computer simulations have been performed and

satisfactory results have been obtained

y More importantly, this work is an attempt to provide an

alternative way to the study of competitive learning

26