Witold Pedrycz Department of Electrical & Computer Engineering University of Alberta, Edmonton,...

Witold PedryczDepartment of Electrical & Computer Engineering University of Alberta, Edmonton, CanadaandSystems Research Institute, Polish Academy of SciencesWarsaw, Poland

wpedrycz@ualberta.ca

Agenda

Introduction: human-centricity of intelligent systems and information granules

Conceptualization and realization of information granules

Information content and its characterization

Context-based information granules

Successive refinements of information granules

Information granules –based architectures

Conclusions

Human-system interactionand system modeling

Perception and processing processing realized at certain level of abstraction

Acceptance of granular (non-numeric) data

Effective two-way communication with the user at the level of information granules

Adjustment of level of detail (abstraction) dependent upon the needs of the individual user (personalization); avoidance of unnecessary details and focus on essentials

Clustering and fuzzy clustering

Discovery of structures and relationships in data

Data analysis

Construction of fuzzy sets

Clustering in fuzzy modeling and modeling…

Information granules: from conceptualization to realization

Implicit information granulesImplicit information granulesExplicit (operational)information granules

Humans Computer realizations

Various points of view (models)Various points of view (models) Fuzzy setsRough setsIntervals (sets)Shadowed setsProbability functions

Information granulesInformation granules

Development of information granules

Usage of all available experimental evidence (numeric and non-numeric;knowledge hints)

Information granules capturing existing domain knowledge (especiallyguidance –knowledge hints provided by the designer/user)

Results of information granulation dependent upon the underlying formalism of Granular Computing

Information granules are context-dependent; the ensuing design framework should incorporate this aspect in an explicit way

Information granules: from conceptualization to realization

Data Information granules Construction of Intelligent systems

auxiliary guidance mechanisms

Content of information granules

Formalisms of sets, fuzzy sets, rough sets

Clustering and fuzzy clustering

Clustering 2,670,000

Fuzzy clustering 443,000

Rough clustering 268,000

Information granulation/granules 158,000

Granular Computing 104,000

Google Scholar, October 29, 2014

Objective function-based clustering

{x1, x2,…, xN}

Objective function Minimize w.r.t. structure

information granules G1, G2 , …,Gc Prototypes, medoidsv1, v2,…, vc

Partition matrix U

Fuzzy C-Means (FCM) as an exampleof fuzzy clustering

vi – prototypes

U- partition matrix

FCM – representation of information granules (granules)

Partition matrix U

prototypes v1, v2, …, vc

c 1,2,...,i ,Nu0N

1kik

Fuzzy Clustering: Fuzzy C-Means (FCM)

Given data x1, x2, …, xN, determine its structure byforming a collection of information granules – fuzzy sets

Objective function

2ik

N

1k

mik

c

1i||||uQ vx

Minimize Q; structure in data (partition matrix and prototypes)

FCM – flow of optimization

2ik

N

1k

mik

c

1i||||uQ vx

Minimize

subject to

(a) prototypes

(b) partition matrix

Construction of information granules: Fuzzy C-Means (FCM)

Data {x1, x2, …, xN} xk in Rn.

Performance index (objective function)

Construct information granules (clusters) - fuzzy sets A1, A2, …, Ac organized aspartition matrix U

Partition matrix

Prototype vi

Quality of clustering

Cluster validity indexes

Cluster content

Granulation-degranulation: reconstruction criterion

Information content of clusters

Description of information content:

•Variability of data

•Classification content

•Variability with regard to auxiliary (output) data

Variability of data

Description of data residing within a given ith cluster

Variability of data around the prototype vi

Variability in terms of membership grades of data

Classification content of clusters

Applied to classification problems.

Dominant class present in ith cluster

Classification content:

count index

cumulative membership grades of classes

Granular mapping: an architecture

Aggregation of contents of information granules and their activation levels

Auxiliary variable content

Problems in which occur some additional variables (say output variable, y) whose values determine the content of the cluster.

Clustering realized for data in the multivariable input space

Information granulation and degranulation: reconstruction criterion

v1 v2 vc

granulation

u1, u2, …, uc

v1 v2 vc

degranulation

Results of degranulation made more abstract (in the form of information granules):granular clustering

Key challenges of clustering

Selection of distance function (geometry of clusters)

Number of clusters

Quality of clustering results

Landscape of clustering

Graph-oriented and hierarchical (single linkage, complete linkage, average linkage..)

Objective function-based clustering

Variety of formalisms and optimization tools(e.g., methods of Evolutionary Computing)

Diversity

CommonalityData-driven methods

The dichotomy and a paradigm shift

Human-centricityGuidance mechanisms

Knowledge –based clustering

dataknowledge

Partial supervision

Context-based guidance

Proximity –based

Viewpoints

Domain Knowledge:categories of knowledge-oriented

guidance

Context-based guidance: clustering realized in a certain contextspecified with regard to some attribute

Viewpoints: some structural information is provided

Partially labeled data: some data are provided with labels (classes)

Proximity knowledge: some pairs of data are quantified interms of their proximity (resemblance, closeness)

Context-based clustering

Clustering : construct clusters in input space X

Context-based Clustering : construct clusters in input space X given some context expressed in output space Y

Active role of the designer [customization of processing]

Context-based clustering:Conmputational considerations

•computationally more efficient,•well-focused, •designer-guided clustering process

Data

structure

Data

structure

context

Context-based clustering:focus mechanism

Determine structure in input space given the output is high

Determine structure in input space given the output is medium

Determine structure in input space given the output is low

Input space (data)

Context-based clustering:examples

Find a structure of customer data [clustering]

Find a structure of customer data considering customers making weekly purchases in the range [$1,000 $3,000]

Find a structure of customer data considering customers making weekly purchases at the level of

around $ 2,500

Find a structure of customer data considering customers making significant weekly purchases who

are young

no context

context

context

context(compound)

Context-based Fuzzy C-Means

data(xk, yk), k=1,2,…,N

contexts: fuzzy sets W1, W2, …, Wp defined in the output space

wjk = Wj(yk)

c

1i

N

1kikjkikikj iNu0andk wu|0,1u)(WU

Context-drivenpartition matrix

Context-based clustering:the use of context

xk

Context Wj

yk

Wj(yk)

xkContext-based fuzzy clustering

Context-oriented FCM:Optimization flow

Objective function

Iterative adjustment of partition matrix and prototypes

2ik

c

1i

N

1k

mik ||||uQ vx

c

1j

1m

2

jk

ik

jkik

wu

vx

vx

N

1k

mik

N

1kk

mik

i

u

u xv

Subject to constraint U in U(Wj)

Successive refinements of information granules

Information granules constructed in a successive manner forming a hierarchy of refined constructs of higher specificity

The refinement applied to information granules based on their information content

Successive usage of context-based fuzzy clustering

Successive refinements of information granules

information granuleto be refined

membership function Ai [1] used as a contextrefinement process

membership function Aj [2] used as a context

Expansion formulas:Context-based FCM

information granuleto be refined

membership function Ai [1] used as contextrefinement process

property of fuzzy partition

membership function Aj [2] used as context

Expansion formulas:Context-based FCM

information granuleto be refined

membership function Ai [1] used as context

membership function Aj [2] used as context

Successive fuzzy partitions

Fuzzy clustering with viewpoints

Viewpoints: definitionDescription of entity (concept) which is deemed essential in describing phenomenon (system) and helpful in castingan overall analysis in a required setting

“external” , “reinforced” clusters

Viewpoints: examples

-150

-100

-50

0

50

100

150

200

0 100 200 300 400 500

x1

x2

a

b

x1

x2

a

viewpoint (a,b) viewpoint (a,?)

Viewpoints in fuzzy clustering

x1

x2

a

b

otherwise 0,

viewpointby the determined is B of rowth -i theof featureth -j theif 1,b ij

0

0

1

0

0

1

B

0

0

b

0

0

a

F

B- Boolean matrix characterizing structure: viewpoints prototypes (induced by data)

Viewpoints in fuzzy clustering

Q = 2ijkj

n

1:bji,1j

mik

c

1i

N

1k

2ijkj

n

0:bji,1j

mik

c

1i

N

1k

)f(xu)v(xu

ijij

1b if f

0bif vg

ijij

ijijij

2ijkj

n

1j

mik

c

1i

N

1k

)g(xuQ

Fuzzy clustering with proximity guidelines

Proximity hints

Characterization in terms of proximity degrees:

Prox(k, l), k, l=1,2, …., N

and supervision indicator matrix B = [bkl], k, l=1,2,…, N

Prox(k,l)

Prox(s,t)

Proximity measureProperties of proximity:

(a)Prox(k, k) =1

(b)Prox(k,l) = Prox(l,k)

Proximity induced by partition matrix U

Linkages with kernel functions K(xk, xl)

Augmented objective function

> 0

Granular fuzzy clustering

Granular prototypes and reconstruction criterion

prototypes granular prototypes

Selection/construction of prototypes

Forming granular prototypes to capture existing structural variability and satisfyingdegranulation criterion

(a) information granules of prototypes built around prototypes(b) optimization of allocation of granularity by minimizing reconstructioncriterion

Formation of granular (interval) membership grades – details

xVi

ui-(x)=min(w1(x), w2(x))

ui+(x)=max(w1(x), w2(x))

Overview

Information granules Blueprint of model

content

Model development,refinements, augmentations

Conclusions

Fuzzy clustering as a conceptual and algorithmic backbone ofdesign of information granules

Human-centric (knowledge-oriented) design of information granules

Emergence of higher type granular constructs

Needs for further advancements in optimization frameworks of fuzzy clustering