Linear and Non-Linear ICA-BSS

• I C A -------- Independent Component Analysis • B S S -------- Blind Source Separation

Carlos G. PuntonetDept.of Architecture and Computer TechnologyCircuits and system for information processing groupUniversity of Granada (Spain)

The Problem of “linear” blind separation of p sources:

Original signals: s(t)=[s1(t),....,sp(t)]T

Mixture: e(t)=[e1(t),...,ep(t)]T

Mixture matrix: A(t) pxp

The goal is to estimate A(t) by means of W(t) such that the output vector, s*(t) is:

s(t)W1(t)e(t)

W(t)A(t)PD

e(t)A(t)s(t)

*( ) ( )s t s t

REAL APLICATIONS

BSS is Independent Component Analysis (ICA)

Noise Elimination in general

Speech Processing (Cocktail Party, Noise environment,...)

Sonar, Radar

Sismic waves

Preprocessing recognition

Image Processing

Biomedicine (ECG, EEG, fMRI,...)

Geometric methods I: Digital

* Binary SignalsBinary Signals

S1u = (1,...,0,...,0)t

..................... Si

u = (0,...,1,...,0)t

..................... Sp

u = (0,...,0,...,1)t

  The image of a base vector Siu is the vector Aoi, i.e.

the column i of the unknown mixture matrix Ao:

h(Siu) = Aoi

p

k k i ki=1

( n -1,..., n -1) = (n -1) = (n -1) (1,...,1,...,1)h a h

* n-valued Signals Signals

Geometric methods II: Slopes

Fij(t)ei(t).ej(t)1 ej(t)0,i,j{1,...,p}

Fij/sk 0Fijaik/ajk i,j,k{1,...,p}

S(0,0,...,sl,...,0,0)T l{1,...,p}

For input Vectors:

Slope Function:

Extreme values:

aij = min { ei . ej-1 } ej > 0 œi,j{1,..,p}

** Fast method for p=2 signals

** Valid for random or bounded sources

** Slopes are the independent components

** Modifiable for p>2

s1

s2 e2

1

s1

s2

e

p1

p3

p2

q1

q2

q3

O O

a / a 12 22

a / a21 11

M

M

ei

ej

ei

ej

q1

q2

q3

q4

q5

a / a il jl

a / a ij jj

a / a ij jj

O O

e 'i

e 'j

Simulation example (p=3, 1000 samples)

GENERAL p-DIMMENSIONAL METHOD

11 1 1j j 1p p

i1 1 ij j ip p

p1 1 pj j pp p

... ... a s a s a s... ... ... ... ...

... ... W = a s a s a s... ... ... ... ...

... ... a s a s a s

Obtained matrix W:

p-1 p

i j i j k ki=1 j=i+1

cos , ) > cos ( , ) , k {1,...,p}v v w w v wp-1 p

i=1 j=i+1

= (

For p points verifying minimum value of:

ADAPTIVE NETWORK

** Geometric method for p signals

** Valid for random or bounded sources

** Slopes are the independent components

** No order statistics

** Probability of obtaining p points close to the hiper- parallelepiped edges ?

Geometric methods III: Speech

- For Linear mixtures

- Unimodal p.d.f.’s (non-uniform’s)

- Detection of max.density points in the mixture space

- Normalization and detection in the sphere radius-unit.

3

0.8 0.4

0.5 0.9A

SOURCESPACE

MIXTURESPACEFROM

Detection of 2 maxima ( 2 ICA components, p=2 )

Amari index

NEW GEO-METHOD with KURTOSIS.

• (K(e1)>0) and (K(e2)>0) • (K(e1)<0) or (K(e2)<0)

Lattice of Space

• M1*M2 Cells. Threshold (TH), and Red-Cells with points > TH.

ICA COMPONENTS FROM KURTOSIS:

• If K(e1)>0 and K(e2)>0 Maximum Density Zones

• If K(e1)<0 or K(e2)<0 Border Detection

Separation of Sources usingSimulated Annealing and Competitive Learning

Univ. Regensburg and Univ. Granada

- New adaptive procedure for the linear and “non-linear” separation - Signals with non-uniform, symmetrical probability distributions- Simulated annealing, competitive learning, and geometric methods- Neural network, and multiple linearization in the mixture space- Simplicity and rapid convergence - Validated by speech signals or biomedical data.

Geometric methods IV: Heuristic + Neural networks

e1

e2

w4 (D)

w3 (D)

w2 (D)

w1 (D)

D1

D2

D3

D4

eD (t)

Observation space with n p-spheres (n=4, p=2)

k1<||e(t)||<k 00k{1,...,n}Space Quantization:

d(i,k)||wi(k,t)e(k,t)||i{1,...,2p}k{1,...,n}

wi(k,t1)wi(k,t)(k,t)sgn[e(k,t)wi(k,t)]Ki(t)Ki(t)exp(1(t)||wi(k,t)wi(k,t)||2) ii{1,...,2p}

W ρk

w1 1 ρk... w1 p ρk

wp 1 ρk... wp p ρk

k {1,...,n }

Competitive Learning:

Simulated Annealing:

EEij(t)<(cum22[si(t)sj(t)])2> i,j{1,...,p}

Energy Function:

Fourth-order cumulant :

Wsijk(t)2rij1 i,j{1,...,p}ijk{1,...,n}Wsijk(t)2rij1 i,j{1,...,p}ijk{1,...,n}

Weights generation:

2 2 2 2 222 ( ( ), ( )) ( ) ( ) ( ) ( ) 2 ( ) ( )i j i j i j i jcum s t s t s t s t s t s t s t s t

Simulated Annealing and Competitive Learning

Wijk(t1)Wsijk(t)(t)Wcijk(t)(1(t))ij{1,...,p}k{1,...,n}

1

0

time SA CL

1

2

e (t)

e1

e2

NON-LINEAR: Contour for where the mixture can be considered linear

( ) ( )

( ) ( )

w ( , ) w ( , )( ) , { ,..., } { ,..., }

w ( , ) w ( , )

( ) { (1) (2) ... ( ) ( ( ), ) ( ( ), )} {1...2 }

k

j i k j j ki

j j k j j k

k k

t tt i j p k n

t t

j p d j d m m p m j

W

1

1

1 1

e 1

e 2

x 3 component

Simulation 1: 3 signals

e 1

e 2

W S21

W S12

W S13 / W

S23

e 1

e 2

W21

W12

W13 / W23

s 1

s 2

e 1

e 2

Simulation 2: Non-linear mixture of 2 digital 32-valued signals

e 1

e 2

w4 (4)

w2 (4)

w1(4)

w3 (4)

e1(t)2sgn[x1(t)]x1(t)21.1x1(t)x2(t)e2(t)2sgn[x2(t)]x2(t)21.1x2(t)x1(t)

Wρ(1)1 1.7

1.6 1Wρ (2)

1 0.25

0.22 1

Wρ(3)1 0.2

0.22 1Wρ(4)

1 0.1

0.15 1

Simulation 3: EEG signals

Eye blink -->

Low wave 1 -->

Musc. Spik. -->

Low wave 2 -->

Cardi. Contam. -->

Neural network for the separation

s1

s2

sp

e2

ep

e1

Wp p-1

W12 W

21W

p1

W2p

W1p

1 1 1

1...

s ( 1) e ( , ) W ( ) s ( ) { } { }ki i k ij j

j p

t t t t i p i j k n

Real Time Simulation

GENETIC ALGORITHMS FOR NON LINEAR ICA

)()( tAsFtx

n

jjjiji txgwty

1

))(()(

12

1

)(

k

j

P

kjkjj xgxg

n

j

kj

P

kjkiji xgwy

1

12

1

Genetic Algorithms are one of the most popular stochastic optimisation techniques. Inspired by natural genetics and the biological evolutionary process:

* A scheme for encoding solutions to a problem in the form of a chromosome (chromosomal representation).

 * An evaluation function which indicates the fitness of each chromosome relative to the others in the current set of chromosomes (referred to as population).

 *  An initialisation procedure for the population of chromosomes.

 * A set of parameters that provide the initial settings for the algorithm: the population size and probabilities employed by the genetic operators.

*The GA evaluates a given population and generates a new one iteratively, with each successive population referred to as a generation, from genetic operations: reproduction, crossover and mutation.

ICA and BSS have LOCAL MINIMA

SIMULATIONS

¡¡¡¡¡ THE END !!!!!

THANK YOU VERY MUCH

DANKESHÖN

GRACIAS