RTC 2010 1

REAL-TIME INDEPENDENT COMPONENT ANALYSIS IMPLEMENTATION AND APPLICATIONS

By

MARCOS DE AZAMBUJA TURQUETIturqueti@fnal.gov

FERMILAB

May 2010

RTC 2010 2

What is it?

•Independent component analysis or ICA is a mathematical technique used for extracting hidden parameters that underlie in sets of random variables or signals.•ICA is a type of blind source separation

method and common inputs sources are signals originated from audio, images or telecommunications.

RTC 2010 3

ICA Applications

RADAR

• A wide variety of systems can make use of ICA algorithms;

CCD signal processing SONAR

PIXEL detectors Medical ultrasonography Positron Emissions Scan machines

RTC 2010 4

Assumptions

•This technique is based on the assumption that signals from different sources are statistically independent and non-Gaussian.

•At most one signal can be Gaussian otherwise this technique does not work.

•Information about magnitude of the signal is lost.

•ICA can provide redundant outputs.

5RTC 2010

ICA

Prep

roce

ssin

g

Centering

Whitening

High pass filter

Measure Gaussianity

Compute weight vector

Correlation filter

ICA

Algo

rithm

Post

Pro

cess

ing

Algo

rithm

Application Preprocessing

RedundancyElimination

Converged?

Yes

No

Algorithm implementation

J (Y) = H(Ygauss) - H(Y)

cov( X Y )= I

differential entropy (negentropy)

Dn= J(yn)-J(yn-1)

X= WTY

population correlation coefficient

sample correlation coefficient

make input variables zero mean

RTC 2010 6

A B

M1 M20 10 20 30 40 50 60 70 80 90 100

-1

-0.5

0

0.5

1

A M

agni

tude

Signal A

0 10 20 30 40 50 60 70 80 90 100-1

-0.5

0

0.5

1

Sample number

B M

agni

tude

Signal B

0 10 20 30 40 50 60 70 80 90 100

-0.5

0

0.5

M1

Mag

nitu

de

Variable M1

0 10 20 30 40 50 60 70 80 90 100-1

-0.5

0

0.5

1

Sampe number

M2

Mag

nitu

de

Variable M2

Algorithm simulation

Linear combination

RTC 2010 7

Whitening of the signal (decorrelating)

Variable M1

Var

iabl

e M

2

Variable M1 x M2

-0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.40

50

100

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

050100 Variable W1

Var

iabl

e W

2

Data whiten

-4 -2 0 2 4 60

50

100

-5

-4

-3

-2

-1

0

1

2

3

4

050100

cov( X Y )= I

Joint probability distribution of the mix signals before and after whitening.

Mix signals

RTC 2010 8

CAPTAN Network Compatible Hardware

Signal A recovered

Sig

nal B

reco

vere

d

Signal AxB recovered

-0.6 -0.4 -0.2 0 0.2 0.40

100

200

-0.5

0

0.5

0100200

Minimizing Gaussianity by rotating the axis and using Negentrophy to have a indication of Gaussianity.

Variable W1

Var

iabl

e W

2 Data whiten

-4 -2 0 2 4 60

50

100

-5

-4

-3

-2

-1

0

1

2

3

4

050100

Joint probability distribution of the whiten signal and ICA output.

X= WTY

J (Y) = H(Ygauss) - H(Y)

RTC 2010 9

Recovered signal

0 10 20 30 40 50 60 70 80 90 100-2

-1

0

1

2

A M

agni

tude

Signal B recovered

0 10 20 30 40 50 60 70 80 90 100-2

-1

0

1

2

Sample numberB

Mag

nitu

de

Signal A recovered

0 10 20 30 40 50 60 70 80 90 100-1

-0.5

0

0.5

1

A M

agni

tude

Signal A

0 10 20 30 40 50 60 70 80 90 100-1

-0.5

0

0.5

1

Sample number

B M

agni

tude

Signal B

Original signal Recovered signal

RTC 2010 10

Hardware platform for ICA implementation

•Due to the large number of operations involving arrays this algorithm is proper to be implemented by multi-core vectorial processors. •FPGA’s are also specially suitable due to flexibility and parallelism capabilities.•On this work parallel FPGA’s are being used as hardware platform for the algorithm implementation.

CAPTAN System (FERMILAB) Sensor Array

RTC 2010 11

Implementation constrains• In order for the algorithm to converge fast a maximum number of interactions to maximize the non-Gaussianity is allowed;•This maximum number of interactions depends on several factors such as:

• The system speed;• Type of signals being input into the algorithm;• Number of mixed sources;• Amount of noise on the system; (Gaussian

source)

RTC 2010 12

Real-time application example

Source separation can be used in many different applications such as:

•Video conference•Cell phones•Sonar•Medical

RTC 2010 13

0 20 40 60 80 100 120 140 160 180 2001600

1800

2000Capture signals

Mag

nitu

de

0 20 40 60 80 100 120 140 160 180 2001000

2000

3000

Mag

nitu

de

0 20 40 60 80 100 120 140 160 180 2001600

1800

2000

Mag

nitu

de

0 20 40 60 80 100 120 140 160 180 2001500

2000

2500

Mag

nitu

de

0 20 40 60 80 100 120 140 160 180 2001700

1800

1900

Mag

nitu

de

Sample Number

Test I - Four Sine Waves Test

RTC 2010 14

Test I - Results

0 20 40 60 80 100 120 140 160 180 2000

1

2

3

4

5

0 20 40 60 80 100 120 140 160 180 200-1

0

1

2

3

4

Sample Number

0 20 40 60 80 100 120 140 160 180 200-5

0

5

0 20 40 60 80 100 120 140 160 180 200-10

0

10

0 20 40 60 80 100 120 140 160 180 200-5

0

5

0 20 40 60 80 100 120 140 160 180 2000

5

10

0 20 40 60 80 100 120 140 160 180 200-5

0

5

Sample Number

Two microphones fastICA results:

Four microphones fastICA results:

Four microphones fastICA/ AIIA results:

RTC 2010 15

Test I - Results

0 20 40 60 80 100 120 140 160 180 200-5

0

5

0 20 40 60 80 100 120 140 160 180 200-2

0

2

0 20 40 60 80 100 120 140 160 180 200-5

0

5

0 20 40 60 80 100 120 140 160 180 200-5

0

5

0 20 40 60 80 100 120 140 160 180 200-5

0

5

Sample Number

Fast ICA algorithm results: AIIA algorithm results:

RTC 2010 16

0 1 2 3 4 5 6 7 8 9 10

x 104

1000

1500

2000

2500Chicken

AD

C c

ount

s

0 2 4 6 8 10 12

x 104

0

1000

2000

3000Pelican

Sample number

AD

C c

ount

s

Test II – Pelican x Chicken

0 2 4 6 8 10 12

x 104

500

1000

1500

2000

2500

3000Chicken+Pelican

Sample number

AD

C c

ount

s

RTC 2010 17

0 2 4 6 8 10 12 14

x 104

-20

0

20

0 2 4 6 8 10 12 14

x 104

-10

0

10

Test II - Results

Mix signals before fastICA: Signals after fastICA:

Signals after fastICA (8 microphones):

0 2 4 6 8 10 12 14

x 104

-10

0

10

0 2 4 6 8 10 12 14

x 104

-20

0

20

Sample Number

Signals after fastICA (12 microphones):

RTC 2010 18

0 1 2 3 4 5 6 7 8 9

x 104

-1

-0.5

0

0.5

1AK47 Rifle

0 1 2 3 4 5 6 7 8 9

x 104

-1

-0.5

0

0.5

1A10 Aircraft

Sample Number

0 1 2 3 4 5 6 7 8 9

x 104

0

500

1000

1500AK47 Rifle

Mag

nitu

de

0 1 2 3 4 5 6 7 8 9

x 104

0

500

1000

1500A10 Aircraft

Sample Number

Mag

nitu

de

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 105

1000

1200

1400

1600

1800

2000

2200

2400

2600

2800

AD

C c

ount

s

Sample Number

Test III – AK47 x Turbine

(A)

(A)

(B)

(B)

Red – configuration 1Green – configuration 2 (0.5 s delay)

RTC 2010 19

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 105

-20

0

20

40

60

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 105

-30

-20

-10

0

Test III – Recovered Signals

0 0.5 1 1.5 2 2.5

x 105

-20

0

20

40

60

0 0.5 1 1.5 2 2.5

x 105

0

10

20

30

fastICA/AIIAConfiguration 1

fastICA/AIIAConfiguration 2

AK47

AK47

Turbine

Turbine

RTC 2010 20

Conclusion

•This work shows that it is possible to use the ICA algorithm on real-time applications;

•It also demonstrates the capabilities of the algorithm as well as its limitations;

•Currently the algorithm is being implemented for applications that demand faster convergence time.

RTC 2010 21

Thank you!