Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"

Eras

mus

+ s

emin

ar, 18

/04/

2016

1 / 26

Faculty of ComputerScience andInformationTechnology

West Pomeranian University of Technology,Szczecin

Dimensionality reduction methods applied to digital image processing

and recognition

Paweł ForczmańskiChair of Multimedia Systems, Faculty of Computer Science and Information

Technology, West Pomeranian University of Technology, Szczecin

Vilnius University, Institute of Mathematics and Informatics, 18/04/2016

Eras

mus

+ s

emin

ar, 18

/04/

2016

2 / 26



AgendaAgenda

Subspace concept in computer visionSubspace concept in computer vision

Application to image recognition: Eignefaces approach

Application to image recognition: Eignefaces approach

One-dimensional linear dimensionality reduction: PCA/KLT, LDA/KLT

One-dimensional linear dimensionality reduction: PCA/KLT, LDA/KLT

Two-dimensional linear dimensionality re-duction: 2DPCA/2DKLT, 2DLDA /2DKLT

Two-dimensional linear dimensionality re-duction: 2DPCA/2DKLT, 2DLDA /2DKLT

Application to image processing: watermarking, scrambling

Application to image processing: watermarking, scrambling

Algo

rytm

y Roz

pozn

awan

ia W

zorc

ów

3 / 26

1 1 2 2

1 2

ˆ ...

where , ,..., is a basein the -dimensionalsub-space (K<N)K K

K

x b u b u b u

u u u K

= + + +

x̂ x=

1 1 2 2

1 2

...

where , ,..., is a basein theoriginal N-dimensionalspaceN N

n

x a v a v a v

v v v

= + + +

The problem of determining a basis in low-dimensional sub-space:− Approximation of vectors by projecting them into a new, low-dimensional sub-

space:

(1) Initial representation:

(2) Low-dimensional representation:

• Remark: if K==N, then

Subspace? (1/2)Subspace? (1/2)

where is a basis in N-dimensional space

is a basis in K-dimensional subspace (K<N)where

Algo

rytm

y Roz

pozn

awan

ia W

zorc

ów

4 / 26

Subspace? (2/2)Subspace? (2/2)

Example (K==N):

Eras

mus

+ s

emin

ar, 18

/04/

2016

5 / 26



PCAPCA

●Karhunen-Loève Transform●Principal Component Analysis = Hoteling Transform●How? Data decorrelation●Why? Reduce dimensionality●What for? Many applications...

Eras

mus

+ s

emin

ar, 18

/04/

2016

6 / 26



introductionintroduction

1998

1902 (Pearson)

1936 (Hoteling)1987(Kirby, Sirowich)

1991 (Turk, Pentland)

One-dimensional

Two-dimensional

PCA(Principal Component Analysis)

1998 (Tsapatsoulis N.,Alexopoulos V. Kollias S.)2000, 2001, 2004

(Kukharev G., Forczmanski P.), Faculty report 2000, PRIP'2001, MG&V 2004

Eras

mus

+ s

emin

ar, 18

/04/

2016

7 / 26



2DKLT/ PCArc2DKLT/ PCArc

On the input we assume L images X in grayscale of M×N pixels.

1.

2.

3.

Then we calculate a matrices of eigenvalues and a matrices of eigenvectors on the basis of covariance matrices RM i CN :

Transformation is performed as follows, where V(R) and V(C) are submatricesof W(R) and W(C):

Λ(R) ,Λ(C)

W (R ) , W (C)

Vector or matrix repre-sentation is possible

X̄ M ×N=1L∑k=1

L

X M ×N( l)

X̂ M ×N( l)

=X M×N( l )

− X̄ M×N ∀ l=1,2 ,… , L

RM=1L∑l=1

L

X̂ M×N( l ) [ X̂ M ×N

( l ) ] T ; C N=1L∑l=1

L

[ X̂ M×N(l ) ] T X̂ M ×N

( k ) ;

Y p×q l =[V M×p

R ] T X M×N l V N ×q

C

G. Kukharev, P. Forczmański, Data dimensionality reduction for face recognition, Ma-chine Graphics & Vision, vol. 13, no. 1/2, 2005, s. 99-122

Eras

mus

+ s

emin

ar, 18

/04/

2016

8 / 26



Exemplary application to face Exemplary application to face recognitionrecognition

G. Kukharev, P. Forczmański, Data dimensionality reduction for face recognition, Ma-chine Graphics & Vision, vol. 13, no. 1/2, 2005, s. 99-122

X

Eras

mus

+ s

emin

ar, 18

/04/

2016

9 / 26



General scheme ofGeneral scheme of2DPCA/2DKLT application2DPCA/2DKLT application

inputimage

blockdecomposition blocks 2DKLT transformed

blocks Eigenvectors

Quantization

Coding

Outputfile/stream

inverse2DKLT

blocks

composition

embedding message

message embedding (1)

rearrangement

P. Forczmański 2DKLT-Based image compression and scrambling, Congress of Young IT Scientists, 2007, s. 86-89 (Polish Journal of Environmental Studies, vol. 16, no. 4a)

P. Forczmański Information Embedding in Remotely sensed images by means of two-Two-dimensional Karhunen-Loeve Transform, Advanced Computer Systems: 14th International Conference: ACS’2007, Ol-sztyn: HARD, 2007, s. 275-279 (Polish Journal of Environmental Studies, vol. 16, no. 5B)

Eras

mus

+ s

emin

ar, 18

/04/

2016

10 / 26



ArtifactsArtifacts

original 2DKLT

JPEG JPEG 2000

Eras

mus

+ s

emin

ar, 18

/04/

2016

11 / 26



DCT vs 2DKLTDCT vs 2DKLT

DCT (JPEG)

2DKLT

Eras

mus

+ s

emin

ar, 18

/04/

2016

12 / 26


West Pomeranian University of Technology,Szczecin Watermarking / steganographyWatermarking / steganography

➔ embedding watermarks and copyright information in multimedia (Digital Rights Management – DRM),

➔ hiding secret information for the safe transfer,➔ protection of data against changes.

➔ All methods work either in spatial or spectral domain (FFT, DFT, DCT, Wavelets).

➔ The most popular, yet the least sophisticated method is "Least Significant Bit (LSB) insertion"

➔ The basic problem with the LSB is a low resistance to typical image manipulations

Eras

mus

+ s

emin

ar, 18

/04/

2016

13 / 26


West Pomeranian University of Technology,Szczecin Modification in each blockModification in each block

➔ Modification of block after 2DKLT

Message 00010101000101010001...

Original (carier) Modifier block

Bit-wide decomposition

key:

{ 4,1,

2, 3,.

..}

Eras

mus

+ s

emin

ar, 18

/04/

2016

14 / 26


West Pomeranian University of Technology,Szczecin ExamplesExamples

Carrier image +watermark

P. Forczmański, M. Węgrzyn, Virtual Steganographic Laboratory for Digital Ima-ges, Information systems architecture and technology, Polska 2008, s. 163- 173P. Forczmański, M. Węgrzyn, Open Virtual Steganographic Laboratory Elektronika, nr 11, 2009, s. 60-65

Eras

mus

+ s

emin

ar, 18

/04/

2016

15 / 26



ExperimentsExperiments

10

0

94

88

82

76

70

64

58

52

46

40

34

28

22

16

10

5

10

15

20

25

30

35

40

0%

20%

40%

60%

80%

100%

120%JPEG Compression

PSNR [dB]

Information [%]

Quality

PS

NR

-10

0-9

0

-80

-70

-60

-50

-40

-30

-20

-10 0 10

20

30

40

50

60

70

80

90

10

0

5

10

15

20

25

30

35

40

0%

20%

40%

60%

80%

100%

120%Brightness

PSNR [dB]

Information [%]

Brightness Coefficient

PS

NR

P. Forczmański Information Embedding in Remotely sensed images by means of twodimensional Karhunen-Lo-eve Transform, Advanced Computer Systems: 14th International Conference: ACS’2007, Olsztyn: HARD, 2007, s. 275-279 (Polish Journal of Environmental Studies, vol. 16, no. 5B)

0,0

2,0

4,0

6,0

8,0

10

,01

2,0

14

,01

6,0

18

,02

0,0

22

,02

4,0

26

,02

8,0

30

,03

2,0

34

,03

6,0

38

,04

0,0

42

,04

4,0

46

,04

8,0

50

,0

5

10

15

20

25

30

35

40

0%

20%

40%

60%

80%

100%

120%Additive noise

PSNR [dB]

Information [%]

Noise Amplitude

PS

NR

0,0

5

0,2

5

0,4

5

0,6

5

0,8

5

1,0

5

1,2

5

1,4

5

1,6

5

1,8

5

2,0

5

2,2

5

2,4

5

2,6

5

2,8

5

3,0

5

3,2

5

3,4

5

3,6

5

3,8

5

5

10

15

20

25

30

35

40

0%

20%

40%

60%

80%

100%

120%Contrast

PSNR [dB]

Information [%]

Contrast Coefficient

PS

NR

Eras

mus

+ s

emin

ar, 18

/04/

2016

16 / 26



ScramblingScrambling

original

Scrambled image

Recovered image #1

Recovered image #2?

?

Eras

mus

+ s

emin

ar, 18

/04/

2016

17 / 26



Linear Discriminant Analysis (5/6)Linear Discriminant Analysis (5/6)

D. Swets, J. Weng, "Using Discriminant Eigenfeatures for Image Retrieval", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 831-836, 1996

PCA LDA

Eras

mus

+ s

emin

ar, 18

/04/

2016

18 / 26



LDA : Algorithm (1)LDA : Algorithm (1)

Let us assume input images X are grayscale, gathered in K classes, each one having L objects.We calculate means for each K class and one common, for all classes:

Then, covariance matrices are calculated:

Eras

mus

+ s

emin

ar, 18

/04/

2016

19 / 26




Total covariance matrix is:

Which is decomposed using eigen-approach:

where Ω – diagonal of eigen values and U – orthogonal matrix having eigenvectors

Transform matix is created from U by selecting sub-matrix with s columns related to the highest values in Ω.

Ω pxp

→ F sxs

Eras

mus

+ s

emin

ar, 18

/04/

2016

20 / 26




Dimensionality reduction is applied in two-step process:

1. initial reduction (down-sampling, DCT/DFT, PCA)2. final LDA transformation LDA:

Eras

mus

+ s

emin

ar, 18

/04/

2016

21 / 26



LDA: classification of texturesLDA: classification of textures

K. Okarma, P. Forczmański, 2DLDA-based texture recognition in the aspect of objec-tive image quality assessment Annales Universitatis Mariae Curie-Skłodowska. Sectio AI Informatica, vol. 8, no. 1, 2008, s. 99-110

Distortion Recognition accuracy

Nearest element Centers of classes

Median 3x3 81.33 % 71.59 %

Median 5x5 62.18 % 57.79 %

Low-pass 3x3 71.47 % 63.37 %

Low-pass 5x5 46.83 % 47.32 %

5% impulse noise 64.29 % 60.88 %

10% impulse noise 47.24 % 47.89 %

15% impulse noise 38.47 % 38.80 %

20% impulse noise 30.03 % 31.33 %

JPEG 60% 89.77 % 78.08 %

JPEG 40% 90.10 % 77.11 %

JPEG 20% 89.95 % 76.82 %

JPEG 10% 88.96 % 76.14 %

Eras

mus

+ s

emin

ar, 18

/04/

2016

22 / 26



LDA : limitationsLDA : limitations

Classical LDA method requires to carry out a preliminary dimensionality re-duction of input data, eg. by means of sampling (down-sampling) or PCA / PCArc. It is required to meet the condition:

where K – no. classes, L- no. Images in class, DIM – dimensionality of fe-ature-space.

G. Kukharev, P. Forczmański, Two-Dimensional LDA Approach to Image Compression and Re-cognition, Computing, Multimedia and Intelligent Techniques, vol.2, no. 1, 2006, s.87-98

G. Kukharev, P. Forczmański, Face Recognition by Means of Two-Dimensional Direct Linear, Discriminant Analysis Pattern recognition and information processing: PRIP ’2005: Proceedings of the Eighth International Conference, 18-20 Maj, Mińsk, Białoruś 2005, s.280-283

Eras

mus

+ s

emin

ar, 18

/04/

2016

23 / 26



2DLDA/LDArc (1)2DLDA/LDArc (1)

The solution to this problem is to use 2DLDA (LDArc), which involves the decomposition of the image into a set of rows and columns and calculating 2 sets of covariance matrices:

Eras

mus

+ s

emin

ar, 18

/04/

2016

24 / 26



2DLDA/LDArc (5)2DLDA/LDArc (5)

Transformation is done using the following formula:

Exemplary LDA spectra and the reconstruction is presented below:

Eras

mus

+ s

emin

ar, 18

/04/

2016

25 / 26



2DLDA/LDArc: Facial recognition2DLDA/LDArc: Facial recognition

G. Kukharev, P. Forczmański, Facial images dimensionality reduction and recognition by means of 2DKLT, Machine Graphics & Vision, vol. 16, no. 3/4, 2007, s. 401-425

Eras

mus

+ s

emin

ar, 18

/04/

2016

26 / 26



Thank you for your attenttion!

Any questions?

Paweł ForczmańskiChair of Multimedia Systems, Faculty of Computer Science and Information

Technology, West Pomeranian University of Technology, Szczecin

Vilnius University, Institute of Mathematics and Informatics, 18/04/2016

??