Image Texture Analysis

"All truths are easy to understand once they are discovered; the point is to discover them." - Galileo

Work done in IIT Madras

Image Texture Analysis

Lalit Gupta,Scientist, Philips Research



Texture Analysis

Region based texture segmentation

+

Texture Edge DetectionTextured image



Region Based Texture Segmentation



Image histograms

R1 R2

R3 R4

R1 R2

R3 R4



Classification using Proposed Methodology

A1

V1

H1

D1

1ST level Decomposition

DWT (Daubechies)

Dj

Dj

Filtering

FCM

Unsupervised classification

Image

DCT(9 masks)

DCT(9 masks)

.

.

Gaussian filtering

Gj

Gj

Smoothing

.

.

Mean

Fj

Fj

Feature extraction

.

.

DWT: Discrete wavelet transformDCT: Discrete cosine transform Ref: [Randen99]



Input Image

Steps of Processing

DWT

A1 V1 H1 D1

FCM

.. .. ..DCT

. . .

.. .. ..Smoothing

. . .

.. .. ..Mean

36 Feature images

. . .



(a) Input Image (b) DWT (c) Gabor filter (b) DWT+Gabor

(d) GMRF (e) DWT + MRF (f) DCT (f) DWT+DCT

Results using various Filtering Techniques

Ref: [Ng92], [Rao2004], [Cesmeli2001]



Results (Cont.)

I1 I2 I3 I4 I5

Input images

I6 I7 I8 I9 I10



Results (Cont.)

05

10152025303540

1 2 3 4 5 6 7 8 9 10Image Index

Err

or

in c

las

sif

ica

tio

n

(%)

DWT+Gabor DWT+MRF DCT DWT+DCT

Number of pixels incorrectly classifiedError in classification =

Total number of pixels



Texture Edge Detection



Proposed Methodology

Filtering using 1-D Discrete Wavelet Transform and 1-D Gabor filter bank

16 dimensional feature vector is mapped onto one dimensional feature map

Self-Organizing feature Map (SOM)

Smoothed image

Smoothing using 2-D symmetricGaussian filter

Edge mapEdge detection using Canny operator

Final edge map

Edge Linking

Input image

Smoothed images

Smoothing using 2-D asymmetric Gaussian filter

. . .

16 filtered images, 8 each along horizontal and vertical parallel lines of image

. . .

Ref: [Liu99], [Canny86], [Yegnanarayana98]



Steps of Processing

Input image

Filtered images

...

...Smoothed

images

Feature map

Smoothed images

Edge map



Results

Input image Edge map Input image Edge map Input image Edge map



Integrating Region and Edge Information for Texture

Segmentation

We have used a modified constraint satisfaction neural networks termed as Constraint Satisfaction Neural Network for Complementary Information Integration (CSNN-CII), which integrates the region and edge based approaches.

+



Dynamic Window

Image Window



Constraint Satisfaction Neural Networks For Image Segmentation

1 < i < n1 < j < n1 < k < m

i

j

k

Size of image: n x nNo. of labels/classes: m

Ref: [Lin92]



Constraint Satisfaction Neural Network for Complementary Information

Integration (CSNN-CII)Each neuron in CSNN-CII contains two fields: Probability and Rank.

Probability: probability that the pixel belongs to the segment represented by the corresponding layer.

Rank: Rank field stores the rank of the probability in a decreasing order, for that neuron.

0.1

0.5

0.4Probabilities

3

1

2Rank



The weight between kth layer’s (i, j)th, Uijk, neuron and lth layer’s (q, r)th, Uqrl, neuron is computed as:

, , ,

211 ijk qrl

ij qr k l

R RW

p m

Weights in the CSNN can be interpreted as constraints. Weights are determined based on the heuristic that a neuron excites other neurons representing the labels of similar intensities and inhibits other neurons representing labels of quite different intensities.

Where,p: number of neurons in 2D neighborhood (dynamic window).m: number of layers (classes).Uijk: represents kth layer’s (i, j)th neuron.Rijk: Rank for (i, j)th neuron in kth layer or Uijk neuron.

Ref: [Lin 92]

Uijk

Uqrl

Wij,qr,k,l



Algorithm• Phase 1:

– Initialize the CSNN neurons using fuzzy c-means results.• The probability values obtained from FCM are assigned to

the nodes of CSNN. Ranks for each neuron are also computed on the basis of initial class probabilities.

0.2 0.2 0.8

0.3 0.6 0.2

0.6 0.3 0.6

0.8 0.8 0.2

0.7 0.4 0.8

0.4 0.7 0.4

0.2, 2 0.2, 2 0.8, 1

0.3, 2 0.6, 1 0.2, 2

0.6, 1 0.3, 2 0.6, 1

0.8, 1 0.8, 1 0.2, 2

0.7, 1 0.4, 2 0.8, 1

0.4, 2 0.7, 1 0.4, 2 RankProbabilityCSNN-CII

Layer-1

Layer-2

FCM output



Uijk

Hijk

1

, , ,1 1

qr ij

ij qr k qrU N

W O

, , ,

qrm ij

ij qr k m qrmU N

W O

1tijkO

tijkO

Hijk: sum of inputs from all neighboring neurons.

Oijk: the probability of (i,j)th pixel having a label k (Probability value assigned

to the Uijk neuron).

Nij: a set of neurons in the 3D neighborhood of (i,j)th neuron (considering

Dynamic window).

, , ,

qrl ij

t tijk ij qr k l qrl

U N

H W O

i j

k– Iterate and update the probabilities, edge map and determine the winner label

Algorithm (Cont.)



0.2, 2 0.2, 2 0.8, 1

0.3, 2 0.6, 1 0.2, 2

0.6, 1 0.3, 2 0.6, 1

0.8, 1 0.8, 1 0.2, 2

0.7, 1 0.4, 2 0.8, 1

0.4, 2 0.7, 1 0.4, 2

CSNN-CII

Layer-1

Layer-2

1 2 |1 1| 11

5 2 5W

For neurons with rank=1

1 2 |1 2 |1 0

5 2W

For neurons with rank=2

, , ,

qrl ij

t tijk ij qr k l qrl

U N

H W O

1 15 50*0.2 *0.8 ... *0.8 ...aH

0.74aH

, , ,

211 ijk qrl

ij qr k l

R RW

p m

0.26bH

Algorithm (Cont.)

1 0 0

1 0 0

1 0 0

Edge information



1

1

( )

( )

t tijk ijkt

ijk mt tijl ijl

l

Pos O OO

Pos O O

if 0( )

0 otherwise

X XPos X

if max

otherwise

t tijk ijlt l

ijk

H HO

0.74aH

0.26bH

0.6aO 0.4bO

a bH H

Algorithm (Cont.)

0.2, 2 0.2, 2 0.8, 1

0.3, 2 0.6, 1 0.2, 2

0.6, 1 0.3, 2 0.6, 1

0.8, 1 0.8, 1 0.2, 2

0.7, 1 0.4, 2 0.8, 1

0.4, 2 0.7, 1 0.4, 2

CSNN-CII

Layer-1

Layer-2



1ijarg max O ( ) t t

ijl

Y l

Labels to each pixel of an image are assigned as:

Where, l l m

0.1aO 0.1bO

Updated probability values:

0.7aO 0.3bO

0.2, 2 0.2, 2 0.8, 1

0.3, 2 0.6, 1 0.2, 2

0.6, 1 0.3, 2 0.6, 1

0.8, 1 0.8, 1 0.2, 2

0.7, 1 0.4, 2 0.8, 1

0.4, 2 0.7, 1 0.4, 2

2 2 1

2 1 2

1 2 1

Layer-1

Layer-2

Y

Where,

Algorithm (Cont.)



Updating Edge Map:

B : Edge map obtained using lower threshold.E : Edge map obtained using higher threshold.

Mij : the set of pixels in the neighborhood of pixel (i, j) in the output image Y of size 2v+1, excluding edge pixels in E.

0

tij qr

tqr

q v q q v

M Y r v r r v

E

Algorithm (Cont.)

Ymin( ) max( )ij ijM M

E

1

1 1

1 1 and min( ) max( )

0 otherwise

tij

tij ij ij ij

E

E B M M

Edge map at each iteration is computed as:



1

1 1


0 otherwise

tij

tij ij ij ij

E

E B M M

Edge map at each iteration is computed as:

– Check the convergence condition, i.e., the number of pixels updated in Y, at each iteration. If there is any update go to second step.

B Y Updated edge map (E)EM

Algorithm (Cont.)



• Phase 2– Iterate, and update edge map E, by removing extra

edge pixels and by adding new edge pixels.

1 1

1 1

0ij qr

tqr

q q q

L Y r r r

E

Lij is considered as:


otherwise

ij ij ij

ij

ij

E L LE

E

Edge map E is updated as:

Algorithm (Cont.)

Ymin( ) max( )ij ijL L



– Merge Edge map and Segmented map to get final output.

Finally, new edge pixels are added where Eij = 0 and min(Lij) max(Lij)


otherwise

ij ij ij

ij

ij

E L LE

E

E Y Updated edge map (E)

E Y Updated edge map (E)

Algorithm (Cont.)


Work done in IIT MadrasFinal Output

Segmented map Edge map

– Merge Edge map and Segmented map to get final output.



Input Image

Segmented map before integration (Ref: [Rao2004])

Edge map before integration

(Ref: [Lalit2006])

Segmented map and Edge map

after integration

Results



ResultsInput Image

Segmented map before integration (Ref: [Rao2004])

Edge map before integration

(Ref: [Lalit2006])

Segmented map and Edge map

after integration

Technology

Image Texture Analysis