An Efficient Wavelet Dictionary for Texture Separation

Mohamed Anis Loghmari, Faten Katlane and Mohamed Saber Naceur Laboratoire de Télédétection et Système d’Informations à Référence Spatiale

Ecole Nationale d’Ingénieurs de Tunis BP 37- Tunis le Belvédère 1002 TUNISIE

MohamedAnis.Loghmari@isi.rnu.tn Faten.Katlane@enit.rnu.tn Saber.naceur@insat.rnu.tn

Abstract – In this paper, our goal is to highlight the importance of the source separation method on remote sensing data analysis when dealing with urban areas characterized by spatial concept like texture. Source separation has become an attractive tool used to compensate physical information deficiency by statistical assumptions. The method’s key comes from the fact that the blind signal separation can be achieved by restoring statistical independence. In this work, we try to design a statistical generative model, based on a wavelet dictionary, composed of atoms which are automatically selected to maximize the sparseness of the underlying texture type. This application is of utmost importance in the classification process and should minimize the misclassification risk of urban areas.

Keywords-source separation; wavelets; texture analysis.

I. INTRODUCTION Source separation consists in recovering a set of

unobservable signals from a set of observed mixtures. This technique has received significant attention due to its suitability to recover sources when no information is available about mixture. Typically, this problem is known as blind source separation (BSS). Promising applications can already be found in the processing of communication signals [1], biomedical signals [2] and astrophysical data analysis [3]. Regarding remote sensing, this technique is recently adapted to obtain more accurate representation of the land-covers [4]. In this context, BSS method is used to recover sources by giving only sensor observations with unknown linear mixtures of the unobserved source signals. The method’s key comes from the fact that the blind signal separation can be achieved by restoring statistical independence. When dealing with heterogeneous areas, like urban zones, the independence assumption should be less plausible.

In this article, we try to depart from this difficult real-word scenario and make less realistic assumptions about the environment so as to make the problem more tractable. One increasingly popular and powerful assumption is that the sources have a sparse representation in a given basis. The advantage of a sparse signal representation is that the probability of two or more sources being simultaneously active

is low. Thus, sparse representations lead themselves to good separation.

The aim of this research is to model a wavelet dictionary, composed of atoms, to perform texture separation. The selected atoms are those that maximize the sparseness of the underlying texture components. Thus, the independence or equivalently the sparseness is not required for the source signals but for their wavelet coefficients, which is more plausible.

This application is of utmost importance in the classification process. It should constitute the continuity of our previous work and regularizes the classification results related to urban zones [5].

This paper will be organized as follows: First, we will investigate the independence and sparseness equivalence. Then, we will design an efficient wavelet dictionary by reducing higher order statistical dependencies between textural feature vectors. In the last part, we will discuss the results and compare them to other methods.

II. SOURCE SEPARATION PRINCIPLE The Blind Signal Separation consists in recovering n

unobserved signals or sources s1(t), …, sn(t) from m observed mixtures x1(t),…, xm(t). The simplest BSS form can be represented by the following linear instantaneous model

x(t) = As(t) + b(t), (1) where s(t) is an n × 1 column vector collecting the source signals, vector x(t) similarly collects the m observed signals, A is m × n unknown mixing matrix, each of its columns is called the directional vector associated to the corresponding source and b(t) is an additive m × 1 noise vector corrupting the signal. Second-order information can be used to reduce the BSS problem to a simpler form. By whitening the signal part, it is able to do about half the BSS job. The whitened process z(t) still obeys to a linear model given by

z(t) = WA s(t) + W b(t) = U s(t) + W b(t), (2) where z(t) is a m × 1 column vector, W is n × m whitening matrix and U is n × n unitary matrix. Hence, instead of estimating m × n coefficients of the original mixing matrix A,

1-4244-1212-9/07/$25.00 ©2007 IEEE. 266

we only need to estimate n × (n - 1)/2 coefficients of the unitary matrix U.

The Maximum Likelihood (ML) principle can be used as a starting point of the BSS problem, via the Kullback divergence, which should be understood as a statistical way of quantifying the closeness of estimated source distributions to the true ones [6].

The ML principle can be associated with the following contrast function

)/()( syKyML =φ , (3) where K(./.) is the Kullback divergence and y = A-1x. From the ML principle, we can define a quantitative measure of independence called the Mutual Information (MI) contrast and given by

)()/()/(minmin~

yyyKsyK MIsMLs φφ === , (4)

where~y is a random vector with independent entries

distributed as the corresponding entry of y. The sparse-independence link can be described from the MI contrast. Under the whiteness constraint, we can set the MI contrast under the following form

)()(1∑ =

=T

i iMI yHyφ , (5)

where H(. ) is the Shannon marginal entropy. So, the minimization of the mutual information is equivalent to the minimization of the marginal entropy of each component. A minimum of entropy corresponds to the sparsest distribution which is the farest from a Gaussian distribution that has, for a same variance, a maximum of entropy.

III. GABOR WAVELETS PRINCIPLE Texture analysis is a significant challenge due to the

complexity of the textural patterns that must be taken into account. Textures are generally modelled as a pattern dominated by a narrow band of spatial frequencies and orientations. Bovik and al. [7] chose the Gabor filters and propose the characterization of textures through their dominant frequency and orientation components.

The 2-D Gabor function, is a complex sinusoidal modulated by a 2-D Gaussian envelope in the spatial domain,

sjts

eetsg ts 02

2

2

2

2)

22(

0 ),,( πνσσν+−

= . (6) Every dictionary element can be modelled using the 2-D Gabor basis function as follows

)','(),( tsgatsg mmn

−= , (7) with a > 1, m = 0, 1, …, S-1, )sincos(' θθ tsas m += − ,

)cossin(' θθ tsat m +−= − and θ = nπ/h for n = 0, 1,…, h-1. a is the scale factor, S is the total number of scales and h is the total number of orientations. However, the most difficult part remains choosing Gabor filter parameters adapted to each texture pattern. By decomposing the images according to this dictionary, we can transform the source separation problem to the wavelet domain to take the following structure

wx = Aws + wb, (8)

where wx = [ wx1, wx

2,…,wxk] is m × k matrix, with wx

. representing the wavelet coefficient of the observed signal x, ws= [ ws

1, ws2,…,ws

k] is n × k matrix, with ws. representing the

wavelet coefficient of the source signal s and wb = [ wb1,

wb2,…,wb

k] is m × k matrix, with wb. representing the wavelet

coefficients of an error term b. The source separation problem consists now of searching m × n matrix A, such that the ws components are as independent as possible. Thus, the independence assumption is not required for the source signals s but for their wavelet coefficients ws, which is more plausible thanks to the sparse property of the wavelet coefficients. Thanks to the last property, most columns of ws contain at most one significant term, so we have

Ts

ls ww ]0,...0,1,0,...,0[≈ , (9)

with the number 1 at the l_th position. From the approximate equality sx Aww ≈ , we can argue that the columns of wx are proportional to the mixture matrix columns, and the source separation problem consists now on estimating the independent directions of A, for which we can associate the most sparse representation of ws.

IV. ALGORITHM In this work, we should manipulate higher order statistics to

extract non stationary information related to texture. To ensure the identifiably and to improve the statistical efficiency, we estimate the unitary matrix U by a joint diagonalization of several cumulant matrices. To provide a proper decomposition of the signal in the Gabor dictionary, we have chosen to adapt a blind and automated procedure, that rely on an optimal decomposition of the signal on redundant dictionary based on the independence criterion. The algorithm details are described on the following steps:

- To initiate our algorithm, four main directions are used, vertical, horizontal, right diagonal and left diagonal.

.32,1,04/ andnforn == πθ - Three fundamental frequencies are used too, to analyse large, medium and small-periodic texture. To reach the wavelet property in the 2-D frequency space, Gabor spectral support should be proportional to the mid-frequency. Then, we set Gabor dictionary basis function on the form

]

)()([2/1

021

22

202

21

201

),,( σ

νν

σ

νν

νννii

eG−

+−

−

= with ic 01 νσ = and ic 02 'νσ = for i = 1, 2 and 3. - Initialization of wx according to main Gabor dictionary related to the fundamental directions and frequencies

GwX x .= where X is the observed image matrix which size is m × T and G is k × T dictionary matrix.

- Estimation of covariance matrix xwR^

)()(1,1

*^

twtwT

RTt

xxwx ∑ ==

and its n large eigen-values and their corresponding eigen-vectors according to

1-4244-1212-9/07/$25.00 ©2007 IEEE. 267

*,1

^

mmMm mw vvR x ∑ == λ

- Estimation of the noise variance 2^

σ

∑ >−=

nm mnmλσ 12^

- Estimation of the whitened matrix W H

nnW ])(,...,)[( 2/12^

12/1

2^

1 νσλνσλ −− −−= - Form the set Qwz of the fourth-order cumulants of the whitened process z(t), which is defined for a n × n matrix M by

lklzkzjziznlkij

def

w mwwwwCumnMQN z ),,,()( **,1,∑ =

=⇔=

then we determine their n large eigen-values λr 1≤ r ≤ n and their n eigen-matrices Mr 1≤ r ≤ n

2,1),()(

)(

nsrsrMMTracewith

MMQHsr

kkkwz

≤≤=

=

δ

λ

- Approximate joint diagonalization of the set {λrMr avec 1≤ r ≤ n} by a unitary matrix U. - Estimation of source coefficients with the largest sparse representation.

xH

s WwUw = - These sources are represented according to independent line directions given by columns of the mixture matrix A, which is given by

)()( )( lnln UWA #= with n(l) is the source index that engendered significant coefficient at the l_th column.

V. RESULTS AND EVALUATION

A. Gabor Source Filters and Gabor Source Images An example of the Gabor source filters created from the

fourth-order cumulant with a mask size of 20 × 20 are presented on Fig. 1.

Figure 1. Gabor filter sources (3D-space representation) (a) source 1 (b) source 2 (c) source 3 (d) source 4.

Many filter sizes have been tested, the most adapted for our context have a mask size of 6 × 6 (Fig. 2(a)). (a) (b)

12

34

56

0

2

4

60

0.5

1

1.5

2

1 1 5 2 2 5 3 3 5 4 4 5 5 5 5 61

1.5

2

2.5

3

3.5

4

4.5

5

5.5

6

Figure 2. (a) Gabor source atom. (b) Scatter plot.

Fig. 2(b) shows the scatter plot of one of these atoms. This scatter plot shows Gabor spectral support, presenting different aspects with different sizes, each specific to certain texture pattern at a given level. Thanks to higher order statistics, the spectral supports related to different frequencies are separated and turned to the dominant frequency and orientation which permit to characterize texture with complex behaviour. To evaluate these atoms, an experiment is performed on multispectral images with really heterogeneous data (Spot-4). Fig. 3 shows the texture feature generated by three different atoms with three different scales and orientations. (a) (b) (c)

Figure 3. Source images. (a) (θ1, ν01). (b) (θ2, ν02). (c) (θ3, ν03).

The decomposition of four Spot-4 bands according to Gabor wavelets gives 48 source images (four independent orientations and three different octaves engendering 12 source images for each band).

B. Fusion and Segmentation Gabor coefficient selection scheme is based on the local

energy. The local energy in a mask size of M × M, related to the (i,j)-th pixel and the k-th atom is given by (10)

∑ ∑+

−=

+

−==

2/

2/

2/

2/2 ),()(

1 Mp

Mpi

Mq

Mqjkssk jiw

McardE . (10)

The local energies related to different sources are compared and for each pixel the largest local energy is chosen to represent the corresponding element of the fused image. In this work, image fusion is applied with the purpose of reducing the complexity for segmentation and classification tasks. The segmentation result on real word texture is presented on Fig. 4(a) and (b). The pixels belonging to urban zone are grouped to form a single class.

(a) (b)

(c) (d)

1-4244-1212-9/07/$25.00 ©2007 IEEE. 268

Then, we will compare our approach with other orthogonal wavelet representation, which permit to reduce the wavelet redundancy and give a comparison framework with our approach. The kept wavelets are: Haar, Daubechies, Symmlets and Coifflets. The segmentation results according to these wavelets basis are presented on Fig. 4(c)-(f).

Figure 4. Segmentation results. (a) Gabor source. (b) Mask Gabor source. (c) Haar. (d) Daubechies. (e) Symmlets. (f) Coifflets.

Apparently, source Gabor wavelets have better segmentation results than the chosen orthogonal wavelet basis. Quantitative comparison is given by Table I.

TABLE I. GOOD IDENTIFICATION RATES

Source Gabor atoms 0.90 Haar 0.25

4 6 8 10 Daubechis 0.57 0.46 0.50 0.47

4 6 8 10 Symmlets 0.57 0.47 0.58 0.49

1 2 3 4 Coifflets 0.53 0.54 0.63 0.6

By adjusting the aspects and orientations of the wavelets’

envelope to the texture pattern of the images, higher order statistics lead to optimal separation capability between close texture features. The used criterion provides independent textural information tuned to several frequencies and orientations. This approach adjusts the aspect and orientation of the filters’ envelope to extract texture information at a finer

level of precision. Because textures are modelled as a pattern dominated by a narrow band of spatial frequencies and orientations, the giving Gabor atoms characterize textures through their dominant frequency and orientation components, which gives the best identification rate for the source Gabor dictionary.

VI. CONCLUSION In this paper we present a novel statistical generative

dictionary to perform texture separation from remote sensing data. The designed dictionary is very attractive due to its optimal separation both in spatial and spectral domain. It is composed by wavelet atoms automatically tuned, by higher order statistics, to the dominant frequency and orientation components of textures. To illustrate our approach, an experiment on real data will be carried out. The experiment results presented in this paper show that the source channel outputs provide a reliable tool for texture segmentation.

REFERENCES

[1] E. Chaumette, P. Comon, and D. Muller, “ICA-based technique for radiating sources estimation: Application to airport surveillance,” Proc. Inst. Electr. Eng.—F, vol. 140, no. 6, pp. 395–401, Dec. 1993.

[2] A. Bijaoui and D. Nuzillard, “Blind source separation of multispectral astronomical images,” in Proc. MPA/ESO/MPE Joint Astronomy Conf., A. J. Banday, A. Zaroubi, and A. Bartelmann, Eds. Garching, Germany: Springer-Verlag, 2000, pp. 571–581. 31/7-4/8.

[3] L. De Lathauwer, B. De Moor, and J. Vandewalle, “Fetal electrocardiogram extraction by source subspace separation,” in Proc. HOS, Aiguablava, Spain, pp. 134–138, Jun. 1995.

[4] M. S. Naceur, M. A. Loghmari, and M. R. Boussema, “The contribution of the sources separation method in the decomposition of mixed pixels,” IEEE Trans. Geosci. Remote Sens., vol. 42, no. 11, pp. 2642–2653, Nov. 2004.

[5] M. A. Loghmari, M. S. Naceur and M. R. Boussema, ‘‘A Spectral and Spatial Sources Separation of Mulispectal Images,’’ IEEE Trans. on Geoscience and Remote Sensing, vol. 44, no. 12, pp. 3659-3673, Dec. 2006.

[6] J. F. Cardoso, “Blind signal separation: Statistical principles,” Proc. Inst. Elect. Eng., vol. 9, pp. 2009–2025, 1998. [7] A. C. Bovik, M. Clark and W. S. Geisler, “Multichannel texture analysis

using localized spatial filters,” IEEE Trans. Patt. Anal. Match. Int., (12)1, pp. 55-73, 1990.

(d) (e) (f)

(a) (b) (c)

1-4244-1212-9/07/$25.00 ©2007 IEEE. 269