23
Unsupervised Texture Segmentation Via Gaussian Markov Random Field (GMRF) Model And Filter Bank Decomposition by Zhou Shaohua March 15, 2001

Texture Segmentation

Embed Size (px)

DESCRIPTION

Gaussian Markov Random Field (GMRF) ModelAndFilter Bank Decomposition

Citation preview

Page 1: Texture Segmentation

Unsupervised Texture Segmentation

Via

Gaussian Markov Random Field (GMRF) Model

And

Filter Bank Decomposition

by

Zhou Shaohua

March 15, 2001

A project report for partial requirement of ENEE739J, Image Understanding

Page 2: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Abstract

This report deals with texture segmentation under no supervision. With absence

of training patterns in unsupervised segmentation, typically, K-Means clustering

technique is invoked to make the segmentation in the feature space. Two kinds of

features are extracted from the mosaic texture in this study. GMRF model is used to

capture the statistical properties of textures, while filter bank decomposition to attain the

frequency properties.

Page 3: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Chapter 1: Introduction

Due to its widespread presence all over the world, texture is served as an

important cue in human visual system (HVS). Texture-related research topics, such as

texture analysis, classification, and segmentation, attract a significant amount of attention

in the literature. In this report, we focus on the problem of texture segmentation.

Despite of long history of research effort on texture, no exact definition of texture

can be found in the literature, because texture has multi-fold characteristics, which are

hard to be described in one single sentence. For instance, some textures have strong

structural organization, some have random natures, and some have both properties. Thus,

approaches to texture analysis, according to their emphases, can be roughly divided into

three groups: structural, statistical [1, 2, 3], and spectral [4, 5] ones. Also, there are

alternative approaches, such as normalized cut [6], available in the literature. In this

report, we study both statistical and spectral approaches. In particular, GRMF model is a

statistical approach, and filter bank decomposition is a spectral one.

Texture segmentation can be achieved under no supervision. With absence of

training patterns in unsupervised segmentation, unsupervised clustering techniques such

as K-means algorithm are common practices. However, clustering image intensities

produces nothing meaningful, especially for textural images. Therefore, feature should be

extracted before clustering. The ideal feature, which is hard to extract in practice, should

possess the capability of minimizing within-cluster distance while maximizing between-

cluster one. Two features proposed in this report are (i) GMRF parameters, and (ii) post-

processed results of filter responses after filter bank decomposition.

The rest of this report is organized as follows. Chapter 2 introduces the GMRF

model and its implemental details. Chapter 3 describes the filter bank approach and

discusses the issues of filter selection and feature extraction. Experimental results are also

included in according chapters. Chapter 4 concludes the report.

Page 4: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Chapter 2: Gaussian Markov Random Field (GMRF) Model

Markov random field (MRF) is an extension of 1-D causal Markov chain to 2-D,

unnecessarily causal any more. An MRF is depicted in terms of

local conditional probability density function (PDF), i.e.,

where is the neighborhood of pixel . If this PDF happens to be a Gaussian

distribution, we call this MRF as a GMRF. Fig. 2.1 illustrates the order of GMRF, which

is related to the location of neighbors.

Figure 2.1: Illustration of the order of GMRF.

In this report, we use a second-order GMRF, which can be expressed as follows:

where , and

with zero mean and variance . For each pixel, we train the mean ,

(which is defined in the covariance matrix), and the parameters, ,

over a window centered on this, by Least Square Estimation (LSE), i.e.,

Page 5: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

where is the number of pixel in the window . Note that, due to the symmetry of

correlation function, only four ’s are estimated. We now form the feature space

, and K-Means clustering algorithm is then applied to make

segmentation in this space.

Fig. 2.2 shows two mosaic images used in the experiment, Fig. 2.3 the

corresponding feature spaces of Fig. 2.2(a), and Fig. 2.4 the segmented results. From the

given figures, we observe the following:

A reasonable unsupervised segmentation is achieved. See, for example, Fig. 2.3

where the feature spaces are distinguishable, and Fig 2.4(a) where five patches are

obtained. Also, the upper part of Fig. 2.4(b) is not bad.

However, misclassifications are there due to (i) that some regions of the given

mosaic images are too non-stationary to be captured by GMRF model, or in other

word, the trained parameters have large variations, and (ii) that estimation from a

window centered at the pixel has de facto smoothing effect, which blurs the

boundary.

In addition, Fig. 2.2(b) is much more different to segment than Fig. 2.2(a) because

of the small patch of each class in the down part, which make it extremely hard to

train GMRF parameters.

Page 6: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

(a)

(b)

Figure 2.2: Two mosaic images used in the experiment.

Page 7: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Figure 2.3: Feature spaces of Fig. 2.2(a).

Page 8: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

(a)

(b)

Figure 2.4: Segmented results using GMRF model.

Page 9: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Chapter 3: Filter Bank Decomposition

A general filter bank scheme is presented in Fig. 3.1. A mosaic image is

first decomposed using a filter bank , then useful features are

extracted from the filter responses by nonlinear operations, and finally clustering

algorithm is applied to produce segmentation. Given such a scheme, two issues should be

addressed: (i) filter selection and (ii) feature extraction.

Figure 3.1: a general scheme of filter bank analysis of image.

Three basic guidelines on how to choose a good filter bank are as follows. (1) It

should have a good localization in both spatial and spectral domains, though there is an

uncertainty between them. (2) It should be able to cover multi-channels or multi-

frequency components in order to provide a kaleidoscopic view. (3) It is hopefully HVS-

motivated because HVS performs much better than any computer system does in visual

processing. Based on the above guidelines, we choose Gabor wavelets (or ‘Gabor jets’ of

[7]) as the filter bank used in this report. It is well known that Gabor function minimizes

joint uncertainty, and simulates some functionalities of primary visual cortex in HVS,

while wavelets are typical multi-resolution filters. One more advantage of Gabor function

is that it is also orientation-selective.

The Gabor wavelets as defined as follows:

Page 10: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Fig. 3.2 shows some examples of filters and their spectrums. There are 24 filters in this

wavelet family, which covers almost all high-frequency components. In addition, a

simple Gaussian averaging filter is introduced in order to cover low-frequency

components.

Filter responses themselves are features. However, as argued in [4], purely linear

mechanism such as convolution is inadequate for discrimination, and thus nonlinear

operations are needed. In this report, we introduce two nonlinear operations: (i) full-wave

rectification, and (ii) adaptive pooling. Full-wave rectification is mathematically defined

as taking the absolute operation, i.e.,

.

Adaptive pooling is defined as follows:

,

where is a Gaussian averaging function with its variance proportional to the

sample variance of . The idea of adaptive pooling stems from the fact: the filter

responses are non-stationary even within one class, thus need pooling before feeding to

K-Means clustering.

Fig. 3.3 shows the segmented results using Gabor filter bank decomposition. We

can observer the following:

Filter bank approach seems to produce a better segmentation than GMRF model

algorithm. The benefit may arise from the adaptive pooling to overcome the

irregularity within one texture class.

There exist boundary artifacts (as shown in Fig. 3.3), which are inherent to

filtering approaches.

It is still very difficult to segment the down part of Fig. 2.2(b).

Page 11: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Figure 3.2 Gabor filters and their spectrum

(a)

Page 12: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

(a)

(b)

Figure 3.3 Segmented results of Gabor Filter Bank

Page 13: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Chapter 4: Conclusion

In this report, texture segmentation is achieved in unsupervised way. GMRF

model is fitted on the mosaic images, and the model parameters are used to form a multi-

dimensional feature space. In filter bank approach, nonlinear operations are applied to

filter responses to extract features. K-Means clustering algorithm is then invoked to

perform the segmentation. Experimental results are presented and observations are made

to demonstrate the efficiency of our approaches.

Reference

[1] B. S. Manjunath, T. Simchony, and R. Chellappa, “Stochastic and Deterministic

Networks for Texture Segmentation,” IEEE Acoustic, Speech, and Signal Processing,

Vol. 38, No. 5, July 1990.

[2] Fure-Ching Jeng, J. W. Woods, and S. Rastogi, “Compound Gauss-Markov Random

Fields for Parallel Image Processing,” Markov Random Fields Theory and Application,

Academic Press, 1993.

[3] S. Krishnamachari, and R. Chellappa, “Multiresolution Gauss-Markov Random Field

Models for Texture Segmentation,” IEEE Trans. Image Processing, Vol. 6, No. 2,

February 1997.

[4] J. Malik and P. Perona, “Preattentive texture discrimination with early vision

mechanisms,” J. Opt. Soc. Am. A, Vol. 7, No. 5, May 1990

[5] B. S. Manjunath and W. Ma, “Texture features for browsing and retrieval of image

data,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 18, No. 8, August

1996.

[6] Jianbo Shi and J Malik, “Normalized Cuts and Image Segmentation,” IEEE

Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 8, August 2000.

Page 14: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

[7] L. Wiskott, J.-M. Fellous, N. Kruger, and C. von der Malsburg, “Face recognition by

elastic graph matching,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.

19, No. 7, July 1997.

[8] Part of Matlab source is attached in the appendix.

Page 15: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Appendix: Part of Matlab Source Code

% GMRF modelfunction para = gmrf(in)

[M,N]= size(in);S = 1;T = 32;R = T/S-1;mno = M/S;nno = N/S;for m=1:mno-R for n=1:nno-R left = (m-1)*S+1; right = (m-1)*S+T; top = (n-1)*S+1; bottom = (n-1)*S+T; a = in(left:right, top:bottom); u = mean2(a); a = a - mean2(a); r00 = corrcoef2(a, 0, 0); r10 = corrcoef2(a, 1, 0); r01 = corrcoef2(a, 0, 1); r11 = corrcoef2(a, 1 ,1); r1_1 = corrcoef2(a, 1, -1); r02 = corrcoef2(a, 0, 2); r1_2 = corrcoef2(a, 1, -2); b = [r00, r01, r02, r10; ... r01, r00, r01, r1_1; ... r02, r01, r00, r1_2; ... r10, r1_1, r1_2, r00]; c = [r11; r10; r1_1; r01]; d = b\c; para(m,n,1:4) = d; para(m,n,5) = u;

para(m,n,6) = sqrt(r00-d(1)*r11-d(2)*r10-d(3)*r1_1-d(4)*r01); endend

for m=mno-R:mno for n=nno-R:nno if m ~= mno-R | n~= nno-R para(m,n,:) = para(mno-R,nno-R,:); end endend

if (mno>R+1)for m=1:mno-R-1

for n=nno-R+1:nno para(m,n,:) = para(m, nno-R, :);

end endend

Page 16: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

if (nno>R+1)for m=mno-R+1:mno

for n=1:nno-R-1 para(m,n,:) = para(mno-R, n, :);

endend

end

%CORRCOEF2 Correlation coefficient for the image% CORRCOEF2(X, DX, DY) computes the correlation coefficient % r(dx, dy).function r = corrcoef2(x, dx, dy)

[M,N] = size(x);

if dx >= 0 & dy >= 0 x1 = x(dx+1:M, dy+1:N); x2 = x(1:M-dx, 1:N-dy);elseif dx >= 0 & dy < 0 x1 = x(dx+1:M, 1:N+dy);

x2 = x(1:M-dx, -dy+1:N);elseif dx < 0 & dy >= 0 x1 = x(1:M+dx, dy+1:N);

x2 = x(-dx+1:M, 1:N-dy);else x1 = x(1:M+dx, 1:N+dy);

x2 = x(-dx+1:M, -dy+1:N);end

r = sum(sum(x1.*x2))/(M-abs(dx))/(N-abs(dy));

%function gw = gaborw(v, u, sigma, scale, sz)% u -- controls the center freq% v -- controls the orientaion% sigma -- controls the standard variation% scale -- set to 1 by default% sz -- filter sizefunction gw = gaborw(v, u, sigma, scale, sz)

k_v = pi/2/power(2, v/2);phi_u = u*pi/8;s = sigma;y = scale;rows = sz(1);cols = sz(2);gw = zeros([rows,cols]);for r=1:rows for c = 1:cols r1 = (r-1-rows/2)*y; c1 = (c-1-cols/2)*y; a = y*y; a = a*(k_v*k_v)/(s*s)*exp(-k_v*k_v*(r1*r1+c1*c1)/(2.0*s*s)); a = a*(cos(k_v*(r1*cos(phi_u)-c1*sin(phi_u)))-exp(-s*s/2.0)); gw(r,c) = a; endend

Page 17: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

% out = JETS(in) -- compute gabor jets for input imagefunction out = jets(in)

[M,N] = size(in);out = zeros([M,N,24]);fft2_in = fft2(in);

for u=0:2 for v=0:7 fprintf('jet (%d %d)\n', u, v); k = u*8+v+1; gw = gaborw(u,v,2*pi,1.0, [M,N]); out(:,:,k) = abs(fftshift(real(ifft2(fft2_in.*fft2(gw))))); endend

x = [1:M] - M/2 -1;y = gauss(x, 0, 2*pi);gw = y'*y;out(:,:,25) = abs(fftshift(real(ifft2(fft2_in.*fft2(gw)))));

% jets poolingfunction out = jpool(in)[M,N,L] = size(in);for l=1:L s = std2(in(:,:,l)); if s>4 K = floor(s); x = -K:K; y = gauss(x, 0, s); B = y'*y; else B = 1; end out(:,:,l) = conv2(in(:,:,l), B, 'same');end

%function cmap = kmeans(in, cno, error)% cmap -- cluster map% in -- input matrix% cno -- class number% epsilon -- convengence errorfunction cmap = kmeans(in, cno, epsilon)

if nargin ~= 3 error('Usage: kmeans(in, cno, epsilon)');end

[M,N,L] = size(in);cmap = zeros([M,N]);

for k=1:cno a(k,:) = in(floor(rand(1)*M), floor(rand(1)*N), :);end

for i=1:L s(i) = std2(in(:,:,i));

Page 18: Texture Segmentation

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

end

prev_error = 0;while (1) disp(a); curr_error = 0; for m=1:M for n=1:N v(1,:) = in(m,n,:); [id, dist] = nnb(v, a, s); %nearest neighbor cmap(m,n)=id; curr_error = curr_error + dist; end end curr_error = curr_error/(M*N); error = abs(curr_error - prev_error); fprintf('curr_error = %f, prev_error = %f, error=%f\n', ... curr_error, prev_error, error); if error < epsilon break; else prev_error = curr_error; a(:,:)= 0.0; b(1:cno)= 0; for m=1:M for n=1:N k = cmap(m,n); b(k)=b(k)+1; v(1,:) = in(m,n,:); a(k,:) = a(k,:) + v; end end for k=1:cno a(k,:)=a(k,:)/b(k); end end end

% nearest neighborfunction [id,dist] = nnb(in, a, s)dist = 1000;[cno,x] = size(a); for i=1:cno d = sum(abs((in-a(i,:))./s)); if dist>d id = i; dist = d; endend