Edge Preserving Spatially Varying Mixturesfor Image Segmentation

Giorgos Sfikas, Christophoros Nikou, Nikolaos Galatsanos

(CVPR 2008)

Presented by Lihan HeECE, Duke University

Feb 23, 2009

by

Introduction

Edge preserving spatially varying GMM

Inference using MAP-EM

Experimental results

Conclusion

Outline

2/15

Introduction

3/15

Image segmentation

GMM: no prior knowledge is exploited

Adjacent pixels most likely belong to the same cluster; Edge of objectives.

SVGMM (spatially variant GMM):

Clustering pixels or super pixels such that the same group has common characteristics (same objective, similar texture)

Spatial smoothness is imposed in the neighborhood of each pixel based on the Markov random field;

Without considering the edge of textures

Introduction

4/15

In this paper

Hierarchical Bayesian model; Spatially varying GMM: mixing weights are different for different

pixels; Difference of mixing weights for two neighbored pixels follows a

student-t distribution; Heavy tailed student-t preserves edges of textures; MAP-EM is used for model inference.

St-SVGMM

Nnxn ,...,1, Feature vector for each pixel:

SVGMM:

Each pixel has its own mixing weights

Each pixel xn: },,...,,{ 21nK

nn

},,...,,{ 21nK

nn zzz

weights

indicator variables

Likelihood:

Prior:

5/15

K

jnj

nj zz

11 },1,0{

St-SVGMM

Prior for mixing weight π:

nx

d=2

d=1

d: neighborhood adjacency type

d=1: horizontald=2: vertical

γd(n): the set of neighbors of pixel n, with respect to the dth adjacency type

K×D different student-t distributions are introduced, with hyperparameters

Joint prior for π:

6/15

St-SVGMM

The student-t distribution can be modeled by introducing the latent variable

)(,:1,:1,:1}{ nkDdKjNnnkj d

uU

nkju plays an important role:

,nkju ,k

jnj neighboring pixels n, k belong to the same cluster

,0nkju ,k

jnj n, k are at the edge of two clusters

n – edge location k (d) – adjacency type (horizontal or vertical)j – cluster index (edges of which cluster)

7/15

St-SVGMM

),;(~,,| jjnnn xNzx

Model summary

)(~ nn lmultinomiaz

8/15

Inference

MAP-EM algorithm for model inference.

Complete log-likelihood

Model parameters:

E-step (update Z, U)

9/15

Inference

M-step ( update )

=

,,,,

10/15

Results

U-variable maps

nkju

j=1: sky

j=2: roof & shadows

j=3: building

d=1: horizontal d=2: vertical

n – edge location k (d) – adjacency typej – cluster index

K=3 clusters

Brighter regions represent lower values – edges.

11/15

Results

Comparison on 300 images of the Berkeley image database

Statistics on the Rand Index (RI) (measuring the consistency between the ground truth and the segmentation map); higher is better.

Statistics on the boundary displacement error (BDE) (measuring error of boundary displacement with respect to the ground truth); lower is better.

12/15

Results

Segmentation examples

K=5 K=15K=10original image

13/15

Results

K=5 K=15K=10original image 14/15

Conclusion

15/15

Proposed a GMM-based clustering algorithm for image segmentation;

Used smoothness prior to consider the adjacent pixels belonging to the same cluster;

Also captured the image edge structure (no smoothness enforced across segment boundaries);

All required parameters are estimated from the data (no requirement of empirical parameter selection).

Next: automatically estimating the number of components K.