Mean ShiftA Robust Approach to

Feature Space Analysis

Kalyan Sunkavalli

04/29/2008

ES251R

An Example Feature Space

An Example Feature Space

An Example Feature Space

Parametric Density Estimation?

Mean Shift

• A non-parametric technique for analyzing complex multimodal feature spaces and estimating the stationary points (modes) of the underlying probability density function without explicitly estimating it.

Outline

• Mean Shift– An intuition– Kernel Density Estimation– Derivation– Properties

• Applications of Mean Shift– Discontinuity preserving Smoothing– Image Segmentation

Outline

• Mean Shift– An intuition– Kernel Density Estimation– Derivation– Properties

• Applications of Mean Shift– Discontinuity preserving Smoothing– Image Segmentation

Intuitive Description

Distribution of identical billiard balls

Region ofinterest

Center ofmass

Mean Shiftvector

Objective : Find the densest regionSlide Credit: Yaron Ukrainitz & Bernard Sarel

Intuitive Description

Distribution of identical billiard balls

Region ofinterest

Center ofmass

Mean Shiftvector

Objective : Find the densest region

Intuitive Description

Distribution of identical billiard balls

Region ofinterest

Center ofmass

Mean Shiftvector

Objective : Find the densest region

Intuitive Description

Distribution of identical billiard balls

Region ofinterest

Center ofmass

Mean Shiftvector

Objective : Find the densest region

Intuitive Description

Distribution of identical billiard balls

Region ofinterest

Center ofmass

Mean Shiftvector

Objective : Find the densest region

Intuitive Description

Distribution of identical billiard balls

Region ofinterest

Center ofmass

Mean Shiftvector

Objective : Find the densest region

Intuitive Description

Distribution of identical billiard balls

Region ofinterest

Center ofmass

Objective : Find the densest region

Outline

• Mean Shift– An intuition– Kernel Density Estimation– Derivation– Properties

• Applications of Mean Shift– Discontinuity preserving Smoothing– Image Segmentation

Assumed Underlying PDF

Estimate from data

Data Samples

Parametric Density Estimation

The data points are sampled from an underlying PDF

Assumed Underlying PDF Data Samples

Data pointdensity

Non-parametric Density Estimation

PDF value

Assumed Underlying PDF Data Samples

Non-parametric Density Estimation

Parzen Windows

Kernel Properties

1. Bounded

2. Compact support

3. Normalized

4. Symmetric

5. Exponential decay

6.

Kernels and Bandwidths

• Kernel Types

• Bandwidth Parameter

(product of univariate kernels) (radially symmetric kernel)

Various KernelsEpanechnikov

Normal

Uniform

Outline

• Mean Shift– An intuition– Kernel Density Estimation– Derivation– Properties

• Applications of Mean Shift– Discontinuity preserving Smoothing– Image Segmentation

Density Gradient Estimation

Epanechnikov Uniform

Normal Normal

Modes of the probability density

Mean Shift

KDE Mean Shift

Mean Shift Algorithm

• compute mean shift vector

• translate kernel (window) by mean shift vector

Mean Shift

• Mean Shift is proportional to the normalized density gradient estimate obtained with kernel

• The normalization is by the density estimate computed with kernel

Outline

• Mean Shift– An intuition– Kernel Density Estimation– Derivation– Properties

• Applications of Mean Shift– Discontinuity preserving Smoothing– Image Segmentation

Properties of Mean Shift• Guaranteed convergence

– Gradient Ascent algorithms are guaranteed to converge only for infinitesimal steps.

– The normalization of the mean shift vector ensures that it converges.

– Large magnitude in low-density regions, refined steps near local maxima Adaptive Gradient Ascent.

• Mode Detection– Let denote the sequence of kernel locations.– At convergence– Once gets sufficiently close to a mode of it will

converge to the mode.– The set of all locations that converge to the same mode define

the basin of attraction of that mode.

Properties of Mean Shift

• Smooth Trajectory– The angle between two consecutive mean shift vectors

computed using the normal kernel is always less that 90°– In practice the convergence of mean shift using the normal

kernel is very slow and typically the uniform kernel is used.

Mode detection using Mean Shift

• Run Mean Shift to find the stationary points– To detect multiple modes, run in parallel starting with

initializations covering the entire feature space.

• Prune the stationary points by retaining local maxima– Merge modes at a distance of less than the bandwidth.

• Clustering from the modes– The basin of attraction of each mode delineates a cluster of

arbitrary shape.

Mode Finding on Real Data

initialization

detected mode

tracks

Mean Shift Clustering

Outline

• Mean Shift– Density Estimation– What is mean shift?– Derivation– Properties

• Applications of Mean Shift– Discontinuity preserving Smoothing– Image Segmentation

Joint Spatial-Range Feature Space

• Concatenate spatial and range (gray level or color) information

Discontinuity Preserving Smoothing

Discontinuity Preserving Smoothing

Discontinuity Preserving Smoothing

Discontinuity Preserving Smoothing

Outline

• Mean Shift– Density Estimation– What is mean shift?– Derivation– Properties

• Applications of Mean Shift– Discontinuity preserving Smoothing– Image Segmentation

Clustering on Real Data

Image Segmentation

Image Segmentation

Image Segmentation

Image Segmentation

Image Segmentation

Acknowledgements

• Mean shift: A robust approach toward feature space analysis. D Comaniciu, P Meer Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 24, No. 5. (2002), pp. 603-619.

• http://www.caip.rutgers.edu/riul/research/papers.html

• Slide credits: Yaron Ukrainitz & Bernard Sarel

Thank You