Probability density function (pdf) estimation using
isocontours/isosurfaces Application to Image Registration
Application to Image Filtering Circular/spherical density
estimation in Euclidean space 2
Slide 3
3 Histograms Kernel density estimate Mixture model Parameter
selection: bin-width/bandwidth/number of components Bias/variance
tradeoff: large bandwidth: high bias, low bandwidth: high variance)
Sample-based methods Do not treat a signal as a signal
Slide 4
4 Continuous image representation: using some interpolant.
Trace out isocontours of the intensity function I(x,y) at several
intensity values.
Slide 5
Slide 6
6 Assume a uniform density on (x,y) Random variable
transformation from (x,y) to (I,u) Integrate out u to get the
density of intensity I Every point in the image domain contributes
to the density. Published in CVPR 2006, PAMI 2009. u = direction
along the level set (dummy variable)
Slide 7
7
Slide 8
8 Relationships between geometric and probabilistic
entities.
Slide 9
Similar density estimator developed by Kadir and Brady (BMVC
2005) independently of us. Similar idea: several differences in
implementation, motivation, derivation of results and applications.
9
Slide 10
10 Densities (derivatives of the cumulative) do not exist where
image gradients are zero, or where image gradients run parallel.
Compute cumulative interval measures.
Randomized/digital approximation to area calculation. Strict
lower bound on the accuracy of the isocontour method, for a fixed
interpolant. Computationally more expensive than the isocontour
method. 14
Slide 15
15 128 x 128 bins
Slide 16
Simplest one: linear interpolant to each half- pixel (level
curves are segments). Low-order polynomial interpolants: high bias,
low variance. High-order polynomial interpolants: low bias, high
variance. 16
Slide 17
17 Polynomial Interpolant Accuracy of estimated density
improves as signal is sampled with finer resolution. Assumptions on
signal: better interpolant Bandlimited analog signal,
Nyquist-sampled digital signal: Accurate reconstruction by sinc
interpolant! (Whitaker-Shannon Sampling Theorem)
Slide 18
Probability density function (pdf) estimation using isocontours
Application to Image Registration Application to Image Filtering
Circular/spherical density estimation in Euclidean space 18
Slide 19
19 Given two images of an object, to find the geometric
transformation that best aligns one with the other, w.r.t. some
image similarity measure. Mutual Information: Well known image
similarity measure Viola and Wells (IJCV 1995) and Maes et al (TMI
1997). Insensitive to illumination changes: useful in multimodality
image registration
24 PD slice T2 sliceWarped T2 sliceWarped and Noisy T2 slice
Brute force search for the maximum of MI
Slide 25
25 MI with standard histograms MI with our method Par. of
affine transf.
Slide 26
26 MethodError in Theta (avg., var.) Error in s (avg.,var.)
Error in t (avg., var.) Histograms (bilinear)
3.7,18.10.7,00.43,0.08 Isocontours0,0.060,0 PVI1.9,
8.50.56,0.080.49,0.1 Histograms (cubic) 0.3,49.40.7,00.2,0
2DPointProb0.3,0.220,0 32 BINS
Slide 27
Probability density function (pdf) estimation using isocontours
Application to Image Registration Application to Image Filtering
Circular/spherical density estimation in Euclidean space 27
Slide 28
Anisotropic neighborhood filters (Kernel density based
filters): Grayscale images 28 Central Pixel (a,b): Neighborhood
N(a,b) around (a,b) K: a decreasing function (typically Gaussian)
Parameter controls the degree of anisotropicity of the
smoothing
Slide 29
Anisotropic Neighborhood filters: Problems 29 Sensitivity to
the parameter Sensitivity to the SIZE of the Neighborhood Does not
account for gradient information
Slide 30
Anisotropic Neighborhood filters: Problems 30 Treat pixels as
independent samples
Slide 31
Continuous Image Representation 31 Interpolate in between the
pixel values
Slide 32
Areas between isocontours at intensity and + (divided by area
of neighborhood)= Pr( < Intensity < +|N(a,b)) Continuous
Image Representation 32
Slide 33
Areas between isocontours: contribute to weights for averaging.
33 Published in EMMCVPR 2009
Slide 34
Extension to RGB images 34 Joint Probability of R,G,B = Area of
overlap of isocontour pairs from R, G, B images
Slide 35
Mean-shift framework A clustering method developed by Fukunaga
& Hostetler (IEEE Trans. Inf. Theory, 1975). Applied to image
filtering by Comaniciu and Meer (PAMI 2003). Involves independent
update of each pixel by maximization of local estimate of
probability density of joint spatial and intensity parameters.
35
Slide 36
Mean-shift framework One step of mean-shift update around
(a,b,c) where c=I(a,b). 36
Slide 37
Our Method in Mean-shift Setting 37 I(x,y)X(x,y)=x
Y(x,y)=y
Slide 38
Our Method in Mean-shift Setting 38 Facets of tessellation
induced by isocontours and the pixel grid = Centroid of Facet #k. =
Intensity (from interpolated image) at. = Area of Facet #k.
Slide 39
Experimental Setup: Grayscale Images Piecewise-linear
interpolation used for our method in all experiments. For our
method, Kernel K = pillbox kernel, i.e. For discrete mean-shift,
Kernel K = Gaussian. Parameters used: neighborhood radius =3, =3.
Noise model: Gaussian noise of variance 0.003 (scale of 0 to 1). 39
If |z|
Slide 40
Original Image Noisy Image Denoised (Isocontour Mean Shift)
Denoised (Gaussian Kernel Mean Shift) MSE Noisy Image181.27
Isocontour (=3, =3) 110.95 Std. Mean shift (=3, =3) 175.27 Std.
Mean shift (=5, =5) 151.27 40
Slide 41
Original Image Noisy Image Denoised (Isocontour Mean Shift)
Denoised (Std.Mean Shift) Noisy image Isocontour Mean shift (=3,
=3) Std. mean shift (=3, =3) Std. mean shift (=5, =3)
MSE190113.8184.77153.5 41
Slide 42
Experiments on color images Use of pillbox kernels for our
method. Use of Gaussian kernels for discrete mean shift. Parameters
used: neighborhood radius = 6, = 6. Noise model: Independent
Gaussian noise on each channel with variance 0.003 (on a scale of 0
to 1). 42
Slide 43
Experiments on color images Independent piecewise-linear
interpolation on R,G,B channels in our method. Smoothing of R, G, B
values done by coupled updates using joint probabilities. 43
Slide 44
Original Image Noisy Image Denoised (Isocontour Mean Shift)
Denoised (Gaussian Kernel Mean Shift) MSE Noisy Image572.24
Isocontour (=3, =3) 319.88 Std. Mean shift (=3, =3) 547.96 Std.
Mean shift (=5, =5) 496.7 44
Slide 45
Original Image Noisy Image Denoised (Isocontour Mean Shift)
Denoised (Gaussian Kernel Mean Shift) Noisy image Isocontour Mean
shift (=3, =3) Std. mean shift (=3, =3) Std. mean shift (=5, =5)
MSE547.9306.14526.8477.25 45
Slide 46
Observations Discrete kernel mean shift performs poorly with
small neighborhoods and small values of . Why? Small sample-size
problem for kernel density estimation. Isocontour based method
performs well even in this scenario (number of isocontours/facets
>> number of pixels). Large or large neighborhood values not
always necessary for smoothing. 46
Slide 47
Observations Superior behavior observed when comparing
isocontour-based neighborhood filters with standard neighborhood
filters for the same parameter set and the same number of
iterations. 47
Slide 48
Probability density function (pdf) estimation using isocontours
Application to Image Registration Application to Image Filtering
Circular/spherical density estimation in Euclidean space 48
Slide 49
Examples of unit vector data: 1. Chromaticity vectors of color
values: 2. Hue (from the HSI color scheme) obtained from the RGB
values. 49
Slide 50
50 Convert RGB values to unit vectors Estimate density of unit
vectors voMF mixture models Banerjee et al (JMLR 2005) Other
popular kernels: Watson, cosine.
Slide 51
51 Estimate density of RGB using KDE/Mixture models Density of
(magnitude,chromaticity) using random-variable transformation
Density of chromaticity (integrate out magnitude) Projected normal
estimator: Watson,Statistics on spheres, 1983, Small,The
statistical theory of shape, 1995 Density of chromaticity:
conditioning on m=1. Variable bandwidth voMF KDE: Bishop, Neural
networks for pattern recognition 2006. Whats new? The notion that
all estimation can proceed in Euclidean space.
Slide 52
52 Estimate density of RGB using KDE/Mixture models Use random
variable transformation to get density of HSI (hue,
saturation,intensity) Integrate out S,I to get density of hue
Slide 53
Consistency between densities of Euclidean and unit vector data
(in terms of random variable transformation/conditioning).
Potential to use the large body of literature available for
statistics of Euclidean data (example: Fast Gauss Transform
Greengard et al (SIAM Sci. Computing 1991), Duraiswami et al (IJCV
2003). Model selection can be done in Euclidean space. 53