Multiscale Analysis of Images

Multiscale Analysis of Images

Gilad LermanMath 5467

(stealing slides from Gonzalez & Woods, and Efros)

The Multiscale Nature of Images

Recall: Gaussian pre-filtering

G 1/4

G 1/8

Gaussian 1/2

Solution: filter the image, then subsample• Filter size should double for each ½ size reduction.

Subsampling with Gaussian pre-filtering

G 1/4 G 1/8Gaussian 1/2

Solution: filter the image, then subsample• Filter size should double for each ½ size reduction.

Image Pyramids

Known as a Gaussian Pyramid [Burt and Adelson, 1983]• In computer graphics, a mip map [Williams, 1983]• A precursor to wavelet transform

A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose

Figure from David Forsyth

Gaussian pyramid construction

filter mask

Repeat• Filter• Subsample

Until minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid)

The whole pyramid is only 4/3 the size of the original image!

What does blurring take away? (recall)

original

What does blurring take away? (recall)

smoothed (5x5 Gaussian)

High-Pass filter

smoothed – original

Band-pass filtering

Laplacian Pyramid (subband images)Created from Gaussian pyramid by subtraction

Gaussian Pyramid (low-pass images)

Laplacian Pyramid

How can we reconstruct (collapse) this pyramid into the original image?

Need this!

Originalimage

Image resampling (interpolation)So far, we considered only power-of-two subsampling

• What about arbitrary scale reduction?• How can we increase the size of the image?

Recall how a digital image is formed

• It is a discrete point-sampling of a continuous function• If we could somehow reconstruct the original function, any new image could be generated, at any resolution and

scale

1 2 3 4 5

d = 1 in this example

Image resamplingSo far, we considered only power-of-two subsampling

• What about arbitrary scale reduction?• How can we increase the size of the image?

Recall how a digital image is formed

• It is a discrete point-sampling of a continuous function• If we could somehow reconstruct the original function, any new image could be generated, at any resolution and

scale

1 2 3 4 5

d = 1 in this example

Image resamplingSo what to do if we don’t know

1 2 3 4 52.5

1 d = 1 in this example

• Answer: guess an approximation• Can be done in a principled way: filtering

Image reconstruction• Convert to a continuous function

• Reconstruct by cross-correlation:

Resampling filtersWhat does the 2D version of this hat function look like?

Better filters give better resampled images• Bicubic is common choice

Why not use a Gaussian?

What if we don’t want whole f, but just one sample?

performs linear interpolation

(tent function) performs bilinear interpolation

Bilinear interpolationSmapling at f(x,y):

Applications (more efficient with wavelets)

•Coding High frequency coefficients need fewer bits, so one can collapse a compressed Laplacian pyramid•Denoising/Restoration Setting “most” coefficients in Laplacian pyramid to zero•Image Blending

Application: Pyramid Blending

0

1

0

1

0

1

Left pyramid Right pyramidblend

Image Blending

Feathering

01

01

+

=Encoding transparency

I(x,y) = (R, G, B, )

Iblend = left Ileft + right Iright

Ileft Iright

leftright

Affect of Window Size

0

1 left

right0

1

Affect of Window Size

0

1

0

1

Good Window Size

0

1

“Optimal” Window: smooth but not ghostedBurt and Adelson (83): Choose by pyramids…

Pyramid Blending

Pyramid Blending (Color)

Laplacian Pyramid: BlendingGeneral Approach:

1. Build Laplacian pyramids LA and LB from images A and B2. Build a Gaussian pyramid GR from selected region R (black

white corresponding images)3. Form a combined pyramid LS from LA and LB using nodes

of GR as weights:• LS(i,j) = GR(I,j,)*LA(I,j) + (1-GR(I,j))*LB(I,j)

4. Collapse the LS pyramid to get the final blended image

laplacianlevel

4

laplacianlevel

2

laplacianlevel

0

left pyramid right pyramid blended pyramid

Blending Regions

Simplification: Two-band BlendingBrown & Lowe, 2003

• Only use two bands: high freq. and low freq.• Blends low freq. smoothly• Blend high freq. with no smoothing: use binary mask

Can be explored in a project…