Wavelets

Fast Multiresolution Image QueryingJacobs et.al. SIGGRAPH95

Outline

Overview / BackgroundWavelets

2D Image matchingL1, L2 metricsWavelet metricEvaluation

Use in 3D

Image matching

2D analogue of 3D shape matching Raster instead of XYZ

What are we trying to match?

Looking for different images of the same things? Different projections Different colors

Looking for images that look the same? Look for similar shapes and colors

Metric to discern like human eye

Wavelets

Decompose a signal into component partsFourier analysis: a signal can be represented

as a (possibly infinite) sum of sine and cosine functions

Signal becomes a set of wavelet coefficients

Coefficients represent features of signal

Wavelets II

Signal can be completely reconstructed from all the coefficients

Signal can be partially reconstructed from some coefficients

Wavelets for 2D Images

Each color plane in image is signal Coefficients will represent visual features

in the image Store as many coefficients as needed

Image compression (see next slide) c.f. Statistical shape descriptors

Wavelet Reconstruction

SIGGRAPH 96 Course Notes: Wavelets in Computer Graphics

Comparing Images

Develop a metric that describes how closely two images match

Smaller difference in metric = images more similar

Image metrics

Comparing images Q and T, with dimensions i,j L1-Norm

For each pixel in Q, calculate the difference between Q[i,j] and T[i,j]

Add absolute value of differences of all i,j to form metric

ji

jiTjiQTQ,

1],[],[,

Image metrics II

L2-Norm

For each pixel in Q, calculate the square of the difference between Q[i,j] and T[i,j]

Add for all i,j, and take square root Better than L1?

2/1

,

2

2],[],[,

ji

jiTjiQTQ

Image metrics III

Problems with L1 and L2Expensive to compute / compare: O(i*j)Not discriminating in cases with

Color Shift Misregistration Noise / Dithering

In general, not good descriptorsc.f. D1, D2 in 3D

Wavelets as image metrics

Capture features of images e.g. edges in coefficients

Use Haar waveletsSquare basis functionsEasy to implement and compute

Calculate coefficients, truncate, quantize

Truncation

128x128 image has 1282 coefficients Truncation = only storing largest ‘n’

coefficients ‘n’ ~ 40-60 depending on exact use Discarding smaller coefficients discards

high frequency information i.e. detailLoss of that information is desirable

Quantization

Reduce precision of wavelet magnitudeLarge +ve +1Large –ve -1Else 0

Turns out this works well for matching

Wavelet metric

Q,T are query and target image coefficients

w is weighting function

ji

ji jiTjiQwTQwTQ,

~~

,0,0 ],[],[]0,0[]0,0[,

Weighting function

Weighting function applied to give particular pairs of coefficients different significance in comparison

Gives ability to statistically tune the metric Determined experimentally from dataset

(Appendix A) Weights expensive to calculate

Compute fewer Bins to map range of i,j onto a weight

Wavelet metric II

For i=0, j=0 value in Q and T is proportional to the average overall color

From quantization, use ≠

0],[:,

~~

),(0~

],[],[]0,0[]0,0[,jiQji

jibin jiTjiQwTQwTQ

Calculating coefficients

Standard two-dimensional Haar wavelet decomposition

Decompose each row, then decompose each column of the result

Trivial to implement

Wavelet metric III

Final metric isT[0,0]Sign, i and j of n largest coefficients in T

Faster Matching

To speed up matching, use 6 arraysOne for each combination of R,G,B,+,-DR+, DR-, …Each i,j in Dx is a list of all images with a

metric coefficient in that color range, with that sign

Evaluation

Better than L1 and L2 Matches to ~1% of database Compact Fast to compare: similar complexity to an 8x8 pixel

image L1/L2 for any resolution More robust

Misregistration Color shifting Dithering Different resolutions

Evaluation II

Evaluation III

LimitsScaling ~1.5 timesRotation ~20 degreesTranslation ~ 15% of width

Matching in 3D

Can this be used in 3D as well Compares image rather than geometry Render 3D into voxels? Projection of 3D object into 2D?

In general, other 3D specific methods probably much better