Multiresolution Histograms and Multiresolution Histograms and their Use for Texture Classificationtheir Use for Texture Classification

Stathis Hadjidemetriou, Michael Grossberg and Shree NayarStathis Hadjidemetriou, Michael Grossberg and Shree Nayar

CAVE Lab, Columbia UniversityCAVE Lab, Columbia University

Partially funded by NSF ITR Award, DARPA/ONR MURIPartially funded by NSF ITR Award, DARPA/ONR MURI

Fast and Simple FeatureFast and Simple Feature

Q: Is there a fast feature which captures spatial information?

A: Consider multiple resolutions.

Same Histogram

Histograms of Filtered ImagesHistograms of Filtered Images

Graylevel

Graylevel

Graylevel

Graylevel

Bin

C

ount

Bin

C

ount

Bin

C

ount

Bin

C

ount

Graylevel

Graylevel

Graylevel

Graylevel

Bin

C

ount

Bin

C

ount

Bin

C

ount

Bin

C

ount

Histograms

Histograms

Re

solu

tion

h h

Analysis of Multiresolution HistogramsAnalysis of Multiresolution Histograms

Bin

C

ount

Bin

C

ount

Bin

C

ount

Graylevel

Graylevel

Graylevel

Graylevel

Graylevel

Bin

Cou

nt C

hang

eB

in C

ount

Cha

nge

Shape and TextureProperties

Difference Histograms

Multiresolution Histograms

Shape and TextureImages

?

Tools for Analysizing the HistogramTools for Analysizing the Histogram• Shanon Entropy

•Change in Shanon Entropy: Fisher Information

•Generalization: Tsallis Entropy/Generalized Fisher Information

255

0

)(log)()(j

jhjhS

)()(

Sd

dKJ

Resolution

Multiresolution Histogram

Bin

Filter Dependent Constant

Relating Histogram Change to ImageRelating Histogram Change to Image• Fisher Information:

Measure of image sharpness [Stam, 59, Plastino et al, 97]:

dydxLL

LJ

D

2

Image

Image Gradient

Image Domain

L

L 2||

Edge filter never computed: Implicit

Analysis of Multiresolution HistogramsAnalysis of Multiresolution Histograms

Bin

C

ount

Bin

C

ount

Bin

C

ount

Graylevel

Graylevel

Graylevel

Graylevel

Graylevel

Bin

Cou

nt C

hang

eB

in C

ount

Cha

nge

Shape and TextureProperties

Difference Histograms

Multiresolution Histograms

Shape and TextureImages

FisherInformation

Resolution Fis

her

Inf

orm

atio

n •Shape Elongation

•Shape Boundary

•Texel Repetition

•Texel Placement

Shape Elongation and Fisher Shape Elongation and Fisher InformationInformation

Elongation:

.y

x

St. dev. along axes: x, y.

•Gaussian:Sides of base: rx, ry.

.y

x

r

r

•Pyramid:

)1

(

J (analytically)

Elongation:

6

5

4

3

21 2 3 4 5

J

Shape Boundary and Fisher Shape Boundary and Fisher InformationInformation

.)( 15.0 yxRL Superquadrics:

=0.56 =1.00 =1.48 =2.00 =6.67

(numerically)Complex boundary J

J

2

3

4

5

6

2 4 60

Texel Repetition and Fisher Texel Repetition and Fisher InformationInformation

,p

JpJ p 2 (analytically).

1 42 3 5 60

2

4

6

8Tileing

1 42 3 5 6

Tileing p

J

0

2

4

6

8

Tileing p

J

x 103

Texel Placement and Fisher Texel Placement and Fisher InformationInformation

Stand. dev. of perturbation

(numerically)Randomness qJ

Average of 20 trials

0 155 10 20

6.6

6.4

6.2

6

5.8

J

St. Dev (% of Texel Width)

0 155 10 20St. Dev (% of Texel Width)

x 103

2.9

2.8

2.7

2.6

2.5

J

L1 norm

Matching AlgorithmMatching Algorithm

Multiresolution histogram with Burt-Adelson Pyramid

Cumulative histograms

Difference histograms betweenconsecutive resolutions

Concatenate to form feature vector

Com

pute

F

eatu

re

Histograms Bin WidthHistograms Bin Width

•Histogram bin width:

3/16/1 )()8()( nhhw

59.12)(

)( 3/21

i

i

hw

hw

•Subsampling factor in pyramid:

Parameters of Multiresolution HistogramParameters of Multiresolution Histogram

•Histogram smoothing to avoid aliasing:–Database images–Test images

•Histogram normalization– Image size– Histogram size

Databases for MatchingDatabases for Matching

• Database of CUReT textures [Dana et al, 99]:

– 8,046 images; 61 materials– Histogram equalized

• Database of Brodatz textures [Brodatz, 66]:

– 91 images; 7 images– Histogram equalized

Database of Brodatz TexturesDatabase of Brodatz TexturesSamples of equalized images:

Match Results for Brodatz TexturesMatch Results for Brodatz TexturesMatch under Gaussian noise of st.dev. 15 graylevels

Class Matching Sensitivity: Brodatz Class Matching Sensitivity: Brodatz TexturesTextures

0 10 20 30 40 50 600

20

40

60

80

100

St dev. of noise n

Cla

ss m

atch

ed

8

16

32

62

128

256

Number ofbins

Class Matching Sensitivity: Brodatz TexturesClass Matching Sensitivity: Brodatz Textures

smoothing & adaptive bin size

0 10 20 30 40 50 60

100

St dev. of noise n

95

90

85

80

75

70

65

60

256 Constant256, Higher Subsampling= 22/3 256, Lower Subsampling = 21/2

Database of Curet Textures Database of Curet Textures Samples of equalized images:

Match Results for Curet TexturesMatch Results for Curet Textures

Match under Match under Gaussian noiseGaussian noise of st.dev. of st.dev. 15 graylevels15 graylevels..

• Match 100 randomly selected images per noise level

Difference norm & Smoothing

Class Matching Sensitivity: CUReT Class Matching Sensitivity: CUReT TexturesTextures

0 10 20 30 40 5050

60

70

80

90

100

St dev. of noise n

Cla

ss m

atch

ed

256 Constant256, Higher Subsampling= 22/3 256, Lower Subsampling = 21/2

Comparison with Low-level FeaturesComparison with Low-level Features

• Fourier power spectrum annuliFourier power spectrum annuli• Gabor featuresGabor features• Daubechies wavelet featuresDaubechies wavelet features• Auto-cooccurrence matrixAuto-cooccurrence matrix• Markov random field parametersMarkov random field parameters

Comparison with Low-Level FeaturesComparison with Low-Level Features

•Auto-cooccurrence matrix

•Fourier power spectrum annuli:

•Gabor features

r1

r2

Comparison with Low-Level FeaturesComparison with Low-Level Features

•Markov random field parameters

Wavelets decomposition Wavelet packets decomposition

•Wavelet coefficient energies:

Comparison of Computation CostsComparison of Computation Costs

1 Markov random field parameters O(n(2-1)2-(2-1)3/3)

2 Gabor features ( (logn+1)nlogn1/2)

3 Fourier power spectrum features O(n3/2)

4 Auto-cooccurrence matrix O(n)

5 Wavelet coefficient energies O(nl)

6 Multiresolution histograms n

n- number of pixels- window widthl- resolution levels

decreasing cost

Sensitivity Comparison to TransformationsSensitivity Comparison to Transformations FeatureFeature TranslationTranslation RotationRotation Uniform Uniform

ScalingScaling

11 Fourier power Fourier power spectrum annulispectrum annuli

invariantinvariant robustrobust equivariantequivariant

22 Gabor featuresGabor features invariantinvariant variantvariant equivariantequivariant

33 Daubechies wavelet Daubechies wavelet energiesenergies

variantvariant variantvariant variantvariant

44 Multiresolution Multiresolution histogramshistograms

invariantinvariant invariantinvariant equivariantequivariant

55 Auto-cooccurrence Auto-cooccurrence matrixmatrix

invariantinvariant robustrobust equivariantequivariant

66 Markov random field Markov random field parametersparameters

invariantinvariant variantvariant variantvariant

Matching Comparison of Features: Matching Comparison of Features: BrodatzBrodatz

•Brodatz textures database:

0 10 20 30 40 50 600

20

40

60

80

100

St dev. of noise n

Cla

ss m

atch

ed Multiresolution Diff. HistogramsFourier Power SpectrumGabor FeaturesWavelet PacketsCooccurence MatrixMarkov Random Fields

Matching Comparison of Features: Matching Comparison of Features: CUReTCUReT

•Curet textures database:

•Match 100 randomly selected images per noise level

0 10 20 30 40 50St dev. of noise n

0

20

40

60

80

100

Cla

ss m

atch

ed Multiresolution Diff. HistogramsFourier Power SpectrumGabor FeaturesWavelet PacketsCooccurence MatrixMarkov Random Fieldsr1

Sensitivity of Features to RecognitionSensitivity of Features to Recognition

Feature Gaussian Noise

Database size,#classes

Illumination Parameter selection

Fourier power spectrum annuli

sensitive sensitive robust very sensitive

Gabor features robust robust robust sensitive

Daubechies wavelet energies

sensitive robust robust robust

Multiresolution histogram

robust robust robust robust

Auto-cooccurrence matrix

very sensitive

very sensitive very sensitive

very sensitive

Markov random field parameters

very sensitive

very sensitive sensitive N/A