Upload
candace-sherman
View
225
Download
4
Tags:
Embed Size (px)
Citation preview
Signal Processingand
Representation Theory
Lecture 4
Outline:• Review• Fourier Transforms• Applications• Invariant Descriptors
Representation Theory
ReviewCircle:
If G is the group of 2D rotations / reflections, acting on the space of functions on the circle, the irreducible sub-representations of G are the 1D subspaces spanned by the complex exponentials:
Vk=Span{eik}
Representation Theory
ReviewCircle:
Given a function f defined on the circle, we can obtain a rotation / reflection invariant representation by computing the Fourier decomposition:
and storing the norms of the Fourier coefficients:
k
ikkeaf )(
,...,,...,,,)( 110 nn aaaaaf
Representation TheoryReviewCircle:
Given two functions f and h defined on the circle, we can correlate the functions by computing the Fourier decompositions:
multiplying the Fourier coefficients:
and computing the inverse Fourier transform.
k
ikk
k
ikk ebheaf )()(
k
ikkk ebahf )(),(
Representation Theory
ReviewSphere:
If G is the group of 3D rotations / reflections, acting on the space of functions on the sphere, the irreducible sub-representations of G are the (2d+1)-dimensional subspaces spanned by the spherical harmonics of frequency d:
),(),,(,),,(),,( 11 dd
dd
dd
ddd YYYYV Span
Representation Theory
ReviewSphere:
Given a function f defined on the sphere, we can obtain a rotation / reflection invariant representation by computing the spherical harmonic decomposition:
and storing the norms of the frequency components:
0
),(),(l
l
lm
ml
ml Yaf
,,,,)(
22211
201
211
00
dd
dd aaaaaaf
Representation TheoryReviewSphere:
Given two functions f and h defined on the sphere, we can correlate the functions by computing the spherical harmonic decompositions:
multiplying the spherical harmonic coefficients:
and computing the inverse Wigner-D transform.
00
),(),(),(),(l
l
lm
ml
ml
l
l
lm
ml
ml YbhYaf
0 ',
',' )()(,
l
l
lmm
lmm
ml
ml RDbahRf
Outline:• Review• Fourier Transforms• Applications• Invariant Descriptors
Representation Theory
Fourier Transforms2D:
If we have the space of functions in the plane, we can consider the representation obtained by the group of translations.
Since translations are commutative, Schur’s Lemma tells us that the irreducible representations are all one-dimensional.
Representation Theory
Fourier Transforms2D:
The irreducible representations are the sub-spaces spanned by the functions:
Translating each function by (x0,y0) we get:
)(, ),( lykxilk eyxw
),(
),(
,)(
)()(
))()((00,
00
00
00
yxwe
ee
eyyxxw
lklykxi
lykxilykxi
yylxxkilk
Representation Theory
Fourier Transforms2D (Invariance):
If f(x,y) is a function defined on the plane, we can express the function in terms of its Fourier decomposition:
and obtain a rotation invariant representation by storing the energy in each frequency:
lk
lykxilk eayxf
,
)(,),(
,,,
,,,
,,,
)(1,11,10,1
1,11,10,1
1,01,00,0
aaa
aaa
aaa
f
Representation Theory
Fourier Transforms2D (Correlation):
If f and g are functions defined on the plane whose Fourier decompositions are:
the correlation of f with g over the space of translations can be computed by multiplying the Fourier coefficients:
lk
lykxilk
lk
lykxilk ebyxheayxf
,
)(,
,
)(, ),(),(
lk
lykxilklk ebayyxxhyxf
,
)(,,00
00),(),,(
Representation Theory
Fourier Transforms3D:
If we have the space of functions in 3D, we can consider the representation obtained by the group of translations.
Since translations are commutative, Schur’s Lemma tells us that the irreducible representations are all one-dimensional.
Representation Theory
Fourier Transforms3D:
The irreducible representations are the sub-spaces spanned by the functions:
Translating each function by (x0,y0,z0) we get:
)(,, ),,( mzlykximlk ezyxw
),,(
),,(
,,)(
)()(
))()()((000,,
000
000
000
zyxwe
ee
ezzyyxxw
mlkmzlykxi
mzlykximzlykxi
zzmyylxxkimlk
Representation Theory
Fourier Transforms3D (Invariance):
If f(x,y,z) is a function defined in 3D, we can express the function in terms of its Fourier decomposition:
and obtain a rotation invariant representation by storing the energy in each frequency:
mlk
mzlykximlk eazyxf
,,
)(,,),,(
,,,
,,,
,,,
,,,
,,,
,,,
,,,
,,,
,,,
)(1,1,11,1,10,1,1
1,1,11,1,10,1,1
1,0,11,0,10,0,1
1,1,11,1,10,1,1
1,1,11,1,10,1,1
1,0,11,0,10,0,1
1,1,01,1,00,1,0
1,1,01,1,00,1,0
1,0,01,0,00,0,0
aaa
aaa
aaa
aaa
aaa
aaa
aaa
aaa
aaa
f
Representation Theory
Fourier Transforms3D (Correlation):
If f and g are functions defined in 3D whose Fourier decompositions are:
the correlation of f with g over the space of translations can be computed by multiplying the Fourier coefficients:
mlk
mzlykximlk
mlk
mzlykximlk ebzyxheazyxf
,,
)(,,
,,
)(,, ),,(),,(
mlk
mzlykximlkmlk ebazzyyxxhzyxf
,,
)(,,,,000
000),,(),,,(
Outline:• Review• Fourier Transforms• Applications
– Circle– 2D– Sphere– 3D
• Invariant Descriptors
Representation Theory
ApplicationsCircle:
If we have a real-valued function f on the circle, we can express the function in terms of its Fourier decomposition:
where ak,bkℝ and bk=-b-k.
k
ikkk eibaf )(
Representation Theory
ApplicationsCircle:
Given the space of real-valued functions on the circle, the Fourier decomposition can be used for correlation and invariants-extraction with respect to the group of 2D rotations / reflections.
Representation Theory
ApplicationsCircle:
What if we consider the smaller group of axial flips:
that arise due to the ambiguity in PCA alignment?
10
01
G
Representation Theory
ApplicationsCircle:
Initialf()
y-flipf(π-)
x-flipf(-)
x,y-flipf(π+)
10
01
10
01
10
01
10
01
Representation Theory
ApplicationsCircle:
Because axial flips are a subgroup of the rotations / reflections, sub-representations for the entire group of rotations / reflections are also sub-representations for the group of axial flips.
Representation Theory
ApplicationsCircle:
k
ikkeaf )(
Initialf()
y-flipf(π-)
x-flipf(-)
x,y-flipf(π+)
Representation Theory
ApplicationsCircle:
Initialf()
y-flipf(π-)
x-flipf(-)
x,y-flipf(π+)
k
ikk
k
ikk
k
ikk
ea
ea
eaf
)(kk aa
Representation Theory
ApplicationsCircle:
Initialf()
y-flipf(π-)
x-flipf(-)
x,y-flipf(π+)
k
ikk
kk
ikk
kk
ikikk
k
ikk
ea
ea
eea
eaf
)1(
)1(
)( )(
kk aa kk aa
Representation Theory
ApplicationsCircle:
Initialf()
y-flipf(π-)
x-flipf(-)
x,y-flipf(π+)
k
ikk
kk
ikikk
k
ikk
ea
eea
eaf
)1(
)( )(kk aa kk aa k
kk aa )1(
Representation Theory
ApplicationsCircle:
Initialf()
y-flipf(π-)
x-flipf(-)
x,y-flipf(π+)
kk aa kk aa kk
k aa )1( kk
k aa )1(
Representation Theory
ApplicationsCircle:
If f is a real-valued function on the circle expressed in terms of its Fourier decomposition as:
an axial-flip invariant representation can be obtained by storing the norms of the real and imaginary components:
k
ikkk eibaf )(
,...,,,,,)( 111100 bababaf
Representation Theory
ApplicationsCircle:
If f and h are real-valued functions on the circle, we could compute the correlation of f with h over all axial flips by comparing at each axial flip independently.
This would take four times as long.
Representation Theory
ApplicationsCircle:
Instead, if we express f and h in terms of their Fourier decomposition:
then the correlation of f with h becomes:
k
ikkk
k
ikkk eidcheibaf )()(
k
kkk
kkk
kkk
kk dbcadbcahf 121212122222)(),(
Representation Theory
ApplicationsCircle:
By computing the different summations independently:
we can compute the correlation at the different axial flips more efficiently.
k
kkk
kkk
kkk
kk dbcadbcahf 121212122222)(),(
α(real, even)
β(imaginary, even)
γ(real, odd)
δ(imaginary, odd)
Representation Theory
ApplicationsCircle:
α(real, even)
β(imaginary, even)
γ(real, odd)
δ(imaginary,odd)
Initial α β γ δ
x- flip α -β γ -δ
y- flip α -β -γ δ
x,y-flip α β -γ -δ
Initial y-flipx-flip x,y-flip
kk aa kk aa kk
k aa )1( kk
k aa )1(
Representation Theory
ApplicationsCircle:
So by pre-computing the values of α, β, γ, and δ, we can compute the correlation at each of the four axial flips by summing the values of α, β, γ, and δ with the appropriate sign.
Instead of taking 4 times as long, it takes 16 extra arithmetic ops. to compute the correlation values.
Initial x-flip y-flip x,y-flip
Correlation α+β+γ+δ α-β+γ-δ α-β-γ+δ α+β-γ-δ
Representation Theory
Applications2D (Rotation):
If we are given a function f defined on the set of points inside the unit disk (x2+y21), we can express the function in terms of radius and angle:
)sin,cos(),(~
rrfrf
rr
Representation Theory
Applications2D (Rotation):
If we hold the radius fixed, we get a function defined on a circle:
and we can apply methods from functions on a circle to obtain rotation invariants and to correlate.
rrrffr )sin,cos()(
rr
Representation Theory
Applications2D (Rotation):
To get rotation invariants, we can express the initial function f as a collection of circular functions, obtained by restricting f to different radii:
Nk
rfff krr N )(,),(
1
rr
Representation Theory
Applications2D (Rotation):
Computing the Fourier decomposition of each circular restriction:
we can obtain a rotation invariant representation by storing the norms of the different frequency components of the different circular restrictions:
ik
kkrr eaf ,)(
,...,,,...,,, 1,1,0,1,1,0, 222111 rrrrrr aaaaaaf
Representation Theory
Applications2D (Rotation):
To correlate two functions f and h, we can express the initial functions as collections of circular functions, obtained by restricting to different radii:
)(,),()(,),(11
NN rrrr hhhfff
Representation Theory
Applications2D (Rotation):
Then the correlation can be obtained by multiplying the Fourier coefficients of each of the restrictions:
Complexity:1. 2N forward Fourier Transforms: O(N2 logN)
2. Frequency multiplication: O(N2)
3. One inverse Fourier Transform: O(N logN)
2/
2/ 1,,
1
2/
2/,,),(),,(
N
Nk
ikN
lkrkr
N
l
N
Nk
ikkrkr ebaebarhrf
llll
Representation Theory
Applications2D (Axial Flips):
Given two functions f and h defined on the plane, we can express the two functions in terms of the Fourier decomposition of their radial restrictions:
with ar,k,br,k,cr,k,dr,kℝ.
k
ikkrkrr
k
ikkrkrr eidcheibaf ,,,, )()(
Representation Theory
Applications2D (Axial Flips):
By computing the different summations independently:
the correlation at all four axial flips can be computed with only 16 extra arithmetic operations.
k
N
lkrkr
k
N
lkrkr
k
N
lkrkr
k
N
lkrkr llllllll
dbcadbcahf1
12,12,1
12,12,1
2,2,1
2,2,)(),(
α(real, even)
β(imaginary, even)
γ(real, odd)
δ(imaginary, odd)
Representation Theory
ApplicationsSphere:
If we have a real-valued function f on the sphere, we can express the function in terms of its spherical harmonic decomposition:
where alm,blmℝ, al-m=(-1)malm, and bl-m=(-1)mblm.
l
l
lm
ml
ml
ml Yibaf ),(),(
Representation Theory
ApplicationsSphere:
Given the space of real-valued functions on the sphere, the spherical harmonic decomposition can be used for correlation and invariants-extraction with respect to the group of 3D rotations / reflections.
Representation Theory
ApplicationsSphere:
What if we consider the smaller group of axial flips:
that arise due to the ambiguity in PCA alignment?
100
010
001
G
Representation Theory
ApplicationsSphere:
Because axial flips are a subgroup of the rotations / reflections, sub-representations for the entire group of rotations / reflections are also sub-representations for the group of axial flips.
Representation Theory
ApplicationsSphere:
Using the facts that:
and the harmonic of frequency l are even (resp. odd) when l is even (resp. odd), we get:
ml
ml
l
l
lm
ml
ml
ml
mlml
l
l
lm
ml
ml
l
ml
mml
l
l
lm
ml
ml
ml
ml
l
l
lm
ml
ml
aaYay
aaYax
aaYaxy
aaYa
flip-z
flip-z
flip-
Initial
),(
)1(),()1(
)1(),(
),(
imml
ml eP
mlmll
Y cos)!(
)!(
4
12,
Representation Theory
ApplicationsSphere:
An axial flip invariant representation can be obtained by computing the spherical harmonic decomposition:
and separately storing the norms of the real and imaginary components of the harmonic coefficients:
l
l
lm
ml
ml
ml Yibaf ),(),(
22
22
12
12
02
02
12
12
22
22
11
11
01
01
11
11
00
00
,,,,,,,,,
,,,,,
,
)(bababababa
bababa
ba
f
Representation Theory
ApplicationsSphere:
and in a similar manner as before, we can compute the correlation at all eight axial flips with only 64 extra arithmetic operations.
Representation Theory
Applications3D (Rotation):
If we are given a function f defined on the set of points inside the unit disk (x2+y2+z21), we can express the function in terms of radius and angle:
)sinsin,cos,cossin(),,(~
rrrfrf
Representation Theory
Applications3D (Rotation):
If we hold the radius fixed, we get a function defined on a sphere:
and we can apply methods from functions on a sphere to obtain rotation invariants and to correlate.
rrrrffr )sinsin,cos,cossin(),(
Representation Theory
Applications3D (Rotation):
To get rotation invariants, we can express the initial function f as a collection of spherical functions, obtained by restricting f to different radii:
Nk
rfff krr N )(,),(
1
Representation Theory
Applications3D (Rotation):
Computing the spherical harmonic decomposition of each spherical restriction:
we can obtain a rotation invariant representation by storing the norms of the different frequency components of the different spherical restrictions.
l
l
lm
ml
mlrr Yaf ),(),( ,
Representation Theory
Applications3D (Rotation):
To correlate two functions f and h, we can express the initial functions as a collection of spherical functions, obtained by restricting to different radii: ),(,),,(),(,),,(
11
NN rrrr hhhfff
Representation Theory
Applications3D (Rotation):
Then the correlation can be obtained by multiplying the Fourier coefficients of each of the restrictions:
Complexity:1. 2N forward spherical harmonic transforms: O(N3 log2N)
2. N intra-frequency multiplication: O(N4)
3. One inverse Wigner-D transform: O(N4)
2/
2/',
', 1
',, )()(,
N
Nl
lmm
l
lmm
N
l
mlr
mlr RDbahRf
ll
Representation Theory
Applications3D (Axial Flips):
Given two functions f and h defined in 3D, we can express the two functions in terms of the spherical harmonic decomposition of their radial restrictions and obtain a method that computes the correlation at each axial flip with only 64 extra arithmetic operations.
Representation Theory
ApplicationsSummary:
Translation Rotation Axial Flip
Circle 1D Fourier Transform Real/Imaginary Even/Odd Fourier Transform
2D 2D Fourier Transform Circular Restrictions1D Fourier Transform
Circular RestrictionsReal/Imaginary Even/Odd Fourier Transform
Sphere Spherical Harmonics Wigner-D Transform
Real/Imaginary Even/Odd Spherical Harmonic Transform
3D 3D Fourier Transform Spherical RestrictionsSpherical HarmonicsWigner-D Transform
Spherical RestrictionsReal/Imaginary Even/Odd Spherical Harmonic Transform
Outline:• Review• Fourier Transforms• Applications• Invariant Descriptors
– Shape Histograms (Shells)
– Shape Distributions (D2)
– Extended Gaussian Images (EGI)?
Representation Theory
Shape Histograms (Shells)Obtain a rotation invariant by storing the amount of the shape that resides within each spherical shell.
Model Shape Histogram (Shells)
Representation Theory
Shape Histograms (Shells)This amounts to storing the constant order component of a spherical restriction.
We could get a more discriminating descriptor by storing other frequency information.
Representation Theory
D2 DistributionsObtain a translation and rotation invariant by storing the distribution of distances between pairs of points on the model.
D2 DistributionD2 Distribution3D Model3D Model
p
q
DistanceD
istr
ibut i
on
Representation Theory
D2 DistributionsWe can decompose the computation of the D2 descriptor into two steps:
– Voting on displacement
– Computing the average over spherical restrictions
Representation Theory
D2 DistributionsStep 1:
For each pair (p,q) of surface points– Vote on bin corresponding to the vector p-q
3D Model3D Model
p
q
Representation Theory
D2 DistributionsStep 2:
Compute the distance distribution by counting up the number of points in each spherical restriction.
D2 DistributionD2 DistributionDistance
Dis
trib
ut i
on
Representation Theory
D2 DistributionsRotation Invariance:
Obtained by storing the amount of information in the constant order component.
More rotation invariant information can be obtained by storing the norms of the non-constant frequencies as well.
Representation Theory
D2 DistributionsTranslation Invariance:
The value in a bin can be obtained by translating the model and counting the amount of overlap between the original model and the translated one.
Representation Theory
D2 DistributionsTranslation Invariance:
Set f(x,y,z) to be the function that is equal to 1 at points on the boundary and 0 everywhere else.
Then the function:
is equal to 1 if and only if (x,y,z) and (x-x0,y-y0,z-z0) are both on the boundary, otherwise the value is 0.
),,(),,( 000 zzyyxxfzyxf
Representation Theory
D2 DistributionsTranslation Invariance:
For a fixed offset (x0,y0,z0), the number of points which satisfy the property that both the point (x,y,z) and the point (x-x0,y-y0,z-z0) are on the boundary is equal to the number of points at which:
is equal to 1.
),,(),,( 000 zzyyxxfzyxf
Representation Theory
D2 DistributionsTranslation Invariance:
Thus the number of points (x,y,z) which satisfy the property that both (x,y,z) and (x-x0,y-y0,z-z0) are on the boundary is equal:
),,(),,,(
),,(),,(),,(
000
000000
zzyyxxfzyxf
dxdydzzzyyxxfzyxfzyx
Bin
Representation Theory
D2 DistributionsTranslation Invariance:
If we write out f in terms of its frequency decomposition:
the auto-correlation is:
mlk
mzlykximlk eazyxf
,,
)(,,),,(
mlk
mzlykximlk eazyx
,,
)(2
,,),,(Bin
Representation Theory
D2 DistributionsTranslation Invariance:
So translation invariance is obtained by representing the frequency components by their (square) norms.
mlk
mzlykximlk eazyx
,,
)(2
,,),,(Bin