Upload
buddy-powers
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
Dorin Comaniciu, Visvanathan Ramesh and Peter Meer “Kernel-Based Object Tracking”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.25, No 5, May 2003
Dorin Comaniciu and Peter Meer, “Mean shift: a robust approach toward feature space analysis”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.25, no 5, May 2002
Dorin Comaniciu and Peter Meer, “Mean shift analysis and applications", The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol.2, pp.1197-1203, September 1999
Mean-shift and its application for object tracking
Andy {[email protected]}
2Intelligent Systems
Lab.
What is mean-shift?
Non-parametricDensity Estimation
Non-parametricDensity GRADIENT Estimation
(Mean Shift)
Data
Discrete PDF Representation
PDF Analysis
A tool for: finding modes in a set of data samples, manifesting an underlying probability density function (PDF) in RN
3Intelligent Systems
Lab.
Intuitive description
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
4Intelligent Systems
Lab.
Intuitive descriptionObjective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
5Intelligent Systems
Lab.
Intuitive descriptionObjective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
6Intelligent Systems
Lab.
Intuitive descriptionObjective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
7Intelligent Systems
Lab.
Intuitive descriptionObjective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
8Intelligent Systems
Lab.
Intuitive descriptionObjective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
9Intelligent Systems
Lab.
Intuitive descriptionObjective : Find the densest region
Region ofinterest
Center ofmass
10Intelligent Systems
Lab.
Modality analysisTessellate the space with windows
Run the procedure in parallel
11Intelligent Systems
Lab.
Modality analysis
The blue data points were traversed by the windows towards the mode
12Intelligent Systems
Lab.
Modality analysis
Window tracks signify the steepest ascent directions
13Intelligent Systems
Lab.
Mean shift for data clustering
14Intelligent Systems
Lab.
Data clustering example
Simple Modal Structures
Complex Modal Structures
15Intelligent Systems
Lab.
Data clustering example
Initial windowcenters
Modes found Modes afterpruning
Final clusters
Feature space:L*u*v representation
16Intelligent Systems
Lab.
Image segmentation examples
17Intelligent Systems
Lab.
General framework: target representation
Current frame
… …
Choose a feature space
Represent the model in the
chosen feature space
Choose a reference
model in the current frame
18Intelligent Systems
Lab.
General framework: target localization
Search in the model’s
neighborhood in next frame
Start from the position of the model in the current frame
Find best candidate by maximizing a similarity func.
Repeat the same process in the next pair
of frames
Current frame
… …Model Candidate
19Intelligent Systems
Lab.
Target representation
Choose a reference
target model
Quantized Color Space
Choose a feature space
Represent the model by its PDF in the
feature space
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 . . . m
color
Pro
bab
ility
20Intelligent Systems
Lab.
PDF representation
,f y f q p y Similarity
Function:
Target Model(centered at 0)
Target Candidate(centered at y)
1..1
1m
u uu mu
q q q
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 . . . m
color
Pro
bab
ility
1..
1
1m
u uu mu
p y p y p
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3 . . . m
color
Pro
bab
ility
21Intelligent Systems
Lab.
Smoothness of the similarity function
f ySimilarity Function: ,f y f p y q
Problem:Target is
represented by color info only
Spatial info is lost
Solution:Mask the target with an isotropic kernel in the spatial domain
f(y) becomes smooth in y
f is not smooth
Gradient-based
optimizations are not robust
Large similarity variations for
adjacent locations
22Intelligent Systems
Lab.
PDF with spatial weighting
1..i i nx
Target pixel locations
( )k x A differentiable, isotropic, convex, monotonically decreasing kernel• Peripheral pixels are affected by occlusion and background interference
( )b x The color bin index (1..m) of pixel x
2
( )i
u ib x u
q C k x
Normalization factor
Pixel weight
Probability of feature u in model
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3 . . . m
color
Pro
bab
ility
Probability of feature u in candidate
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3 . . . m
colorP
rob
abili
ty
2
( )i
iu h
b x u
y xp y C k
h
Normalization factor Pixel weight
0
model
y
candidate
23Intelligent Systems
Lab.
Similarity function
1 , , mp y p y p y
1, , mq q q
Target model:
Target candidate:
Similarity function: , ?f y f p y q
1 , , mp y p y p y
1 , , mq q q
q
p yy1
1
1
cosT m
y u uu
p y qf y p y q
p y q
The Bhattacharyya Coefficient
m
uuu qpqypff
1
)(]),([)( yy
24Intelligent Systems
Lab.
Target localization algorithm
,f p y q
Start from the position of the model in the current frame
q
Search in the model’s
neighborhood in next frame
p y
Find best candidate by maximizing a similarity func.
25Intelligent Systems
Lab.
Approximating similarity function
01 1 0
1 1
2 2
m mu
u u uu u u
qf y p y q p y
p y
Linear
approx.(around y0)
0yModel location:yCandidate location:
2
12
nh i
ii
C y xw k
h
2
( )i
iu h
b x u
y xp y C k
h
Independent of y
Density estimation
(as a function of y)
1
m
u uu
f y p y q
26Intelligent Systems
Lab.
Computing gradient of similarity function
i
n
i
ii
hn
i
ii
hm
uuu w
h
xykxy
h
C
h
xykw
Cqypyf
hh
12
110 '
22ˆ)(ˆ
2
1
xkxg '
y
hxy
gw
hxy
gwx
h
xygw
h
Cw
h
xygyx
h
Ch
h
hh
n
i
ii
n
i
iiin
i
ii
hi
n
i
ii
h
1
1
12
12 22
2
1
2
1
( )
ni
ii
ni
i
gh
gh
x - xx
m x xx - x
Simple Mean Shift procedure:• Compute mean shift vector
•Translate the Kernel window by m(x)
27Intelligent Systems
Lab.
Kernels and kernel profiles
y
h
xygw
h
xygwx
h
xygw
h
Cw
h
xykxy
h
Cyf
h
h
hh
n
i
ii
n
i
iiin
i
ii
hi
n
i
ii
h
1
1
12
12 2
'2
xkxg ' k – kernel profile
2xckxK K– kernel
• Epanechnikov Kernel
• Uniform Kernel
• Normal Kernel
21 1
( ) 0 otherwise
E
cK
x xx
1( )
0 otherwiseU
cK
xx
21( ) exp
2NK c
x x
28Intelligent Systems
Lab.
Epanechnikov kernel
Epanechnikov kernel:
1 if x 1
0 otherwise
xk x
A special class of radially symmetric kernels: 2
K x ck x
11
1
n
i iin
ii
x wy
w
2
0
1
1 2
0
1
ni
i ii
ni
ii
y xx w g
hy
y xw g
h
1 if x 1
0 otherwiseg x k x
Uniform kernel:
29Intelligent Systems
Lab.
Tracking with mean-shift1. Initialize the location of the target in the current frame (y0) with location of model on previous frame, compute candidate target model p(y0) and evaluate �����������������
2. Compute weights3. Find the next location of the target candidate according to
4. Compute candidate model at new position p(y1) �����������������5. If ||y0-y1||<ε Stop. Else – set y0 = y1 and go to step 2.
m
uuu qypqypf
100 )(]),([
2
0
1
1 2
0
1
ni
i ii
ni
ii
y xx w g
hy
y xw g
h
30Intelligent Systems
Lab.
Conclusions
Advantages :
• Application independent tool
• Suitable for real data analysis
• Does not assume any prior shape (e.g. elliptical) on data clusters
• Can handle arbitrary feature spaces
• Only ONE parameter to choose
• h (window size) has a physical meaning, unlike K-Means
Disadvantages:
• The window size (bandwidth selection) is not trivial
• Inappropriate window size can cause modes to be merged, or generate additional “shallow” modes Use adaptive window size