Upload
goldy
View
36
Download
0
Embed Size (px)
DESCRIPTION
Contour-Based Joint Clustering of Multiple Segmentations. Daniel Glasner * 1 Shiv N. Vitaladevuni * 2 Ronen Basri 1 * equal contribution authors. 1. 2. Objective: 1. Similar shape across frames 2. Coherent regions within frames . - PowerPoint PPT Presentation
Citation preview
Contour-Based Joint Clustering of Multiple Segmentations
Daniel Glasner *1 Shiv N. Vitaladevuni *2
Ronen Basri 1
* equal contribution authors
1 2
1
2 3
1
2 3
1
2 3
1
2 3
Joint-Clustering of Image SegmentsObjective: 1. Similar shape across frames 2. Coherent regions within frames
Segments /Super-pixels
Ourclusteringresult
Inputframe
Object contours can be matchedOversegmentation artifacts can not
• Large deformation histograms are similar
• Single object
• Inter image similarity:• Overlap• Color
Related Work This WorkCo-segmentation
[Rother et al.06, Bagon et al.08, Hochbaum et al.09, Joulin et al.10]
Co-clustering of image segments[Vitaladevuni and Basri
2010]• Small deformation
shapes are similar• Full image segmentation
# objects unknown• Inter image similarity:
• Shape• Intra Image similarity:
• Color / optical flow etc.
Different lighting conditions
Some ApplicationsVideo segmentation
EM slices[Vitaladevuni &Basri CVPR10]
( )
+ Coherency + Coherency( )
F(union) = Shape-sim( , )
Shape similarityacross frames ×Coherencywithin frame ✓
✓
Searching for a Good ClusterConvex functional of unions of segments 1. Bounding contours match - across frames2. Coherent regions - within frameExterior bounding contours match
Incorporating Shape
Input frames Shape-basedjoint clustering
IntersectionShape-basedsimilarity
Descriptor (segment) = bounding contour
A Novel Contour Descriptor
3
dimension =
# contour samples
in image 1
Image 1
0ei(θ+)
0eiα
ei
eiθ
3
3
Descriptor (union) = its bounding contour
Additive Descriptor & Contour Cancellation
ei
eiθ
0
ei(θ+)
union
ei
0+ =
dimension =
#contour samples
Comparing Shapes Across Images
B2B1
0
0eiθ
# contour elements image 2
# segments image 2
# contour elements image 1
# segments image 1
Image 2Image 1
Comparing Shapes Across Images
B2x2B1
= =
Contour descriptorin image 2
Contour descriptorin image 1
x1
Image 2Image 1
Binary indicator of segmentsin image 1
Binary indicator of segmentsin image 2
B2x2B1
= =x1
Shape-sim( , )
= ?
Correspondence
Matrix
W1,2
B1Hx1
T
?
# contour elements image1X
# contour elements image2
Comparing Shapes Across Images
Comparing Shapes Across Images
B2 x2
W1,2
x1T B1
H
Contour descriptor of shape in image 1
Contour descriptor of shape in image 2
Q1,2 =
# segments image1X
# segments image2
# contour elements image1X
# contour elements image2
1. Similar shape across frames2. Coherent regions within frames
2 wkl cos(k,l external contour
k l )=
= Shape-sim( , )
xT
x
Q1,2
Q1,2H
xTQx =
For arbitrary selection of segments
x x1
x2
Q1,1
Q2,2
( )
+ Coherency + Coherency( )
Optimization
maxX x 1 ... x c ),c
x jTQx j
j 1
c
s.t. x ij {0,1}
x ij2
j =1
c
1 i
Vitaladevuni and Basri CVPR 2010
NP-hard [Garey and Johnson 1979]
Convex Relaxation
• Efficient linear programming relaxation[Vitaladevuni & Basri 2010, Charikar et al. 2003]
• Q is hermitian objective is real
Video Dataset for Occlusion/Object Boundary Detection
A. Stein and M. Hebert. Occlusion boundaries from motion: low-level detection and mid-level reasoning. IJCV, 82(3), 2009. 2http://www.cs.cmu.edu/~stein/occlusion_data/
referenceframe
ground truth
Qualitative ResultsReference frame:
Our result:
Qualitative ResultsReference frame:
Our result:
Summary
• Joint segmentation of closely related images• Additive contour representation
– Ignores internal contours of unions of segments• Efficient convex optimization to find subsets
of segments with a similar shape• Combine inter and intra image similarity
Thank you!