View
90
Download
1
Category
Preview:
DESCRIPTION
Spectral Matting. A. Levin D. Lischinski and Y. Weiss. A Closed Form Solution to Natural Image Matting. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June 2006, New York - PowerPoint PPT Presentation
Citation preview
Spectral Matting
A. Levin D. Lischinski and Y. Weiss. A Closed Form Solution to Natural Image Matting. IEEE Conf. on Computer Vision and
Pattern Recognition (CVPR), June 2006, New York
A. Levin, A. Rav-Acha, D. Lischinski. Spectral Matting. Best paper award runner up. IEEE Conf. on Computer Vision and Pattern
Recognition (CVPR), Minneapolis, June 2007
A. Levin1,2, A. Rav-Acha1, D. Lischinski1. Spectral Matting. IEEE Trans. Pattern Analysis and Machine Intelligence, Oct 2008.
1School of CS&Eng The Hebrew University2CSAIL MIT
1
Hard
segmentation compositing
Matte compositing
Source image
Hard segmentation and matting
2
Previous approaches to segmentation and matting
Unsupervised
Input Hard output Matte output
Spectral segmentation:Spectral segmentation: Shi and Malik 97 Yu and Shi 03 Weiss 99 Ng et al 01 Zelnik and Perona 05 Tolliver and Miller 06
3
Previous approaches to segmentation and matting
Unsupervised
Input Hard output Matte output
Supervised
0
1
July and Boykov01 Rother et al 04 Li et al 04
4
Previous approaches to segmentation and matting
Unsupervised
Input Hard output Matte output
Supervised
0
1
Trimap interfaceTrimap interface: Bayesian Matting (Chuang et al 01) Poisson Matting (Sun et al 04) Random Walk (Grady et al 05)Scribbles interface:Scribbles interface: Wang&Cohen 05 Levin et al 06 Easy matting (Guan et al 06)
?
5
User guided interface
TrimapScribbles Matting result
6
Generalized compositing equation
iiiii BFI )1( 2 layers compositing
= x x+ 1 2L1L
7
Generalized compositing equation
iiiii BFI )1( 2 layers compositing
= x x+ 1 2L1L
Ki
Kiii LLLI
iii ...2211
K layers compositing
= x x+
+ x x+3 4 4L3L
1 2 2L1L
Matting components
8
Generalized compositing equation
1...21 K
iii
“Sparse” layers- 0/1 for most image pixels
Matting components:
Ki
Kiii LLLI
iii ...2211
K layers compositing
= x x+
+ x x+
10 ki
1
3 4
2 2L
4L3L
1L
9
Automatically computed matting components
Input
1 2 3 4
8765
Unsupervised matting
10
Building foreground object by simple components addition
=+ +
11
Spectral segmentation
22/
),(ji CC
ejiW
WDL
j
jiWiiD ),(),(
Spectral segmentation: Analyzing smallest eigenvectors of a graph Laplacian L
E.g.: Shi and Malik 97 Yu and Shi 03 Weiss 99 Ng et al 01 Maila and shi 01 Zelnik and Perona 05 Tolliver and Miller 0612
Problem Formulation
= x x+ 1 2L1L
Assume a and b are constant in a small window
13
Derivation of the cost function
14
Derivation
LJ T )(
15
The matting Laplacian
LJ T )(
• semidefinite sparse matrix
• local function of the image:),( jiL
L
16
The matting affinity
17
The matting affinity
Color Distribution
Input
18
Matting and spectral segmentation
Typical affinity function Matting affinity function
19
Eigenvectors of input image
Input
Smallest eigenvectors 20
Spectral segmentationFully separated classes: class indicator vectors belong to Laplacian nullspace
General case: class indicators approximated as linear combinations of smallest eigenvectors
Null
Binary indicating
vectors
Laplacian matrix
21
Spectral segmentation
Fully separated classes: class indicator vectors belong to Laplacian nullspace
General case: class indicators approximated as linear combinations of smallest eigenvectors
Smallest eigenvectors- class indicators only up to linear transformation
33
RZero eigenvectors
Binary indicating
vectors
Laplacian matrix
Smallest eigenvecto
rs
Linear transformati
on
22
From eigenvectors to matting components
linear transformat
ion
23
From eigenvectors to matting components
Sparsity of matting components
Minimize
24
From eigenvectors to matting components
Minimize
Newton’s method
with initialization
25
From eigenvectors to matting components
Smallest eigenvectors
Projection into eigs space kCTk mEE
....
K-means
..
kCmle
1) Initialization: projection of hard segments
2) Non linear optimization for sparse components26
Extracted Matting Components
27
Brief Summary
LJ T )(
Construct Matting Laplacian
Smallest eigenvectors
Linear Transformation
Matting components
28
Grouping Components
=+ +
29
Grouping Components
Unsupervised matting User-guided matting
Complete foreground matte
=+ +
30
Unsupervised matting
LJ T )(
Matting cost function
Hypothesis:Generate indicating vector b
31
Unsupervised matting results
32
User-guided matting Graph cut method
Energy function
Unary term Pairwise termConstrained components
33
Components with the scribble interface
Components (our
approach)
Levin et al cvpr06
Wang&Cohen 05
Random Walk
Poisson 34
Components with the scribble interface
Components (our
approach)
Levin et al cvpr06
Wang&Cohen 05
Random Walk
Poisson 35
Direct component picking interface
=+ +
Building foreground object by simple components addition
36
Results
37
Quantitative evaluation
38
Spectral matting versus obtaining trimaps from a hard segmentation
39
Limitations Number of eigenvectors
Ground truth matte Matte from 70 eigenvectors
Matte from 400 eigenvectors40
Limitations Number of matting components
41
Conclusion Derived analogy between hard spectral
segmentation to image matting Automatically extract matting components
from eigenvectors Automate matte extraction process and
suggest new modes of user interaction
42
Recommended