Upload
doanmien
View
215
Download
2
Embed Size (px)
Citation preview
Tim Althoff, Philip Ly 1
Region-Filling and Object Removal by Exemplar-Based Inpainting. Criminisi, Pérez, and Toyama. CVPR 2004 Image Completion with Structure Propagation. Sun, Yuan, Jia and Shum SIGGRAPH 2005
CS294-69 Image Manipulation and Computational Photography
Tim Althoff, Philip Ly 2
Problem
Region filling, object removal, image completion, …
Tim Althoff, Philip Ly 3
Another Example
Tim Althoff, Philip Ly 4
Key ideas (1)
Exemplar-based synthesis
+ execution speed
+ accuracy of propagated structures
Tim Althoff, Philip Ly 5
Key ideas (2)
Filling order is critical
Tim Althoff, Philip Ly 6
Algorithm
1) Computing patch priorities at fill front → see next slide
2) Find patch with max priority 3) Find most similar exemplar (source region)
→ SSD in CIE lab color space 4) Copy image data & update confidences
Tim Althoff, Philip Ly 7
Patch priorities
P(p) = C(p) * D(p) Confidence term: amount of reliable information
surrounding pixel p Data term: encourages linear structures to be
synthesized first
Tim Althoff, Philip Ly 8
Limitations
Need similar patches for synthesis Only linear structures (curves?)
Cannot handle
depth ambiguities
Tim Althoff, Philip Ly 9
Competitive work & Impact
Diffusion-based techniques [Bertalmio et al., 2000]
No blur / faster → Patch-based approach wide-spread
Tim Althoff, Philip Ly 10
Image completion with Structure Propagation
Previous approaches have difficulties with salient structures in unknown regions
→ Structure propagation along user-specified curves
Tim Althoff, Philip Ly 11
Key Ideas
Synthesis ordering − Salient structures first − Then texture propagation into other regions
→ Graph labeling problem
Tim Althoff, Philip Ly 12
Key Ideas (2)
Formulate structure propagation as global optimization problem
− Similar structure − Overlapping patches
should match
Solve effectively using
− Dynamic Programming − Belief Propagation
Tim Althoff, Philip Ly 13
Results & Competitive Work
Criminsi et al., 2003 Sun et al., 2005
Tim Althoff, Philip Ly 14
Discussion
Questions?
Tim Althoff, Philip Ly 15
Discussion
Differences? − Combination structure and textures vs.
separation of structure and texture − Linear vs. curves − Amount of user input
Tim Althoff, Philip Ly 16
Discussion
Limitations? − Depth − Areas for sampling − Guidance by user
Tim Althoff, Philip Ly 17
Discussion
Solutions? − Layers and Bayesian Matting − Patch Matching
Tim Althoff, Philip Ly 18
Discussion
Future work? − Applying Belief Propagation to other graphical
work − Video/Meshes