View
223
Download
2
Tags:
Embed Size (px)
Citation preview
Background Removal of Multiview Imagesby Learning Shape Priors
Yu-Pao Tsai, Cheng-Hung Ko, Yi-Ping Hung, and Zen-Chung Shih
ABSTRACTABSTRACT
Image-based rendering has been successfully used to display 3-D objects for many applications. A well-known example is the object movie, which is an image-based 3-D object composed of a collection of 2-D images taken from many different viewpoints of a 3-D object.
INTRODUCTIONINTRODUCTION
MVI ( Multiview Image)MVI ( Multiview Image)
Ο ( omicron )Θ ( thet ) : pan angle Φ ( phi ) : tilt angleMVI has three basic characteristics:
1) When an equi-tilt set of the MVI is captured, a large proportion of the background scene is static.2) Only one interesting object is presented in every image of the MVI.3) The foreground and background color distributions are distinct in most cases.
Proposed Flowchart Proposed Flowchart
The proposed approach aims to let every single image segmentation,rather than only those in neighboring viewing directions
To take the shape prior into account, the user is required to selectd a subset of acceptably segmented images. The 3-D shape is then generatedfrom these selected images.
AUTOMATIC INITIAL AUTOMATIC INITIAL SEGMENTATION :SEGMENTATION :
GraphGraph Cut Image Cut Image SegmentationSegmentation Graph cut image segmentation requires the user to
interactively mark some pixels as being inside the foreground objects, and others as a part of the background scene.
All the other pixels are considered to be unknown, and then they can be classified into the foreground or background by Markov random field (MRF) optimization.
AUTOMATIC INITIAL SEGMENTATION :AUTOMATIC INITIAL SEGMENTATION : Trimap Labeling Trimap Labeling
Trimap consisting of labels drawn from Trimap labeling method = β-labeling and ξ-labeling β : based on the color difference. zero-mean normalized cross correlation (ZNCC) ξ : based on the foreground model.
ξ : xi . β: beta. μ:mu.
1.If the color of a pixel varies, the pixel should be the background and labeled 2.mathematical morphology is applied to filter out the remained noises such that only one region exists
3.Each pixel whose color differs widely from the background model can be labeled .
Where is a strict threshold to ensure that only the pixels that differ widely from the background model are labeled ξ
The β and μ regions are collected and clustered by using K-means.Let denote the mean color of the th cluster for imageEach pixel p with the label μ in the image is examined and labeled ξ
AUTOMATIC INITIAL SEGMENTATION :AUTOMATIC INITIAL SEGMENTATION : Trimap Labeling Trimap Labeling
β Labeling
ξ Labeling
Background: blackForeground : white
Unknow : gray
Mathematical morphology in (a)
MVI
SEGMENTATION WITH SHAPE SEGMENTATION WITH SHAPE PRPRIORS :IORS :
Volumetric Graph CutsVolumetric Graph Cuts
In (6), the first integral tends toward a photo-consistent surface, while the second, called the ballooning term, prefers a fatter reconstructed model.
For each voxel , let be the photo-consistency score of , where a lower value represents a better photo-consistency. Let be the volume between and the base surface. The true surface is determined by finding the global minimum of the energy function among all candidate surfaces .
Discrete Medial Axis Discrete Medial Axis Constraint:Constraint:
Energy Function Energy Function Analysis:Analysis:Let be a neighborhood system defined for , which containing the
set of all pairs of neighboring voxels.
Let be a family of random variables defined on the set , in which each variable takes a label from
D(p) is the penalty according to how well the voxel fits into the given label, while B(p,q) indicates whether the surface is likely to pass through the edge between p and q .
B(p,q) can maintain the smoothness prior
Discrete Medial Axis Constraint:Discrete Medial Axis Constraint:
Imposing theImposing the DMADMA ConstraintConstraintThe medial axis is represented by a set of discrete voxels interior
to the 3-D object, called discrete medial axis (DMA).
To compute the DMA of the base surface, which is assumed to be an adequate approximation of the DMA of the true surface.
let be the set of voxels in the DMA. Let dp be the minimum distance from the voxel to its nearest voxel in
(12) guarantees that the voxels in are always labeled as being inside the surface.
Discrete Medial Axis Constraint:Discrete Medial Axis Constraint:
Segmentation RefinementSegmentation Refinement
C1.C2.C4 built hull.
C3.C5 projection
EXPERIMENTS:EXPERIMENTS:Initial Segmentation Initial Segmentation
ResultsResultsError
MVI
Trimap Labeling
(β Labeling + ξ Labeling)
Error
EXPERIMENTS:EXPERIMENTS:Learning Shape PriorLearning Shape Prior
Without DMA andballooning increased
a. Visual hull
b. DMA of ( a )
c. MVI
d. Our method
a.Visual hull base surface
b. DMA of ( a )
c.d. reconstructed model
EXPERIMENTS:EXPERIMENTS:Rectification of Segmentation Rectification of Segmentation
ErrorsErrors
Projection of the reconstructed
EXPERIMENTS:EXPERIMENTS:Rectification of Segmentation Rectification of Segmentation
ErrorsErrors
Projection of the reconstructed
Trimap Labeling = β Labeling +ξ Labeling
Automatic initial segmentation
MVI
Shape priors
EXPERIMENTS:EXPERIMENTS:Rectification of Segmentation Rectification of Segmentation
ErrorsErrors
Shape prior1 use 10 images
Shape prior2 use 20 images
Thank you for your Thank you for your
listening !listening !
2008.12.092008.12.09