Embed Size (px)
DESCRIPTION
Segmentation Based Multi- V iew Stereo. Michal Jan čošek, Tomáš Pajdla. Problem formulation : input. Input. Output. Problem formulation : output. Input. Output. Previous work. [Furukawa07] Y. Furukawa and J. Ponce. Accurate, Dense, and Robust Multi-View Stereopsis , CVPR 2007 - PowerPoint PPT Presentation
Citation preview
Center for Machine PerceptionDepartment of Cybernetics
Faculty of Electrical EngineeringCzech Technical University in Prague
Segmentation Based Multi-View Stereo
Michal Jančošek, Tomáš [email protected], [email protected]
/
Previous work
[Furukawa07] Y. Furukawa and J. Ponce. Accurate, Dense, and Robust Multi-View Stereopsis, CVPR 2007 prematching, growing, filtering
[Tao00] H. Tao and H.S. Sawhney. Global Matching Criterion and Color Segmentation Based Stereo, ACV 2000 using segmentation for hypothesizing the continuous parts of a
scene
[Felzenszwalb04] Pedro F. Felzenszwalb and Daniel P. Huttenlocher. Efficient graph-based image segmentation, IJCV 2004. color segmantation
[Jancosek08] Jancosek M. and Pajdla T. Effective seed generation for 3D reconstruction, CVWW 2008 optimal 3D segment orientation
34
[email protected], [email protected]
4
/
Pipeline overview : Prematching34
[email protected], [email protected]
5
InputInput Prematching(3D seeds)
Hypothesizing(3D segments)Hypothesizing(3D segments)
Final mesh constructionFinal mesh
constructionFiltering
(3D patches)Filtering
(3D patches)OutputOutput =
/
Pipeline overview : 3D segments34
[email protected], [email protected]
6
InputInput Prematching(3D seeds)
Prematching(3D seeds)
Hypothesizing(3D segments)
Final mesh constructionFinal mesh
constructionFiltering
(3D patches)Filtering
(3D patches)OutputOutput =
/
Pipeline overview : Filtering34
[email protected], [email protected]
7
InputInput Prematching(3D seeds)
Prematching(3D seeds)
Hypothesizing(3D segments)Hypothesizing(3D segments)
Final mesh constructionFinal mesh
constructionFiltering
(3D patches)OutputOutput =
/
Pipeline overview : Final mesh construction 34
[email protected], [email protected]
8
InputInput Prematching(3D seeds)
Prematching(3D seeds)
Hypothesizing(3D segments)Hypothesizing(3D segments)
Final mesh construction
Filtering(3D patches)
Filtering(3D patches)Output =
Poisson Surface Reconstruction
/
Pipeline overview : Prematching34
[email protected], [email protected]
9
InputInput Prematching(3D seeds)
Hypothesizing(3D segments)Hypothesizing(3D segments)
Final mesh constructionFinal mesh
constructionFiltering
(3D patches)Filtering
(3D patches)OutputOutput =
/
Prematching
Cameras are known (or computed [Martinec et al.]) Feature points detection (harris on a grid [Furukawa07]) Feature points are matched in each image pair (guided matching
by epigeoms). Matching is based on the NCC score of two 5x5 windows Mutually best matches are selected Tracks are constructed by grouping mutually best matches
34
[email protected], [email protected]
10
r = ²i( )
= center of gravity of the points in S
S
/
Pipeline overview : 3D segments34
[email protected], [email protected]
11
InputInput Prematching(3D seeds)
Prematching(3D seeds)
Hypothesizing(3D segments)
Final mesh constructionFinal mesh
constructionFiltering
(3D patches)Filtering
(3D patches)OutputOutput =
/
3D segments34
[email protected], [email protected]
12
For each segment with some seed projections, we create an optimal 3D segment and set it as explored
Next we do a greedy searching of unexplored segments to explore more
/
Optimal 3D segment creation34
[email protected], [email protected]
13
TP FPTP
TP – true positiveFP – false positive - explored segment - unexplored segment
/
Optimal 3D segment creation34
[email protected], [email protected]
14
MNCC( )=
Ф = 45
Ѳ = 90
Ф = 75
Ѳ = 120
,
The goal is to find the global maximum of the criteria function
)),,(),,,((,,
,180,0180,0:
kvpTkvpTMNCCkf
RKKf
mkd
d
k = 0
/
Optimal 3D segment creation
First, we estimate 3D segment orientation 3D segment orientation is estimated by gradient descent optimization
from the best sample out of 10x10 regular orientation samples [Jancosek08]
Next, we find the optimum position of 3D segment on the ray from reference camera center for the estimated orientation
34
[email protected], [email protected]
15
/
Greedy searching of unexplored segments34
[email protected], [email protected]
16
TP FPTPTPTP
TP – true positiveFP – false positive - explored segment - unexplored segment
/
Greedy searching of unexplored segments34
[email protected], [email protected]
17
TP FPTPTP TP TP
TP – true positiveFP – false positive - explored segment - unexplored segment
/
Pipeline overview : Filtering34
[email protected], [email protected]
18
InputInput Prematching(3D seeds)
Prematching(3D seeds)
Hypothesizing(3D segments)Hypothesizing(3D segments)
Final mesh constructionFinal mesh
constructionFiltering
(3D patches)OutputOutput =
/
Filtering34
[email protected], [email protected]
19
TP FPTP
TP – true positiveFP – false positive - explored segment - unexplored segment
/
Pipeline overview : Final mesh construction 34
[email protected], [email protected]
20
InputInput Prematching(3D seeds)
Prematching(3D seeds)
Hypothesizing(3D segments)Hypothesizing(3D segments)
Final mesh construction
Filtering(3D patches)
Filtering(3D patches)Output =
Poisson Surface Reconstruction
/
Conclusions
Advantages Complete models Lack of texture is explained by planes Speed Possible to implement on GPU
Disadvantages Low accuracy
Future work MRF on volume around 3D segments
34
[email protected], [email protected]
33
![Kuo-Chin Lien and Jerry D. Gibsonvivonets.ece.ucsb.edu/lien_ICME2013.pdf · [6] proposed a stereo inpainting algorithm based on stereo bi-layer segmentation. Lo et al. [5] introduced](https://img.dokumen.tips/doc/110x75/6041bed85e6d031734537bf3/kuo-chin-lien-and-jerry-d-6-proposed-a-stereo-inpainting-algorithm-based-on-stereo.jpg)
