Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
SEOUL | Oct.7, 2016
Gwang-Soo Hong, Sookmyung women’s university
고속 스테레오 매칭 알고리즘 및 GPU 기반 병렬처리 기법 연구
2
CONTENTS
Stereo vision
Conventional stereo matching
Fast stereo matching
Evaluation
3
STEREO VISON
4
STEREO VISION Vision-Based Driver Assistance
http://ae-plus.com/technology/autoliv-develops-stereo-camera-system
Pedestrian detection
Vehicle detection
Lane detection
Stereo camera
5
STEREO VISION Computer stereo vision
Stereo vision is the extraction of 3D information from images. 3D information can be extracted by examination of the relative positions of objects in the two image planes.
Left image Right image
Disparity
6
STEREO VISION Relationship between disparity and depth
Triangulation
10/11/2016
𝑍: depth 𝑓: focal length 𝑂𝑅 , 𝑂𝑇: center positions of lens 𝐵: baseline 𝑃: object
𝑍: 𝐵 = 𝑍 − 𝑓: 𝑥
disparity
7
STEREO VISION Overview of a stereo vision system
10/11/2016
Stereo matching
Calibration Rectification
Triangulation
Stereo pair
Rectified stereo pair
Disparity map
Depth map
offline
online
8
CONVENTIONAL STEREO MATCHING
9
STEREO MATCHING Taxonomy for stereo algorithms [1]
1) Matching cost computation
2) Cost aggregation
3) Disparity computation / optimization (local and global algorithm)
4) Disparity refinement
10/11/2016
[1] D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms” Int. Jour.
Computer Vision, 47(1/2/3):7–42, 2002
10
STEREO MATCHING Matching cost computation
Basic Idea
Start at pixel 𝑝, consider its neighborhood defined by a square window
Compare with neighborhoods around pixels on the epipolar line for best match of pixel neighborhoods 10/11/2016
𝑑𝑚𝑎𝑥 𝑑𝑚𝑖𝑛
Left image Right image
𝑑𝑚𝑎𝑥
𝑑𝑚𝑖𝑛
Initial matching cost
𝑝
11
STEREO MATCHING Cost aggregation
Basic Idea
Color differences and a variation exist in the depth discontinuous
The variation in the disparity value is small between adjacent pixels
10/11/2016
Initial matching cost Aggregated matching cost
12
STEREO MATCHING Disparity computation and refinement
Disparity computation
Disparity Refinement
10/11/2016
Disparities
Matc
hin
g c
ost
Select minimum matching cost
Winner-take-all [1]
• Left-Right Consistency Check
• Median filtering
13
FAST STEREO MATCHING
14
FAST STEREO MATCHING Cost volume filtering approach for stereo matching
Stereo pair Initial matching cost Matching cost filtering Disparity computation
15
FAST STEREO MATCHING Matching cost computation
10/11/2016
𝑑𝑚𝑎𝑥
𝑑𝑚𝑖𝑛
Initial matching cost
𝑑𝑚𝑎𝑥 𝑑𝑚𝑖𝑛
Left image Right image
𝑝
𝑝
16
FAST STEREO MATCHING Cost aggregation
10/11/2016
𝑑𝑚𝑎𝑥
𝑑𝑚𝑖𝑛
𝑑𝑚𝑎𝑥
𝑑𝑚𝑖𝑛
Initial matching cost (C) Aggregated matching cost (C’)
Filtering
Guided image filtering
Filter input C
Guide G
Filtering output C’
𝐶′ = 𝐶𝑖 − 𝑛𝑖
𝐶′ = 𝑎𝐺𝑖 + 𝑏
Guide G
17
FAST STEREO MATCHING Cost aggregation
10/11/2016
Weighted guided image filtering
Filter input C
Guide G
Filtering output C’
𝐶′ = 𝐶𝑖 − 𝑛𝑖
𝐶′ = 𝑎𝐺𝑖 + 𝑏
𝐸 = ( 𝑎𝑘 ⋅ 𝐺 𝑝 + 𝑏𝑘 − 𝐶 𝑝2 + 𝜆 ⋅ 𝑎𝑘
2)
𝑝∈𝜔𝑘
𝐸 = ( 𝑎𝑘 ⋅ 𝐺 𝑝 + 𝑏𝑘 − 𝐶 𝑝2 +
𝜆
𝑊(𝐺, 𝑝)⋅ 𝑎𝑘2)
𝑝∈𝜔𝑘
Guided image filtering
18
FAST STEREO MATCHING Cost aggregation
10/11/2016
𝐸 = ( 𝑎𝑘 ⋅ 𝐺 𝑝 + 𝑏𝑘 − 𝐶 𝑝2 +
𝜆
𝑊(𝐺, 𝑝)⋅ 𝑎𝑘2)
𝑝∈𝜔𝑘
Linear regression
19
FAST STEREO MATCHING Cost aggregation
10/11/2016
The weights in tsukuba image
20
FAST STEREO MATCHING Cost aggregation
10/11/2016
Integral image technique
21
EXPERIMENTAL RESULTS
22
COMPARATIVE EVALUATIONS OF STEREO MATCHERS
Test data and Performance of stereo matching
KITTI: http://www.cvlibs.net/datasets/kitti/index.php
Middlebury Stereo Vision: http://vision.middlebury.edu/stereo/
10/11/2016
23
EXPERIMENTAL RESULTS Quantitative Middlebury evaluation
10/11/2016
TSUKUBA VENUS CONES TEDDY AVERAGE PERCENT OF
BAD PIXELS NON-OCC
ALL DISC NON-OCC
ALL DISC NON-OCC
ALL DISC NON-OCC
ALL DISC
Prop. Algorightm 1.37 1.79 6.55 0.21 0.48 2.40 5.98 11.2 15.5 2.29 8.51 7.69 5.33
Inforpermeable[2] 1.06 1.53 5.64 0.32 0.88 4.15 5.60 13.0 14.5 2.65 9.16 7.69 5.51
CostAggr[3] 1.38 1.85 6.90 0.71 1.19 6.13 7.88 13.3 18.6 3.97 9.79 8.26 5.54
GeoSup[4] 1.45 1.83 7.71 0.14 0.26 1.90 6.88 13.2 16.1 2.94 8.89 8.32 5.80
AdaptWeight[5] 1.37 1.79 6.55 0.21 0.48 2.40 5.98 11.2 15.5 2.29 8.51 7.69 6.66
[2] C. Ciglaa and A. A. Alatanb, “Information permeability for stereo matching," Signal Processing: Image Communication, vol. 28, no. 9, pp. 1072-1088,October, 2013.
[3] C. Rhemann, A. Hosni, M. Bleyer, C. Rother and M. Gelautz, “Fast cost-volume filtering for visual correspondence and beyond," IEEE Conference on Computer Vision and Pattern
Recognition, pp. 3017-3024, June, 2011.
[5] K. 300 J. Yoon and I. s. Kwoen, “Adaptive support-weight approach for correspondence search," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4,
pp. 650-656, April, 2006.
[4] A. Hosni, M. Bleyer, M. Gelautz and C. Rhemann, “Local stereo matching using geodesic support weights," IEEE 16th International Conference on Image Processing, pp. 2093-
2096, November, 2009.
24
EXPERIMENTAL RESULTS Qualitative Middlebury evaluation
10/11/2016
25
EXPERIMENTAL RESULTS Qualitative KITTI evaluation
10/11/2016
26
EXPERIMENTAL RESULTS Computational complexity
3.2
0.16 0.11
0
0.5
1
1.5
2
2.5
3
3.5
CPU TK1 TX1
20x
1.4x
Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375)
1.9
0.08 0.06
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
CPU TK1 TX1
1.3x 23x
Experiment environments
CPU: Intel Core i7 3.40GHz
TK1: Jetson TK1 Kepler GPU
TX1: Jetson TX1 Maxwell GPU
SEOUL | Oct.7, 2016
THANK YOU