Upload
norishige-fukushima
View
501
Download
1
Tags:
Embed Size (px)
DESCRIPTION
presentation at 3DTV-CON2014.
Citation preview
Comparison between Blur Transfer and Blur Re-
Generation in Depth Image Based Rendering
Norishige Fukushima†, Naoki Kodera†, Yutaka Ishibashi†, Masayuki Tanimoto‡
July 2-4, 2014 Budapest, Hungary 3DTV-CON 2014
†Graduate School of Engineering, Nagoya Institute of Technology,Japan
‡Nagoya Industrial Science Research Institute,Japan
Outline
Background Related Works View Synthesis Methods
Blur erasing type Blur re-generation type Blur transfer type
Experimental Results Conclusion and Future Works
Free Viewpoint
Image
Depth Image Based
Rendering
Original Image Depth Map
Background (1/3)
Background (2/3)
Input view and
depth map
3D warping
Hole filling
Free viewpoint image
Background (2/3)
Input view and
depth map
3D warping
Hole filling
Free viewpoint image
Background (3/3)
Blurred region
FB
Background (3/3)
Blurred region
FB
3D warp to left
Split
Background (3/3)
Blurred region
FB
3D warp to left
Split
Filled
Background (3/3)
Blurred region
FB
3D warp to left
Split
Filled
Boundary regions are degraded.
View synthesis method
3 types of blur treatment Blur erasing type Blur re-generation type Blur transfer type
Improved blur transfer method
Blur erasing type Blur re-generation type Blur transfer type
Improved blur transfer method (proposed method)
View synthesis method
3 types of blur treatment Blur erasing type Blur re-generation type Blur transfer type
Improved blur transfer method
Blur erasing type Blur re-generation type Blur transfer type
Improved blur transfer method (proposed method)
Blur erasing type
3D warp
Erase
Artifact
Blur erasing type
3D warp
Erase
Filled
Blurs are broken.
Artifact
View synthesis method
3 types of blur treatment Blur erasing type Blur re-generation type Blur transfer type
Improved blur transfer method
Blur erasing type Blur re-generation type Blur transfer type
Improved blur transfer method (proposed method)
Blur re-generation type
Generating blurred region using alpha matting
Splitting input image into three images→ foreground, background and alpha mask
Alpha blending three images after DIBR
Input image
Split
Foreground
Alpha mask
Background
Blur re-generation type
Fore
Alpha
Back
3D warp blendMatting
Blur re-generation
View synthesis method
3 types of blur treatment Blur erasing type Blur re-generation type Blur transfer type
Improved blur transfer method (proposed method)
Blur transfer type
Dilation
Color image
Depth map
Blur transfer type
Dilation
Color image
Depth map
Foreground depth value can cover almost fuzzy/blurred region.
Blur transfer type
Blur keep
3D warping
Blur transfer type
3D warping
interpolating
Blur keep
Canny and Gaussian Filtering
Dilation
DIBR
Blur transfer type
Improved blur transfer method (proposed method)
Adding a simple process
Improved blur transfer method
(proposed method)
Improved blur transfer method (proposed method)
Generating mask by canny filter Smoothing masked region by Gaussian
filterGenerate
mask
Smooth
Masked region
Comparison
Blur erasing type × Blurs are broken. ◎ Fastest
Blur re-generation type ○ Blurs are reconstructed. ×Slowest
Blur transfer type ○ Blurs are kept. ○ Faster
For better boundary treatment, we compare blur re-generation type with blur transfer type.
Experimental Results (1/6)
Evaluating PSNR of these methods Basic type [1] Blur re-generation type [2] Blur transfer type [3] Improved blur transfer method (proposed
method)
: TeddyH (950×750)
[1]: Y. Mori et al., Signal Process. Image Commu., vol. 24, no. 1, pp. 65-72, Jan. 2009. [2]: N. Kodera et al., IEEE VCIP 2013. [3]: X. Xu et al., IEEE ICASSP 2012.
: Bowling1 (671×555)
: Reindeer (671×555) Input depth map is ground truth
Experimental Results (2/6) PSNR
BasicRe-
generation
Transfer Proposed
average 35.62 38.85 37.93 38.54
TeddyH 32.74 35.03 34.97 35.13
Reindeer 34.37 38.14 36.75 38.15
Bowling1 34.42 39.72 35.46 35.93
. . . . .
. . . . .
. . . . .
. . . .
(dB)
#Average: using 30 data set
Experimental Results (3/6)
Basic
TeddyH: 950 x 750
Re-generation Proposed
Proposed method can soften contour artifacts, look like re-generation method.
Experimental Results (4/6)
Reindeer: 671 x 555
Re-generation Proposed
In proposed method, blurs around object boundary blend background color. Proposed method dilates depth value to
the unnecessary region.
Experimental Results (5/6)
Bowling1: 671 x 555
Re-generation Proposed Proposed method interpolates using
foreground or mixed color. Depth maps are not dilated enough.
Experimental Results (6/6)
Computational cost Proposed method ≒ Basic method (15ms) +
4ms 2 dilations for input depth map < 1ms Canny filter for depth map on the synthesized image
< 3ms Gaussian filter with a small kernel < 1ms
Blur re-generation type > 100ms Matting ≒ 55ms 3 times DIBR ≒ 45ms Blending < 2ms
Conclusions Comparing 3 types of DIBR method in blur
treatment. Proposed method improves subjective quality at object
boundaries, and objectively has 0.61dB improvement. This is the second best (0.31dB lower than the blur re-
generation). Proposed method reaches the state of the arts of blur
regeneration method, and computational cost is about x5 effective.
Proposed (19 ms) vs Blur re-generation (>100 ms)Future Work Investigating effect of proposed method using
estimated depth maps. Considering effect of coding distortions.