Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf ·...

Inferring Differential Structure from Defocused Images

Fahim Mannan

School of Computer ScienceMcGill University

fmannan@cim.mcgill.ca

April 8, 2014

Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 1 / 26

Overview

1 Introduction

2 Overview of ApproachDepth From DefocusQuadric Surface FittingVariational RelaxationSurface Re-estimation

3 Conclusion

Objective

Estimate differential structure from a pair of defocused images.

(a) Near Focused (b) Far Focused

Output

(c) Depthmap (d) 3D

along with Normal and Principal Curvatures at every pixel.

Darboux Frame

Local representation of a surface.

N: Normal

M: Principle curvature(max direction)

m: Principle curvature(min direction)

Darboux Frame

Local representation of a surface.

N: Normal

M: Principle curvature(max direction)

m: Principle curvature(min direction)

Overview of the Algorithm

1 Initial depth estimate using Depth from Defocus (DFD)

2 Quadric surface fitting at every pixel and compute initial Darbouxframe

3 Refine initial estimate using variational relaxation

4 Use the refined Darboux frame estimates to re-estimating the surface

Related Work

1 Based on Sander and Zucker’s “Inferring Surface Trace andDifferential Structure” [1].

2 Ferrie et al. “Darboux Frames, Snakes and Super-Quadrics” [2]

3 Mathur et al. Curvature Consistency using Optimal EstimationTheory [3]

4 Savadjiev et al. Surface Recovery and Geometric Flow [4]

Note1 [1] considers volumetric images.

2 [2, 3, 4] 3D point cloud acquired using laser scanner.

3 This project uses two defocused images, and

4 Depth (noisy and low resolution) estimated using DFD.

Related Work

Overview of Depth from DefocusModeling Defocus Blur

f : Focal length.

A: Aperture.

N: f-number f /A.

s: Sensor distance.

u: Object distance.

r : Blur width.

ρ: pixels/mm.

σ: Blur in pixels.

Image Plane

Defocus Blur Equation

σ = ρr = ρfs

f− 1

u− 1

s) (1)

f : Focal length.

A: Aperture.

N: f-number f /A.

s: Sensor distance.

u: Object distance.

r : Blur width.

ρ: pixels/mm.

σ: Blur in pixels.

Image Plane

σ = ρr = ρfs

f− 1

u− 1

s) (1)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

1/u m−1

focus = 1 m, f =50 mm, f/A = 22

0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

u (meters)bl

focus = 1 m, f =50 mm, f/A = 22

|σ| = ρfs

2N|1f− 1

u− 1

Overview of Depth from DefocusModeling Defocused Images

I0 : True sharp image

I : Observed image

H(|σ|) : Defocus PSF

N (0, σ2N): Gaussian noise

(a) Pillbox

(b) Gaussian

Defocused Image Formation Equation

I = I0 ∗ H(|σ|) + η, where, η ∈ N (0, σ2N) (2)

Overview of Depth from DefocusDepth Estimation

IS : Sharper image

IB : Blurrier image

HS & HB : PSFs for the sharper and blurrier images

ηS & ηB : Gaussian noise from N (0, σ2N)

Pair of Defocused Images

IS = I0 ∗ HS + ηS (3)

IB = I0 ∗ HB + ηB (4)

IS : Sharper image

IB : Blurrier image

IS = I0 ∗ HS + ηS (3)

IB = I0 ∗ HB + ηB (4)

IS : Sharper image

IB : Blurrier image

IS = I0 ∗ HS + ηS (3)

IB = I0 ∗ HB + ηB (4)

Relative Blur

HB = HS ∗ HR (5)

Relative Blur Radius

σS = min{|σ1|, |σ2|}σB = max{|σ1|, |σ2|}

σR =√σ2B − σ2S (6)

Relative Blur

HB = HS ∗ HR (5)

Relative Blur Radius

σS = min{|σ1|, |σ2|}σB = max{|σ1|, |σ2|}

σR =√σ2B − σ2S (6)

A Simplification

Recall Eqs. 3 and 4,

IS = I0 ∗ HS + ηS

IB = I0 ∗ HB + ηB

Assume,

IB ≈ IS ∗ HR + ηB (7)

Relative Blur Estimation Problem

arg minHR

‖IB − IS ∗ HR‖ (8)

A Simplification

Recall Eqs. 3 and 4,

IS = I0 ∗ HS + ηS

IB = I0 ∗ HB + ηB

Assume,

IB ≈ IS ∗ HR + ηB (7)

Relative Blur Estimation Problem

arg minHR

‖IB − IS ∗ HR‖ (8)

Overview of Depth from DefocusImplementation

1 Apply isotropic diffusion to the sharper image i.e. IS ∗ HR

2 For every pixel choose HR that minimizes ‖IB − IS ∗ HR‖3 Compute depth corresponding to HR

2 For every pixel choose HR that minimizes ‖IB − IS ∗ HR‖

3 Compute depth corresponding to HR

2 For every pixel choose HR that minimizes ‖IB − IS ∗ HR‖3 Compute depth corresponding to HR

Initial Quadric Surface Fitting

Quadric Surface Model

φ(u, v) = (u, v , h(u, v))

where, h(u, v) =1

2au2 + buv +

2cv2 + eu + fv + g

Quadric Fitting for a Patch

1 For every 7× 7 patch.

2 Fit the quadric in the perspective projection space usingLeast-Squares.

Variational RelaxationExample

Figure: Neighborhood Support

Variational Relaxation

Variational Relaxation Update Equation (Sander and Zucker [1])

D̂i (v) =1

|Ni |∑j∈Ni

Dj(v) (9)

Information Theoretic Based Update (Mathur et al. [3])

D̂i (v) =1

|Ni |∑j∈Ni

λjDj(v) (10)

where λj is a measure of confidence in node j .

Proposed Weight

λ′j = λj f (‖IB − IS ∗ HR‖). (11)

D̂i (v) =1

|Ni |∑j∈Ni

Dj(v) (9)

D̂i (v) =1

|Ni |∑j∈Ni

λjDj(v) (10)

Proposed Weight

λ′j = λj f (‖IB − IS ∗ HR‖). (11)

D̂i (v) =1

|Ni |∑j∈Ni

Dj(v) (9)

D̂i (v) =1

|Ni |∑j∈Ni

λjDj(v) (10)

Proposed Weight

λ′j = λj f (‖IB − IS ∗ HR‖). (11)

Surface Re-estimation

Energy Minimization Model

E (s) = arg mins

∑‖IB − IS ∗ HR(s)‖+ α

∑(r ,θ)

(‖∇2

(r ,θ)s‖ − κ(r ,θ))

Where, κ(r ,θ) is the estimated curvature in a direction (r , θ).

Results

Figure: 3D point cloud with normal

Results

(a) Initial (b) Iteration 10

Figure: XY-plane projection of normals. Starting with noisy depthmap.

Results

(a) Initial (b) Iteration 10

Figure: XY-plane projection of normals. Starting with MAP-MRF depthmap.

Observations/Limitations

Noise needs to be low to get meaningful estimates.

Low resolution depthmap =⇒ mostly planar.

Boundary needs to be handled carefully.

Next Steps

Use λ′ weighting in variational relaxation.

Re-estimate surface using the refined estimates.

References

[1] P. T. Sander and S. W. Zucker, “Inferring surface trace and differential structurefrom 3-d images,” IEEE PAMITC, vol. 12, no. 9, pp. 833–854, 1990.

[2] F. Ferrie, J. Lagarde, and P. Whaite, “Darboux frames, snakes, and super-quadrics:geometry from the bottom up,” PAMI, IEEE Transactions on, vol. 15, no. 8, pp.771–784, Aug 1993.

[3] S. Mathur and F. Ferrie, “Edge localization in surface reconstruction using optimalestimation theory,” in CVPR, 1997., Jun 1997, pp. 833–838.

[4] P. Savadjiev, F. Ferrie, and K. Siddiqi, “Surface recovery from 3d point data using acombined parametric and geometric flow approach,” in EMMCVPR, 2003, vol.2683, pp. 325–340.

The End

Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf ·...

Documents

Palero Food Corp. v Zucker

Zucker sucht faulen Zahn- wie süchtig macht Zucker? keiner Becher Joghurt: ca. 15 Gramm Zucker. 18.12.2013 14 So schmeckt Naschen

JAKUB ZUCKER - storage.googleapis.com

Dem Zucker verfallen - WordPress.com · 2017. 10. 20. · Dem Zucker verfallen Expertenbericht. Die Diskussion, ob Zucker süchtig macht oder Süß-stoffe zu einem erhöhten Konsum

Wasserstudio-Bodensee...Ausschüttung von Glückshormon beim Genuss von Zucker; das Bild links lässt allerdings eine eher koboldhafte Wirkung ahnen. Rohr-Zucker Rohr-Zucker 100 fach

Zucker 2014 Agrana

Tolbert e Zucker 1983

ZUCKER STÄRKE STARCH FRUCHT SUGAR

Social Media Newsrooms by Zucker

Zucker Pastisseria

A Hybrid Restoration Approach of Defocused Image Using

Cruz v Zucker

KRANK DURCH ZUCKER

Die Zucker-Rangliste

Gewürze, Zucker, Dextrose, Antioxidations- · Gewürze, Zucker, Dextrose, Antioxidations-mittel: Natriumascorbat, Konservierungs-stoffe: Natriumnitrit, Kaliumnitrat, Tannen-rauch

Straßen aus Zucker #3

1 3O6 , · 2020-05-13 · Ïk ( ) ã GUYRèUYU1 @Ô4 é ¾ @qDÁ8fÁ D¾ @/ %(* v Þ+b3 * % ÖlDÉ w ä¥j2n qX ÍX µ®Ç, Ú+ QUYMPKHLN å¥j2sÌ XJTVHV1®Ç, Ú+ ;E EmÐ æ¥j2n

Coherent defocused orientation imaging of nano-crystals in

Kohlenhydrate (Zucker) - Lessing-Gymnasium Norderstedt

Jeff Zucker