Numerical Methods on the Image Processing Problemshl107.math.msstate.edu/pdfs/khu_06.pdfNumerical Methods on the Image Processing Problems Hyeona Lim Department of Mathematics and

Numerical Methods on the Image ProcessingProblems

Hyeona Lim

Department of Mathematics and StatisticsMississippi State University

December 13, 2006

Hyeona Lim Numerical Methods on the Image Processing Problems

Objective

I Develop efficient PDE (partial differential equations) basedmathematical models and their numerical algorithms for

1 Noise removal

Enhance the quality of images

2 Image segmentation

Edge (2D) or surface (3D) detection


Objective


1 Noise removal





Objective


1 Noise removal





Applications - Noise Removal

Image with 20% impulse noise (left) and denoised image (right)



Image with 10% impulse noise (left) and denoised image (right)



Color image with impulse noise (left) and denoised image (right)


Applications - Edge detection



Medical Imaging



Image Analysis in Materials Science


Outline

1 History

- PDE based Mathematical Image Processing2 Image Denoising

I Conventional approach

- Total variation (TV) minimization

I New models and their numerical procedure

- Non-convex diffusion model - Kim and Lim (’05)- Anisotropic diffusion model - Lim and Williams (’06)

3 Image Segmentation

I Conventional ApproachI Method of Background Subtraction (MBS) for Medical Image

Segmentation - Kim and Lim (’05)

4 Conclusions


Outline

1 History









4 Conclusions


Outline

1 History









4 Conclusions


Outline

1 History









4 Conclusions


Outline

1 History









4 Conclusions


History of PDE based Mathematical Image Processing

I Short history, but has strong impact

I Image denoising, deconvolution (deblurring), segmentation(edge/surface detection)

I Image denoising

- Total variation (TV) minimization (Osher (’92), Lions (’97),Chan (’98), Kim (’01))

- Weakness: Edges of images can be easily smeared out due todiffusion property of TV minimization

original image (left) and smeared image (right)





I Image denoising








I Image denoising








I Image denoising






I Color image denoising (Kim (’02), Osher (’03))- Use RGB color component

Color image with 15% impulse noise (left) and denoised image (right)


Image Denoising

Conventional approach

TV minimization model

ut − σ|∇u|γ+1∇ ·(

∇u

‖∇u‖

)= β (uo − u)

I Efficiently removes noiseI Image loses sharpness since the frequency of noise and edges

are similarI Produces a staircasing (locally constant) effect and

nonphysical dissipation


Image Denoising



ut − σ|∇u|γ+1∇ ·(

∇u

‖∇u‖

)= β (uo − u)





Image Denoising



ut − σ|∇u|γ+1∇ ·(

∇u

‖∇u‖

)= β (uo − u)





Image Denoising



ut − σ|∇u|γ+1∇ ·(

∇u

‖∇u‖

)= β (uo − u)





Image Denoising - NC Model

Non-convex diffusion model

Control of nonphysical dissipationI Consider min

uFε,p(u), where

Fε,p(u) =∫Ω|∇εu|pdx + λ

2 ‖f − u‖2. Then

−p∇ ·(

∇u

|∇εu|2−p

)− λ(f − u) = 0,

where |∇εu| = (u2x + u2

y + ε2)1/2.

Sharp (left) and blurry image (right)




Fε,p Sharp image Blurry image

F0,2 0 + 0 + 12 + 0 = 1 0 + 0.52 + 0.52 + 0 = 0.5F0,1 0 + 0 + 1 + 0 = 1 0 + 0.5 + 0.5 + 0 = 1

F0.1,1 0.1 + 0.1 +√

1.01 + 0.1 ≈ 1.31 0.1 +√

0.26 +√

0.26 + 0.1 ≈ 1.22F0,0.9 0 + 0 + 10.9 + 0 = 1 0 + 0.50.9 + 0.50.9 + 0 ≈ 1.07

I The strictly convex minimization (p > 1) makes imagesblurrier.

I The TV model itself (p = 1 and ε = 0) may not introduce“blur” but its regularization (p = 1 and ε > 0) does.

I When p < 1(non-convex), the model can make the imagesharper.





F0,2 0 + 0 + 12 + 0 = 1 0 + 0.52 + 0.52 + 0 = 0.5F0,1 0 + 0 + 1 + 0 = 1 0 + 0.5 + 0.5 + 0 = 1

F0.1,1 0.1 + 0.1 +√

1.01 + 0.1 ≈ 1.31 0.1 +√

0.26 +√

0.26 + 0.1 ≈ 1.22F0,0.9 0 + 0 + 10.9 + 0 = 1 0 + 0.50.9 + 0.50.9 + 0 ≈ 1.07








F0,2 0 + 0 + 12 + 0 = 1 0 + 0.52 + 0.52 + 0 = 0.5F0,1 0 + 0 + 1 + 0 = 1 0 + 0.5 + 0.5 + 0 = 1

F0.1,1 0.1 + 0.1 +√

1.01 + 0.1 ≈ 1.31 0.1 +√

0.26 +√

0.26 + 0.1 ≈ 1.22F0,0.9 0 + 0 + 10.9 + 0 = 1 0 + 0.50.9 + 0.50.9 + 0 ≈ 1.07








F0,2 0 + 0 + 12 + 0 = 1 0 + 0.52 + 0.52 + 0 = 0.5F0,1 0 + 0 + 1 + 0 = 1 0 + 0.5 + 0.5 + 0 = 1

F0.1,1 0.1 + 0.1 +√

1.01 + 0.1 ≈ 1.31 0.1 +√

0.26 +√

0.26 + 0.1 ≈ 1.22F0,0.9 0 + 0 + 10.9 + 0 = 1 0 + 0.50.9 + 0.50.9 + 0 ≈ 1.07







New non-convex (NC) model

ut−|∇εu|1+ω∇·(

∇u

|∇εu|1+ω

)= β (f − u) , ω ∈ (−1, 1), β ≥ 0

Numerical proceduresI Linearized θ- method.I Alternating Direction Implicit (ADI) method

Theorem (Stability)

The θ- method for the new NC model is stable and holds themaximum principle.





ut−|∇εu|1+ω∇·(

∇u

|∇εu|1+ω

)= β (f − u) , ω ∈ (−1, 1), β ≥ 0


Theorem (Stability)






ut−|∇εu|1+ω∇·(

∇u

|∇εu|1+ω

)= β (f − u) , ω ∈ (−1, 1), β ≥ 0


Theorem (Stability)



Numerical Experiments - NC Model

Image with 20% mean zero noise (left), conventional method (middle), new NC method (right)



Image with 20% mean zero noise (left), conventional method (middle), new NC method (right)



Original brain image (left) and enhanced image by new NC method (right)



Horizontal line cuts of brain image and its restored images


Image Denoising - AD Model

Anisotropic diffusion model

I Speckle noise is multiplicative and it can be modeled as(Krissian et. al. ’04, ’05):

f = u +√

un,

I Corresponding time marching equation:

∂u

∂t− u2

f + u|∇u|∇·

( ∇u

|∇u|

)= λ |∇u| (f − u).

• u2

f +u ≈ u/2 makes the diffusion faster in the lighter region(where the image values are high) and slower in the darkerregion (where the image values are low).⇒ unrealistic and ineffective in practice!





f = u +√

un,


∂u

∂t− u2

f + u|∇u|∇·

( ∇u

|∇u|

)= λ |∇u| (f − u).

• u2






f = u +√

un,


∂u

∂t− u2

f + u|∇u|∇·

( ∇u

|∇u|

)= λ |∇u| (f − u).

• u2






f = u +√

un,


∂u

∂t− u2

f + u|∇u|∇·

( ∇u

|∇u|

)= λ |∇u| (f − u).

• u2




I Considerf = u +

(√u − us

)n,

where us : smoothed version of the noised image f .

New Anisotropic Diffusion (AD) Model

∂u

∂t− C |u − us |α|∇u|∇·

( ∇u

|∇u|

)= β (f − u), C > 0, 1/2 < α < 2

• On the regions where noise is present, |u − us | is relatively big.⇒ Diffusion is big enough to reduce the noise efficiently.

• On the regions where noise is not present, |u − us | is small.⇒ Diffusion is relatively slower.



I Considerf = u +

(√u − us

)n,



∂u

∂t− C |u − us |α|∇u|∇·

( ∇u

|∇u|

)= β (f − u), C > 0, 1/2 < α < 2





I Considerf = u +

(√u − us

)n,



∂u

∂t− C |u − us |α|∇u|∇·

( ∇u

|∇u|

)= β (f − u), C > 0, 1/2 < α < 2





I Considerf = u +

(√u − us

)n,



∂u

∂t− C |u − us |α|∇u|∇·

( ∇u

|∇u|

)= β (f − u), C > 0, 1/2 < α < 2




Numerical Procedure - AD Model

1 TFR (texture-free residual) parametrization

1 Set β as a constant: β(x, 0) = β0.2 For n = 2, 3, · · ·

I Compute the absolute residual and a quantity G n−1Res :

Rn−1 = |f − un−1|,

G n−1Res = max

0, Sm(Rn−1)− Rn−1

,

where Sm is a smoothing operator and Rn−1 denotes theL2-average of Rn−1.

I Update:βn = βn−1 + γn G n−1

Res ,

where γn is a scaling factor having the property: γn → 0 asn →∞.


Numerical Experiments - AD Model

Cuba Missile Crisis: The original (left above) and restored images by using ITV (right

above), ITV-TFR (left below), and AD-TFR (right below) model


Medical Image Segmentation - MBS

Motivation

I Medical images can involve noise, diverse artifacts, andunclear edges.

I Conventional segmentation methods show difficulties whenapplied to medical imagery.

I When an appropriate background is subtracted from the givenimage, the residue can be considered as an essentially binaryimage.



Procedure of method of background subtraction



Procedure of the construction of background

1 Select a coarse mesh Ωij for the image domain Ω andchoose a coarse image Uc on Ωij. Each element Ωij in thecoarse mesh corresponds to mx ×my pixels.

2 Smooth Uc .

3 Prolongate Uc to the original mesh Ω, for Uf .

4 Smooth Uf . Assign the result for the background U.

Strategies for background construction

I In step I, choose Uc on Ωij as raij + (1− r)mij , 0 ≤ r ≤ 1,aij : arithmetic average, mij : minimum of U0 on Ωij .

I U must contain only background Information, not objectsinformation. Thus select m = mx = my such that number ofblocks in Uc corresponding to objects are smaller than thenumber of smoothing iterations in step II.



Procedure of the construction of background

1 Select a coarse mesh Ωij for the image domain Ω andchoose a coarse image Uc on Ωij. Each element Ωij in thecoarse mesh corresponds to mx ×my pixels.

2 Smooth Uc .

3 Prolongate Uc to the original mesh Ω, for Uf .

4 Smooth Uf . Assign the result for the background U.

Strategies for background construction

I In step I, choose Uc on Ωij as raij + (1− r)mij , 0 ≤ r ≤ 1,aij : arithmetic average, mij : minimum of U0 on Ωij .

I U must contain only background Information, not objectsinformation. Thus select m = mx = my such that number ofblocks in Uc corresponding to objects are smaller than thenumber of smoothing iterations in step II.


Numerical Experiments - MBS

Heart: Original (left), Conventional Method (middle), Conventional approach with MBS (right)



Hand: Original (left), Conventional Method (middle), Conventional approach with MBS (right)



Leukemia: Original (left), Conventional Method (middle), Conventional approach with MBS (right)


Conclusions

1 Regularization of |∇u| can be a significant source ofnonphysical dissipation in TV-based models.

2 Non-convex (NC) diffusion model has been introduced inorder to simultaneously suppress the noise and enhance edges.

3 Anisotropic diffusion (AD) model is more efficient on SARimage denoising than conventional models.

4 Numerical procedures of NC and AD models are stable.5 The method of background subtraction (MBS) can be

efficiently used as a pre-process of various segmentationmethods for medical image segmentation.


Conclusions







Conclusions







Conclusions







Conclusions







Conclusions







Documents

Numerical Methods on the Image Processing Problemshl107.math.msstate.edu/pdfs/khu_06.pdfNumerical Methods on the Image Processing Problems Hyeona Lim Department of Mathematics and