Upload
britney-nichols
View
226
Download
3
Tags:
Embed Size (px)
Citation preview
PDE-based Methods for Image and Shape Processing Applications
Alexander BelyaevSchool of Engineering & Physical Sciences
Heriot-Watt University, Edinburgh
Institute of Sensors, Signals & Systems
Very active research area
+ dozens of books and thousands of research papers
Joachim Weickert, Anisotropic Diffusion in Image Processing
Tony Chan & Jianhong Shen, Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods.
Guillermo Sapiro, Geometric Partial Diffrential Equations and Image Analysis.
Gilles Aubert & Pierre Kornprobst, Mathematical Problems in Image Processing.
Equations
073 23 xxx An algebraic equation
dt
tduutF
dt
tud,,
2
2
An ordinary differential equation
tyxuuy
u
x
u
t
u,,,
2
2
2
2
A partial differential equation
Usually it is not possible to solve partial differential equations (PDEs) analytically and they are solved numerically.
I. Partial Differential Equations (PDEs)
PDEs are equations involving partial derivatives of an unknown function.
For example, the so-called heat or diffusion equation is given by
yxuyxuy
u
x
u
t
u
tyxuu
,0,,
,,
0
2
2
2
2 Describes temperature distribution in a material or concentration of particles in a medium or a random walk.
Fourier transform, Gaussian smoothing, and linear diffusion
,, 22 fyxf FF
c1It explains why boosts high frequencies
yxuyxuy
u
x
u
t
u
tyxuu
,0,,
,,
0
2
2
2
2
Fourier transform w.r.t. x and y
,0,,
,,
0
22
UU
Ut
U
tUU
tUtyxu ,,,, F
A very simple ordinary differential equation. Can be easily solved analytically
I. Linear diffusion (heat/diffusion equation)
xfKtxu
xftxux
u
t
u
t
t
*,
,,
2
02
2
Proof: apply Fourier transform w.r.t. x, solve the resulting ordinary differential equation, apply inverse Fourier transform to the solution.
Thus linear diffusion is equivalent to Gaussian smoothing (convolution with Gaussian). This leads to a simple way to solve the heat equation on a plane (in space).
In practice the heat equation is usually solved numerically by using finite difference approximations or finite element methods.
I. A Brief History of PDE Methods in IP
1955
I. Kovasznay & Joseph: inverting diffusion for image sharpening purposes
I. Image enhancement/deblurring
, 1 ,I x y c I x y
2 3
12 6 !
n
c c c ce c
n
Unsharp masking (a popular image enhancement technique)
, 1 ,N
I x y c I x y
Iterated unsharp masking
I. Dennis Gabor on image enhancement
Dennis Gabor (Nobel prize in physics for inventing holography, 1971): “Information theory and electron microscopy”, 1965
Simple image sharpening
Original
111
111
111
Convolution with mask
Blurred
I. Simple image sharpening
Original
1111-81111
000010000
-
c1
Boosting high frequencies2
2
2
2
yx
Sharpened
I. Image enhancement with stabilized inverse diffusion
Can be used for deblurring Gaussian blur
tyxItyxItyxI h ,,,,pass-low,,
utu is ill posed (unstable). So a regularization is needed
A. Belyaev, ”Implicit image differentiation and filtering with applications to image sharpening.” SIAM Journal on Imaging Sciences, 6(1):660–679, 2013.
I. Stabilized inverse diffusion 2
, , implicit low-pass filtering , , , ,hI x y t dt I x y t dt I x y t
I. Stabilized inverse diffusion 3
I. Very recent use of heat (diffusion) equation
PDE: Hopf-Cole transformation
on 0,1 esapproximat 01
on 0,10
,
exp :onSubstituti
ynumericall solve easy to PDELinear :Poisson Screened
0,on 1,in 0
22
2
2
22
2
2
uuutu
uutuvvtv
x
u
t
v
x
u
t
v
x
v
x
u
t
v
x
v
txuxv
tvvtv
iiiii
eikonal equation
Hopf-Cole transformation
I. PDE: Hopf-Cole transformation
xwtxu
wwtw
xwxv
vvtv
txuxv
tuutu
1ln
on 0,in 1
1 :onSubstituti
ynumericall solve easy to PDELinear
on 1,in 0
exp :onSubstituti
1 if 1,0122
rhs = ones(N,1);
u = -sqrt(t)*log(1-(t*D+eye(N))\rhs);
Laplacian
I. PDE: Hopf-Cole transformation
20t 2t 2.0t
I. Applications: Dynamic distance-based shape features for gait recognition
T. P.Whytock, A. Belyaev, and N. M. Robertson, ”Dynamic distance-based shape features for gait recognition.” Journal of Mathematical Imaging and Vision. 2014.
I. Dynamic distance-based shape features for gait
recognition
I. Dynamic distance-based shape features for gait
recognition
II. Perona-Malik Diffusion
II. Perona-Malik diffusion with Matlab
P. Perona, T. Shiota, and J. Malik, “Anistropic Diffusuion.” Geometry-Driven Diffusion in Computer Vision, 1994.
KIIgIIgtI exp,div
II. Repeated averaging and nonlinear diffusion
Gray-scale image
Iterative local averaging:
Gaussian smoothing edge-enhancing averaging
),( yxIz
ij
ij
wkjyixIw
kyxI ),,(1
)1,,(
)|),,(|exp(),,( 2kyxIckyxwij 1),,( kyxwij
II. Repeated averaging and nonlinear diffusion
Gray-scale image
Iterative local edge-enhancing averaging:
),( yxIz
ij
ij
wkjyixIw
kyxI ),,(1
)1,,(
)),,(exp(),,( KkyxIkyxwij
Perona-Malik diffusion:
KIIg
IIgtI
exp
div Efficient numerical schemes Possibilities for various generalizations and improvements
II. Perona-Malik diffusion and its extensions
nonlinear diffusion
can be used for enhancing small-scale details
II. Nonlinear diffusion for mesh processing
2D Image Triangle mesh
),( yxI )(Tn
II. Nonlinear diffusion for surface denoising
Smoothing normals
Updating vertex positions
)(
)(
ineij
j
ineij
oldjj
newi w
w n
n
2
expT
kw j
j
0 jk
jw
Y. Ohtake, A. Belyaev, and I. A. Bogaevski, “Mesh Regularization and Adaptive Smoothing.” Computer-Aided Design, Vol. 33, No. 11, 2001, pp. 789–800.
II. Perona-Malik nonlinear diffusion for surface denoising
Nonlinear diffusion of mesh normals
Gaussian like smoothing
Adding noise
II. Perona-Malik nonlinear diffusion for surface denoising
Nonlinear diffusion of mesh normals
Conventional mesh smoothing
II. Perona-Malik nonlinear diffusion for surface denoising
II. Image compression with nonlinear diffusion
I. Galić, J. Weickert, M. Welk, M. Bruhn, A. Belyaev, H.-P. Seidel: “ “Image compression with anisotropic diffusion”. Journal of Mathematical Imaging and Vision. . 31(2-3): 255-269, 2008.
)(0,,1
11,,gdiv: 222
xfxufuxcLuxcu
ssgGuuuuuLu
t
T
III. Intro to Variational Image Processing: gradient
2
01
11
sup
,,
,,,,
h
hff
htxfdt
dhf
x
f
x
ff
xxxxxf
tn
nn
III. Intro to Variational Image Processing: max / min
function a is wheremax/min uuE
hhxxx
xx
any for 0 if at extremuman has
,,
000
1
t
d
tfdt
df
xxf
vtvuEdt
d
t
any for 00
III. Membrane energy
00
0,22
1
2
1
2
2
0
2
2
222
2
dx
udtvuE
dt
d
vdx
dudx
dx
udvdx
dx
dv
dx
du
t
uEtvuE
bvavdxdx
dvtdx
dx
dv
dx
dutdx
dx
dutvuE
dxdx
duuE
t
bx
ax
b
a
b
a
b
a
b
a
b
a
b
a
Minimizing E(u) by gradient descent:
xIxux
u
t
u
0,2
2
const, ttxuSo we have to stop this gradient descent flow at some t=T
III. Membrane Energy
Minimizing Eλ(u) by gradient descent: uI
x
u
t
u
2
2
xutxu t
,
uIx
u2
2
0
00
0,
2
1
2
2
0
2
2
2
22
Iudx
udtvuE
dt
d
uvdxIudx
udvdxvIudx
dx
dv
dx
du
t
uEtvuE
bvavtdxvIudxdx
dv
dx
dutuEtvuE
dxIudxdx
duuE
t
bx
ax
b
a
b
a
b
a
b
a
b
a
b
a
b
a
02
2
Iudx
ud
III. Variational Approach to Image Smoothing
22
22
1
,~
,~
in ,,,
min,,,
Iu
yxIyxuyxu
dxdyyxuyxuyxI
s
ssgdu
uugdivt
u
u
,min
Links to robust statistics
III. Variational Approach to Image Smoothing
yu
xuu
Resembles least-square fitting
Given image I(x,y), we approximate it by u(x,y)
min,,,22
dxdyyxuyxuyxIuE
data fitting term smoothing term
Energy (functional)
We have to learn how to differentiate E(u) w.r.t. u(x)
λ controls the amount of smoothing we add to I(x,y)
III. Edge-preserving image smoothing
min,,,2
dxdyyxuyxuyxIp
ppdxdyyxu
u
1~,
1~
preserved are edges image1
smoothed are edges image1
p
p
III. Total Variation Energy
ynumericall 0 Solving
222
,,,
2322
22
2222
uE
uu
uuuuuuuIu
u
uIuE
uuIuuuuLdxuIuuE
TV
yx
xyyxyyxyxx
yxyxTV
pppppkkkdxu
u
250250,505
50,50,50,50,50
1
ppdxu
u
250
0,0,250,0,0
edges sharp prefers 1 blur, prefers 1 pp
III. The Rudin-Osher-Fatemi (ROF) model
min,,, 2
dxdyyxuyxuyxI
02
div
Iuu
u
III. The Rudin-Osher-Fatemi (ROF) model
B.Goldlücke, Foundations of Variational Image Analysis, Lecture Notes, 2011
III. The Rudin-Osher-Fatemi (ROF) model
III. The Rudin-Osher-Fatemi (ROF) model
02
div,,
2div
2div,,,,
,,,,,,
2div,,
1
Iuu
uyxuu
Iuu
uuu
Iuu
utyxutyxu
tyxutyxutyxu
t
Iuu
utyxu
t
kk
kk
kkk
III. TV image processing models
(TV) Variation Total
22
dxdyyuxudu xx
III. Gradient descent minimization
min,
dxdyyxu
u
uuE
t
udiv uuE
t
u
min,2
dxdyyxu
Curvature flowLinear diffusion
III. The Rudin-Osher-Fatemi (ROF) model
Diffusion (heat) Total variation
Original signal
III. TV image inpainting
B.Goldlücke, Foundations of Variational Image Analysis, Lecture Notes, 2011
Original image I(x,y) Removed region R Inpainted result
min\
2
xxxx duduIR
IV. Image Deblurring
Image restoration is to restore a degraded image back to the original image
Linear image degradation model
( , ) ( , ) ( , ) ( , )g x y f x y h x y n x y blur additive noise
IV. A variational approach to image deblurring
GH
HF
FHGHF
dFdGFH
fgAf
gAffhAf
2
*
*
22
22
equation LagrangeEuler0
min
min
,
Wiener filtering
IV. Variational image deblurring
ω
ωω
ωω
ωωωωωωω
ωωω
GbaH
HF
dFbadGFH
dFbafL
fLgAfgAffhAf
2
*
22
2
2
min
min,,
IV. TV deblurring
iterationspoint fixed
0
equation LagrangeEuler
0
min,
12
1
2
22
gfhhf
f
gfhhf
f
dxffLfLgfh
n
n
n
non-blind deblurring
blind deblurring
IV. TV deblurring
V. Snakes: Active Contour Models
V. Geodesic active contours
V. Geodesic active contours
22, dydxyxgdl
Riemannian metric (conformal, for the sake of simplicity)
),(1
1),(
yxIyxg
V. Geodesic active contours
V. Geodesics in heat
Possibly this approach can be used for a very efficient implementation of geodesic active contours.
VI. B.K.P.Horn: Shape from Shading
Berthold Klaus Paul Horn, Robot Vision. The MIT Press. 1986.
VI. B.K.P.Horn: Shape from Shading
VII. Mumford-Shah Approach
VII. Blake-Zisserman = Mumford-Shah
VII. Chan-Vese active contours without contours
CoutsideCinside
dxdycyxudxdycyxuCFCF2
20
2
1021 ,,
The end.Thank you!