10
8/11/2019 06423910 http://slidepdf.com/reader/full/06423910 1/10 2056 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 5, MAY 2013 Nonseparable Shearlet Transform Wang-Q Lim  Abstract—Over the past few years, various representation systems which sparsely approximate functions governed by anisotropic features, such as edges in images, have been pro- posed. Alongside the theoretical development of these systems, algorithmic realizations of the associated transforms are pro- vided. However, one of the most common shortcomings of these frameworks is the lack of providing a unified treatment of the continuum and digital world, i.e., allowing a digital theory to be a natural digitization of the continuum theory. In this paper, we introduce a new shearlet transform associated with a nonseparable shearlet generator, which improves the directional selectivity of previous shearlet transforms. Our approach is based on a discrete framework, which allows a faithful digitization of the continuum domain directional transform based on compactly supported shearlets introduced as means to sparsely encode anisotropic singularities of multivariate data. We show numerical experiments demonstrating the potential of our new shearlet transform in 2D and 3D image processing applications.  Index Terms—Discrete shearlet transform, multiresolution analysis, sparse approximation, shearlets, wavelets. I. I NTRODUCTION M ULTIVARIATE data are typically dominated by anisotropic features such as singularities on lower dimensional embedded manifold. For instance, edges in digital images are typically located along smooth curves owing to smooth boundaries of physical objects. In order to efficiently analyze and process these data, several directional repre- sentation systems have been introduced (see [2], [5]). The main idea, in all of these constructions, is that, in order to obtain efficient representations of multi-variable functions with spatially distributed discontinuities, such representations must contain basis elements parametrized by scaling, trans- lation and direction so that they provide many more shapes and directions than the classical wavelet bases. The main difficulty for digital implementation of such representations is how directional information can be identified in digital domain. Recall that, in the context of wavelet theory, faithful digitization is achieved by the concept of multiresolution analysis, where scaling and translation are digitized by dis- crete operations: downsampling, upsampling and convolution. In the case of directional transforms however, three types of operators: Scaling, translation and direction, need to be digitized. None of digital directional transforms developed Manuscript received October 26, 2012; revised December 31, 2012; accepted January 10, 2013. Date of publication January 30, 2013; date of current version March 19, 2013. This work was supported by DFG under Grant SPP-1324 and Grant KU 1446/13-1. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Chun- Shien Lu. The author is with the Institute of Mathematics, TU Berlin, Berlin 10623, Germany (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIP.2013.2244223 so far can provide fully faithful digitization of directional operator to the best of our knowledge. Shearlets have been introduced as means to sparsely encode piecewise smooth data and they are obtained by applying anisotropic scaling, shearing and translation to some fixed generating functions [7], [13]. In particular, unlike other directional representation schemes, directionality is obtained by shear matrix with integer entries and digital domain Z 2 (and Z 3 ) is invariant under this operator. In fact, this leads to faithfully digitized shearlet transform which allows a unified treatment of the continuum and digital world like the discrete wavelet transform. Such a discretization concept has been considered for the 2D shearlet transform with band-limited shearlets in [12] and with compactly supported shearlets generated by separable generating functions in [14]. This paper extends the work of [14] to establish a natural dis- cretization of the continuum shearlet transform with compactly supported shearlets. This paper is organized as follows. In Section II, we briefly review the definition of shearlets and the optimal sparse approximation properties of compactly supported shearlets. In Section III, we discuss how to discretize the shear operator so that the directional information of digital data can be faithfully encoded. Based on this, we will derive a discrete shearlet transform using compactly supported shearlets in Sections IV and V, In Section VI, we will compare our algorithmic approach with other existing schemes. Then we will show various applications of the shearlet transform in 2D and 3D image processing applications. Concluding remarks are drawn in Section VIII. II. SHEARLETS IN 2D  AND 3D In this section, we briefly overview the basic theory of shearlets in the continuum domain. Before this, let us start with a general mathematical notion of representation system, so called a frame. When designing representation systems of functions, it is sometimes beneficial to go beyond the setting of orthonormal bases and consider redundant systems. In particular, relaxing orthonormality of representation system can offer more flexibility of constructing system so that the elements of the system are well adapted for a signal  f  to provide sparse representation of  f  . Let us recall the basic definition of a frame in a Hilbert space  H.  Definition 1:  A sequence  (φ i ) i  I  in a Hilbert space  H  is called a frame for H, if there exist constants 0 <  A ≤ B < ∞ such that  A f   2 i  I |  f i | 2  B  f   2 for all  f  ∈ H. The notion of a frame guarantees stable reconstruction of a signal  f  decomposed into frame coefficients   f i  as follows. The main operator associated with a frame is the 1057-7149/$31.00 © 2013 IEEE

06423910

Embed Size (px)

Citation preview

Page 1: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 1/10

2056 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 5, MAY 2013

Nonseparable Shearlet TransformWang-Q Lim

 Abstract— Over the past few years, various representationsystems which sparsely approximate functions governed by

anisotropic features, such as edges in images, have been pro-posed. Alongside the theoretical development of these systems,algorithmic realizations of the associated transforms are pro-vided. However, one of the most common shortcomings of theseframeworks is the lack of providing a unified treatment of the continuum and digital world, i.e., allowing a digital theoryto be a natural digitization of the continuum theory. In thispaper, we introduce a new shearlet transform associated with anonseparable shearlet generator, which improves the directionalselectivity of previous shearlet transforms. Our approach is basedon a discrete framework, which allows a faithful digitization of the continuum domain directional transform based on compactlysupported shearlets introduced as means to sparsely encodeanisotropic singularities of multivariate data. We show numericalexperiments demonstrating the potential of our new shearlettransform in 2D and 3D image processing applications.

 Index Terms— Discrete shearlet transform, multiresolutionanalysis, sparse approximation, shearlets, wavelets.

I. INTRODUCTION

MULTIVARIATE data are typically dominated byanisotropic features such as singularities on lower

dimensional embedded manifold. For instance, edges in digital

images are typically located along smooth curves owing to

smooth boundaries of physical objects. In order to efficiently

analyze and process these data, several directional repre-

sentation systems have been introduced (see [2], [5]). Themain idea, in all of these constructions, is that, in order

to obtain efficient representations of multi-variable functions

with spatially distributed discontinuities, such representations

must contain basis elements parametrized by scaling, trans-

lation and direction so that they provide many more shapes

and directions than the classical wavelet bases. The main

difficulty for digital implementation of such representations

is how directional information can be identified in digital

domain. Recall that, in the context of wavelet theory, faithful

digitization is achieved by the concept of multiresolution

analysis, where scaling and translation are digitized by dis-crete operations: downsampling, upsampling and convolution.

In the case of directional transforms however, three types

of operators: Scaling, translation and direction, need to bedigitized. None of digital directional transforms developed

Manuscript received October 26, 2012; revised December 31, 2012;accepted January 10, 2013. Date of publication January 30, 2013; date of current version March 19, 2013. This work was supported by DFG underGrant SPP-1324 and Grant KU 1446/13-1. The associate editor coordinatingthe review of this manuscript and approving it for publication was Dr. Chun-Shien Lu.

The author is with the Institute of Mathematics, TU Berlin, Berlin 10623,Germany (e-mail: [email protected]).

Color versions of one or more of the figures in this paper are availableonline at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TIP.2013.2244223

so far can provide fully faithful digitization of directional

operator to the best of our knowledge. Shearlets have been

introduced as means to sparsely encode piecewise smooth dataand they are obtained by applying anisotropic scaling, shearing

and translation to some fixed generating functions [7], [13].

In particular, unlike other directional representation schemes,

directionality is obtained by shear matrix with integer entries

and digital domain Z2 (and Z

3) is invariant under this operator.

In fact, this leads to faithfully digitized shearlet transform

which allows a unified treatment of the continuum and digital

world like the discrete wavelet transform. Such a discretizationconcept has been considered for the 2D shearlet transform with

band-limited shearlets in [12] and with compactly supportedshearlets generated by separable generating functions in [14].

This paper extends the work of [14] to establish a natural dis-

cretization of the continuum shearlet transform with compactlysupported shearlets.

This paper is organized as follows. In Section II, we briefly

review the definition of shearlets and the optimal sparseapproximation properties of compactly supported shearlets. In

Section III, we discuss how to discretize the shear operator sothat the directional information of digital data can be faithfully

encoded. Based on this, we will derive a discrete shearlet

transform using compactly supported shearlets in Sections IV

and V, In Section VI, we will compare our algorithmic

approach with other existing schemes. Then we will show

various applications of the shearlet transform in 2D and 3D

image processing applications. Concluding remarks are drawn

in Section VIII.

I I . SHEARLETS IN  2 D   AN D 3 D

In this section, we briefly overview the basic theory of 

shearlets in the continuum domain. Before this, let us start

with a general mathematical notion of representation system,

so called a frame. When designing representation systemsof functions, it is sometimes beneficial to go beyond the

setting of orthonormal bases and consider redundant systems.

In particular, relaxing orthonormality of representation system

can offer more flexibility of constructing system so that the

elements of the system are well adapted for a signal   f    to

provide sparse representation of   f  . Let us recall the basicdefinition of a frame in a Hilbert space  H.

 Definition 1:   A sequence   (φi )i∈ I   in a Hilbert space  H   is

called a frame for H, if there exist constants 0 <   A ≤  B  < ∞such that

 A f  2 ≤i∈ I 

| f , φi |2 ≤  B f  2 for all   f  ∈ H.

The notion of a frame guarantees stable reconstruction of 

a signal   f    decomposed into frame coefficients   f , φi   as

follows. The main operator associated with a frame is the

1057-7149/$31.00 © 2013 IEEE

Page 2: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 2/10

LIM: NONSEPARABLE SHEARLET TRANSFORM 2057

frame operator

S  : H → H,   S f  =i∈ I 

 f , φi φi .

In general, a signal   f  ∈  H   can be recovered from its framecoefficients through the reconstruction formula

 f 

 = i∈ I  f , φi

S −1φi .   (1)

The   N -term approximation of a frame  (φi )i∈ I  is given by the

index set   I  N   associated with the   N   largest frame coefficients f , φi   in magnitude as follows.

 f  N  =i∈ I  N 

 f , φi S −1φi .

With this generalized notion of representation system, we are

now ready to define shearlet systems for   L2(R2).

 Definition 2:   For   j ≥ 0 and  k  ∈  Z, define

 A2 j

 = 2 j 0

0 2

 j/2

  and   S k 

 = 1   k 

0 1.

Also, let   ˜ A2 j  =   diag(2 j/2, 2 j ).   For   c =   (c1, c2) ∈   (R+)2,

we now define

ψ  j,k ,m (·) = 2( j+ j/2)/2ψ(S k  A2 j · −diag(c1, c2)m)

and

ψ  j,k ,m (·) = 2( j+ j/2)/2ψ(S T k   ˜ A2 j · −diag(c2, c1)m).

Then, the   (cone-adapted) regular discrete shearlet system

generated by a   scaling function   φ ∈   L2(R2)   and   shearlets

ψ, ψ  ∈  L2(R2)   is defined by

SH(c; φ,ψ, ψ) = (c1, φ) ∪ (c, ψ) ∪ (c, ψ)

where

(c1, φ) = {φm =  φ (· − c1m) : m ∈  Z2},

(c, ψ) = {ψ  j,k ,m :   j ≥  0, |k | ≤ 2 j/2, m ∈  Z2}and

(c, ψ) = { ψ  j,k ,m :   j ≥  0, |k | ≤ 2 j/2, m ∈  Z2}.

One key property of the shearlet system  SH(c; φ,ψ, ψ)   is

that directionality is achieved by the shear matrix   S k   which

not only provides directionality but also leaves the integer

grid   Z2 invariant. In fact, this invariance property leads to a

unified digitization concept for the shearlet transform betweencontinuum and digital world as we will discuss in the next

section. The following theorem shows sufficient conditions forthe shearlet generators  ψ   and  ψ  to generate a frame [10].

Theorem 1:   Let  φ, ψ  ∈  L2(R2)  be such that

φ(ξ 1, ξ 2) ≤ C 1ρ(ξ 1)ρ(ξ 2)

and

| ψ(ξ 1, ξ 2)| ≤ C 2 min{1, |ξ 1|α}ρ(ξ 1)ρ(ξ 2)   (2)

with   ρ(ξ 1)   =   min{1, |ξ 1|−γ }   for some positive constants

C 1, C 2   < ∞   and   α > γ >  3. Define  ψ( x 1, x 2) =  ψ ( x 2, x 1).

(a) (b)

Fig. 1. 2D shearlets   ψ  j,k ,m   in the spatial and frequency domain. (a) 2D

plot of  | ψ 2,0,0|. (b) 2D plot of  | ψ 2,1,0|. (c) 2D plot of   ψ 2,0,0   (d) 2D plotof   ψ 2,1,0 . Here, a nonseparable shearlet generator (18) with the Symmletwavelet eight-tap filter and 17 ×  17 maximally flat 2D fan filter are used.A MATLAB   routine dfilters from NSCT toolbox is used to compute the 2Dfan filter.

We also assume that there exists a positive constant   A   >   0such that

| φ(ξ)|2 + j,k 

| ψ(S T k   A2− j ξ )|2 + | ˆψ(S k 

 ˜ A2− j ξ )|2 >   A.   (3)

Then there exists a sampling parameter  c = (c1, c2) ∈ (R+)2

such that the shearlet system   SH(c; φ,ψ, ψ)   forms a framefor   L2(R2).

Notice that there are many compactly supported functionψ  satisfying the conditions (2). In particular, one can choose

compactly supported wavelets as shearlet generators  ψ . This

choice will guarantee stable reconstruction (1) from the shear-

let coefficients   f , ψ  j,k ,m   provided that   ψ    is sufficiently

smooth and has enough vanishing moments. Fig. 1 shows

frequency covering associated with shearlets which covers2D frequency plane except for low frequency part. The low

frequency part is covered by the scaling functions  φm   like the

wavelet system. Taking the essential supports of   ˆψ  j,k ,m  covers

a region of the horizontal cones as illustrated in Fig. 1(a). In

this case, the directional angles between −π/4 and   π/4 arecovered since the shear parameters satisfy   k  ≥ −2 j/2   and

k  ≤ 2 j/2. Similarly, a region of the vertical cones is covered

by the essential supports of   ˆψ  j,k ,m  as illustrated in Fig. 1(b).

Note that shear matrices  S k  and S T k    behave similar to rotations

for |k | ≤ 2 j/2  which gives directionality.We now extend 2D shearlets to 3D case. 3D shearlets are

scaled according to the paraboloidal scaling matrices A2 j   and

and directionality is encoded by the   shear matrices S k   (now

with  k  = (k 1, k 2)) where   A2 j   and   S k  are defined as follows.

 A2 j = 2 j 0 0

0 2 j/2   0

0 0 2 j/2   and   S k  = 1  k 1   k 2

0 1 00 0 1

.

The translation lattices will be defined through the following

matrices:   M c =  diag(c1, c2, c2). From this, we define

ψ  j,k ,m (·) =  2 j+2 j/2

2   ψ( S k  A2 j · − M cm)   (4)

for   j  ≥   0, |k 1| ≤   2 j/2, |k 2| ≤   2 j/2   and   m  ∈   Z3. We

also define  ψ  j,k ,m ( x ) =   ψ  j,k ,m ( x 2, x 1, x 3)   and  ψ  j,k ,m ( x ) =ψ  j,k ,m ( x 3, x 2, x 1) for  x  = ( x 1, x 2, x 3) ∈ R3. Then 3D shearlet

systems consisting of  ψ  j,k ,m , ψ  j,k ,m   and  ψ  j,k ,m  are defined in

a similar way to 2D case. We refer to [8] for more details. In

Page 3: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 3/10

2058 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 5, MAY 2013

this case, 3D frequency domain is partitioned into three pairs

of pyramids and restricting the range of the shear parameters

k 1  and  k 2   to [−2 j /2, 2 j/2]  makes shear matrices  S k   behavesimilar to rotations as in 2D case. For the 3D shearlet generator

ψ , we choose  ψ  ∈  L2(R3)  such that

| ψ(ξ)| ≤ C 2 · min{1, |ξ 1|δ } · ρ(ξ 1)ρ(ξ 2)ρ(ξ 3)   (5)

with   ρ(ξ 1

) =

  min{1,

|ξ 

1|−γ

}  for some constant   C 

2  >   0 and

δ >   2γ >   6. Then the 3D shearlet system forms a frame

for   L2(R3)  provided that a 3D version of covering condition(3) is satisfied. Finally, it can be shown that compactly

supported shearlet systems provide nearly optimally sparseapproximations for   C 2 piecewise smooth functions   f   which

are   C 2 smooth apart from  C 2 discontinuity curve or surface.

This property holds for both 2D and 3D — see [9] and [8]

for more details. In particular, for such 2D piecewise smooth

functions   f  ∈  L 2(R2), we have

 f  −   f  N 22 ≤ C (log( N ))3 N −2

where   f  N   is the   N −term shearlet approximation.

III. FAITHFUL  D ISCRETIZATION OF S HEAR  O PERATOR

In this section, we will derive a fast algorithm which

explicitly computes the shearlet coefficients  f , ψ  j,k ,m. In

particular, we will discuss how the shear operator   S k   can be

naturally adapted to the digital domain Z2. First, we define the

discrete time Fourier transform of  {a(n)}n∈Zd  ∈  2(Zd )  by

a(ξ) =

n∈Zd 

a(n)e−2π in ·ξ 

and let   a(n)   =   a(−n)   for   n   ∈   Zd . We will use those

notations throughout this paper. Let   ψ 1 and   φ1 ∈   L2(R)   bea wavelet and an associated scaling function satisfying two

scale equations

φ1( x 1) =

n1∈Zh(n1)

√ 2φ1(2 x 1 − n1)   (6)

andψ 1( x 1) =

n1∈Z

g(n1)√ 

2φ1(2 x 1 − n1).   (7)

Then wavelets  ψ 1 j,n1are defined by

ψ 1 j,n1( x 1) = 2 j/2ψ 1(2 j x 1 − n1).

For   j   >  0, let {h j (n1)}n1∈Z   and {g j (n1)}n1∈Z  be the Fourier

coefficients of trigonometric polynomials

h j (ξ 1) = j−1k =0

h(2k ξ 1)   and   g j (ξ 1) = g

2 j ξ 1

2

 h j−1(ξ 1)

(8)

with h0 ≡ 1. Suppose that

 f  1D( x 1) =

n1∈Z f  1D

 J    (n1)2 J /2φ1(2 J  x 1 − n1).

Then the wavelet coefficients  f  1D, ψ 1 j,n1   are computed by

the following discrete formula:

 f  1D, ψ 1 j,m1 = w  j ∗ ( f  1D

 J    ∗ 1)(2 J − j · m1)   (9)

where   w j   ≡   g J − j   and   1(n1)  = φ1(·), φ1(· −   n1). In

particular, when φ1 is an orthonormal scaling function we have

 f  1D, ψ 1 j,m1 = w  j ∗ (  f  1D

 J    )(2 J − j · m1).   (10)

The discrete wavelet transform given by (9) (or (10)) connects

the wavelet transform in the continuous domain and filter

banks in the discrete domain and such a connection allows the

wavelet transform, initially defined in the continuous domain,

to be computed with a fast algorithm based on filter banks.Such a connection was set up by the multiresolution analysis

and it makes wavelets have had a growing impact on many

fields. Our aim in this section is to derive a fast algorithm

which connects continuous and digital shearlet transforms. In

particular, this will lead to an explicit formula calculating the

shearlet coefficients  f , ψ  j,k ,m  in the discrete domain like the

wavelet transform.

For this, we will only consider shearlets  ψ  j,k ,m  for the hor-izontal cone, i.e., belonging to  (ψ ; c). Notice that the same

procedure can be applied to compute the shearlet coefficients

for the vertical cone, i.e., those belonging to  ( ψ ; c), except

for switching the order of variables. We first choose a 2Dscaling function and wavelet,  φ   and  ψ   as follows.

φ( x ) = φ1 ⊗ φ1( x )   and   ψ( x ) = ψ 1 ⊗ φ1( x )   (11)

for x  =  ( x 1, x 2) ∈ R2. Notice that this choice of  ψ  satisfies (2)

and generates a shearlet frame for   L2(R2)   provided that   φ1

and  ψ 1   are sufficiently smooth and  ψ 1   has enough vanishing

moments. We refer to [10] for more details. Now aiming

towards a fast algorithm for computing the shearlet coefficients f , ψ  j,k ,m  for   j = 0, . . . , J  − 1, we first assume that

 f ( x ) =

n∈Z2

 f  J (n)2 J φ(2 J  x 1 − n1, 2 J  x 2 − n2).   (12)

Then observe that

 f , ψ  j,k ,m =  f (S −k 2− j/2 (·)),ψ  j,0,m (·)   (13)

and, WLOG we will from now on assume that   j/2 is integer;

otherwise  j/2 would need to be taken. Our observation (13)

shows us how to compute the shearlet coefficients  f , ψ  j,k ,min the digital domain: by applying the wavelet transform

associated with the anisotropic sampling matrix   A2 j   to the

sheared version of the data   f (S 2− j/2k (·)). Therefore, the main

issue is how to digitize the shear operator   S 2− j/2k , which will

provide the faithful computation of   f , ψ  j,k ,m  in the discrete

domain and our task is to define the discrete shear operator

S d 2−

 j/2k 

 : 2(Z2)

 → 2(Z2)   from   DS 

2 j/2k 

:  L2(R2)

 →  L2(R2)

defined by   DS 2 j/2k 

( f )(·) =   f (S 2 j/2k (·)). One problem for thisis that in general, the shear matrix   S 2− j/2k   does not preserve

the regular grid   Z 2, i.e.,

S 2− j/2k (Z2) =  Z

2.

In order to resolve this problem, we refine the regular gridZ

2 along the horizontal axis   x 1  by a factor of 2 j/2. With thismodification, the new grid 2− j/2

Z×Z is now invariant under

the shear operator   S 2− j/2k , since with   Q = diag(2, 1),

2− j/2Z × Z =   Q− j/2(Z2) =  Q− j /2(S k (Z

2))

=   S 2− j/2k (2− j /2Z × Z).

Page 4: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 4/10

LIM: NONSEPARABLE SHEARLET TRANSFORM 2059

This indicates that the shear operator   S 2− j/2k   is well defined

on the refined grid 2− j/2Z × Z, which will give us a natural

discretization of  S 2− j/2k . From this simple observation, we nowcompute sampling values of   f (S k 2− j/2 ·) for sampled data   f  J  ∈2(Z2)  from   f  ∈   L2(R2)  as follows.

Step 1:   For given input data   f  J   ∈   2(Z2), apply the 1D

upsampling operator ↑ 2 j/2 along the horizontal axis

 x 1  by a factor of 2

 j/2

.Step 2:   Apply 1D convolution to the upsampled input data

 f  J  with 1D lowpass filter   h j /2  along the horizontal

axis   x 1. This gives   ˜ f  J .

Step 3:  Resample   ˜ f  J   to obtain   ˜ f  J (S k (n))   according to the

shear sampling matrix   S k .

Step 4:   Apply 1D convolution to   ˜ f  J (S k (n))   with   h  j/2   fol-

lowed by 1D downsampling by a factor of 2 j /2 alongthe horizontal axis   x 1.

Here the low pass filter  h  j /2  is defined in (8). In Step 1–2, we

basically compute   ˜ f  J   on the refined grid 2− j/2Z × Z   which

is invariant under  S k 2− j/2 . Here,   ˜ f  J (n) are interpolated sample

values from   f  J (n). Note that on this new grid 2− j /2Z

 × Z,

the shear operator   S k 2− j/2   becomes   S k   with integer entries.This allows   ˜ f  J    to be resampled by   S k   which performs the

application of   S k 2− j/2   to discrete data   f  J    in Step 3. Let  ↑2 j /2, ↓   2 j/2 and ∗1   be 1D upsampling, downsampling andconvolution operator along the horizontal axis  x 1   respectively.

We now define the discrete shear operator which performs Step

1–4 as follows. For   f  J  ∈  2(Z2), define

S d 2− j/2k 

( f  J ) :=

(( ˜ f  J )(S k ·)) ∗1  h  j/2

↓2 j/2

(14)

where˜ f  J  =  ((  f  J )↑2 j/2 ∗1  h  j /2).

To start our analysis of the relation between the shear operatorS 2− j/2k  and the associated digital shear operator  S d 

2− j/2 k , let us

consider the following simple example: Set   f c =   χ{ x : x 1=0}.

Then digitize   f c   to obtain a function   f d   defined on   Z2 by

setting   f d (n) =   f c(n) for all n ∈ Z2. For fixed shear parameter

s   ∈   R, apply the shear transform   S s   to   f c   yielding the

sheared function   f c(S s (·)). Next, digitize also this function byconsidering   f c(S s (·))|Z2 . The functions   f d   and   f c(S s (·))|Z2

are illustrated in Fig. 2 for   s  = −1/4. We now focus on

the problem that the integer lattice is not invariant under the

shear matrix   S 1/4. This prevents the sampling points   S 1/4(n),

n ∈  Z2 from lying on the integer grid, which causes aliasing of 

the digitized image   f c(S 

−1/4(

·))

|Z2  as illustrated in Fig. 3(a).

In order to avoid this aliasing effect, the grid needs to berefined by a factor of 4 along the horizontal axis followed bycomputing sample values on this refined grid.

More generally, when the shear parameter is given by

s = −2− j/2k , one can essentially avoid this directional alias-

ing effect by refining the regular grid  Z2 by a factor of 2 j /2

along the horizontal axis followed by computing interpolated

sample values on this refined grid. This ensures that the

resulting grid contains the sampling points   ((2− j/2k )n2, n2)

for any n2 ∈  Z and is preserved by the shear operator  S −2− j/2k .

This procedure precisely coincides with the application of the

digital shear operator  S d 2− j/2k 

. Before we finish this section, we

(a) (b)

Fig. 2. (a) Original image   f d (n). (b) Sheared image   f c(S −1/4(·))|Z2 .

(a) (b) (c)

Fig. 3. (a) Aliased image: DFT of   f c(S −1/4(·))|Z2 . (b) De-aliased image:

S d −1/4( f d )(n). (c) De-aliased image: DFT of   S d 

−1/4( f d )(n).

show how the discrete shear operator   S d k 2− j/2   can be used to

compute the shearlet coefficients   f , ψ  j,k ,m   in the discrete

domain. We first define 2D separable wavelet filter   W  j   as

follows. W  j (ξ ) = g J − j (ξ 1) · h J − j/2(ξ 2)   (15)

where the 1D filters  h  J − j /2  and  g J − j  are defined in (8). Then

we have the following.

Theorem 2 ( [14]):   Retain the notations and definitionsfrom this subsection, and let   φ   and   ψ   as in (11). Suppose

that   f  ∈  L2(R2)  given as in (12). Then we obtain

 f , ψ  j,k ,m = W  j ∗ ( ˜ f  J (S −k ·) ∗ −k ) ∗1 h   j2 ↓2

 j2 ( m)

(16)

where

k (n) = φ( S k (·)),φ(· − n)and

m = 2 J  A−12 j  diag(c1, c2)m.

Here, (c1, c2) ∈ R2+  needs to be chosen so that  m ∈  Z2. When

φ  is an orthonormal scaling function and   k  =  0, (16) simply

becomes the 2D discrete wavelet transformW  j ∗   f  J 

(2 J − j c1 · m 1, 2 J − j/2c2 · m2)

which is a 2D version of (10) associated with   A2

 j . Also, if we

remove  k  which behaves like the low-pass filter   h j/2, then

(16) becomes

W  j ∗

S d −k 2− j/2 ( f  J )

(2 J − j c1 · m1, 2 J − j/2c2 · m 2).   (17)

This naturally extends the discrete wavelet transform (10)

to compute the shearlet coefficients by adapting the discrete

shear operator   S d k 2 j/2 . Note that (17) computes the shearlet

coefficients given in (13) in the digital domain by the following

two basic digitization steps:

1)   ψ  j,0,m   is discretized by the wavelet filter  W  j  associated

with refinement equations (6) and (7).

Page 5: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 5/10

2060 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 5, MAY 2013

2) The shear operator  S −k 2− j/2  is discretized by   S d −k 2− j/2 .

Of course, it is possible to compute   k    so that (16) canbe explicitly implemented. However, we found that both

formulations (16) and (17) perform similarly and   k   is not

practically relevant for discrete implementation.

IV. 2D SHEARLET  T RANSFORM  W IT H  N ON-S EPARABLE

SHEARLET G ENERATOR

Although the separable shearlet generator   ψ   of the form

(11) generates a shearlet frame for   L2(R2)   and simplifies

implementation, this choice of shearlet generator is not a good

choice for directional representations. Fig. 4(a) illustrates how

shearlets   ψ  j,k ,m   generated by separable functions cover 2D

frequency domain. Because of the separability of the shearletgenerator ψ , there is significant overlap between supp( ψ  j,k ,m )

and supp( ψ  j,k +1,m ). In fact, this makes the ratio of frame

bounds,  B/ A relatively large. On the other hand, as illustratedin Fig. 4, wedge shaped support is well adapted for covering

the frequency domain by the application of the shear and scale

operators while improving directional selectivity. Motivated bythis, we now consider a nonseparable shearlet generator so that

its essential frequency support provides better frame bounds

as well as better directional selectivity. We start by introducingthe nonseparable shearlet generator  ψ non as follows.

ψ non(ξ) =  P(ξ 1/2, ξ 2) ψ(ξ)   (18)

where the trigonometric polynomial   P   is a 2D fan filter

(c.f. [5]) and  ψ   is the 2D separable shearlet generator definedin (11). Observe that   ψ non satisfies (2) and by Theorem 1,

it can be shown that the nonseparable shearlet generator

ψ non generate a shearlet frame for   L2(R2). The proof of this

immediately follows from the proof of Theorem 4.9 in [10]

with a fan filter   P   satisfying inf ξ ∈|P(ξ)| ≥ C 1   where

 =

ξ  ∈ [−1/2, 1/2]2 : 1/4 ≤ |ξ 1| ≤ 1/2, |ξ 2||ξ 1|

 ≤  1

for some   C 1   >   0. This in turn defines shearlets   ψ non j,k ,m

generated by  ψ non by setting

ψ non j,k ,m ( x ) = 2

34  j ψ non(S k  A2 j x  − M c j

m)

where   M c j  is a sampling matrix given by   M c j =  diag(c

 j1 , c

 j2 )

and  c j1   and   c

 j2   are sampling constants for translation.

Next, for a function   f  given in (12), we now derive a digital

formulation for computing the shearlet coefficients  f , ψ non j,k ,m

for   j =   0, . . . , J  −  1. For this, we only discuss the case of 

the shearlet coefficients associated with   A2 j   and  S k ; the same

procedure can be applied to compute the shearlet coefficients

associated with   ˜ A2 j   and S k  except for switching the order of variables   x 1   and   x 2. First, it should be pointed out that one

major drawback of the discrete algorithm (17) is that filtering

needs to be applied to the upsampled data   ˜ f  J , which requires

additional computational cost by a factor of 2  j/2 for each shear

parameter k  and scale   j . To avoid this additional cost, we now

consider  f , ψ non

 j,k ,m

 =  f (·), ψ non j,0,m (S k /2 j/2 ·)

(a) (b)

Fig. 4. Frequency covering by shearlets ψ  j,0,m  and  ψ  j,1,m  (a) with separablegenerator and (b) with non-separable generator.

Transform   A B B/A   Size of filter for j j = 0   j = 1   j = 2   j = 3

DNST1 0.24 1.99 8.19 122× 86 66× 86 38× 178 24× 178DNS T2 0.23 2 .08 9.03 22× 28 14× 28 10× 56 8× 56D SST 0.48 7.74 16.14 106× 22 50× 22 22× 50 8× 50

Fig. 5. Numerical estimates for fame bounds   A   (upper) and   B   (lower) forthe shearlet transforms with 8 8 16 16 directions. The DNST1 and DNST2are the nonseparable shearlet transforms. For the DNST1, we use the 17 ×17maximally flat 2D fan filter   P   and Symmlet wavelet eight-tap filter for   W  j .For the DNST2, we use a 2D fan filter   P  generated by the 9 − 7 CDF filterand Haar wavelet filter for   W  j . For the DSST, we use the Symmlet waveleteight-tap filter for   W  j . We use a MATLAB   routine   dfilters   from an NSCTtoolbox to obtain 2D fan filters we used here.

in stead of (13) so that the shear operator   S k /2 j/2   is directly

applied to shearlets  ψ  j,0,m . We note that using (15) and (18),

we have

 f , ψ non j,0,m = (  f  J  ∗ ( p j ∗ W  j )(2 J − j c

 j1 m1, 2 J − j /2c

 j2 m2)

where  p j (n) are the Fourier coefficients of a 2D fan filter given

by   P(2 J − j−1ξ 1, 2 J − j/2ξ 2). This defines the discretization of 

ψ non j,0,m   by   p j ∗ W  j . Using the discrete shear operator   S d 

k 2− j/2 ,

we now define digital shearlet filters which discretize   ψ non j,k ,m

originally defined in the continuum domain as follows.

ψ d j,k (n) = S d k 2− j/2

 p j ∗ W  j

(n).   (19)

The discrete nonseparable shearlet transform (DNST) associ-

ated with the non-separable shearlet generators   ψ non is then

given by

 D N ST  j,k ( f  J )(n) = (  f  J ∗ψ d j,k )(2 J − j c

 j

1 n1, 2 J − j /2c j

2 n2)   (20)

for   f  J    ∈   2(Z2).   Also, if we replace   p j ∗   W  j   by the

separable filter  W  j  in (19), this will give digital shearlet filters

associated with shearlets   ψ  j,k ,m   generated by the separable

shearlet generator ψ . Then we call the discrete transform (20)

associated with the separable filters  W  j   the discrete separableshearlet transform (DSST). In both cases, the shearlet filters

ψ d j,k   can be pre-computed and (20) can be then efficiently

computed based on the 2D FFT without the additional compu-tational cost especially to compute the discrete shear operator

S d k 2− j/2 . In Fig. 5, we compare upper and lower frame bounds

for the DNST and DSST, for which we simply compute

maxξ  d(ξ)/ minξ  d(ξ )  where  d is defined in (21). Fig. 5

shows that better frame bounds are obtained for the DNST

even when it uses shearlet filters comparable with filters for

the DSST with respect to the size of support. In addition to

this, one major advantage of shearlets   ψ non j,k ,m   generated by

Page 6: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 6/10

LIM: NONSEPARABLE SHEARLET TRANSFORM 2061

nonseparable shearlet generator is the fact that the fan filter

P   improves directional selectivity in the frequency domain at

each scale.By setting c

 j1 =  2 j− J  and  c

 j2 =  2 j/2− J , the DNST becomes

a 2D convolution with shearlet filters and this gives a shift

invariant linear transform. As indicated before, the digital

shearlet filters  ψ d j,k   are also derived by switching the order

of variables. Finally, we define separable low-pass filter by

φd(ξ ) = h J (ξ 1) · h J (ξ 2)

and let

d(ξ ) = | φd(ξ )|2 + J −1 j=0

|k |≤2 j/2

| ψ d j,k (ξ)|2 + | ˆψ d j,k (ξ)|2

.

(21)

If   c j1 =   2 j− J  and   c

 j2 =   2 j/2− J  in (20), dual shearlet filters

can be easily computed by,

ˆϕd(ξ)

 =

φd(ξ)

d

(ξ)

,

  ˆγ d

 j,k (ξ )

 =

ψ d j,k (ξ)

d

(ξ )

,   ˆ˜

γ d j,k (ξ)

 =

ˆψ d j,k (ξ)

d

(ξ)

.

(22)

Then we obtain the reconstruction formula

 f  J  =  (  f  J  ∗ φd

) ∗ ϕd + J −1 j=0

|k |≤2 j/2

( f  J  ∗ ψ d  j,k ) ∗ γ d j,k 

+ J −1 j=0

|k |≤2 j/2

( f  J  ∗ ψ d j,k ) ∗ γ d j,k .   (23)

This reconstruction formula is well defined by the frame

property of shearlets generated by the nonseparable shearletψ non (see [10]). In particular,  ψ non can be chosen so that

C 1 ≤ d(ξ) ≤ C 2   for all  ξ  ∈  R2

with some positive constants 0  <  C 1 ≤ C 2  < ∞.

V. 3D SHEARLET T RANSFORM

In this section, we derive a discrete algorithm which com-

putes the shearlet coefficients   f , ψ  j,k ,m , where   ψ  j,k ,m   are

3D shearlets defined as in (4) and assume that

 f ( x ) =

n∈Z3

 f  J (n)2 J ·3/2φ1 ⊗ φ1 ⊗ φ1(2 J  x  − n).

Note that we will retain the notations from the previoussections except that   ψ  j,k ,m ,   ψ   and   ψ d j,k   are now shearlets,

shearlet generator and discrete shearlet filters in 3D. Following

the idea of 2D case, we choose the 3D shearlet generator  ψ 

as follows:

ψ(ξ) =

P

ξ 1

2 , ξ 2

 ψ 1(ξ 1)φ1(ξ 2)

·

P

ξ 1

2 , ξ 3

 φ1(ξ 3)

.

Fig. 6 shows the essential support of  ψ . It should be noticed

that this choice of  ψ  satisfies (5) and generates a shearlet frame

for   L2(R3)   provided that   φ1   and   ψ 1   are sufficiently smooth

and 1D wavelet  ψ 1   has enough vanishing moments. Let   Q =

ξ 1

ξ 2

ξ 1

ξ 3   ξ 3

ξ 1ξ 2

(a) (b) (c)

Fig. 6. 3D shearlet generator ψ  in the frequency domain. (a) Essential supportof   P(ξ 1/2, ξ 2) ψ 1(ξ 1)φ1(ξ 2). (b) Essential support of   P(ξ 1/2, ξ 3)φ1(ξ 3).(c) Essential support of  ψ .

diag(2, 1) and  λ = ( j, k , m), then the Fourier transform of 3D

shearlets ψ  j,k ,m  can be written as

ψ λ(ξ) =   1

2 jψ 1

ξ 1

2 j

  j,k 1 (ξ 1, ξ 2)  j,k 2 (ξ 1, ξ 3) E λ(ξ)   (24)

where

 j,k 1 (ξ 1, ξ 2) =   (φ1 · P )

Q−1

S T −k 1

 A−12 j   (ξ 1, ξ 2)T 

,

 j,k 2 (ξ 1, ξ 3) =   (φ1 · P )Q−1S 

−k 2 A−1

2 j   (ξ 1, ξ 3)T ,

 E λ(ξ ) =  exp(−2π i · γλ(ξ))

and

γλ(ξ ) =m1

2 j − k 1

m2

2 j − k 2

m3

2 j

ξ 1 + m2

ξ 2

2 j/2 + m3

ξ 3

2 j/2.

Note that    j,k 1   and    j,k 2   are functions of the form of 2Dshearlets. Therefore, we discretize them in the same way as

(19) (without convolution with high-pass filter   g J − j ). This

gives

d j,k 1

(n1, n2) =

S d k 1 2− j/2 (h J − j /2 ∗1   p j )

(n1, n2)   (25)

and

d j,k 2

(n1, n3) = S d 

k 2 2− j/2 (h J − j/2 ∗1   p j )

(n1, n3).

Here, ∗1   is 1D convolution along the   x 2   axis for the first

equation and the   x 3   axis for the second one. Finally, by

multiresolution analysis, 1D wavelet 2 j/2ψ 1(2 j ·) is discretized

by 1D filter   g J − j . Define the discrete time Fourier transform

F d  :  2(Zd ) → L2([0, 1]d )  by

F d (a)(ξ) =

n∈Zd 

a(n)e−2π iξ ·n for  a ∈ 2(Zd ).

Then, we obtain 3D digital shearlet filters   ψ d j,k   which dis-

cretize ψ  j,k ,m   as follows.

ψ d j,k (ξ) = g J − j (ξ 1)F 2(d j,k 1

)(ξ 1, ξ 2)F 2(d j,k 2

)(ξ 1, ξ 3)   (26)

where  ψ d j,k  =   F 3(ψ d j,k )   (see Fig. 7). The same procedure

is applied to derive 3D shearlet filters for  ψ  j,k ,m   and  ψ  j,k ,m

except for switching the order of variables. Now we have the

3D discrete shearlet transform associated with the nonsepara-

ble shearlet generator  ψ :

DNST3d j,k ( f  J )(n) = (  f  J  ∗ ψ 

d j,k )(n)   (27)

for   f  J  ∈  2(Z3).  Here,

n = (2 J − j c j

1 n1, 2 J − j /2c j

2 n2, 2 J − j/2c j

3 n3).

Page 7: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 7/10

2062 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 5, MAY 2013

(a) (b)

Fig. 7. (a) and (b) DFT of 3-D digital shealrets  ψ d  j,k   with different shearparameters  k  = (k 1, k 2).

By setting  c j1 =  2 j− J ,  c

 j2 =  2  j/2− J  and  c

 j3 =  2 j/2− J , we have

a shift invariant shearlet transform as in 2D case. In this case,

dual shearlet filters can be explicitly computed in the same way

as (22). From this, a reconstruction formula is easily obtained

as in (23).

VI . RELATED  WOR K

There has been extensive study in developing discrete direc-

tional transforms aiming at sparse representations of piecewisesmooth images. In particular, the various constructions of 

directional atoms compactly supported in the spatial domainhave been introduced. We now give a short overview of some

of previous works for this.

1)   Contourlet transform: Do and Vetterli first intro-

duced compactly supported directional atoms con-

structed based on directional filter banks in [5]. After

this, several variants have been proposed to producedirectional frequency partitioning similar to those of 

curvelets. As the main advantage of this approach,it allows a tree-structured filter bank implementation,

in which aliasing due to subsampling is allowed toexist. Consequently, one can achieve great efficiency

in terms of redundancy and good spatial localization.

However, directional selectivity in this approach is arti-

ficially imposed by the special sampling rule of filterbanks which often causes artifacts. In particular, there is

no theoretical guarantee for sparse approximations forpiecewise smooth images based on this approach unless

band-limited directional atoms are used.

2)   Separable shearlet transform: The discrete shearlettransform associated with compactly supported shearlets

generated by separable shearlet generator has been pro-

posed in [14]. In this approach, the shear operator  S k 2− j/2

is discretized in a very crude way — it is simply givenby replacing   k 2− j/2n1   by k 2− j/2n1   for   n1 ∈   Z. As

consequence of this crude discretization, digital shearlet

filters are severely aliased in this case as illustrated

in Fig. 8.

In this paper, we have introduced a new algorithmic

approach for directly discretizing shearlets originally defined

in the continuous domain. In particular, our discretization

strategy based on the discrete shear operator   S d k 2− j/2   provides

digital shearlet filters with essentially no aliasing as illustrated

in Fig. 8. Also, our numerical examples in the next sectionindicate our digital shearlet filters can capture edges more

(a) (b)

Fig. 8. Magnitude of the DFT of shearlet filters generated by a separablewavelet filter. (a) Digital shearlet filter proposed in [14]. (b) Digital shearletfilter with the discrete shear operator   S d

k 2− j/2 . Here, we use the Symmlet

wavelet eight-tap filter for both cases.

(a) (b)

(c) (d)

(e) (f)

Fig. 9. Image denoising results with various transforms. (a) Noisy Boat image(σ =  30). (b) FDCT (27.72 dB). (c) NSST (28.65 dB). (d) NSCT (28.61 dB).(e) DNST1 (29.01 dB). (f) SWT (27.27 dB).

effectively compared with directional transforms based on

directional filter banks due to superior directional selectivity.

VII. APPLICATIONS

In this section, we will provide the numerical test results

of our proposed directional transform DNST in variousapplications including image denoising and inpainting. For all

numerical examples, we test our proposed shearlet transformgiven in (20) for 2D examples and (27) for 3D examples. For

our tests, we use the Symmlet wavelet 8-tap filter for  W  j   and

17 × 17 maximally flat fan filter for   P  obtained from NSCTtoolbox to compute digital shearlet filters for the DNST. For

2D case, we test two types of 4 level shearlet decompositions

which are denoted by DNST1 and DNST2. For the DNST1,

we apply the DNST with 8, 8, 16, 16 directions across scales.

Page 8: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 8/10

LIM: NONSEPARABLE SHEARLET TRANSFORM 2063

Transform NSCT SWT FDCT NSST DNST1 DNST2 DSTRunning time 266.24 sec 0.57 sec 0.48 sec 5.98 sec 1.48 sec 0.78 sec 1.57 secRedundancy 53 13 5.64 49 49 25 20

Fig. 10. Comparison of running times for denoising schemes with various transforms for a 512 ×   512 image. Running times for computing shearlet filtersfor the DNST1 and DNST2 are 1.25 and 0.50 s, respectively.

(a) (b)

(c) (d)

(e) (f)

Fig. 11. Image inpainting results with various transform. (a) Original image.(b) 80% missing pixels. (c) FDCT (28.75 dB). (d) DST (28.82 dB). (e) DSST(27.61 dB). (f) DNST1 (29.37 dB).

Surf Ba nd -S he ar   DNST3d1   DNST3d

2Redundancy 6.4 208 42 154

Ru nn in g t ime 30 .9 0sec 2 63 se c 4 5. 12sec 16 5. 97 se c

Fig. 12. Redundancy and running time for each of 3D transforms.

On the other hand, for the DNST2 we apply the DNST with4, 4, 8, 8 directions across scales. For 3D case, we test two

types of 3 level shearlet decompositions which are denoted

by DNST3d1   and DNST3d

2   . For the DNST3d1   , we apply the

3D shearlet transform with 3, 19, 19 directions for scales j =  0, 1, 2, which gives redundancy factor of 42. On the other

hand, for the DNST3d2   , we apply the 3D shearlet transform

with 19, 67, 67 directions, which gives redundancy factor of 

154. In this case, we pre-compute only 2D shearlet filters.Then all 3D shearlet filters are easily obtained as illustrated

in Section V. All numerical tests are performed on an Intel

CPU 2.70 GHz. For 2D examples, 512 × 512 images are usedwhile 192 × 192 ×  192 images are used for 3D examples.

We also refer to [11] for various numerical examples to show

the effectiveness of our proposed shearlet transform in image

separation.

Mobile

Noise (σ) 10 20 30 40 50Surf 33.05 30.25 28.56 27.34   26.40

Band-Shear NA NA 28.68 27.15 25.97

DNST 3d1   34.87 30.93 28.60 26.96 25.69DNST 3d2   35.42 31.55 29.30 27.66   26.38

CoastguardNoise (σ) 10 20 30 40 50

Surf 31.02 28.42 27.05 26.08 25.33Band-Shear NA NA 27.36 26.10 25.12

DNST 3d1

  32.59 29.04 27.16 25.85 24.85DNST 3d

2  33.58 29.92 27.93 26.55 25.45

Fig. 13. Video denoising results.

Fig. 14. Video denoising of mobile video sequence. The figure compares thedenoising performance of the denoising algorithm based on the 3D shearlettransform, DNST3d

2   , on a representative frame of the video sequence Mobileagainst video denoising routine based on the surfacelet transform denoted asSurf. Top: original frame (left) and noisy frame (18.62 dB) (right). Bottom:denoised frame using Surf (28.56 dB) (left) and DNST3d

2   (29.30 dB) (right).

 A. 2D Image Denoising

First, we show comparison results in image denoising with

additive Gaussian noise. For this, we compare the DNST(and the discrete separable shearlet transform DSST) with

other existing directional transforms, the fast discrete curvelettransform FDCT1 [1], nonsubsampled contourlet transform

NSCT2 [4] and stationary wavelet transform SWT fromMATLAB   wavelet toolbox. We also compare our proposed

transform with other shearlet transforms, the nonsubsampled

shearlet transform NSST3 [3] and discrete shearlet transform

1CurveLab (Version 2.1.2) is available from http://www.curvelet.org.2NSCT is available from http://www.mathworks.com/matlabcentral/fileex

change/10049.3NSST is available from http://www.math.uh.edu/ ∼dlabate/software.html.

Page 9: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 9/10

2064 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 5, MAY 2013

Lena Barbara Peppers

σ   10 20 30 40 10 20 30 40 10 20 30 40

NSCT 35.43 32.35 30.51 29.17 33.07 29.16 27. 01 25.54 33.99 31. 57 29.97 28.73

SWT 34.52 31.05 29.00 27.57 31.96 27.57 25.36 24.05 33.48 30.67 28.69 27.32

FDCT 34. 00 31. 31 29. 54 28. 24 29. 14 25. 62 24. 58 24. 02 32.60 29. 93 28.45 27.43

NSST 35.50 32.57 30.71 29.31 32.97 29.45 27. 31 25.84 34.12 31. 86 30. 23 28.98

DNST1   35.90 32.75 30.84 29.41   33.64   30.03 27.84 26.33 34.44 32.17 30.59 29.31

DNST2 35. 79 32.38 30.25 28. 64 33.38 29.45 27.08 25.41 34.40 31.99 30.26 28.87

DSST 35. 79 32. 38 30. 25 28. 64   33.65   29. 70 27.27 25.54 34. 38 31.84 30. 04 28. 56

DST 35.74 32.27 30.28 28.81 33.71 29.78 27.56 26.07 34.21 31.58 29.87 28.59

Man F16 Boat

σ   10 20 30 40 10 20 30 40 10 20 30 40

NSCT 32.46 29.61 28.07 27.02 34.59 31.49 29.58 28.17 33.63 30.37 28.61 27.39

SWT 31.80 28.51 26.81 25.61 33.85 30.47 28.27 26.79 32.91 29.18 27.27 25.93

FDCT 31. 06 28. 63 27. 19 26. 09 33. 37 30. 05 28. 06 26. 83 31.56 29. 18 27.72 26.56

NSST 32.62 29.73 28.09 26.94 34.68 31.57 29. 50 28.19 33.77 30. 47 28. 65 27.30

DNST1 33.05   30.04 28.34 27.13   34.99   31.88 29.89 28.60   34.23   30.87 29.01 27.63

DNST2 33. 10 29.96 28.20 26. 93 35.03 31.90 29.82 28.48 34.09 30.63 28.71 27.27

DSST   33.29   30.00 28.11 26.73   35.11   31.92 29.75 28.35   34.29   30.69 28.64 27.11

DST 32.96 29.67 27.93 26.70 34.96 30.57 29.43 28.08 34.24 30.56 28.59 27.17

Fig. 15. Comparison results (PSNR values) for various denoising schemes.

DST4

[14]. It should be pointed out that the NSST is the firstpublished implementation of the 2D shearlet transform. In this

task, an observed image   g  is given by

g =   f  +  n,

where  n  is additive Gaussian noise and our aim is to approx-

imate the image   f   from the noisy image   g. For simple com-

parison, we apply hard threshold on the transform coefficients.For each scale   j , we choose the threshold parameter  T  j   as

T  j =  K i · σ   (28)

where   σ   is the standard deviation of Gaussian noise and we

choose   K  j =  2.5 for   j =  0, . . . , 2 and   K  j =  3.8 for   j =  3for all directional transforms, the NSCT, NSST, DST, DSST,

DNST1 and DNST2. And choose   K  j =   3 for   j =   0, . . . , 2

and   K  j =  4 for   j =  3 for the SWT. For the DSST, DNST1,NSCT and NSST, 8, 8, 16, and 16 directional filters are used

for scales   j = 0, 1, 2, 3 while 4, 4, 8, and 8 directional filters

are used for the DNST2. As illustrated in Figs. 9, 10, and 15,

the shearlet transform outperforms other denoising schemes in

terms of PSNR (peak signal to noise ratio) as well as visual

quality while its running time is comparable with others.

 B. Image Inpainting

Image inpainting refers to the filling-in of missing datain digital images based on the information available in the

observed region. The applications of this technique include

film restoration, text or scratch removal, and digital zooming.

Assume that the missing pixels are indicated by an   n ×  n

diagonal mask matrix   M . The main diagonal of   M   encodes

the pixel status, namely 1 for an existing pixel and 0 for a

missing one. Then, we consider the following 1  minimization

problem.

arg minY ∈Rn

T (Y )1 + λ M ( X  − Y )22   (29)

4DST is available from http://www.shearlab.org.

where   T   is a linear transform   T   :   Rn

→   Rm

with   m ≥   n.Our aim here is to approximate the original image   X   from

incomplete data   M ( X )  by minimizing the   1 norm of trans-

form coefficients associated with   T   in this optimization prob-

lem (29). Then restored image will be obtained by a solution

to (29). In this approach, it is crucial to choose the linear

transform   T   which can sparsify the original image   X   in the

transform domain in order to successfully restore   X   from

 M ( X ). This sparsity model (29) for image inpainting was

introduced and several numerical experiments showed the

effectiveness of this approach in [6]. We apply the same

iterative threshold scheme described in [6] to approximately

solve (29). We now present numerical experiments with var-

ious test images to compare the performance of inpaintingschemes using the FDCT(MCALab)5, DNST1, DSST and

DST. For our tests, we simply apply the DNST1 (and DSST)with 8, 8, 16, 16 directions in (29), where   T   is the DNST1

(and DSST). Our numerical tests show that DNST1 outper-

forms other transforms. In particular, the edges of inpainted

images are well captured due to the superior directional

selectivity of DNST1. The numerical results are illustrated inFigs. 11 and 16.

C. Video Denoising

We finally present the numerical test results of the 3D

shearlet transform in 3D video denoising tasks. As in 2Ddenoising tests, we apply various linear transforms followed

by applying hard threshold to remove additive Gaussian noise— here, basic settings for the tests are the same as 2D image

denoising tests. For the 3D discrete shearlet decomposition, weapply a 3-level decomposition. We have tested two different

types of the shearlet transforms, the DNST3d1   and DNST3d

2— they differ only by the number of directions across scales aswe explained in the beginning of this section. We compared the

performance of our shearlet based denoising scheme with other

5MCALab (Version 120) is available from http://www.greyc.ensicaen.fr/ ∼ jfadili/demos/WaveRestore/downloads/mcalab/Home.html.

Page 10: 06423910

8/11/2019 06423910

http://slidepdf.com/reader/full/06423910 10/10

LIM: NONSEPARABLE SHEARLET TRANSFORM 2065

Lena Barbara Peppers

PSNR Iter Time PSNR Iter Time PSNR Iter Time

FDCT 31.23dB 300 590.26sec 27.59dB 300 590.26sec 30.61dB 300 590.26

DNST1   32.54dB   126 202.92sec   28.12dB   126 202.92sec   31.89dB   126 202.92sec

DSST 31.27dB 126 202.92sec 27.11dB 126 202.92sec 30.34dB 126 202.92sec

DST 31.77dB 135 225.23sec 27.96dB 135 225.23sec 31.15dB 135 225.23sec

Man F16 Boat

PSNR Iter Time PSNR Iter Time PSNR Iter Time

FDCT 27.45dB 300 590.26sec 29.26dB 300 590.26sec 28.75dB 300 590.26

DNST1   28.69dB   126 202.92sec   29.89dB   126 202.92sec   29.37dB   126 202.92sec

DSST 28.07dB 126 202.92sec 28.27dB 126 202.92sec 27.61dB 126 202.92sec

DST 28.34dB 135 225.23sec 29.74dB 135 225.23sec 28.82dB 135 225.23sec

Fig. 16. Inpainting results: PSNR, the number of iterations and running time.

state-of-the-art directional transforms: the surfacelet transform

(denoted by Surf)6 [15] and 3D band-limited shearlet trans-

form (denoted by Band-Shear) [16]. It should be noted that

Band–Shear is the first published implementation of the 3D

shearlet transform. The numerical results of Band-Shear are

taken from [16]. We use two test video sequences Mobile and

Coastguard (192 × 192 ×  192) and choose   K  j  =   4 for the

finest scale and   K  j  =   3 for the rest of scales in (28). Thenumerical results are illustrated in Figs. 12–14.

VIII. CONCLUSION

In this paper, we have developed a new discrete shearlet

transform associated with compactly supported shearlets

generated by a nonseparable generator which provides good

localization in space domain as well as improved directional

selectivity. It is based on the discrete shear transform   S d k 2− j/2

which faithfully digitizes the shear operator   S k 2− j/2   in

the continuum domain. In fact, combining multiresolution

analysis in theory of wavelets, this framework allows the

shearlet coefficients  f , ψ  j,k ,m   to be explicitly computed inthe discrete domain. We also showed the various applicationsof our directional transform in image processing. In numerical

tests, our proposed transform compares favorably with other

existing directional transforms. In particular, the superior

directional selectivity and localization property of compactly

supported shearlets reduces artifacts to enhance visual quality

of restored image.

REFERENCES

[1] E. J. Candès, L. Demanet, D. L. Donoho, and L. Ying, “Fast discretecurvelet transforms,”  SIAM Multiscale Model. Simul., vol. 5, no. 3, pp.861–899, 2006.

[2] E. J. Candès and D. L. Donoho, “New tight frames of curvelets andoptimal representations of objects with  C 2 singularities,”  Commun. Pure

 Appl. Math., vol. 57, no. 2, pp. 219–266, 2003.[3] G. Easley, D. Labate, and W. Lim, “Sparse directional image represen-

tations using the discrete shearlet transform,”   Appl. Comput. Harmon. Anal., vol. 29, no. 2, pp. 232–250. 2010.

[4] A. L. Cunha, J. Zhou, and M. N. Do, “The nonsubsampled con-tourlet transform: Theory, design, and applications,”  IEEE Trans. ImageProcess., vol. 15, no. 10, pp. 3089–3101, Oct. 2006.

[5] M. N. Do and M. Vetterli, “The contourlet transform: An efficientdirectional multiresolution image representation,”   IEEE Trans. ImageProcess., vol. 14, no. 12, pp. 2091–2106, Dec. 2005.

6SurfBox is available from http://www.mathworks.com/matlabcentral/ fileexchange/14485.

[6] M. Elad, J.-L. Starck, D. Donoho, and P. Querre, “Simultaneous Cartoonand Texture Image Inpainting using Morphological Component Analysis(MCA),”   Appl. Comput. Harmon. A., vol. 19, no. 3, pp. 340–358, Nov.2005.

[7] K. Guo, G. Kutyniok, and D. Labate, “Sparse multidimensional repre-sentations using anisotropic dilation and shear operators,”   Wavelets and Splines (Athens, GA, 2005), Nashville, TN, USA: Nashboro Press, 2006,pp. 189–201.

[8] G. Kutyniok, J. Lemvig, and W.-Q. Lim, “Optimally sparse approxima-

tions of 3-D functions by compactly supported shearlet frames,”  SIAM  J. Math. Anal., vol. 44, no. 4, pp. 2962–3017, 2012.[9] G. Kutyniok and W.-Q. Lim, “Compactly supported shearlets are opti-

mally sparse,” J. Approx. Theory, vol. 163, no. 11, pp. 1564–1589, 2011.[10] P. Kittipoom, G. Kutyniok, and W.-Q. Lim, “Construction of compactly

supported shearlet frames,”  Construct. Approx., vol. 35, no. 1, pp. 21–72,Feb. 2012.

[11] G. Kutyniok and W.-Q. Lim, “Image separation using wavelets andshearlets,” in   Lecture Notes in Computer Science, New York, USA:Springer-Verlag, 2012.

[12] G. Kutyniok, M. Shahram, and X. Zhuang, “ShearLab: A rational designof a digital parabolic scaling algorithm,”  SIAM J. Imag. Sci., vol. 5, no.4, pp. 1291–1332, 2012.

[13] D. Labate, W.-Q. Lim, G. Kutyniok, and G. Weiss, “Sparse multidimen-sional representation using shearlets,” in   Proc. Int. Soc. Opt. Photon.vol. 5914. Bellingham, WA, USA, 2005, pp. 254–262.

[14] W.-Q. Lim, “The discrete shearlet transform: A new directional trans-

form and compactly supported shearlet frames,”   IEEE Trans. ImageProcess., vol. 19, no. 5, pp. 1166–1180, May 2010.[15] Y. M. Lu and M. N. Do, “Multidimensional directional filter banks and

surfacelets,”   IEEE Trans. Image Process., vol. 16, no. 4, pp. 918–931,Apr. 2007.

[16] P. S. Negi and D. Labate, “3-D discrete shearlet transform and videoprocessing,”  IEEE Trans. Image Process., vol. 21, no. 6, pp. 2944–2954,Jun. 2012.

Wang-Q Lim  received the Ph.D. degree in mathe-matics from Washington University, St. Louis, WA,USA, in 2006.

He has been a Post-Doctoral Research Fellowwith the Department of Mathematics, TechnischeUniversität Berlin, Berlin, Germany, since 2011. Hiscurrent research interests include applied harmonicanalysis, compressed sensing, and signal and imageprocessing.