Pattern Recognition 76 (2018) 367–379

Contents lists available at ScienceDirect

Pattern Recognition

journal homepage: www.elsevier.com/locate/patcog

Weighted variational model for selective image segmentation with

application to medical images

Chunxiao Liu

a , Michael Kwok-Po Ng

b , Tieyong Zeng

b , c , ∗

a Department of Mathematics, Hangzhou Normal University, Hangzhou 310036, PR China b Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong c HKBU Institute of Research and Continuing Education Shenzhen Virtual University Park, Hong Kong

a r t i c l e i n f o

Article history:

Received 18 January 2017

Revised 1 November 2017

Accepted 16 November 2017

Available online 20 November 2017

Keywords:

Selective segmentation

Mumford-Shah model

Thresholding

Medical images

Iterative algorithm

a b s t r a c t

Selective image segmentation is an important topic in medical imaging and real applications. In this pa-

per, we propose a weighted variational selective image segmentation model which contains two steps.

The first stage is to obtain a smooth approximation related to Mumford-Shah model to the target region

in the input image. Using weighted function, the approximation provides a larger value for the target

region and smaller values for other regions. In the second stage, we make use of this approximation and

perform a thresholding procedure to obtain the object of interest. The approximation can be obtained

by the alternating direction method of multipliers and the convergence analysis of the method can be

established. Experimental results for medical image selective segmentation are given to demonstrate the

usefulness of the proposed method. We also do some comparisons and show that the performance of the

proposed method is more competitive than other testing methods.

© 2017 Elsevier Ltd. All rights reserved.

1

i

t

c

b

[

s

o

fl

T

s

n

t

a

i

e

t

o

t

s

m

a

s

t

d

s

a

g

s

p

t

s

M

p

f

c

m

t

h

0

. Introduction

Image segmentation is a fundamental task in image process-

ng. In this field, models based on variational methods and par-

ial differential equations are popular due to their flexibility and

omputational convenience. Variational segmentation models can

e classified into edge based models [1–3] and region based ones

4] . Edge based methods utilize the local properties of the first and

econd order derivatives of the image. The main weakness is a lack

f robustness in dealing with image noise. The well-known and in-

uential region based model is the Mumford–Shah (MS) model [4] .

he variational problem leads to a piecewise smooth solution with

mooth boundaries. In numerical computation, parameterization is

eeded and topological changes are not allowed during the itera-

ions.

In some applications, such as surgery simulation, medical di-

gnosis, object tracking, etc, people aim to extract only objects of

nterest from an image. In such cases, selective segmentation mod-

ls are more appropriate. There is relatively less work done for this

opic. Gout et al. [5] introduced a geometry constraint using a set

f marker points on the contour of interest to the geodesic ac-

∗ Corresponding Author.

E-mail addresses: mng@math.hkbu.edu.hk (M.K.-P. Ng), 20110065@hznu.edu.cn

(T. Zeng).

p

n

p

t

l

ttps://doi.org/10.1016/j.patcog.2017.11.019

031-3203/© 2017 Elsevier Ltd. All rights reserved.

ive contour model. Later, Badshah and Chen [6] improved the re-

ults by combining a region term. With the region information, the

odel became more robust especially for noisy images. In [7] , the

uthors proposed an adaptively varying narrow band algorithm to

egment a larger class of image segmentation problems and cap-

ure more complex features. A new variational model was intro-

uced in [8] with two level set functions which can do two tasks

imultaneously. One is to find the segmentation of all boundaries

nd the other focuses on the selected object that is close to the

eometry constraints. An intensity term was proposed for 3D liver

egmentation in [9] . In [10] , a selective segmentation model was

roposed by utilizing the average image of channels, which can ex-

ract textural and inhomogeneous objects.

The main purpose of this paper is to propose a new two-

tage weighted selective image segmentation model based on the

umford–Shah model. In the first stage, we choose several marker

oints around the objects of interest and construct a weighted

unction by using an edge function and a distance function. By in-

orporating this weighted function into a convex second-order seg-

entation model, we can obtain a smooth approximation to the

arget region. Existence and uniqueness of the minimizer of the

roposed segmentation model can be guaranteed. Many effective

umerical algorithms can be used by virtue of the convexity of the

roposed model. We illustrate clearly how to use alternating direc-

ion method to get a suitable numerical algorithm and prove the

inear convergence under mild conditions. In the second stage, we

368 C. Liu et al. / Pattern Recognition 76 (2018) 367–379

2

i

s

i

p

d

d

w

z

f

a

d

r

d

g

w

m

w

t

�

w

fi

C

t

p

m

3

m

t

m

s

3

m

t

u

w

i

ω

make use of this approximation function and perform a threshold-

ing procedure to obtain the object of interest.

This paper is organized as follows. In Section 2 , we briefly re-

view the related segmentation models. In Section 3 , we present

our model and develop the related basic mathematical properties.

The numerical algorithm and its convergence analysis are given

in Section 4 . Numerical experiments are then listed in Section 5 ,

which clearly show the good performance of our approach. Finally,

we conclude the paper in Section 6 .

2. Related works

2.1. Global segmentation models

The level set method [11] is a classical method to solve curve

evolution problems. The interface is represented implicitly by the

zero level set of a Lipschitz continuous level set function (LSF) and

topological changes are allowed. Discretization schemes can be im-

plemented on fixed mesh grids. For images with piecewise con-

stant intensities, Chan and Vese proposed the two-phase MS model

[12] and the multiphase case [13] . Piecewise constant level set

methods [14–16] use discontinuous functions to represent different

phases. Methods of fuzzy membership function [17,18] use contin-

uous functions ranging from 0 to 1 to represent the probability of

belonging to some region. Convex relaxation methods [19,20] and

graph cut method [21] were proposed to overcome the local min-

imum problem of the MS model. Fast continuous max-flow meth-

ods were proposed in [22] . Recently, Cai et al. [23] proposed a two-

stage segmentation model using a convex variant of the MS model,

which can be regarded as a unification of image restoration and

image segmentation. They first found a unique smooth minimizer

by the split-Bregman algorithm [24,25] and then segmented the

image by a thresholding strategy. Furthermore, they introduced a

K-means method to choose the thresholds automatically. In [26] ,

Cai proposed a new multiphase segmentation model by com-

bining image restoration and image segmentation models. Multi-

atlas segmentation methods have been proposed in [27,28] and

proved to be useful. In [29,30] , nonparametric statistical segmenta-

tion methods were proposed to handle texture features. A wavelet

method with shape prior was proposed in [31] for individual image

and video segmentations. For more about segmentation methods,

please refer to [32–40] and the references therein.

Let � ⊂ R

2 be a bounded connected open set with Lipschitz

boundary. Let � be a compact curve in � and f : � → R be a

given image. The MS model aims to solve the following minimiza-

tion problem [4] :

inf u, �

{H

1 (�) +

α

2

∫ �\ �

|∇u | 2 dx +

β

2

∫ �

| u − f | 2 dx

},

where α and β are positive constants and H

1 is the one-

dimensional Hausdorff measure. For regular curves, H

1 (�) denotes

the length of �.

In [23] , a two-stage variational image segmentation was pro-

posed. The first stage is to find a smooth approximation to the

original image by solving the following convex minimization prob-

lem:

min

u ∈ W

1 , 2 (�)

{∫ �

|∇ u | dx +

α

2

∫ �

|∇ u | 2 dx +

β

2

∫ �

| u − f | 2 dx

}, (1)

where the Sobolev space W

1 , 2 (�) = { v ∈ L 2 (�) | ∂ i v ∈ L 2 (�) , i =1 , 2 } with L 2 (�) = { f (x )

∣∣( ∫ � f 2 (x ) dx

)1 / 2 < ∞} . Once u is obtained,

then in the second stage, the segmentation is obtained by thresh-

olding u properly. The thresholds can be provided by the users or

obtained automatically by any clustering methods such as K-means

or convex K-means methods [41,42] .

.2. Selective segmentation models

Selective segmentations that extract objects of interest from an

mage are important and challenging in a number of areas, such as

ecurity monitoring and medical diagnosis. Assume that K points

nside the target are available on the image. Using these marker

oints X = { x 1 , x 2 , · · · , x K } , a distance function d(x ) : � → R can be

efined as [6] ,

(x ) =

K ∏

i =1

(1 − e −

| x −x i | 2 2 h 2

), x ∈ �, (2)

here h is a positive constant. The distance function is close to

ero near the marker points and approximates to one far away

rom the marker points. There are other ways to define d ( x ), such

s

(x ) = min

y ∈ X | x − y | . (3)

To use the edge information, an edge detector is often incorpo-

ated into segmentation models. A popular way to define an edge

etection function is

(x ) =

1

1 + β|∇ f (x ) | 2 , (4)

here β > 0 adjusts the strength of edge.

In [5] , an edge-based model was proposed as follows:

in

�

∫ �

d · gds,

ith d defined by (3) .

Badshah and Chen [6] improved the model by introducing in-

ensity fitting terms similar to Chan-Vese model [12] as follows:

min

,C 1 ,C 2

{

α

∫ �

d · gds + λ1

∫ �in

| f − C 1 | 2 dx + λ2

∫ �out

| f − C 2 | 2 dx

}

,

(5)

here α, λ1 and λ2 are constants to adjust the smoothing and the

delity term, � is the boundary between �in and �out and C 1 and

2 are constants to be optimized. Under the level set framework,

hey need to solve the Euler-Lagrange equation of the LSF.

Later in [7] , a new adaptive local band level set method was

roposed. Nguyen [43] combined a marker set and another anti-

arker set to obtain a selective segmentation model.

. The proposed weighted model

In this section, we propose our new model and present the

athematical analysis. Our method consists of two stages. In

he first stage, we solve a minimization problem based on some

arker points and obtain a smooth minimizer. Then in the second

tage, segmentation can be carried out by simple thresholding.

.1. Proposed model

Inspired by the convex model (1) and the above selective seg-

entation models, we propose the following weighted model for

he task of selective image segmentation:

min

∈ W

1 , 2 (�)

{

E(u ) :=

∫ �

|∇ u | dx+

α

2

∫ �

|∇ u | 2 dx +

β

2

∫ �

ω

2 | u − f | 2 dx

}

,

(6)

here ω is a weight function to adjust the fidelity and the smooth-

ng terms. In this paper, we use

2 (x ) = 1 − d(x ) g(x ) , ω(x ) ∈ (0 , 1] , (7)

C. Liu et al. / Pattern Recognition 76 (2018) 367–379 369

Fig. 1. First row: segmentation results of [5] ; Second row: segmentation results of [10] with initial LSF Fig. 1 e; Third row: segmentation results of [10] with another initial

LSF Fig. 1 i; Fourth row: segmentation results of our model.

370 C. Liu et al. / Pattern Recognition 76 (2018) 367–379

Fig. 2. || u k +1 −u k || 2 || u k || 2 (left) and || v k +1 −v k || 2

|| v k || 2 (right) vs. the iteration number.

T

|

a

|

M

f

|

C

w

v

e

E

P

m

P

b

t

where d ( x ) and g ( x ) are defined by (2) and (4) , respectively.

In practice, d ( x ) is small around the marker points, while g ( x ) is

small around the boundaries. Since the marker points are chosen

near the boundaries, the weight function ω( x ) in (7) is large near

the boundaries and becomes smaller far away from the boundaries.

Therefore, we conclude that:

(I) Near the boundaries, the third term in (6) , which is the fi-

delity term, plays an important and details are remained.

(II) Far away from the boundaries, smoothing plays more and

more important roles. As we can seen from the following

numerical results, complicated structures far away from the

edge are smoothed out and only the outline of the target

object remains.

We make some comparisons between our model and other ex-

isting selective segmentation models here:

• The existing selective models need to use level set methods

or membership functions in numerical computation, which can

bring additional computational efforts. • In the performed experiments, references [5,10] show sensitiv-

ity to the initialization, differently from the proposed model

whose convexity is demonstrated in Section 3.2 . For our model,

existence and uniqueness of the minimizer can be guaranteed.

Many stable and effective numerical algorithms can be devel-

oped.

Once we obtain the solution of (6) , we can obtain the segmen-

tation by thresholding in the second stage.

3.2. Mathematical analysis

Proposition 1. Suppose f ∈ L 2 ( �) and inf x ∈ �ω( x ) > 0 . Then model

(6) is strictly convex and there exists a unique minimizer

u ( x ) ∈ W

1, 2 ( �) .

Proof. By the definition of ω( x ) in (7) and the condition

inf x ∈ �ω( x ) > 0, we have the boundness of ω( x ). We suppose

M 1 ≤ω( x ) ≤ M 2 , where M 1 and M 2 are two positive constants.

Choose u = 0 , we have

0

0 ≤ inf u ∈ W

1 , 2 (�) E(u )

≤ E(u 0 ) =

β

2

∫ �

ω

2 f 2 dx

≤ M

2 2 β

2

|| f || 2 L 2 (�)

< + ∞ .

hus inf u ∈ W

1 , 2 (�) E(u ) must exist.

We now prove that E ( u ) is coercive. It is obvious that

|∇u || L 2 (�) ≤√

2

αE(u ) (8)

nd

| u || L 2 (�) ≤ || u − f || L 2 (�) + || f || L 2 (�) . (9)

eanwhile,

0 ≤ M

2 1 β

2

∫ �

| u − f | 2 dx

≤ β

2

∫ �

ω

2 | u − f | 2 dx ≤ E(u ) ,

rom which we obtain

| u − f || L 2 (�) ≤√

2

M

2 1 β

E(u ) . (10)

ombining (8), (9) and (10) , we have

|| u || W

1 , 2 (�) ≤ || u || L 2 (�) + ||∇u || L 2 (�)

≤( √

2

α+

√

2

M

2 1 β

) √

E(u ) + || f || L 2 (�) ,

hich means that E ( u ) is coercive.

Note that W

1, 2 ( �) is a reflective Banach space, and E ( u ) is con-

ex, lower semicontinuous (l.s.c.) and coercive. We conclude the

xistence of the minimizer of E ( u ) in W

1, 2 ( �) [23,44] .

Finally, uniqueness can be guaranteed by the strict convexity of

( u ). �

roposition 2. Suppose f ∈ L 2 ( �), inf x ∈ �ω( x ) > 0, the unique mini-

izer u ∗( x ) of model (6) satisfies inf x ∈ � f (x ) ≤ u ∗(x ) ≤ sup x ∈ � f (x ) .

roof. From Proposition 1 , it is clear that E ( u ) is proper. Let { u n }

e a minimizing sequence. Then there exists a constant M > 0 such

hat E ( u n ) ≤ M for all n ∈ N . Therefore, we have ||∇u n || L 2 (�) are

C. Liu et al. / Pattern Recognition 76 (2018) 367–379 371

Fig. 3. First row: segmentation results of [5] ; Second row: segmentation results of [10] ; Third row: segmentation results of our model.

u

f

|i

W

s

E

u

d

w

ω

L

∫

niformly bounded. Moreover,

βM

2 1

2

|| u n − f || 2 L 2 (�) ≤β

2

∫ �

ω

2 | u n − f | 2 dx ≤ M, for all n ∈ N ,

rom which we conclude that || u n − f || L 2 (�) is uniformly bounded.

Then we have

| u n || L 2 (�) ≤ || u n − f || L 2 (�) + || f || L 2 (�)

s uniformly bounded.

Therefore, up to a subsequence, u n converges strongly in

1, 2 ( �) to some u ∗ and ∇u n converges weakly as a mea-

ure to ∇u ∗. Since E ( u ) is lower semicontinuous, we have

( lim inf n →∞

u n ) ≤ lim inf n →∞

E(u n ) , which implies u ∗ is the

nique solution to (6) .

Let α = inf f and β = sup f . We remark that x → ω

2 | x − f | 2 is

ecreasing in (0, f ) and increasing in ( f, + ∞ ) . Therefore, if C ≥ f ,

e have

2 | min (x, C) − f | 2 ≤ ω

2 | x − f | 2 .

et C = β = sup f, we obtain

�ω

2 | min (u

∗, β) − f | 2 dx ≤∫ �

ω

2 | u

∗ − f | 2 dx. (11)

372 C. Liu et al. / Pattern Recognition 76 (2018) 367–379

Fig. 4. (a) The original liver CT image with six marker points. (b) The smoothed solution u . (c) Thresholding of u . (d) Segmentation.

Fig. 5. (a) The original CT image with eight marker points. (b) The smoothed solution u . (c) Thresholding of u . (d) Segmentation.

Fig. 6. (a) The original image with two marker points. (b) The smoothed solution u . (c) Thresholding of u . (d) Segmentation.

4

g

m

w

a

i

c

m

W

s

m

In the same way we can prove that ∫ �

ω

2 | sup (u

∗, α) − f | 2 dx ≤∫ �

ω

2 | u

∗ − f | 2 dx. (12)

On the other hand, from Proposition 15 in [45] , both min ( u ∗, β)

and sup (u ∗, α) are members of W

1, 2 ( �) and

|∇ ( min (u

∗, β)) | ≤ |∇ u

∗| , |∇ ( sup (u

∗, α)) | ≤ |∇u

∗| . (13)

Combing (11), (12) and (13) , we have

E( min (u

∗, β)) ≤ E(u

∗) , E( sup (u

∗, α)) ≤ E(u

∗) ,

which implies that α ≤ u ∗ ≤β . �

4. Numerical optimization

Since model (6) is convex, there can be many efficient numeri-

cal methods to solve it [17,24,46–50] . In this section, we present an

alternating direction method (ADM) [17,51] to solve it and provide

convergence analysis of the algorithm.

.1. Algorithm

Rewriting the minimization problem (6) into a matrix form

ives

in

u

{|| Au || 1 +

α

2

|| Au || 2 2 +

β

2

|| ω(u − f ) || 2 2

}, (14)

here A denotes the matrix of the gradient operator ∇ and u is

n n 2 -by-1 vector corresponding to an n -by- n image. By introduc-

ng an auxiliary variable v = Au , (14) is equivalent to the following

onstrained optimization problem:

in

u , v

{|| v || 1 +

α

2

|| v || 2 2 +

β

2

|| ω(u − f ) || 2 2

}subject to v = Au .

e apply the quadratic penalty method [4 8,4 9] to enforce the con-

traint and solve the following unconstrained problem:

in

u , v

{|| v || 1 +

α

2

|| v || 2 2 +

β

2

|| ω(u − f ) || 2 2 +

μ

2

|| v − Au || 2 2

}, (15)

C. Liu et al. / Pattern Recognition 76 (2018) 367–379 373

Fig. 7. First row: segmentation results of [23,26] ; Second row: segmentation of the triangle using our method; Fourth row: segmentation of the ring using our method. Third

row: segmentation of the triangle and the ring together.

w

u

s

v

v

here μ> 0 is a penalty parameter.

To solve (15) , we use the ADM and minimize with respect to

and v alternatingly. From an initial guess u

0 for u , we obtain a

equence

1 , u

1 , v 2 , u

2 , . . . , v k , u

k , v k +1 , u

k +1 , . . .

ia the following two subproblems:

v k +1 = R (u

k ) := arg min

v

{

|| v || 1 +

α

2

|| v || 2 2 +

μ

2

|| v − Au

k || 2 2

}

(16)

374 C. Liu et al. / Pattern Recognition 76 (2018) 367–379

Fig. 8. (a) The original image f with eight marker points. (b) The image (1 − f ) with eight marker points. (c) The smoothed solution u . (d) Segmentation.

S

i

m

p

F

f

p

A

4

r

(

v

W

b

L

a

H

s

|

and

u

k +1 = S(v k +1 ) := arg min

u

{β

2

|| ω(u − f ) || 2 2 +

μ

2

|| v k +1 − Au || 2 2

},

(17)

for k = 0 , 1 , 2 , . . . , which we call the iterations the outer-ADM-

iterations. We now solve the subproblems separately.

4.1.1. v -Subproblem

Minimization problem (16) can be reformulated as

v k +1 = R (u

k ) = arg min

v

{

|| v || 1 +

α + μ

2

|| v − μ

α + μAu

k || 2 2

}

.

(18)

Suppose v = (v 1 , v 2 ) ,

y 1 =

μ

α + μ∇ x u

k , y 2 =

μ

α + μ∇ y u

k , | y | =

√

y 2 1

+ y 2 2 ,

then (18) can be solved explicitly using a generalized shrinkage op-

erator [24] as follows:

v k +1 1 = max

(| y | − 1

α + μ, 0

)y 1 | y | ,

v k +1 2 = max

(| y | − 1

α + μ, 0

)y 2 | y | .

(19)

4.1.2. u -Subproblem

In order to solve (17) efficiently, we introduce another auxiliary

variable p = u − f . We can use the split-Bregman method to en-

force the constraint strictly. With b

0 = 0 , we have at step (n + 1)

(p

n +1 , u

n +1 ) = arg min

p , u

{

β

2

|| ωp || 2 2 +

λ

2

|| p − u + f − b

n || 2 2

+

μ

2

|| v k +1 − Au || 2 2

}

,

b

n +1 = b

n − p

n +1 + u

n +1 − f . (20)

The minimization problem can be solved effectively by minimiza-

tion with respect to u and p alternatingly. Firstly, the minimization

with respect to u reads

u

n +1 = arg min

u

{λ

2

|| p

n − u + f − b

n || 2 2 +

μ

2

|| v k +1 − Au || 2 2

},

which gives

(λI + A

T A ) u

n +1 = λ(p

n + f − b

n ) + μA

T v k +1 . (21)

ince the matrix (λI + A

T A ) is positive definite and hence invert-

ble, we can use the fast Fourier transforms to solve (21) . Then, the

inimization for p is

n +1 = arg min

p

{β

2

|| ωp || 2 2 +

λ

2

|| p − u

n +1 + f − b

n || 2 2

}.

rom its Euler-Lagrange equation we have an explicit expression

or p :

n +1 =

λ(u

n +1 − f − b

n )

βω

2 + λ. (22)

Unifying all the above steps, we obtain Algorithm 1 .

lgorithm 1 Solving (15) by ADM.

Choose μ and λ; Initialize u

0 = 0 . Choose the tolerance ε. Set

k = 1 ;

• Step 1. Update v k by (19).

• Step 2. Set b

0 = 0 and p

0 =0.

Do n = 0 , 1 , 2 , . . . , N until || u k,n +1 −u k,n || 2

|| u k,n +1 || 2 < ε.

(i) Update u

k,n by (21).

(ii) Update p

n by (22).

(iii) Update b

n by (20).

When iterations (i)-(iii) terminate, we set u

k = u

k,N .

• Step 3. Test convergence. If the stopping criterionis satisfied,

the algorithm stops. Otherwise, set k = k + 1 andgo to Step 1.

.2. Convergence analysis

In the following, we will investigate the convergence of Algo-

ithm 1. Recall that the relationship between v k and v k +1 from

16) and (17) can be expressed as

k +1 = R (S(v k )) := T (v k ) .

e will analyze the convergence of the sequence { v k } generated

y T .

emma 1. [48] If φ is a convex, l.s.c. proper function, then the oper-

tor

A φ : x → arg min

y

{φ(y ) +

1

2

|| x − Ay || 2 2

}atisfies:

| A H

A φ(x 1 ) − A H

A φ(x 2 ) || 2 ≤ || x 1 − x 2 || 2 .

C. Liu et al. / Pattern Recognition 76 (2018) 367–379 375

Fig. 9. Influence of the number of marker points. Blue lines: ground truth drawn by an expert; Red lines: segmentations of our model.

D

i

〈w

L

T

|P

q

P

t

|

S

w

|

w

(

S

a

T

f

l

i

t

efinition 1. A convex, l.s.c. proper function φ is called c-strongly

f there exist a positive constant c such that

∂φ(x 1 ) − ∂φ(x 2 ) , x 1 − x 2 〉 ≥ c|| x 1 − x 2 || 2 2 , ∀ x 1 , x 2 ,

here ∂ denotes the partial derivative.

emma 2. [48] Let φ be a convex, l.s.c. proper c-strongly function.

hen the operator H φ : x → arg min y { φ(y ) +

1 2 || y − x || 2 2 } satisfies:

| H φ(x 1 ) − H φ(x 2 ) || 2 ≤ 1

1 + c || x 1 − x 2 || 2 .

roposition 3. If the subproblem (17) is solved exactly, then the se-

uence { v k } converges to the limit point v ∗.

roof. For (17) , it is obviously φ1 (y ) :=

β2 μ || ω(y − f ) || 2

2 satisfies

he conditions of Lemma 1 . Therefore, we obtain immediately that

| A S(v 1 ) − A S(v 2 ) || 2 ≤ || v 1 − v 2 || 2 . (23)

uppose φ2 (y ) :=

1 μ || y || 1 +

α2 μ || y || 2

2 in (16) , then φ2 is c -strongly

ith c =

αμ . Applying Lemma 2 to (16) , we have

|R (u 1 ) − R (u 2 ) || 2 ≤ 1

1 + q || Au 1 − Au 2 || 2 , (24)

here q =

αμ > 0 .

Let T = RS, u 1 = S(v 1 ) and u 2 = S(v 2 ) . Combining (23) and

24) , we have

||T (v 1 ) − T (v 2 ) || 2 = ||RS(v 1 ) − RS(v 2 ) || 2 = ||R (u 1 ) − R (u 2 ) || 2 ≤ 1

1 + q || Au 1 − Au 2 || 2

=

1

1 + q || A S(v 1 ) − A S(v 2 ) || 2

≤ 1

1 + q || v 1 − v 2 || 2 .

ince 1 / (1 + q ) < 1 , the operator T is a contract operator and has

unique fixed point. Denote the fixed point of T by v ∗, we have

(v ∗) = v ∗ and

|| v k +1 − v ∗|| 2 || v k − v ∗|| 2 =

||T (v k ) − T (v ∗) || 2 || v k − v ∗|| 2 ≤ 1

1 + q ,

rom which we conclude that the algorithm converges at least Q-

inearly. �

Once we obtain the smoothed solution u of (6) , the second step

s to segment the target object by thresholding of u . We will illus-

rate the thresholding methods for our model in Section 5 .

376 C. Liu et al. / Pattern Recognition 76 (2018) 367–379

Fig. 10. Influence of the position of marker points. (a) Three marker points are far away from the boundary; SA = 84.96%. (b) Four marker points are outside but near the

boundary; SA = 95.86%. (c) All marker points are inside the boundary; SA = 96.05%.

Table 1

CPU time in seconds for Figs. 1 and 3 .

[5] [10] Our method

Fig. 1 4.40 3.93 3.52

Fig. 3 4.56 4.17 3.18

f

t

s

t

m

(

p

a

|

w

i

s

F

i

f

T

l

t

i

T

b

r

m

5

i

[

H

s

o

t

f

5. Numerical results

We provide several numerical results in MATLAB to demon-

strate the stability and efficiency of our algorithm. Some medical

images are chosen as test images. All the images are scaled into

the interval [0, 1] by the linear-stretch formula:

ˆ f =

f − f min

f max − f min

,

where f max and f min are the maximum and minimum of the pixel

values of f , respectively.

We set α = 1 , μ = 15 and λ = 10 in all experiments. We use

green stars to indicate the marker points. The parameter h in

(2) should be slightly changed according to the size of the target.

Generally, large objects require large values of h and small objects

require smaller ones.

The threshold parameter in our method is similar to that of

[23] . The user can try different thresholds by try and error. Note

that we need not to recompute our convex model (6) when the

thresholds change. This virtue of our method can save much more

time compared with other methods which have to solve the min-

imization problems again if the threshold is changed [12,52] . Be-

sides, the values of threshold are relatively stable for the same tis-

sue, which can help us to choose suitable threshold parameters.

5.1. Experiment 1

In this subsection, we segment some pictures with the property

that the average intensity of the target area is larger than that of

the surrounding area. Results are presented in Figs. 1 to 6 . In Fig. 1 ,

we present the segmentation results of a bone image. We compare

our method with the selective segmentation models of [5,10] . For

comparison, we choose the same three marker points for different

methods. The first row presents the segmentation results of the

method in [5] . We can see that the model of Gout et al. is not able

to capture the object of interest. The model in [10] can successfully

segment the object, the results of which are listed in the second

row of Fig. 1 . But the method of [10] depends on the initialization

of the LSF. We take another different initialization for the LSF and

obtain a different segmentation result, which is listed in the third

row. We can see the segmentation results are largely dependent on

the initialization.

Segmentation results for our method are presented in the

fourth row. We set β = 2 and h = 20 . We choose three markers

points near the edge. We can observe that all the details far away

rom the edge are filtered out by virtue of the weighting func-

ion. Meanwhile, the image are smoothed selectively and the main

tructures are retained. Therefore, it is easy to obtain the segmen-

ation by simply thresholding the solution u . For this example, seg-

entation is obtained by u > 0.75, which is shown in Fig. 1 (o) and

p). For illustration, we display the convergence curves of the pro-

osed numerical method as shown in Proposition 3 . Since the ex-

ct solution is unknown, we provide the convergence curves of

| u

k +1 − u

k || 2 / || u

k || 2 and || v k +1 − v k || 2 / || v k || 2 in Fig. 2 .

In Fig. 3 , the tumor in the brain has complicated details. Again,

e use same marker points to compare our model with methods

n [5,10] . For our method, we set β = 2 and h = 20 . The tumor is

uccessfully segmented. We provide the CPU time comparison of

igs. 1 and 3 in Table 1 , from which we can see that our algorithm

s faster than the two level set based algorithms.

In Figs. 4 and 5 , we show the segmentation results of two dif-

erent organs in a CT image. In the two tests, β = 2 and h = 15 .

he numbers of marker points are six and eight respectively for the

eft and right organ. The CT image includes many complicated de-

ails, some of which have similar intensity distributions. However,

n the solution image, all details are hid except the target object.

herefore, the targets are highlighted and can be easily extracted

y thresholding.

To test the model for small targets, we provide segmentation

esults for a small vessel in Fig. 6 . We set β = 10 and h = 10 . Two

arker points are chosen. The vessel is easily segmented.

.2. Experiment 2

In this subsection, we provide some experiments and compar-

sons with [23,26] in Fig. 7 . In the first row, we can see that both

23,26] can provide good segmentation results for noisy images.

owever, their methods segment the three objects simultaneously

ince their methods provide a two-phase segmentation. In the sec-

nd and the third rows, we present respectively segmentations of

he triangle and the ring using our method. It is shown in the

ourth row that our model can also segment the two parts simulta-

C. Liu et al. / Pattern Recognition 76 (2018) 367–379 377

Fig. 11. Influence of noise; Blue lines: ground truth; Red lines: our segmentation results. (a) σ= 0.5; SA = 95.13%. (b) σ= 1; SA = 94.80%. (c) σ= 2; SA = 94.01%.

n

a

5

p

m

s

p

t

p

q

b

S

e

o

a

l

i

t

p

W

t

c

t

p

p

F

t

t

a

s

i

e

g

t

b

b

w

p

m

n

σ

f

a

t

w

6

o

e

t

t

t

t

i

i

m

f

A

a

t

t

1

G

f

C

i

q

f

R

eously. In a word, our method can segment any part of the image

ccording to different objectives.

.3. Experiment 3

We emphasize that we should use (1 − f ) instead of the in-

ut normalized image f in model (6) when the target has lower

ean intensity than its neighborhood. The image (1 − f ) has the

ame structure as the original image f . Segmentation results are

resented in Fig. 8 .

Now we investigate the influences of three factors on segmen-

ation results: the number of marker points, the position of marker

oints and different noise levels. To compare segmentation results

uantitatively, we consider the Segmentation Accuracy (SA) defined

y:

A =

correctly classified pixels

all pixels of the region of interest × 100% .

(i) Number of marker points. Firstly, we investigate the influ-

nce of the number of marker points. In Fig. 9 , different quantity

f marker points are used to segment the same ultrasound im-

ge. Ground truth is given by an expert, which is drawn in blue

ines. Our segmentations are drawn in red lines. The correspond-

ng SA values are given. The number of marker points is related

o the size of the object to be segment. Small targets need less

oints while big objects require more points, which is reasonable.

e can see that when the number of marker points is from three

o fifteen, the SA value becomes larger and larger. Meanwhile, we

an observe that the segmentation results are improved little when

he marker points are more than nine. In a word, too less marker

oints is infeasible, while too many is unnecessary.

(ii) Position of marker points. Secondly, we choose ten marker

oints but in different positions to segment the same image as in

ig. 9 . First of all, we emphasize that it is definitely not reasonable

o choose marker points that are far away from the target object. In

his situation, the surrounding objects may be wrongly segmented

s can be seen from Fig. 10 (a). In Fig. 10 (b), segmentation is still

atisfactory even four marker points are outside of the object of

nterest because the marker points are near the boundary. Differ-

ntly, all marker points are inside the boundary in Fig. 10 (c), which

ives similar result as Fig. 10 (b). Thus we conclude that there are

wo rules to choose the marker points for better performance:

(I) If the points are outside the object, they should be near the

oundary.

(II) If the points are inside the object, they can be near the

oundary or in the interior. To get better segmentation results,

e can choose relatively more points near the boundary and less

oints in the interior.

(iii) Noise. Finally, we investigate the influence of noise for our

odel in Fig. 11 . The original image is contaminated by Gaussian

oise with different noise levels. Zero mean noise with variance

is added to the image Fig. 9 (a). Ten marker points are chosen

or different experiments. When σ= 0.5, 1 and 2, our method can

chieve good performances. Besides, we also get good segmenta-

ion results for noisy images in Fig. 7 . Therefore, from Figs. 7 and 11

e can see that our method is robust against noise.

. Conclusion

We proposed a two-stage selective segmentation model based

n the MS model. By selecting several marker points around the

dge of the object, a weight function was constructed to adjust

he smoothing term and the fidelity term. Solving the model by

he ADM produced a smooth solution, which filtered out the de-

ails far away from the edge and preserved the main structure of

he target object. Then segmentation can be obtained by threshold-

ng. Numerical experiments performed on some challenging med-

cal images demonstrated the efficiency and effectiveness of our

ethod. We will apply our method to 3D segmentation in the near

uture.

cknowledgment

The first author is supported by National NSFC (No. 11301129 )

nd Zhejiang Provincial NSFC (No. LQ13A010025 ). The second au-

hor is supported in part by Hong Kong RGC GRF 12302715 . The

hird author is supported in part by Hong Kong RGC 211911 ,

2302714 ; the FRGs of Hong Kong Baptist University and NSFC

rant No. 11271049 . The authors would like to thank Prof. Ke Chen

rom University of Liverpool, Dr. Xiaohao Cai from University of

ambridge and Dr. Haider Ali from University of Peshawar for shar-

ng their Matlab codes. They also would like to thank Prof. Xiao-

un Zhang and Dr. Jiulong Liu from Shanghai Jiaotong University

or providing some experimental data.

eferences

[1] M. Kass , A. Witkin , D. Terzopoulos , Snakes: active contour models, Int. J. Com-

put. Vis. 1 (1988) 321–331 . [2] V. Caselles , R. Kimmel , G. Sapiro , Geodesic active contours, Int. J. Comput. Vis.

22 (1997) 61–79 . [3] Y. Xiang , A.C.S. Chung , J. Ye , An active contour model for image segmentation

based on elastic interaction, J. Comput. Phys. 219 (2006) 455–476 .

378 C. Liu et al. / Pattern Recognition 76 (2018) 367–379

[

[4] D. Mumford , J. Shah , Optimal approximations by piecewise smooth functionsand associated variational problems, Commun. Pure Appl. Math. 42 (1989)

577–685 . [5] C. Gout , C.L. Guyader , L. Vese , Segmentation under geometrical conditions with

geodesic active contour and interpolation using level set methods, Numer. Al-gorithms 39 (2005) 155–173 .

[6] N. Badshah , K. Chen , Image selctive segmentation under geometry constrainesusing an active contour approach, Commun. Comput. Phys. 7 (2009) 759–778 .

[7] J. Zhang , K. Chen , B. Yu , D.A. Gould , A local information based variational

model for selective image segmentation, Inverse Prob. Imaging 8 (2014)293–320 .

[8] L. Rada , K. Chen , A new variational model with dual level set functions forselective segmentation, Commun. Comput. Phys. 12 (2012) 261–283 .

[9] J. Peng , F. Dong , Y. Chen , D. Kong , A region-appearance-based adaptive varia-tional model for 3d liver segmentation, Med. Phys. 41 (4) (2014) . 043502.

[10] L. Mabood , H. Ali , N. Badshah , K. Chen , G.A. Khan , Active contours textural and

inhomogeneous object extraction, Pattern Recognit. 55 (2016) 87–99 . [11] S. Osher , J.A. Sethian , Fronts propagating with curvature-dependent speed: al-

gorithms based on Hamilton-Jacobi formulations, J. Comput. Phys. 79 (1988)12–49 .

[12] T.F. Chan , L.A. Vese , Active contour without edges, IEEE Trans. Image Process.10 (2001) 266–277 .

[13] L.A. Vese , T.F. Chan , A multiphase level set framework for image segmetation

using the Mumford and Shah model, Int. J. Comput. Vis. 50 (2002) 271–293 . [14] J. Lie , M. Lysaker , X.-C. Tai , A variant of the level set method and applications

to image segmentation, Math. Comput. 75 (2006) 1155–1174 . [15] S. Zhu , Q. Wu , C. Liu , Shape and topology optimization for elliptic boundary

value problems using a piecewise constant level set method, Appl. Numer.Math 61 (2011) 752–767 .

[16] S. Zhu , C. Liu , Q. Wu , Binary level set methods for topology and shape op-

timization of a two-density inhomogeneous drum, Comput. Methods Appl.Mech. Eng. 199 (2010) 2970–2986 .

[17] F. Li , M. Ng , T. Zeng , C. Shen , A multiphase image segmentation method basedon fuzzy region competition, SIAM J. Imaging Sci. 3 (2010) 277–299 .

[18] B. Mory , R. Ardon , Fuzzy region competition: a convex two-phase segmenta-tion framework, in: Scale Space and Variational Methods in Computer Vision,

2007, pp. 214–226 .

[19] T.F. Chan , S. Esedoglu , M. Nikolova , Algorithms for finding global minimizersof image segmentation and denoising models, SIAM J. Appl. Math. 66 (2006)

1632–1648 . [20] T. Pock , A. Chambolle , D. Cremers , H. Bischof , A convex relaxation approach

for computing minimal partitions, in: Processings of the IEEE Computer So-ciety Conference on Computer Vision and Pattern Recognition (CVPR), 2009,

pp. 810–817 .

[21] L. Grady , C. Alvino , Reformulating and optimizing the Mumford–Shah func-tional on a graph- a faster, lower energy solution, in: Proceedings of the Euro-

pean Conference on Computer Vision (ECCV), 2008, pp. 248–261 . [22] J. Yuan , E. Bae , X.-C. Tai , Y. Boykov , A continuous max-flow approach to Potts

model, in: ECCV 2010: Proceedings of the 11th European Conference on Com-puter Vision, Springer, Berlin, 2010 . 379-39.

[23] X. Cai , R. Chan , T. Zeng , A two-stage image segmentation method using a con-vex variant of the Mumford-Shah model and thresholding, SIAM J. Imaging Sci.

6 (2013) 368–390 .

[24] T. Goldstein , S. Osher , The split Bregman method for l1-regularized problems,SIAM J. Imaging Sci. 2 (2009) 323–343 .

[25] X. Zhang , M. Burger , S. Osher , A unified primal-dual algorithm frameworkbased on Bregman iteration, J. Sci. Comput. 46 (2011) 20–46 .

[26] X. Cai , Variational image segmentation model coupled with image restorationachievements, Pattern Recognit. 48 (2015) 2029–2042 .

[27] S.P. Awate , R.T. Whitaker , Multiatlas segmentation as nonparametric regression,

IEEE Trans. Med. Imaging 33 (2014) 1803–1817 .

[28] E.B. Dam , M. Lillhol , J. Marques , M. Nielsen , Automatic segmentation of high--and low-field knee MRIs using knee image quantification with data from the

osteoarthritis initiative, J. Med. Imaging 2 (2015) . 024001. [29] S.P. Awate , T. Tasdizen , R.T. Whitaker , Unsupervised texture segmentation

with nonparametric neighborhood statistics, Eur. Conf. Comput. Vis. (2006)494–507 .

[30] J. Kim , A. Yezzi , M. Cetin , A.S. Willsky , A nonparametric statistical method forimage segmentation using information theory and curve evolution, IEEE Trans.

Image Process. 14 (2005) 1486–1502 .

[31] J. Liu , X. Zhang , B. Dong , Z. Shen , L. Gu , A wavelet frame method withshape prior for ultrasound video segmentation, SIAM J. Imaging Sci. 9 (2015)

495–519 . [32] Z. Hui , H. Trevor , Regularization and variable selection via elastic net, J. R. Stat.

Soc. B 67 (2005) 301–320 . [33] A. Chambolle , Image segmentation by variational methods: Mumford and Shah

functional and the discrete approximations, SIAM J. Appl. Math. 55 (1995)

827–863 . [34] C. Liu , F. Dong , S. Zhu , D. Kong , K. Liu , New variational formulations for level

set evolution without reinitialization with applications to image segmentation,J. Math. Imaging Vis. 41 (2011) 194–209 .

[35] S. Esedoglu , Y. Tsai , Threshold dynamics for the piecewise constant Mum-ford-Shah functional, J. Comput. Phys. 211 (2006) 367–384 .

[36] Y. Zhao , L. Rada , K. Chen , S.P. Harding , Y. Zheng , Automated vessel segmenta-

tion using infinite perimeter active contour model with hybrid region infor-mation with application to retinal images, IEEE Trans. Med. Imaging 34 (2015)

1797–1807 . [37] F.A. Limberger , M.M. Oliveira , Real-time detection of planar regions in unorga-

nized point clouds, Pattern Recognit. 48 (2015) 2043–2053 . [38] H. Min , W. Jia , X.F. Wang , Y. Zhao , R.X. Hu , Y.T. Luo , F. Xue , J.T. Lu , An in-

tensity-texture model based level set method for image segmentation, Pattern

Recognit. 48 (2015) 1547–1562 . [39] H.K. Ji , Q.S. Sun , Z.X. Ji , Y.H. Yuan , G.Q. Zhang , Collaborative probabilistic labels

for face recognition from single sample per person, Pattern Recognit. 62 (2016)125–134 .

[40] X. Cai , Variational image segmentation model coupled with image restorationachievements, Pattern Recognit. 48 (2015) 2029–2042 .

[41] J. Hartigan , M. Wang , A k-means clustering algorithm: algorithm AS 136, Appl.

Stat. 28 (1979) 100–108 . [42] F. Lindsten , H. Ohlsson , L. Ljung , Just relax and come clustering! a convexifica-

tion of k-means clustering, Technical Report 2992, 2011 . [43] T. Nguyen , J. Cai , J. Zhang , J. Zheng , Robust interactive image segmentation us-

ing convex active contours, IEEE Trans. Image Process. 21 (2012) 3734–3743 . 44] I. Ekeland , R. Temam , Convex analysis and variational problems, Classics Appl.

Math. 28 (1999) . SIAM.

[45] G. David , Sigular sets of minimizers for the Mumford-Shah functional, Pro-gressin Mathematics 233, Birkhäuser, 2005 .

[46] D. Han , X. Yuan , W. Zhang , An augmented Lagrangian based parallel splittingmethod for separable convex minimization with applications to image process-

ing, Math. Comput. 83 (2014) 2263–2291 . [47] Y. Zhang , B. Dong , Z. Lu , l 0 minimization for wavelet frame based image

restoration, Math. Comput. 82 (2013) 995–1015 . [48] Y. Huang , D. Lu , T. Zeng , Two-step approach for the restoration of images cor-

rupted by multiplicative noise, SIAM J. Sci. Comput. 35 (2013) 2856–2873 .

[49] Y. Huang , M. Ng , Y. Wen , A new total variation method for multiplicative noiseremoval, SIAM J. Imaging Sci. 2 (2009) 20–40 .

[50] P. Getreuer , Rudin-osher-fatemi total variation denoising using split Bregman,Imaging Process. on Line 2 (2012) 74–95 .

[51] M. Nikolova , R. Chan , The equivalence of half-quadratic minimization andthe gradient linearization iteration, IEEE Trans. Image Process. 16 (2007)

1623–1627 .

[52] B. Dong , A. Chien , Z. Shen , Frame based segmentation for medical images,Commun. Math. Sci. 32 (2010) 1724–1739 .

C. Liu et al. / Pattern Recognition 76 (2018) 367–379 379

C , in 2006, and the Ph.D. degree in mathematics from Zhejiang University, China, in 2011. S . Her research interests include image processing, the variational methods, and scientific

c

M the University of Hong Kong, and the Ph.D. degree in 1995 from the Chinese University o Kong Baptist University. His research interests include bioinformatics, image processing,

s

T M.S. degree from Ecole Polytechnique, Palaiseau, France, and the Ph.D. degree from the U ining Hong Kong Baptist University as an Assistant Professor, he worked as a postdoctoral

r sing, statistical learning, and scientific computing.

hunxiao Liu received the B.S. degree in mathematics from Linyi University, Chinahe is a lecturer in department of mathematics at the Hangzhou Normal University

omputing.

ichael K. Ng received the B.Sc. degree in 1990 and the M.Phil. degree in 1992 fromf Hong Kong. He is a Professor in the Department of Mathematics at the Hong

cientific computing and data mining.

ieyong Zeng received the B.S. degree from Peking University, Beijing, China, the niversity of Paris XIII, Paris, France, in 20 0 0, 20 04, and 20 07, respectively. Before jo

esearcher in CMLA, ENS de Cachan, France. His research interests are image proces