Segmentation with Graph Cuts

Zhayida SimayijiangStefanie Grimm

Abstract

The aim of this project is to study graph cut meth-ods for segmenting images and investigate how theyperform in practice.

Keywords: Segmentation, Graph Cuts, Maxflow

1 Segmentation

Segmentation is an important part of image analysis.It refers to the process of partitioning an image intomultiple segments. More precisely, image segmenta-tion is the process of assigning a label to every pixelin an image such that pixels with the same label sharecertain visual characteristics. The goal of segmenta-tion is to simplify and/or change the representationof an image into something that is more meaningfuland easier to analyse.

Segmentation can be used for object recognition,occlusion boundary estimation within motion orstereo systems, image compression, image editing, orimage database look-up.

Segmentation by computing a minimal cut in agraph is a new and quite general approach for seg-menting images. This approach guarantees global so-lutions, which always find best solution, and in ad-dition these solutions are not depending on a goodinitialization. In our case the segmentation will bebased on the image gradient with seeds provided bythe user and on the mean intensity of an object.

2 Graph Theory

Graph theory is the study of graphs. A graph is anabstract representation of a set of objects, whereseveral pairs of the objects are connected by links.It is a mathematical structure and is used to modelpairwise relations between objects from a certaincollection.

To give a more mathematical description of agraph, we introduce some definitions:

In a graph G = (V, E), V and E denote the setof vertices and edges of G, respectively. A weightedgraph associates a positive label (weight) with everyedge in the graph. A directed graph G consists ofa set of vertices V and a set of ordered pairs of edges.An s-t graph is a weighted directed graph with two

identified nodes, the source s and the sink t. An s-tcut, c(s, t), in a graph G is a set of edges Ecut suchthat there is no path from the source to the sinkwhen Ecut is removed from G. The cost of a cut Ecut

is the sum of the edge weights in Ecut. The flowf(u, v) is a mapping f : E → R+, (u, v) 7→ f(u, v)which fulfills the conservation of flows and the weight

constraint [2]. The value of the flow is definedby ∣f ∣ =

∑v∈V f(s, v), where s is the source of the

graph. It represents the amount of flow passing fromthe source to the sink.

The maximum flow problem is to maximize ∣f ∣,that means to route as much flow as possible fromthe source to the sink. The minimum cut problemis to minimize c(S, T ) i.e. to find an s-t cut withminimal cost.

The max-flow min-cut theorem states: The max-imum value of an s-t flow is equal to the minimumweight of an s-t cut.

Our goal will be to segment an image by construct-ing a graph such that the minimal cut of this graphwill cut all the edges connecting the pixels of differentobjects with each other.

3 Segmentation with GraphCuts

The goal is to segment the main objects out of animage using a segmentation method based on graphcuts. We used MAXFLOW - software for computingthe mincut/maxflow of a graph. This software libraryimplements the maxflow algorithm described in ”AnExperimental Comparison of Min-Cut/Max-Flow Al-gorithms for Energy Minimization in Vision.”( YuriBoykov and Vladimir Kolmogorov).

A graph-based approach makes use of efficient solu-tions of the maxflow/mincut problem between sourceand sink nodes in directed graphs. To take advan-tage of this we generate a s-t-graph as follows: Theset of nodes is equal to the set of pixels in the image.Every pixel is connected with its d-neighbourhood(d = 4, 8). For the selection of weights, we have cho-sen two different models:

Segmentation based on the gradient ofan image

In our first case we use the fact, that sharp edgesin an image cause high gradient values. To distin-

guish between edges in the background and contoursof our object we give some input information aboutthe position of the object and the background. Thesets S, T ⊂ V (connected to the source and sink, re-spectively) denote user seeds painted in blue and red,respectively. (The choice of the colors simplifies therecognition but is rather arbitrary.) They are cho-sen from different regions of the image (i.e. they aredisjoint sets of pixels) and have a sufficiently largesize. The resulting s-t-min-cut then provides a glob-ally minimum cost cut between the sets of pixels in Sand T.

The weights of the graph are given as follows: Theweight have to be the way that the minimal cut goesalong the borders of the object we want to segment.Therefore the cost of a cut has to be high inside theobject or the background and low at the borders ofthe object. To guarantee this we first smoothed theimage with a Gaussian kernel to avoid sharp edgeswhich are not belonging to the main object. After-wards we computed the gradient of the image to de-tect the brinks. Based on this the weight for theedge between two pixels x1 and x2 becomes: w1,2 =

(M − ∣∣▽I(x1)∣∣+∣∣▽I(x2)∣∣2 )3, where M := max ∣∣I(xi)∣∣

and I is the smoothed, transformed into black andwhite and normalized double format representationof the input image. This choice assures that the dif-ference between the weights is large enough to workusing the maxflow function.

To segment the object, we set edges with very highweight from the source to the user seeds in the objectand from the background user seeds to the sink. Thisensures a flow via these edges in the maximal flowsolution. Applying the MAXFLOW - software givesthe minimal cut of this graph.

Segmentation based on the mean inten-sity

In this case we assume a characteristic mean value ofthe intensity for every object and the background. Tosimplify the computations we act on the assumptionthat there is only one object and we know its themean intensity �1 just as the mean intensity for thebackground �0. The main goal is now to find thepixels of the object by comparing their intensity to �1.Therefore we want to minimize the so called Energyfunction:E(Ω) = �⋅length(∂Ω)+

∑(I(xi)−�0)2+

∑(I(xi)−

�1)2

The sum of the first two terms are the data termwhile the last term acts as a regularization.

In order to apply graph theory to this minimiza-tion problem, we set an edge from every pixel to thesource and set the weights equal to (I(xi) − �1)2.Likewise we connect each pixel with the sink weightedby (I(xi) − �0)2 . The weights between a pixel andits neighbours is equal to � for all edges. A low �emphasises the data term while a high � stress thelength of the object boundary. Again applying the

MAXFLOW - software gives the minimal cut of thisgraph.

4 Examples

Segmentation based on the gradient ofan image

Information about the position and the shape of theobject and background are given by user seeds interms of blue and red labels in the image.

From our experiments, we know that this methodworks well in most images. In some images, the resultdepends highly on accurate user seeds (for example ifsharp edges exist in the background).

When we use 4-neighbor connectivity in the seg-mentation, in some images, we got sharp edges inthe result. A change to 8-neighbor connectivity givesmore smooth result. This is reasonable because the8-neighbor connectivity describe the pixels in imagesrelated to their 8 neighbours. These pixels are con-nected horizontally, vertically, and diagonally, thismeans in the segmentation they have a 4 more edgesto consider than 4-neighbor connectivity.

Segmentation based on the mean inten-sity

In our experiments, we only worked on the few images(mainly graylevel images), we know when � is verylow, the energy function based on only data term,which makes the result has more noise. When � isvery high, the energy function based on both regular-ization term and data term, which makes the resulthas few edges.

5 Conclusion

Segmentation based on graph cuts works very wellfor most of the images, for some issues it becomesmore laborious. This means if we want to base thesegmentation on the gradient of an image we needmore detailed user seeds if the boundaries of the ob-ject don’t differ clearly enough from the edges in thebackground. For the method based on the mean in-tensity we might need some tries to find the valuesfor �1, �0 and �.

Acknowledgement

The authors are grateful to Petter’s supervision of theproject and careful reading of the report and valuablecomments.

Figure 1: Our seeds and segmentation results

Figure 2: Compairing results for 4 and 8 neighbour-hood

Figure 3: Result for bad user seeds

Figure 4: Different result for lamda

(a) �=20 (b) �=2000

(c) �=20000 (d) �=200000

References

[1] http://en.wikipedia.org/wiki/Segmentation(image processing)

[2] http://en.wikipedia.org/wiki/Max-flow min-cut theorem

[3] http://www.cs.ucl.ac.uk/staff/V.Kolmogorov/software/maxflow-v3.0.src.tar.gz