Segmentation via Graph Cuts

Segmentation via Graph Cuts

Balakrishna Thiagarajan

Computer Science and Engineering

State University of New York at Buffalo

Introduction Gestalt approach

(From) Normalized Cuts and Image Segmentation

Image Segmentation Segmentation refers to the process of

partitioning a digital image into multiple segments.

The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze.

Similarity (gray level difference), Proximity and continuity are important factors in segmentation.

Graph representation of Image

Set of points of the feature space represented as a weighted, undirected graph, G = (V, E).

The points of the feature space are the nodes of the graph.

Edge between every pair of nodes.

Graph representation of Image

Weight on each edge, w(i, j), is a function of the similarity between the nodes i and j.

Partition the set of vertices into disjoint sets where similarity within the sets is high and across the sets is low.

Therefore, Segmentation is equivalent to Graph partition.

Graph representation of ImageCut: A cut in a graph is a set of edges whose

removal disconnects the graph.

A minimum cut is a cut with a minimum number of edges. It is denoted by S. In the weighted graph,

cut(A,B) = ∑ w(u,v) u E A,v E B

where A & B are two partitioned disjoint sets

Segmentation by Min Cut method

By Min Cut method, the graph is partitioned into clusters.

Each cluster is considered as an image segment.

Min Cut method uses the HCS (Highly Connected Subgraphs) Algorithm to find the clusters.

Segmentation by Min Cut method

Edge Connectivity is the minimum number of edges whose removal results in a disconnected graph. It is denoted by k(G).

For a graph with vertices n > 1 to be highly connected if its edge-connectivity k(G) > n/2.

A highly connected subgraph (HCS) is an induced subgraph H in G such that H is highly connected.

Segmentation by Min Cut methodExample:

No. of nodes = 5 Edge Connectivity = 1

Not HCS!

A B

D C E

Segmentation by Min Cut methodExample continued:

No. of nodes = 4 Edge Connectivity = 3

HCS!

A B

D C

HCS Algorithm - Min Cut methodHCS(G(V,E))

begin

(H, H’,C) MINCUT(G)

if G is highly connected

then return (G)

else

HCS(H)

HCS(H’)

end if

end

HCS Algorithm - Min Cut method The procedure MINCUT(G) returns H, H’ and

C where C is the minimum cut which separates G into the subgraphs H and H’.

Procedure HCS returns a graph in case it identifies it as a cluster.

Single vertices are not considered clusters and are grouped into singletons set S.

Example

HCS Algorithm - Min Cut method

Example Continued

HCS Algorithm - Min Cut method

Example Continued

Cluster 2

Cluster 1

Cluster 3

HCS Algorithm - Min Cut method

HCS Algorithm - Min Cut method The HCS Algorithm returns clusters which are

nothing but image segments.

The problem with HCS Algorithm – Min Cut method is that there is a chance that many singletons may be separated.

To overcome this we use normalized-cut for image segmentation.

HCS Algorithm – Main ProblemExample

HCS Algorithm – Main ProblemExample – Another possible cut

Max Flow - Min Cut Method Theorem

In graph G, the maximum source-to-sink flow possible is equal to the capacity of the minimum cut in G.

(L. R. Foulds, Graph Theory Applications, 1992 Springer-Verlag New York Inc., 247-248)

Max Flow - Min Cut Method Min Cut can be made by using Ford-Fulkerson

Algorithm to find the max-flow in the graph. In the graph temporarily add Source ‘s’ and

Sink ‘t’. Consider the weight associated with each each

node as a flow capacity. Starting from zero flow, increase the flow

gradually by finding a path from s to t along which more flow can be sent, until a max-flow is achieved.

Max Flow - Min Cut Method

1

2

2

2

3

22

s t

1

2

11

3

22

s t

1

2

2

1

2

21

s t

1/1

1/2

2/2

1/2

1/3

1/22/2

s t

f = 0

f = 1

f = 2

f = 3

1

1

1

1

1

1

1

2

1

2

12

s t

f = 3

1

1

1

1

0/1

0/2

0/2

0/2

0/3

0/20/2

s t

f = 0

(a)

(b)

(c) (f)

(e)

(d)

Max Flow - Min Cut Method After the max-flow is found, the minimum cut

is by

S = {All vertices reachable from s}

T = G \ S

1

1

2

1

2

12

s t1

1

1

1

Max Flow - Min Cut Method - Illustration

(From) Image Segmentation Using Minimal Graph Cuts by Anders P. Eriksson, Olof Barr and Kalle Astrom�

Normalized Cut Normalized Cut (Ncut) :

Ncut(A,B) = cut(A,B) + cut(A,B)

assoc(A,V) assoc(B,V)

where, assoc(A,V) = ∑ w(u,t) u E A, t E V

The cut is made such that the normalized cut value between the two sets A & B is minimum.

Normalized Cut

Minimum cut criteria favors cutting small sets of isolated nodes in the graph

(From) Normalized Cuts and Image Segmentation –

Shi & Malik

Normalized Cut

Exact solution to minimizing normalized cut is an NP-complete problem.

However, approximate discrete solutions can be found efficiently.

Normalized cut criterion can be computed efficiently by solving a generalized eigenvalue problem.

Normalized Cut – Physical Interpretation Think of the weighted graph as a spring mass

system. Graph nodes physical masses Graph edges springs Graph edge weight spring stiffness Total incoming edge weights mass of the

node

Normalized Cut – Physical Interpretation Imagine giving a hard shake to this spring-

mass system, forcing the nodes to oscillate in the direction perpendicular to the image plane.

Nodes that have stronger spring connections among them will likely oscillate together.

Eventually, the group will “pop” off from the image plane.

Normalized Cut

(From) Normalized Cuts and Image Segmentation - Shi & Malik

Average Cut

Average Cut: Averagecut(A,B) = cut(A,B) + cut(A,B) |A| |B|

The average cut measure is proportional to the total link weight cut, normalized by the sizes of the partitions.

The problem with Average Cut is that one cannot ensure the two partitions computed will have tight within-group similarity.

Comparison Different types of graph cuts - MinCut,

Average Cut, and Normalized Cut were analysed by Padmanabhan Soundararajan and Sudeep Sarkar.

Totally 50 images were used. 5 different image classes with 10 images in each class.

Comparison The 5 classes of images were

1. natural objects in indoor backgrounds

2. natural objects in outdoor backgrounds

3. man-made objects in indoor backgrounds

4. man-made objects in outdoor backgrounds

5. aerial images that contained man-made objects in natural surroundings.

Comparison The best possible performances on each

image for the three different cut measures were not significantly different from each other.

The performance with the three measures is worst for man made objects in indoor surroundings and for aerial images.

The normalized cut based partition took significantly larger time to compute than the average and the mincut.

References

1. J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.

2. Erez Hartuv and Ron Shamir, “A Clustering Algorithm based on Graph Connectivity”, Mar. 1999.

3. Padmanabhan Soundararajan and Sudeep Sark, “Analysis of MinCut, Average Cut, and Normalized Cut Measures”, May 2000.

4. Anders P. Eriksson, Olof Barr and Kalle Astrom,” Image Segmentation Using Minimal Graph Cuts”.

Thank You!!