Segment ácia farebného obrazu

SegmentSegmentácia farebného ácia farebného obrazuobrazu

Image segmentationImage segmentation


SegmentationSegmentation

• Segmentation means to divide up the image into a patchwork of regions, each of which is “homogeneous”, that is, the “same” in some sense - intensity, texture, colour, …

• The segmentation operation only subdivides an image;

• it does not attempt to recognize the segmented image parts!

Complete segmentation - divides an image into nonoverlapping regions that match to the real world objects.

Cooperation with higher processing levels which use specific knowledge of the problem domain is necessary.

Complete vs. partial Complete vs. partial segmentationsegmentation

Complete vs. partial Complete vs. partial segmentationsegmentation

Partial segmentation- in which regions do not correspond directly with image objects.

Image is divided into separate regions that are homogeneous with respect to a chosen property such as brightness, color, texture, etc.

GestaltGestalt (celostné) (celostné) laws of laws of perceptual organizationperceptual organization

The emphasis in the Gestalt approach was on the configuration of the elements.

Proximity: Objects that are closer to one another tend to be grouped

together.

Closure: Humans tend to enclose a space by completinga contour and ignoring gaps.

Gestalt laws of perceptual Gestalt laws of perceptual organizationorganization

Similarity: Elements that looksimilar will be perceived as partof the same form. (color, shape,

texture, and motion).

Continuation: Humans tendto continue contours

whenever the elements ofthe pattern establish an

implied direction.

Gestalt lawsGestalt laws A series of factors affect whether elements should be A series of factors affect whether elements should be

grouped together.grouped together. ProximityProximity: tokens that are nearby tend to be grouped.: tokens that are nearby tend to be grouped. SimilaritySimilarity: similar tokens tend to be grouped together.: similar tokens tend to be grouped together. Common fateCommon fate: tokens that have coherent motion tend to be : tokens that have coherent motion tend to be

grouped together.grouped together. Common regionCommon region: tokens that lie inside the same closed : tokens that lie inside the same closed

region tend to be grouped together.region tend to be grouped together. ParallelismParallelism: parallel curves or tokens tend to be grouped : parallel curves or tokens tend to be grouped

together.together.

• ClosureClosure: tokens or curves that tend to lead to closed: tokens or curves that tend to lead to closed curves tend to be grouped together.curves tend to be grouped together.• SymmetrySymmetry: curves that lead to symmetric groups: curves that lead to symmetric groups are are grouped together.grouped together.• ContinuityContinuity: tokens that lead to “continuous” curves tend to : tokens that lead to “continuous” curves tend to be grouped.be grouped.• Familiar configurationFamiliar configuration: tokens that, when grouped, lead to : tokens that, when grouped, lead to a familiar object, tend to be grouped together.a familiar object, tend to be grouped together.

Gestalt lawsGestalt laws




Segmentation criteria: a segmentation is a partition of Segmentation criteria: a segmentation is a partition of an image I into a set of regions S satisfying:an image I into a set of regions S satisfying:1.1. SSii = S = S Partition covers the Partition covers the

whole whole image.image.

2.2. SSii S Sjj = = , i , i j j No regions intersect.No regions intersect.

3.3. SSii, P(S, P(Sii) = true) = true Homogeneity predicate is Homogeneity predicate is

satisfied bysatisfied by each region.each region.

4.4. P(SP(Sii S Sjj) = false,) = false, Union of adjacent regionsUnion of adjacent regions

i i j, S j, Sii adjacent S adjacent Sjj does not satisfy it.does not satisfy it.


So, all we have to do is to define and implement So, all we have to do is to define and implement the similarity predicate.the similarity predicate.But, what do we want to be similar in each But, what do we want to be similar in each

region?region? Is there any property that will cause the Is there any property that will cause the

regions to be meaningful objects?regions to be meaningful objects?

Segmetnation methodsSegmetnation methods

Pixel-based• Histogram• Clustering

Region-based• Region growing

• Split and merge

Edge-based

Model-based

Physics-based

Graph-based

Histogram-based segmentationHistogram-based segmentation How many “orange” pixels areHow many “orange” pixels are

in this image?in this image? This type of question can be answeredThis type of question can be answered

by looking at the histogram.by looking at the histogram.

Histogram-based segmentationHistogram-based segmentation

How many modes are there?How many modes are there? Solve this by reducing the number of colors to K and Solve this by reducing the number of colors to K and

mapping each pixel to the closest color.mapping each pixel to the closest color. Here’s what it looks like if we use two colors.Here’s what it looks like if we use two colors.

Clustering-based segmentationClustering-based segmentation

How to choose the representative colors?How to choose the representative colors?This is a clustering problem!This is a clustering problem!

K-means algorithm can be used for clustering.K-means algorithm can be used for clustering.


K-means clustering of color.


K-means clustering of color.

Clustering-based segmentationClustering-based segmentationClustering can also be used with other Clustering can also be used with other

features (e.g., texture) in addition to color.features (e.g., texture) in addition to color.Original Images

Color Regions

Texture Regions


K-means variants:K-means variants:Different ways to initialize the means.Different ways to initialize the means.Different stopping criteria.Different stopping criteria.Dynamic methods for determining the right Dynamic methods for determining the right

number of clusters (K) for a given image.number of clusters (K) for a given image.Problem: histogram-based and clustering-Problem: histogram-based and clustering-

based segmentation can produce based segmentation can produce messy messy regions.regions.

How can these be fixed?How can these be fixed?


Expectation-Maximization (EM) algorithm can be used Expectation-Maximization (EM) algorithm can be used as a probabilistic clustering method where each cluster is as a probabilistic clustering method where each cluster is modeled using a Gaussian.modeled using a Gaussian.

The clusters are updated iteratively by computing the The clusters are updated iteratively by computing the parameters of the Gaussians.parameters of the Gaussians.

Example from the UC Berkeley’s Blobworld system.


Examples from the UC Berkeley’s Blobworld system.

Region growingRegion growing

Region growing techniques start with one pixel of a Region growing techniques start with one pixel of a potential region and try to grow it by adding adjacent potential region and try to grow it by adding adjacent pixels till the pixels being compared are too dissimilar.pixels till the pixels being compared are too dissimilar.

The first pixel selected can be just the first unlabeled pixel The first pixel selected can be just the first unlabeled pixel in the image or a set of seed pixels can be chosen from in the image or a set of seed pixels can be chosen from the image.the image.

Usually a statistical test is used to decide which pixels can Usually a statistical test is used to decide which pixels can be added to a region.be added to a region.

Region is a population with similar statistics.Region is a population with similar statistics.

Use statistical test to see if neighbor on border fits into Use statistical test to see if neighbor on border fits into the region population.the region population.


Let R be the N pixel region so far and p be Let R be the N pixel region so far and p be a neighboring pixel with gray tone y.a neighboring pixel with gray tone y.

Define the mean X and scatter SDefine the mean X and scatter S22 (sample (sample variance) byvariance) by

Rc)(r,

c)I(r,N1

X

2

Rc)(r,

2 X-c)I(r,N1

S


The T statistic is defined byThe T statistic is defined by

It has a TIt has a TN-1N-1 distribution if all the pixels in R distribution if all the pixels in R

and the test pixel and the test pixel pp are independent and are independent and identically distributed Gaussians identically distributed Gaussians (i(independentndependent assumption). assumption).

1/2

22 /S)X(p1)(N

1)N(NT


For the T distribution, statistical tables give us the For the T distribution, statistical tables give us the probability Pr(T probability Pr(T ≤≤ t) for a given degrees of freedom and a t) for a given degrees of freedom and a confidence level. From this, pick a suitable threshold t.confidence level. From this, pick a suitable threshold t.

If the computed T If the computed T ≤≤ t for desired confidence level, t for desired confidence level, add p add p to region Rto region R and update the mean and scatter using p. and update the mean and scatter using p.

If T is too high, the value p is not likely to have arisen If T is too high, the value p is not likely to have arisen from the population of pixels in R. from the population of pixels in R. Start a new regionStart a new region..


image

segmentation

Split-and-mergeSplit-and-merge

1.1. Start with the whole image.Start with the whole image.

2.2. If the variance is too high, break into quadrants.If the variance is too high, break into quadrants.

3.3. Merge any adjacent regions that are similar enough.Merge any adjacent regions that are similar enough.

4.4. Repeat steps 2 and 3, iteratively until no more splitting Repeat steps 2 and 3, iteratively until no more splitting or merging occur.or merging occur.

Idea: goodIdea: goodResults: blockyResults: blocky




A large connected region formed by

merging pixels labeled as residential after

classification.

A satellite image. More compact sub-regions after the split-and-merge procedure.

Model-based segmentation Model-based segmentation MMarkov arkov RRandom andom FFieldield model model

A lattice of sites = the pixels of an image S = {1,..., N}

The set of possible intensity values D = {1,...,d}An image viewed as a random variable

The set of class labels for each L = {1,...,l}pixel

A classification (segmentation) of the pixels in an image

MMarkov arkov RRandom andom FFieldield model model

Pairwise independencePairwise independence

Classification is independent of image and neighbouring pixels are independent of each other

Assume a parametric (eg. Gaussian) form for distribution of intensity given label l

Markovian assumptions: probability of class label idepends only on the localneighbourhood Ni

Graph-based segmentationGraph-based segmentation

An image is represented by a graph An image is represented by a graph whose nodes are pixels or small groups of whose nodes are pixels or small groups of pixels.pixels.

The goal is to partition the nodes into The goal is to partition the nodes into disjoint sets so that the similarity within disjoint sets so that the similarity within each set is high and across different sets each set is high and across different sets is low.is low.

http://www.cs.berkeley.edu/~malik/papers/SM-ncut.pdf


Let Let G = (V,E)G = (V,E) be a graph. Each be a graph. Each edge (u,v)edge (u,v) has a has a weight w(u,v)weight w(u,v) that represents the similarity that represents the similarity between u and v.between u and v.

Graph G can be broken into 2 disjoint graphs Graph G can be broken into 2 disjoint graphs with node sets A and B by removing edges that with node sets A and B by removing edges that connect these sets.connect these sets.

Let cut(A,B) = Let cut(A,B) = w(u,v). w(u,v).

One way to segment G is to find the One way to segment G is to find the minimal cutminimal cut..

uA, vB



Minimal cut favors cutting off small node Minimal cut favors cutting off small node groups, so Shi and Malik proposed the groups, so Shi and Malik proposed the normalized cut.normalized cut.

cut(A,B) cut(A,B)Ncut(A,B) = --------------- + --------------- assoc(A,V) assoc(B,V)

assoc(A,V) = w(u,t) uA, tV

How much is A connectedto the graph as a whole

Normalizedcut


2

2 2

2 2

41 3

2

2 2

3

2

2

2

1

3 3Ncut(A,B) = ------- + ------ 21 16

A B


Shi and Malik turned graph cuts into an Shi and Malik turned graph cuts into an eigenvector/eigenvalue problem.eigenvector/eigenvalue problem.

Set up a weighted graph G=(V,E).Set up a weighted graph G=(V,E). VV is the set of (N) pixels. is the set of (N) pixels. EE is a set of weighted edges (weight w is a set of weighted edges (weight w ijij gives the gives the

similarity between nodes i and j).similarity between nodes i and j). Length N vector Length N vector dd: d: dii is the sum of the weights from is the sum of the weights from

node i to all other nodes.node i to all other nodes. N x N matrix N x N matrix DD: D is a diagonal matrix with d on its : D is a diagonal matrix with d on its

diagonal.diagonal. N x N symmetric matrix N x N symmetric matrix WW: W: Wijij = w = wijij..


Let x be a characteristic vector of a set A of nodes.Let x be a characteristic vector of a set A of nodes. xxii = 1 if node i is in a set A = 1 if node i is in a set A

xxii = -1 otherwise = -1 otherwise Let y be a continuous approximation to xLet y be a continuous approximation to x

Solve the system of equationsSolve the system of equations(D – W) y = (D – W) y = D y D y

for the eigenvectors y and eigenvalues for the eigenvectors y and eigenvalues .. Use the eigenvector y with second smallest eigenvalue to Use the eigenvector y with second smallest eigenvalue to

bipartition the graph (y bipartition the graph (y x x A). A). If further subdivision is merited, repeat recursively.If further subdivision is merited, repeat recursively.


Edge weights w(i,j) can be defined byEdge weights w(i,j) can be defined by

wherewhere X(i) is the spatial location of node IX(i) is the spatial location of node I F(i) is the feature vector for node IF(i) is the feature vector for node I

which can be intensity, color, texture, motion…which can be intensity, color, texture, motion…

The formula is set up so that w(i,j) is 0 for nodes The formula is set up so that w(i,j) is 0 for nodes that are too far apart.that are too far apart.

otherwise

rjXiXifeejiwX

I

jXiXjFiF

2/)()(

/)()( )()(

0),(

22

22


Documents

Segment ácia farebného obrazu