SEGMENTATION OF

FOREGROUND – BACKGROUND

FROM NATURAL IMAGES

B YAJAL.A.JASSISTANT PROFESSOR UNIVERSAL ENGINEERING COLLEGE

OUTLINE

Introduction Types of segmentation algorithms Evaluations of RGB Color space SEGMENTATION EXPERIMENTAL RESULTS Summary Appendix

ABSTRACT

This paper presents a part of a more challenging research project aimed at developing a computer vision system for a robot capable of identifying all objects from known natural backgrounds such as forest, sky, ocean, under-water scenes and etc.

Segmentation is an import issue in the field of machine vision for detection and recognition of objects.

The success of segmentation is solely depends on the separation of foreground objects from background objects.

We present a simple framework to extract the foreground objects from the known natural backgrounds in still and moving images using pixel based color segmentation in RGB space.

What is an Image?

2D array of pixels Binary image (bitmap)

Pixels are bits Grayscale image

Pixels are scalars Typically 8 bits (0..255)

Color images Pixels are vectors Order can vary: RGB,

BGR Sometimes includes Alpha

What is an Image?

2D array of pixels Binary image (bitmap)

Pixels are bits Grayscale image

Pixels are scalars Typically 8 bits (0..255)

Color images Pixels are vectors Order can vary: RGB,

BGR Sometimes includes Alpha

What is an Image?

2D array of pixels Binary image (bitmap)

Pixels are bits Grayscale image

Pixels are scalars Typically 8 bits (0..255)

Color images Pixels are vectors Order can vary: RGB,

BGR Sometimes includes Alpha

What is an Image?

2D array of pixels Binary image (bitmap)

Pixels are bits Grayscale image

Pixels are scalars Typically 8 bits (0..255)

Color images Pixels are vectors Order can vary: RGB,

BGR Sometimes includes Alpha

What is an Image?

2D array of pixels Binary image (bitmap)

Pixels are bits Grayscale image

Pixels are scalars Typically 8 bits (0..255)

Color images Pixels are vectors Order can vary: RGB,

BGR Sometimes includes Alpha

HSV VS RGB.

In day to day practice, we'll most likely use two models:

HSV and RGB.

HSV stands for

Hue,Saturation, andValue, and it uses these three concepts to describe a color.

RGB the three colors that make up an image on a monitor.

RGB Color cube

Color segmentation

In the problem of segmentation, the goal is to separate spatial regions of an image on the basis of similarity within each region and distinction between different regions.

Approaches to color-based segmentation range from empirical evaluation of various color spaces, to clustering in feature space , to physics-based modeling

The essential difference between color segmentation and color recognition is that the former uses color to separate objects without a priori knowledge about specific surfaces; the latter attempts to recognize colors of known color characteristics

Segmentation: Elephant and Blind Men Syndrome

SEGMENTATION

Segmented image – giving us the outline of her face, hand etc

Colour Image having a bimodal histogram

Results on color segmentation

SEGMENTATION

Partitioning images into meaningful pieces, e.g. delineating regions of anatomical interest.

Edge based – find boundaries between regions

Pixel Classification – metrics classify regions Region based – similarity of pixels within a

segment

minimum cut

“allegiance” = cost of assigning two nodes to different layers (foreground versus background)

foregroundnode

backgroundnode

pixel nodes

allegiance to foreground

allegiance to background

pixel-to-pixelallegiance

minimum cut

“allegiance” = cost of assigning two nodes to different layers (foreground versus background)

foregroundnode

backgroundnode

pixel nodes

allegiance to foreground

allegiance to background

pixel-to-pixelallegiance

Normalized Cuts

• Graph partitioning technique

• Bi-partitions an edge-weighted graph in an optimal sense

• Normalized cut (Ncut) is the optimizing criterion

i j

wij

Edge weight => Similarity between i and j A B

Minimize Ncut(A,B)

Nodes

• Image segmentation

• Each pixel is a node

• Edge weight is similarity between pixels

• Similarity based on color, texture and contour cues

21

Unknown clusters and centers

Maximization step:Find the center (mean)

of each class

Start with random model parameters

Expectation step:Classify each vectorto the closest center

22

Finding the centers from known clustering

Segmentation fault

A segmentation fault (often shortened to

segfault) or access violation is a particular error condition that can occur during the operation of computer software.

A segmentation fault occurs when a program attempts to access a memory location that it is not allowed to access, or attempts to access a memory location in a way that is not allowed (for example, attempting to write to a read-only location, or to overwrite part of the operating system).

Segmentation Methods

Thresholding approaches Region Growing approaches Classifiers Clustering approaches Markov random fields (MRF) models Artificial neural networks Deformable models Atlas-guided approaches

24

Thresholding

Suppose that an image, f(x,y), is composed of light objects on a dark background, and the following figure is the histogram of the image.

Then, the objects can be extracted by comparing pixel values with a threshold T.

25

Region Growing

1. Define seed point

2. Add n-neighbors to list L

3. Get and remove top of L

4. Test n-neighbors pif p not treatedif P(p,R)=True then p→L and add p to region else p marked boundary

5. Go to 2 until L is empty Two Regions R and ¬ R

Seed pointsSeed points Element in Element in L

Border elementBorder elementRegion elementRegion element

Our approach: The Algorithm The left and right images areThe left and right images are

segmented and each area segmented and each area identifies a node of a graphidentifies a node of a graph

A bipartite graph matching A bipartite graph matching between the two graphs is between the two graphs is computed in order to match each computed in order to match each area of the left image with only area of the left image with only one area of the right imageone area of the right image This process yields a list of This process yields a list of reliably matched areas and a list reliably matched areas and a list of so-called don’t care areas.of so-called don’t care areas.

The Outputs of the algorithm The Outputs of the algorithm are the disparity map and the are the disparity map and the performance mapperformance map

GPCAGeneralized Principal Component Analysis (GPCA)

method for.

modeling and segmenting mixed data using a collection of subspaces

done by introducing certain algebraic models into data clustering.

Unique property (applied to images) is that it decomposes images into regions with fundamentally different characteristics and derives an optimal PCA-based transformation for each region.

Computing a principal component analysis

To compute a principal component analysis in SPSS, select the Data Reduction | Factor… command from the Analyze menu.

Segmentation Example

Intelligent Scissors

Fully automatic segmentation is an unsolved problem due to wide variety of images.

Intelligent Scissors is a semi-automatic general purpose segmentation tool.

The efficient and accurate boundary extraction, which requires minimal user input with a mouse, is obtained.

The underlying mechanism for the Intelligent Scissors is the “live-wire” path selection tool.

More Complex Segmentation Methods - snakes

One More Thing

VLSI IMPLEMENTATION

Floor plan of the prototype chip

Layout of the encoder module

Pros & Cons Very useful for rapid prototyping Strongly growing community and code base

Problems: Very complex Overhead -> higher run-times Still under development

Summary / Closing Thoughts

Segmentation is the essential but critical problem in the field of machine vision. At a stretch, robotics can not be done with a complete knowledge about foreground and background objects.

We have proposed pixel based color segmentation approach to segment the known backgrounds such as forest, sky, ocean, underwater scenes and etc. which will be of unique color generally and the results obtained were satisfactory.

This color segmentation process will overcome the main problems with change of pose and occlusion and overcomes the limitation occurs in the motion analysis and background subtraction methods.

Conclusions

Translation (visual to semantic) model for object recognition

Identify and evaluate low-level vision processes for recognition

Feature evaluation

Color and texture are the most important in that order

Shape needs better segmentation methods

Segmentation evaluation

Performance depends on # regions for annotation

Mean Shift and modified NCuts do better than original NCuts for # regions < 6

Color constancy evaluation

Training with illumination helps

Color constancy processing helps (scale-by-max better than gray-world)

Reference Reading

Digital Image ProcessingGonzalez & Woods,Addison-Wesley 2002

Computer VisionShapiro & Stockman,Prentice-Hall 2001

Computer Vision: A Modern ApproachForsyth & Ponce,Prentice-Hall 2002

Introductory Techniques for 3D Computer VisionTrucco & Verri,Prentice-Hall 1998

REFERENCES :

S. Belongie, C. Carson, H. Greenspan, and J. Malik, "Color and texture-based image segmentation using EM and its application to content-based image retrieval," 6th International Conference on Computer Vision, pp.675–682, 1998.

E. Saber, A.M. Tekalp, R. Eschbach, and K. Knox, "Automatic image annotation using adaptive color classification," Graph. Models Image Process., vol.58, no.2, pp.115–126, 1996.

S.C. Pei and C.M. Cheng, "Extracting color features and dynamic matching for image data-base retrieval," IEEE Trans. Circuits Syst. Video Technol., vol.9, no.3, pp.501–512, April 1999.

T. Pavlidis and Y.-T. Liow, "Integrating region growing and edge detection," IEEE Trans. Pattern Anal. Mach. Intell., vol.12, no.3, pp.225–233, March 1990.

C.-C. Chu and J.K. Aggarwal, "The integration of image segmentation maps using region and edge information," IEEE Trans. Pattern Anal. Mach. Intell., vol.15, no.12, pp.1241–1252, Dec. 1993.

J. Fan, D.K.Y. Yau, A.K. Elmagarmid, and W.G. Aref, "Automatic image segmentation by integrating color-edge extraction and seeded region growing," IEEE Trans. Image Process., vol.10, no.10, pp.1454–1466, Oct. 2001.

QUERRIES ?

Thank you.

AJAL.A.JASSISTANT PROFESSORUNIVERSAL ENGINEERING COLLEGETHRISSUR

MAIL : ec2reach@gmail.com MOB : 0890 730 5642