1 | P a g e

BY-KAMAL PRADHAN

2 | P a g e

ABSTRACT

Image processing is a form of signal processing in which the input is an image,

such as a photograph or video frame. The output of image processing may be

either an image or, a set of characteristics or parameters related to the image.

Most image-processing techniques involve treating the image as a two-

dimensional signal and applying standard signal-processing techniques to it. This

project aims at processing the real time images captured by a Webcam for motion

detection and Color Recognition and system automation using MATLAB

programming.

3 | P a g e

INDEX 1) INTRODUCTION 6 2) PROJECT REVIEW

a. Project Management…………………………………………………………………..9

b. Color Based Image Processing, Tracking, Automation……………………………..10

3) RESOURCES a. System Development Life Cycle…………………………………………………….11

b. Scripting Language Selection………………………………………………………..14

c. Library Selection……………………………………………………………………..17

4) THE PSYCHOPHYSICS OF COLOR 18

5) COLOR MODELS 22 a. RGB Color Model…………………………………………………………………....24

b. HSV Color Model…………………………………………………………………....27

6) COLOR SPACE CONVERSION 31 a. RGB to HSV………………………………………………………………………....32

b. HSV to RGB…………………………………………………………………………33

c. HSV to BINARY…………………………………………………………………….35

7) FLOW DIAGRAM

a. Flow Diagram for Color Detection……………………………………………………....36

8) COLOR BASED TRACKING 37

a. Hardware Detection and Selection of Acquisition Tool…………………………………37

b. Setting Up the Acquisition Tool Properties ………………………………......................41

c. Image Acquisition through Connected Device……………………………......................42

d. Defining Color Detection Parameters……………………………………………………42

4 | P a g e

e. Extraction of Frame……………………………………………………………………...42

f. Color Space Transformation, Finding the Connected Component and Calculation of the

Centroid of the Detected Object…………………………………………………………43

g. Position a Circle Around the Object and Display The frame…………………………...46

9) SYSTEM AUTOMATION 47

a. Java Robot Classes………………………………………………………………………49

b. Java Robot Classes and MATLAB……………………………………………………...51

10) CODES

11) CONCLUSION 55

12) BIBILIOGRAPHY 56

5 | P a g e

INTRODUCTION Computer vision, Artificial intelligence and Digital image processing are related

to each other computer vision gives the power of vision to our computer Digital

image processing is used to enhance the vision and Artificial intelligence uses this

vision to make the computer systems able to perform tasks that normally require

human intelligence such as visual perception, object recognition.

Computer vision is a field that includes methods for acquiring, processing, analyzing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions. A theme in the development of this field has been to duplicate the abilities of human vision by electronically perceiving and understanding an image. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory. Computer vision has also been described as the enterprise of automating and integrating a wide range of processes and representations for vision perception.

Computer Vision is the science and technology of obtaining models, meaning and control information from visual data. The two main fields of computer vision are computational vision and machine vision. Computational vision has to do with simply recording and analyzing the visual perception, and trying to understand it. Machine vision has to do with using what is found from computational vision and applying it to benefit people, animals, environment, etc.

Applications range from tasks such as industrial machine vision systems which, say, inspect bottles speeding by on a production line, to research into artificial intelligence and computers or robots that can comprehend the world around them. The computer vision and machine vision fields have significant overlap. Computer vision covers the core technology of automated image analysis which is used in many fields. Machine vision usually refers to a process of combining automated image analysis with other methods and technologies to provide automated inspection and robot guidance in industrial applications.

6 | P a g e

As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner.

Computer Vision has influenced the field of Artificial Intelligence greatly. The Robocup tournament and ASIMO are examples of Artificial Intelligence using Computer Vision to its greatest extent. The Robocup tournament is a tournament for robot dogs playing soccer. To be able to play soccer, these dogs must be able to see the ball, and then react to it accordingly. Engineers of these robot dogs have been challenged to create robot dogs that can beat the best soccer players at soccer in around fifty years.

Artificial Intelligence also uses computer vision to recognize handwriting text and drawings. Text typed down on a document can be read by the computer easily, but handwritten text cannot. Computer vision fixes this by converting handwritten figures into figures that can be used by a computer. An example is shown below. The attempted drawing of a rectangular prism resting on three other rectangular prism is converted by computer vision to a 3-D picture of the same thing, but in a format usable by the computer and more readable by users.

Image processing is often viewed as arbitrarily manipulating an image to achieve an aesthetic standard or to support a preferred reality. However, image processing is more accurately defined as a means of translation between the human visual system and digital imaging devices. The human visual system does not perceive the world in the same manner as digital detectors, with display devices imposing additional noise and bandwidth restrictions. Salient differences between the human and digital detectors will be shown, along with some basic processing steps for achieving translation. Image processing must be approached in a manner consistent with the scientific method so that others may reproduce, and validate, one's results. This includes recording and reporting processing actions, and applying similar treatments to adequate control images.

7 | P a g e

Digital Image processing is in many cases concerned with taking one array of pixels as input and producing another array of pixels as output which in some Way represents an improvement to the original array. Purpose: 1. Improvement of Pictorial Information • improve the contrast of the image, • remove noise, • remove blurring caused by movement of the camera during image acquisition, • it may correct for geometrical distortions caused by the lens. 2. Automatic Machine perception (termed Computer Vision, Pattern Recognition or Visual Perception) for intelligent interpretation of scenes or pictures.

8 | P a g e

Project management

Project management is the discipline of planning, organizing, motivating, and

controlling resources to achieve specific goals. A project is a temporary endeavor

with a defined beginning and end (usually time-constrained, and often

constrained by funding or deliverables), undertaken to meet unique goals and

objectives, typically to bring about beneficial change or added value. The

temporary nature of projects stands in contrast with business as usual (or

operations), which are repetitive, permanent, or semi-permanent functional

activities to produce products or services. In practice, the management of these

two systems is often quite different, and as such requires the development of

distinct technical skills and management strategies.

The primary challenge of project management is to achieve all of the project goals

and objectives while honoring the preconceived constraints. The primary

constraints are scope, time, quality and budget. The secondary —and more

ambitious— challenge is to optimize the allocation of necessary inputs and

integrate them to meet pre-defined objectives.

Project management skills are put to good use for this project. Project

management focuses on achieving the objectives by applying five processes

presented in Figure below.

9 | P a g e

Color Based Image Processing, Tracking

and Automation.

In color based image processing we work with colors instead of object. Color provides powerful information for object recognition. A simple and effective recognition scheme is to represent and match images on the basis of color histograms.

Color based image processing includes Color space transformations

Histogram processing Color edge detection

Noise reduction Color segmentation

Color image sharpening Tracking refers to detection of the path of the color once the color based

processing is done the color becomes the object to be tracked this can be very

helpful in security purposes.

Automation refers to an automated system is any system that does not require

human intervention. In this project I’ve automated the mouse that work with our

gesture and do the desired tasks.

10 | P a g e

Software Development Life Cycle (SDLC)

Software Development Life Cycle (SDLC) is the most common process adopted to

develop a project and not surprisingly, this project is following this model too.

As with most undertakings, planning is an important factor in determining the

success or failure of any software project. Essentially, good project planning will

eliminate many of the mistakes that would otherwise be made, and reduce the

overall time required to complete the project. As a rule of thumb, the more

complex the problem is, and the more thorough the planning process must be.

Most professional software developers plan a software project using a series of

steps generally referred to as the software development life cycle. A number of

models exist that differ in the number of stages defined, and in the specific

activities that take place within each stage. The following example is a generic

model that should give you some idea of the steps involved in a typical software

project.

A generic software development life

cycle

11 | P a g e

Analysis of user requirements

During this stage, the problem is defined so that a clear understanding can be

gained of what the system should do, i.e. what the inputs to the system are, what

the output should be, and the operational parameters within which the system is

expected to work. If the new system is to replace an existing system, the problem

may be defined in terms of the additional or enhanced functionality that is

required.

Program design

In this stage, a solution to the problem is designed by defining a logical sequence

of steps that will achieve each of the stated system objectives. Such a sequence of

steps is often referred to as an algorithm. Some of the methods used to define

program algorithms are described later in this section, and include flowcharts and

pseudo code. These tools allow the program designer to break a given problem

down into a series of small tasks which the computer can perform to solve the

problem. The user interface will also be designed during this stage, and will

determine how input is obtained, how output is displayed, and what controls are

available to the user.

Program coding

This stage, sometimes known as the implementation stage, is where the

algorithms are translated into a programming language, and tends to be the

longest phase of the development life-cycle. In this case, we are using Visual Basic

to write the program.

12 | P a g e

Documentation and testing

The documentation of the program fulfils two main objectives. The first is to

provide a technical reference to facilitate ongoing maintenance and development

of the software itself. The second is to provide user documentation, i.e. a set of

instructions that inform the user about the features of the software and how to

use them. The aim of software testing is to find any errors ("bugs") in the

program, to eliminate those errors (a process known as "debugging"), and as far

as is reasonably practicable should be sufficiently rigorous to ensure that the

software will function as expected under all foreseeable circumstances.

Operating and maintaining the system

Once the software has been "rolled out" and any necessary user training has been

completed, it will be necessary to monitor the performance of the system over

time to ensure that it is behaving as expected. The system will need to be

maintained, and parts of it will need to be upgraded from time to time to handle

evolving user needs or to cope with new problems. Eventually, as the system

ages, it may no longer be able to adequately cope with the demands of a growing

number of users, take advantage of advances in hardware technology, or adapt to

a constantly changing environment. When this time comes, the system will need

to be decommissioned and replaced by a new system. Hence, the software

development life cycle will begin again.

13 | P a g e

Scripting Language Selection

There are many scripting languages available in the market. So it is important

choose the language you know better and that is closer to your development

experience. This will decrease the test creation time, since you will not have to

learn a new language. The rest of this topic contains answers to some typical

questions that users ask when selecting a scripting language.

Computers are increasingly used for a variety of purposes in engineering and

Science including control, data analysis, simulations and design optimization. It Is

therefore becoming more important for engineering students to have a robust

Understanding of computing and to learn how to program. Languages such as

Matlab, C, C++, python and VB.net are commonly used for scripting and

developing software’s.

The C programming language is a low-level Compiled language (sometimes

classified as a 3rd generation language) that is widely used in academia, industry

and commerce. FORTRAN falls into the same Category but while FORTRAN is still

commonly used in academia it appears to be overtaken by C (and C++) in many

industrial applications. C++ provides a Different programming paradigm than C

but for the purpose of this work, C++ is more similar to C than it is to MATLAB or

Python. The main advantage of compiled low-level languages is their execution

speed and efficiency (for example in embedded systems).

MATLAB stands for "Matrix Laboratory" and is a numerical computing

environment and fourth-generation programming language. Developed by Math

Works, MATLAB allows matrix manipulations, plotting of functions and data,

implementation of algorithms, creation of user interfaces, and interfacing with

programs written in other languages, including C, C++, and Fortran.

Although MATLAB is intended primarily for numerical computing, an optional

toolbox uses the MuPAD symbolic engine, allowing access to symbolic computing

14 | P a g e

capabilities. An additional package, Simulink, adds graphical multi-domain

simulation and Model-Based Design for dynamic and embedded systems.

MATLAB is a widely used tool in the electrical engineering community.

It can be used for simple mathematical manipulations with matrices, for

understanding and teaching basic mathematical and engineering concepts, and

even for studying and simulating actual power systems and electric systems in

general.

The original concept of a small and handy tool has evolved to become an

engineering workhorse. It is now accepted that MATLAB and its numerous

Toolboxes can replace and/or enhance the usage of traditional simulation tools

for advanced engineering applications.

This course introduces the benefits of using MATLAB to electrical engineers

involved in the simulation and analysis of power systems and power electronics

circuits. More explicitly, this course will demonstrate the advantages of using

MATLAB for analyzing power system steady-state behavior and its capabilities for

simulating transients in power systems and power electronics, including control

system dynamic behavior.

Matlab in not only a programming language, but a programming environment as

well. You can perform operations from the command line, as a sophisticated

calculator. Or you can create programs and functions that perform repetitive

tasks, just as any other computer language.

Python is another high-level language and at first sight very similar to MATLAB: it

is interpreted, has an interactive prompt, allows dynamic typing and provides

automatic memory management (and comes with in-built complex numbers). We

have included Python in this work because it provides several advantages over

15 | P a g e

MATLAB in the context of teaching: (i) Python has a very clear, unambiguous and

intuitive syntax and uses indentation to group blocks of statements. (ii) Python

has a small core of commands which provide nearly all the functionality beginners

will require. (iii) Python can be used as a fully object-orientated language and

supports different styles of coding. (iv) The Python interpreter is free software

(i.e. readily available), and Python interpreters for virtually all platforms exist

(including Windows, Linux/Unix, Mac OS).

In conclusion, MATLAB is the preferred selection due to the ease of usage and it can be uploaded and run on another platform with minimal change required to be done to the script.

16 | P a g e

Library Selection

All our cameras are compatible with the most advanced Vision and Image

Processing Libraries. List of currently supported libraries and frameworks

supported in Windows are OpenCV, Matlab’s DIP toolbox, Aforge framework,

Matrox mil, Simple CV are some commonly used libraries for digital image

processing.

Since as Matlab is used for scripting we can use both OpenCV and Matlab’s Image processing tool box for our project. We can integrate the OpenCV computer vision library with Matlab, using the MEX interface, to perform algorithm development, data analysis, and numerical computation in MATLAB.

Since image processing toolbox is easy to use with Matlab so we will be using it as our default library.

Image Processing Toolbox provides a comprehensive set of reference-standard

algorithms, functions, and apps for image processing, analysis, visualization, and

algorithm development. You can perform image enhancement, image deblurring,

feature detection, noise reduction, image segmentation, geometric

transformations, and image registration. Many toolbox functions are

multithreaded to take advantage of multicore and multiprocessor computers.

Image Processing Toolbox supports a diverse set of image types, including high

dynamic range, gigapixel resolution, embedded ICC profile, and tomographic.

Visualization functions let you explore an image, examine a region of pixels, adjust

the contrast, create contours or histograms, and manipulate regions of interest

(ROIs). With toolbox algorithms you can restore degraded images, detect and

measure features, analyze shapes and textures, and adjust color balance.

17 | P a g e

THE PSYCHOPHYSICS

OF COLOR

The perception of color starts with a chromatic light source, capable of emitting

electromagnetic radiation with wavelengths between approximately 400 and 700

nm. Part of that radiation reflects on the surfaces of the objects in a scene and

the resulting reflected light reaches the human eye, giving rise to the sensation of

color. An object that reflects light almost equally in all wavelengths within the

visible spectrum is perceived as white, whereas an object that absorbs most of

the incoming light, regardless of the wavelength, is seen as black. The perception

of several shades of gray between pure white and pure black is usually referred to

as achromatic. Objects that have more selective properties are considered

chromatic, and the range of the spectrum that they reflect is often associated

with a color name. For example, an object that absorbs most of the energy within

the 565–590 nm wavelength range is considered yellow.

A chromatic light source can be described by three basic quantities:

• Intensity (or Radiance): the total amount of energy that flows from the light

Source, measured in watts (W).

• Luminance: a measure of the amount of information an observer perceives from

a light source, measured in lumen (lm). It corresponds to the radiant power of

a light source weighted by a spectral sensitivity function (characteristic of the

HVS).

• Brightness: the subjective perception of (achromatic) luminous intensity.

18 | P a g e

The human retina (the surface at the back of the eye where images are projected)

is coated with photosensitive receptors of two different types: cones and rods.

Rods cannot encode color but respond to lower luminance levels and enable

vision under darker conditions. Cones are primarily responsible for color

perception and operate only under bright conditions. There are three types of

cone cells (L cones, M cones, and S cones, corresponding to long (≈ 610 nm),

medium (≈ 560 nm), and short (≈ 430 nm) wavelengths, respectively) whose

spectral responses are shown in Figure 1 The existence of three specialized types

of cones in the human eye was hypothesized more than a century before it could

be confirmed experimentally by Thomas Young and his trichromatic theory of

vision in 1802. Young’s theory explains only part of the color vision process,

though. It does not explain, for instance, why it is possible to speak of ‘bluish

green’ colors, but not ‘bluish yellow’ ones. Such understanding came with the

opponent-process theory of color vision, brought forth by Edward Herring in

1872. The colors to which the cones respond more strongly are known as the

primary colors of light and have been standardized by the CIE

Fig:1 Spectral absorption curves of the short (S), medium (M), and long (L) wavelength pigments in human cone and rod (R) cells. Courtesy of Wikimedia Commons.

19 | P a g e

(Commission Internationale de L’E´clairage—International Commission on

Illumination, an organization responsible for color standards) as red (700 nm),

green (546.1 nm), and blue (435.8 nm).The secondary colors of light, obtained by

additive mixtures of the primaries, two colors at a time, are magenta (or purple) =

red + blue, cyan (or turquoise) = blue +green, and yellow = green + red (Figure

a).For color mixtures using pigments (or paints), the primary colors are magenta,

cyan, and yellow and the secondary colors are red, green, and blue (Figure b). It is

important to note that for pigments a color is named after the portion of the

spectrum that it absorbs, whereas for light a color is defined based on the portion

of the spectrum that it emits. Consequently, mixing all three primary colors of

light results in white(i.e., the entire spectrum of visible light), whereas mixing all

three primary colors of paints results in black (i.e., all colors have been absorbed,

and nothing remains to reflect the incoming light).

Additive (a) and subtractive (b) color mixtures.

20 | P a g e

The use of the expression primary colors to refer to red, green, and blue may

Lead to a common misinterpretation: that all visible colors can be obtained by

mixing different amounts of each primary color, which is not true. A related

phenomenon of color perception, the existence of color metamers, may have

contributed to this confusion. Color metamers are combinations of primary colors

(e.g., red and green) perceived by the HVS as another color (in this case, yellow)

that could have been produced by a spectral color of fixed wavelength (of ≈ 580

nm).

21 | P a g e

COLOR MODELS

A color model is an abstract mathematical model describing the way colors can be represented as tuples of numbers, typically as three or four values or color components (e.g. RGB and CMYK are color models). However, a color model with no associated mapping function to an absolute color space is a more or less arbitrary color system with no connection to any globally understood system of color interpretation.

Adding a certain mapping function between the color model and a certain reference color space results in a definite "footprint" within the reference color space. This "footprint" is known as a gamut, and, in combination with the color model, defines a new color space. For example, Adobe RGB and sRGB are two different absolute color spaces, both based on the RGB model.

A wide range of colors can be created by the subtractive primary colors of pigment (cyan (C), magenta (M), yellow (Y), and black (K)). Those colors then define a specific color space. To create a three-dimensional representation of a color space, we can assign the amount of magenta color to the representations X axis, the amount of cyan to its Y axis, and the amount of yellow to its Z axis. The resulting 3-D space provides a unique position for every possible color that can be created by combining those three pigments.

However, this is not the only possible color space. For instance, when colors are displayed on a computer monitor, they are usually defined in the RGB (red, green and blue) color space. This is another way of making nearly the same colors (limited by the reproduction medium, such as the phosphor (CRT) or filters and backlight (LCD)), and red, green and blue can be considered as the X, Y and Z axes. Another way of making the same colors is to use their Hue (X axis), their Saturation (Y axis), and their brightness Value (Z axis). This is called the HSV color space. Many color spaces can be represented as three-dimensional (X,Y,Z) values in this manner, but some have more, or fewer dimensions, and some, such as Pantone, cannot be represented in this way at all.

22 | P a g e

Each industry that uses color employs the most suitable color model. For example, the RGB color model is used in computer graphics, YUV or YCbCr are used in video systems, PhotoYCC* is used in PhotoCD* production and so on.

Some widely used color models are

1. RGB Color Model.

2. CMYK Color Model.

3. YUV Color Model.

4. YCbCr and YCCK Color Models.

5. PhotoYCC Color Model.

6. YCoCg Color Model.

7. HSV and HSL Color Models.

8. CIE XYZ Color Model.

9. CIE LUV and CIE Lab Color Models.

We will consider two very popular models used in color image processing:

– RGB (Red Green Blue)

– HSV and HSL (Hue Saturation Value/Lightness)

23 | P a g e

RGB COLOR MODEL

The RGB color model is an additive color model in which red, green, and blue light are added together in various ways to reproduce a broad array of colors. The name of the model comes from the initials of the three additive primary colors, red, green, and blue.

The main purpose of the RGB color model is for the sensing, representation, and display of images in electronic systems, such as televisions and computers, though it has also been used in conventional photography. Before the electronic age, the RGB color model already had a solid theory behind it, based in human perception of colors.

The model is based on a Cartesian coordinate system

– RGB values are at 3 corners.

– Cyan magenta and yellow are at three other corners.

– Black is at the origin.

– White is the corner furthest from the origin.

– Different colors are points on or inside the cube represented by RGB

vectors.

24 | P a g e

RGB Color cube

Cartesian RGB Cube

25 | P a g e

Images represented in the RGB color model consist of three component images

one for each primary color.

When fed into a monitor these images are combined to create a composite color

image.

The number of bits used to represent each pixel is referred to as the color depth.

The axes of RGB color model is represented by the three primary colors of light (R,

G, and B), usually normalized to the range [0, 1].

The eight vertices of the resulting cube correspond to the three primary

Colors of light, the three secondary colors, pure white and pure black. Table

below Shows the R, G, and B values for each of these eight vertices.

RGB color coordinates are often represented in hexadecimal notation, with

individual components varying from 00 (decimal 0) to FF (decimal 255). For

example, a pure (100% saturated) red would be denoted FF0000, whereas a

slightly desaturated yellow could be written as CCCC33.

The number of discrete values of R, G, and B is a function of the pixel depth,

Defined as the number of bits used to represent each pixel: a typical value is

24 bits = 3 image planes × 8 bits per plane. The resulting cube—with more than 16

million possible color combinations.

R, G, and B Values for Eight Representative Colors Corresponding to the Vertices of the RGB Cube.

26 | P a g e

HSV and HSL

COLOR MODEL HSL and HSV are the two most common cylindrical-coordinate representations of

points in an RGB color model. The two representations rearrange the geometry of

RGB in an attempt to be more intuitive and perceptually relevant than the cartesian

(cube) representation. Developed in the 1970s for computer graphics applications,

HSL and HSV are used today in color pickers, in image editing software, and less

commonly in image analysis and computer vision.

HSL stands for hue, saturation, and lightness, and is often also called HLS. HSV

stands for hue, saturation, and value, and is also often called HSB (B for

brightness). A third model, common in computer vision applications, is HSI, for

hue, saturation, and intensity. However, while typically consistent, these

definitions are not standardized, and any of these abbreviations might be used for

any of these three or several other related cylindrical models.

In each cylinder, the angle around the central vertical axis corresponds to "hue",

the distance from the axis corresponds to "saturation", and the distance along the

axis corresponds to "lightness", "value" or "brightness". Note that while "hue" in

HSL and HSV refers to the same attribute, their definitions of "saturation" differ

dramatically. Because HSL and HSV are simple transformations of device-

dependent RGB models, the physical colors they define depend on the colors of the

red, green, and blue primaries of the device or of the particular RGB space, and on

the gamma correction used to represent the amounts of those primaries. Each

unique RGB device therefore has unique HSL and HSV spaces to accompany it,

and numerical HSL or HSV values describe a different color for each basis RGB

space.

27 | P a g e

28 | P a g e

The HSV (sometimes called HSB) color model can be obtained by looking at the

RGB color cube along its main diagonal (or gray axis), which results in a hexagon

shaped color palette. As we move along the main axis in the pyramid in the

hexagon gets smaller, corresponding to decreasing values of V, from 1 (white) to

0 (black). For any hexagon, the three primary and the three secondary colors of

light are represented in its vertices. Hue, therefore, is specified as an angle

relative to the origin (the red axis by convention).

Finally, saturation is specified by the distance to the axis: the longer the distance,

the more saturated the color.

__ shows an alternative representation of the HSV color model in which

The hex cone is replaced by a cylinder.

__ shows yet another equivalent three dimensional representation for the HSV

color model, as a cone with circular-shaped base. In summary, the main

advantages of the HSV color model (and its closely related alternatives) are its

ability to match the human way of describing colors and to allow for independent

control over hue, saturation, and intensity (value).

The ability to isolate the intensity component from the other two—which are

often collectively called chromaticity components—is a requirement in many

color image processing algorithms, Its main disadvantages include the

discontinuity in numeric values of hue around red, the computationally expensive

conversion to/from RGB, and the fact that hue is undefined for a saturation of 0.

The HSV color model describes colors according to their Hue, Saturation, and

Value. In some computer graphics programs, it is used as an alternative to the

RGB system to quantify colors.

In HSV, hue is a number in the interval [0, 360). A color's hue is its general positon

on a color wheel, where red is at 0°, green is at 120°, and blue is at 240°. For

example the RGB code of a yellow/orange color has high red and green

29 | P a g e

components and a low blue component, with the red level slightly higher than the

green. On the color wheel, the angle of this hue is a little less than 60°. The hue of

any neutral color--white, gray, or black--is set at 0°.

Value, in HSV, is the highest value among the three R, G, and B numbers. This

number is divided by 255 to scale it between 0 and 1. In terms of perception, HSV

Value represents how light, bright, or intense a color is. Value does not distinguish

white and pure colors, all of which have V = 1.

HSV Saturation measures how close a color is to the grayscale. S ranges from 0 to

1. White, gray, and black all have a saturation level of 0. Brighter, purer colors

have saturation near 1. In other color models that include a saturation

component, the precise mathematical definition of S may vary.

30 | P a g e

COLOR SPACE CONVERSION

Converting one color space to another is known as color space conversion

there are many color spaces but here we will work with only RGB, HSV and

BINARY color spaces.

31 | P a g e

Converting RGB to HSV Given three numbers R, G, and B (each between 0 and 255), you can first define m

and M with the relations

M = max{R, G, B}

m = min{R, G, B}.

And then V and S are defined by the equations

V = M/255

S = 1 - m/M if M > 0

S = 0 if M = 0.

As in the HSI and HSL color schemes, the hue H is defined by the equations

H = cos-1[ (R - ½G - ½B)/√R² + G² + B² - RG - RB - GB ] if G ≥ B, or

H = 360 - cos-1[ (R - ½G - ½B)/√R² + G² + B² - RG - RB - GB ] if B > G.

Inverse cosine is calculated in degrees.

32 | P a g e

Converting HSV to RGB

Given the values of H, S, and V, you can first compute m and M with the equations

M = 255V

m = M(1-S)

Now compute another number, z, defined by the equation

z = (M-m)[1 - |(H/60)mod_2 - 1|],

where mod_2 means division modulo 2. For example, if H = 135, then

(H/60)mod_2 = (2.25)mod_2 = 0.25. In modulo 2 division, the output is the

remainder of the quantity when you divide it by 2.

Now you can compute R, G, and B according to the angle measure of H. There are

six cases.

When 0 ≤ H < 60,

R = M

G = z + m

B = m.

If 60 ≤ H < 120,

R = z + m

G = M

B = m.

If 120 ≤ H < 180,

R = m

G = M

B = z + m.

33 | P a g e

When 180 ≤ H < 240,

R = m

G = z + m

B = M.

When 240 ≤ H < 300,

R = z + m

G = m

B = M.

And if 300 ≤ H < 360,

R = M

G = m

B = z + m.

34 | P a g e

Converting HSV to BINARY In our program we use a function that can convert the hsv image into a binary

image when the desired parameters are given later on we will understand the use

of this function.

A binary image is a digital image that has only two possible values for each pixel. [1] Typically the two colors used for a binary image are black and white though any

two colors can be used. [1] The color used for the object(s) in the image is the

foreground color while the rest of the image is the background color. [1] In the

document scanning industry this is often referred to as bi-tonal.

function output = hsv2binary(input,h,s,v) output = input(:,:,1)>=h(1)&input(:,:,1)<=h(2)&... input(:,:,2)>=s(1)&input(:,:,2)<=s(2)&... input(:,:,3)>=v(1)&input(:,:,3)<=v(2); end

here input is the input frame and h, s, v are the hue, saturation and value

parameters.

35 | P a g e

FLOW DIAGRAM

Flow Diagram for Color Detection.

36 | P a g e

COLOR BASED TRACKING

Hardware Detection and Selection of Acquisition Tool.

Here hardware refers to the webcam or any type of vision device which is

compatible with present day computers for the image acquisition we have to

choose the proper installed adaptor and the required vision device.

Since matlab ver 8.0(R2012b) is compatible with 4 type of adaptors (a) gentl (b)

gige (c) matrox (d) winvideo we have to select the winvideo as our adaptor.

37 | P a g e

imaqhwinfo() this function returns a structure, OUT, which contains image

acquisition hardware information. This information includes the toolbox version,

MATLAB version and names of installed adaptors.

imaqhwinfo(ADAPTORNAME) returns a structure, OUT, which contains

information related to an adaptor, specified by the string ADAPTORNAME. This

information includes adaptor version and available hardware for the specified

adaptor.

imaqhwinfo(ADAPTORNAME,'FIELD') returns the adaptor information for the

specified field name, FIELD. FIELD can be a single string or a cell array of strings. If

FIELD is a cell array, OUT is a 1-by-N cell array where N is the length of FIELD.

imaqhwinfo(ADAPTORNAME, DEVICEID) returns information for the device

identified by the numerical DEVICEID. DEVICEID can be a scalar or a vector. If

DEVICEID is a vector, OUT is a 1-by-N structure array where N is the length of

DEVICEID.

imaqhwinfo(OBJ) where OBJ is an image acquisition object, returns a structure,

OUT, containing information such as adaptor, board information and hardware

configuration limits. If OBJ is an array of device objects then OUT is a 1-by-N cell

array of structures where N is the length of OBJ.

imaqhwinfo(OBJ, 'FIELD') returns the hardware information for the specified

field name, FIELD, to OUT. FIELD can be any of the field names defined in the

imaqhwinfo(OBJ) structure. FIELD can be a single string or a cell array of strings. If

FIELD is a cell array, OUT is a 1-by-N cell array where N is the length of FIELD.

38 | P a g e

Once imaqhwinfo is called, hardware information is cached by the toolbox. To

force the toolbox to search for new hardware that may have been installed while

MATLAB was running, use IMAQRESET.

imaqreset ;

[camera_name, camera_id, format] = camcheck();

video = videoinput(camera_name, camera_id, format);

imaqreset resets the device cache and loads the newly connected devices.

In this project we use a function camcheck() that automatically detects installed

adaptors and selects the proper device to be used in the program from the

multiple device the code below describes how camcheck() works.

Video is a new video object that stores the camera name, camera id and the video

format so that our device is ready for acquisition.

function [ camera_name, camera_id, resolution ] = camcheck()

display('Camera Check Started'); a=imaqhwinfo%a gets the hardware information c=imaqhwinfo('winvideo')%here c stores winvideo info selected b=c.DeviceIDs %here b gets the no of camera output devices

camera_name = char(a.InstalledAdaptors(1))

[p,q]=size(b)

if (q==0) nohw= errordlg('No Compatible Hardware Found','ERROR' );%error if no

Hardware else switch q, case 1, camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(1).DeviceID(end); resolution = char(camera_info.DeviceInfo(1).SupportedFormats(end)); case 2,

NOTE : THIS IS A FUNCTION WE HAVE TO MAKE IT OUT SIDE THE MAIN FILE

39 | P a g e

hw2= questdlg('Select Device For Input Video', ... 'Select The Input Device', ... c.DeviceInfo(1).DeviceName,

c.DeviceInfo(2).DeviceName,c.DeviceInfo(1).DeviceName ); if strcmp(hw2 , c.DeviceInfo(1).DeviceName)==1 camera_name = char(a.InstalledAdaptors); camera_info = imaqhwinfo(camera_name);

camera_id = camera_info.DeviceInfo(1).DeviceID(end); resolution =

char(camera_info.DeviceInfo(1).SupportedFormats(end));

else if strcmp(hw2,c.DeviceInfo(2).DeviceName)==1 camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(2).DeviceID(end); resolution =

char(camera_info.DeviceInfo(2).SupportedFormats(end)); end

end case 3,

hw2= questdlg('Select Device For Input Video', ... 'Select The Input Device', ... c.DeviceInfo(1).DeviceName,

c.DeviceInfo(2).DeviceName,c.DeviceInfo(3).DeviceName,c.DeviceInfo(1).DeviceN

ame ); if strcmp(hw2,c.DeviceInfo(1).DeviceName)==1 camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(1).DeviceID(end); resolution =

char(camera_info.DeviceInfo(1).SupportedFormats(end)); end if strcmp(hw2,c.DeviceInfo(2).DeviceName)==1 camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(2).DeviceID(end); resolution =

char(camera_info.DeviceInfo(2).SupportedFormats(end)); end if strcmp(hw2,c.DeviceInfo(3).DeviceName)==1 camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(3).DeviceID(end); resolution =

char(camera_info.DeviceInfo(3).SupportedFormats(end)); end

end % switch

end

end

40 | P a g e

Here r define the radius of the circle of the circle that is to be done if the objects are connected i.e. if the required color is detected .

r=100; t= 0:pi/50:2*pi;

Setting Up the Acquisition Tool Properties Since we have created a video object video we have to set the properties of video

such as frames per trigger, trigger mode and trigger repetition color space

selection and we can modify according to our need.

video.FramesPerTrigger = 1; video.TriggerRepeat = Inf; triggerconfig(video,'manual'); video.ReturnedColorspace = 'rgb';

Next we create a video source object names source. The Source property is a vector of video source objects that represent the physical data sources connected to a device. When a video input object is created, the toolbox creates a vector of video source objects associated with the video input object.

Each video source object created is provided a unique source name. You can use the source name to select the desired acquisition source by configuring the SelectedSourceName property of the video input object.

A video source object's name is stored in its SourceName property. If a video source object's SourceName is equivalent to the video input object's SelectedSourceName, the video source object's selected property has a value of 'on'. Here we have some properties such as white balance, contrast, saturation and many more these properties are device dependent and all device doesn’t support this properties.

41 | P a g e

source = getselectedsource(video); source.BacklightCompensation = 'off'; source.WhiteBalanceMode = 'manual'; source.Contrast = 32; source.Gain = 64; source.Saturation = 64; source.Sharpness = 0; source.WhiteBalance = 4096;

Image Acquisition through Connected

Device Image acquisition is done by starting the video object by using a function

start(vid obj name)

start(video);

Defining Color Detection Parameters Here we define the parameters in HSV color space as it is preferred as the

accuracy is more. We define the range for the color to be detected where H

determines the hue and S determine the saturation and V determine the value.

h = [0.1020 0.4627]; s = [0.1373 1.0000]; v = [0.1725 0.8118];

Extraction of Frame

Extraction of frame is done by first triggering the video source object and storing

a single frame in a variable using the function getdata(obj). The while loop

controls how long the program will run by counting the no of frames acquired.

42 | P a g e

while (video.FramesAcquired<100) trigger(video); frame = getdata(video);

Color Space Transformation, Finding the Connected

Component and Calculation of the Centroid of the

Detected Object.

These three functions are now carried out using a single function so that the

program is easily understood and debugged.

Here we use a function detectcolor(frame,h,s,v) where it takes 4 parameters the

captured frame and the hue, saturation and value value’s and returns the two

numbers which are the location of the centroid of color in axial plane.

function x = detectcolor(image,h,s,v)

%color space conversion image = rgb2hsv(image);

%converting the hsv image to binary image = hsv2binary(image,h,s,v); %finding allconnected components

cc = bwconncomp(image);

%finding centroid of all connected objects color = regionprops(cc,'Centroid'); if isempty(color) x = [];

% If the is one or more connected componentes, % return the centroid of the biggest one else numPixels = cellfun(@numel,cc.PixelIdxList); [~,idx] = max(numPixels); x(1) = color(idx).Centroid(1); x(2) = color(idx).Centroid(2); end end

First the colorspace conversion is done the frame which we have extracted was in

rgb color space but now we convert it to hsv color space using the function

NOTE : THIS IS A FUNCTION WE HAVE TO MAKE IT OUT SIDE THE MAIN FILE

43 | P a g e

rgb2hsv(frame) which is a predefined matlab function and It is also described in

the chapter colorspace conversion.

Then the frame is converted to hsv color space then the image is converted to

binary format so that separate the detected object from the others. The detected

colored objects are white in color and the function hsv2binary() is described

briefly in the chapter colorspace conversion.

This is not a predefined function so we have to make a function to make the

program run.

Then we store all the connected objects in a structure with 4 fields or properties using the function cc = bwconncomp(image). This function work as follows. CC = bwconncomp(BW) returns the connected components CC found in BW. The binary image BW can have any dimension. CC is a structure with four fields.

Now that we have found all the connected components we need to find the

centroid of all the connected objects so that we can label them in the frame so we

use a function regionprops() to find centroid and this work as follows.

44 | P a g e

STATS = regionprops(BW, properties) measures a set of properties for each

connected component (object) in the binary image, BW. The image BW is a logical

array; it can have any dimension.

STATS is a structure array with length equal to the number of objects in BW,

CC.NumObjects, or max(L(:)). The fields of the structure array denote different

properties for each region, as specified by properties.

properties can be a comma-separated list of strings, a cell array containing strings,

the single string 'all', or the string 'basic'. If properties is the string 'all',

regionprops computes all the shape measurements, listed in Shape

Measurements. If called with a grayscale image, regionprops also returns the pixel

value measurements, listed in Pixel Value Measurements. If properties is not

specified or if it is the string 'basic', regionprops computes only the 'Area',

'Centroid', and 'BoundingBox' measurements. You can calculate the following

properties on N-D inputs: 'Area', 'BoundingBox', 'Centroid', 'FilledArea',

'FilledImage', 'Image', 'PixelIdxList', 'PixelList', and 'SubarrayIdx'.

'Centroid' – 1-by-Q vector that specifies the center of mass of the region. Note

that the first element of Centroid is the horizontal coordinate (or x-coordinate) of

the center of mass, and the second element is the vertical coordinate (or y-

coordinate). All other elements of Centroid are in order of dimension.

Now that we have the centroid of all the connected object we check for the

biggest connected component and return the centroid and if our frame has no

connected component then we return an empty vector.

45 | P a g e

Position a Circle Around the Object and

Display The frame

Here the frame is shown in the figure window using the function imshow() then the frame is kept in hold in the figure window such that the circle can be plotted around it. Here we check if there are any connected components by the value of variable x Then we assign the coordinates of centroid the variables z and y respectively then the circle is made by plotting the the zunit and yunit and then the figure is hold of and the scope of it and while loop ends. Finally we stop the video acquisition and flush the data and clear all variables using appropriate function.

imshow(frame); hold on; if(x) z=x(1); y=x(2); zunit=r*cos(t)+z; yunit=r*sin(t)+y; plot(zunit,yunit); hold off; end end stop(video); flushdata(video); clear all

46 | P a g e

SYSTEM AUTOMATION

An automated system is any system that does not require human intervention. Computer Automation is the use of control systems such as computers to control industrial machinery and processes, replacing human operators. In the scope of industrialization, it is a step beyond mechanization. Computer Automation greatly reduces the need for human sensory and mental requirements as well. Computer Automation plays an increasingly important role in the global economy and in daily experience. Engineers strive to combine automated devices with mathematical and organizational tools to create complex systems for a rapidly expanding range of applications and human activities.

System automation is common everywhere some of the examples are below:

Automated waste management

Automated waste collection trucks prevent the need for as many workers as well as easing the level of Labor required to provide the service.

Home automation

Home automation (also called domotics) designates an emerging practice of increased automation of household appliances and features in residential dwellings, particularly through electronic means that allow for things impracticable, overly expensive or simply not possible in recent past decades.

Automated video surveillance

The Defense Advanced Research Projects Agency (DARPA) started the research and development of automated visual surveillance and monitoring (VSAM) program, between 1997 and 1999, and airborne video surveillance (AVS) programs, from 1998 to 2002. Currently, there is a major effort underway in the vision community to develop a fully automated tracking surveillance system. Automated video surveillance monitors people and vehicles in real time within a busy environment. Existing automated surveillance systems are based on the environment they are primarily designed to observe, i.e., indoor, outdoor or

47 | P a g e

airborne, the amount of sensors that the automated system can handle and the mobility of sensor, i.e., stationary camera vs. mobile camera. The purpose of a surveillance system is to record properties and trajectories of objects in a given area, generate warnings or notify designated authority in case of occurrence of particular events.

Automated manufacturing

Automated manufacturing refers to the application of automation to produce things in the factory way. Most of the advantages of the automation technology has its influence in the manufacture processes.

The main advantages of automated manufacturing are higher consistency and quality, reduced lead times, simplified production, reduced handling, improved work flow, and increased worker morale when a good implementation of the automation is made.

Lights out factories are factories that require no workers in order to run.

In this project I tried to automate my pc to move its cursor and made it capable for performing all the tasks that a mouse can do through our gestures and signs.

Hence In this project we call it as SYSTEM AUTOMATION.

48 | P a g e

Java Robot Class

This class is used to generate native system input events for the purposes of test automation, self-running demos, and other applications where control of the mouse and keyboard is needed.

This class has three main functionalities: mouse control, keyboard control, and screen capture. Here are some of the important member functions:

Mouse control functions

void mouseMove(int x, int y)

This function moves the cursor to the coordinate (x, y) which is defined with respect to the top-left screen corner (in contrast to Matlab’s coordinate origin at the bottom-left corner).

void mousePress(int buttons)

void mouseRelease(int buttons)

This pair of functions performs the button click. Their input argument is an OR’ed combination of java.awt.event.InputEvents:

java.awt.event.InputEvent.BUTTON1_MASK // left mouse button

java.awt.event.InputEvent.BUTTON2_MASK // middle mouse button

java.awt.event.InputEvent.BUTTON3_MASK // right mouse button

Keyboard control functions

Keyboard action is emulated by the following pair of functions. Their keycodes are defined in java.awt.event.KeyEvent:

void keyPress(int keycode)

void keyRelease(int keycode)

49 | P a g e

NOTE: Although java.awt.event.KeyEvent constants defines most of the US QWERTY keys, not all can actually be used with java.awt.Robot. Specifically, it appears that only the KeyEvent constants for unmodified keys can be used. See the following section for an example.

Utility functions

The robot can be put to sleep for a desired duration (in milliseconds). Also, the

calling routine can be blocked until the robot exhausts its command queue.

void delay(int ms)

void waitForIdle()

50 | P a g e

Java Robot Classes And MATLAB

To create the Robot object in Matlab, simply run

robot = java.awt.Robot;

Moving the mouse cursor

With Matlab’s root (0) handle, the mouse cursor can be repositioned by

set(0,'PointerLocation',[x y]);

The mouse cursor is placed on the x-th pixel from left edge and the y-th pixel from the bottom edge of the screen.

Alternately, we could use the semi-documented moveptr function, which was described here last year.

The same operation can also be done using Robot as follows:

scrSize = get(0,'ScreenSize'); robot.mouseMove(x,scrSize(2)-y);

The extra step must be taken to convert from Matlab to Java screen coordinates.

Depending on the specific case (for example, whether or not we know the absolute or only relative screen coordinates), we may prefer using any of these equivalent mouse-movement alternatives.

Clicking mouse buttons

Unfortunately, we have few alternatives for automating mouse-clicks – Robot is basically our only option. Matlab recognizes 4 different mouse click types as specified in the Figure’s Selection Type property:

51 | P a g e

Normal: Click left mouse button Extend: SHIFT-click left mouse button Alternate: CTRL-click left mouse button Open: Double click left mouse button

To observe the mouse click, open a figure and set its WindowButtonDownFcn callback:

myCallback = @(hObj,event) disp(get(hObj,'SelectionType')); set(gcf,'WindowButtonDownFcn', myCallback);

To trigger the callback, the figure must have the focus and the mouse cursor must be on the figure. To ensure the callback invocation, “figure(gcf); drawnow;” commands are included in the code examples below. Make sure to place the mouse cursor on the figure before running these codes.

Here’s how the Robot can be used to perform each type of clicking. First, normal click:

figure(gcf); drawnow;

robot.mousePress (java.awt.event.InputEvent.BUTTON1_MASK);

robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK);

For “open” or double click, repeat it twice:

figure(gcf); drawnow;

robot.mousePress (java.awt.event.InputEvent.BUTTON1_MASK);

robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK);

robot.mousePress (java.awt.event.InputEvent.BUTTON1_MASK);

robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK);

For the other two click types, we need to use the keyboard functions. First, “extend” or SHIFT-click:

figure(gcf); drawnow;

robot.keyPress (java.awt.event.KeyEvent.VK_SHIFT);

robot.mousePress (java.awt.event.InputEvent.BUTTON1_MASK);

robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK);

robot.keyRelease (java.awt.event.KeyEvent.VK_SHIFT);

52 | P a g e

Lastly, “alternate” or CTRL-click:

figure(gcf); drawnow;

robot.keyPress (java.awt.event.KeyEvent.VK_CONTROL);

robot.mousePress (java.awt.event.InputEvent.BUTTON1_MASK);

robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK);

robot.keyRelease (java.awt.event.KeyEvent.VK_CONTROL);

Clicking keyboard keys

Just as we have shown above how to press a modifier key (Ctrl or Alt key) to set the figure’s SelectionType property, keyPress and keyRelease functions can be used with character keys to type text. For example, with the focus on Matlab’s Command Prompt, running the following code executes the ver command:

robot.keyPress (java.awt.event.KeyEvent.VK_V);

robot.keyRelease (java.awt.event.KeyEvent.VK_V);

robot.keyPress (java.awt.event.KeyEvent.VK_E);

robot.keyRelease (java.awt.event.KeyEvent.VK_E);

robot.keyPress (java.awt.event.KeyEvent.VK_R);

robot.keyRelease (java.awt.event.KeyEvent.VK_R);

robot.keyPress (java.awt.event.KeyEvent.VK_ENTER);

robot.keyRelease (java.awt.event.KeyEvent.VK_ENTER);

In the previous section, I mentioned that not all KeyEvent constants are supported by java.awt.Robot, and keys must be defined with their unmodified characters. For example, a semicolon (‘;’) can be typed by:

robot.keyPress (java.awt.event.KeyEvent.VK_SEMICOLON);

robot.keyRelease (java.awt.event.KeyEvent.VK_SEMICOLON);

However, doing the same for a colon (‘:’ or SHIFT-’;') causes an error:

robot.keyPress (java.awt.event.KeyEvent.VK_COLON);

robot.keyRelease (java.awt.event.KeyEvent.VK_COLON);

??? Java exception occurred:

java.lang.IllegalArgumentException: Invalid key code

at sun.awt.windows.WRobotPeer.keyPress(Native Method)

at java.awt.Robot.keyPress(Unknown Source)

53 | P a g e

Instead of using the VK_COLON constant, VK_SEMICOLON constant must be used while SHIFT modifier key is pressed:

robot.keyPress (java.awt.event.KeyEvent.VK_SHIFT);

robot.keyPress (java.awt.event.KeyEvent.VK_SEMICOLON);

robot.keyRelease (java.awt.event.KeyEvent.VK_SEMICOLON);

robot.keyRelease (java.awt.event.KeyEvent.VK_SHIFT);

once we have got the centroid of the connected object in our color tracking

program we use a function movems() that take the x and y coordinates of the

connected object and moves the mouse pointer to the desired position.

function [ output_args ] = movems( x,y) import java.awt.Robot; mouse = Robot; mouse.mouseMove(3.5*x-300,2.77*y-277); end

In this project I’ve only moved the mouse pointer but we can have the full control

over the system by using the above mentioned methods such as keyboard.

NOTE : THIS IS A FUNCTION WE HAVE TO MAKE IT OUT SIDE THE MAIN FILE.

54 | P a g e

CODES IN MATLAB

Main File

% Script that detects green in a video

% reset acquisition device imaqreset ;

% radius of circle around green color r=30; t= 0:pi/50:2*pi;

% Create a video input object from a camera [camera_name, camera_id, format] = camcheck(); video = videoinput(camera_name, camera_id, format); % Set the properties of the video object video.FramesPerTrigger = 1; video.TriggerRepeat = Inf; triggerconfig(video,'manual'); video.ReturnedColorspace = 'rgb'; % Create video source object from a video input object source = getselectedsource(video); % Define the HSV parameters to isolate the green color h = [0.3 0.4]; s = [0.6 1]; v = [0.5 1]; % Start video aquisition start(video); % define the frames for loop while (video.FramesAcquired<400) % Trigger the camera trigger(video); % Get the current frame data from the camera frame = getdata(video); frame = flipdim(frame,2); % Detect green color position in current frame x = detectcolor(frame,h,s,v); % Show the current frame imshow(frame); % Plot the center of the color in the current frame hold on; if(x) z=x(1); y=x(2) ; movems(z,y); zunit=r*cos(t)+z; yunit=r*sin(t)+y; plot(zunit,yunit); hold off; end

55 | P a g e

end % Stop the video aquisition stop(video); % Flush all the image data stored in the memory buffer flushdata(video); % Clear all variables clear all

movems(); function function [ output_args ] = movems( x,y) %import java robot class

import java.awt.Robot;

%create a robot object mouse mouse = Robot;

% move mouse to desired location mouse.mouseMove(3.5*x-350,2.7*y-270); end

hsv2binary(); function

% Function that converts a color image into a binary % image such that the pixels that are in specified % color range become 1 and all others become 0 function output = hsv2binary(input,h,s,v) output = input(:,:,1)>=h(1)&input(:,:,1)<=h(2)&... input(:,:,2)>=s(1)&input(:,:,2)<=s(2)&... input(:,:,3)>=v(1)&input(:,:,3)<=v(2); end

56 | P a g e

Detectcolor(); function

% Function that detects the center of color %finding centroid and returning coordinates

function x = detectcolor (image,h,s,v) % Convert the RGB color space image to an HSV % color space format image image = rgb2hsv(image); % Threshold the HSV color space image to an % binary image image = hsv2binary(image,h,s,v); % Label all the connected components in the % thresholded binary image cc = bwconncomp(image); % Mesures the centroid for each all the % connected components color = regionprops(cc,'Centroid'); % If there is no connected components, return % an empty vector if isempty(color) x = []; % If the is one or more connected componentes, % return the centroid of the biggest one else numPixels = cellfun(@numel,cc.PixelIdxList); [~,idx] = max(numPixels); x(1) = color(idx).Centroid(1); x(2) = color(idx).Centroid(2); end end

57 | P a g e

Camcheck(); function

function [ camera_name, camera_id, resolution ] = camcheck()

display('Camera Check Started'); a=imaqhwinfo%a gets the hardware information c=imaqhwinfo('winvideo')%here c stores winvideo info selected b=c.DeviceIDs %here b gets the no of camera output devices

camera_name = char(a.InstalledAdaptors(1))

[p,q]=size(b)

if (q==0) nohw= errordlg('No Compatible Hardware Found','ERROR' );%error if no

Hardware else switch q, case 1, camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(1).DeviceID(end); resolution = char(camera_info.DeviceInfo(1).SupportedFormats(end)); case 2,

hw2= questdlg('Select Device For Input Video', ... 'Select The Input Device', ... c.DeviceInfo(1).DeviceName,

c.DeviceInfo(2).DeviceName,c.DeviceInfo(1).DeviceName ); if strcmp(hw2 , c.DeviceInfo(1).DeviceName)==1 camera_name = char(a.InstalledAdaptors); camera_info = imaqhwinfo(camera_name);

camera_id = camera_info.DeviceInfo(1).DeviceID(end); resolution =

char(camera_info.DeviceInfo(1).SupportedFormats(end));

else if strcmp(hw2,c.DeviceInfo(2).DeviceName)==1 camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(2).DeviceID(end); resolution =

char(camera_info.DeviceInfo(2).SupportedFormats(end)); end

end

58 | P a g e

case 3,

hw2= questdlg('Select Device For Input Video', ... 'Select The Input Device', ... c.DeviceInfo(1).DeviceName,

c.DeviceInfo(2).DeviceName,c.DeviceInfo(3).DeviceName,c.DeviceInfo(1).DeviceN

ame ); if strcmp(hw2,c.DeviceInfo(1).DeviceName)==1 camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(1).DeviceID(end); resolution =

char(camera_info.DeviceInfo(1).SupportedFormats(end)); end if strcmp(hw2,c.DeviceInfo(2).DeviceName)==1 camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(2).DeviceID(end); resolution =

char(camera_info.DeviceInfo(2).SupportedFormats(end)); end if strcmp(hw2,c.DeviceInfo(3).DeviceName)==1 camera_name = char(a.InstalledAdaptors(end)); camera_info = imaqhwinfo(camera_name); camera_id = camera_info.DeviceInfo(3).DeviceID(end); resolution =

char(camera_info.DeviceInfo(3).SupportedFormats(end)); end

end % switch

end

end

59 | P a g e

CONCLUSION

From this project we conclude that Color plays a vitally important role in the

world in which we live. Color can sway thinking, change actions, and cause

reactions. Colors tend to be most important part in object detection. Colors can

be expressed in many ways and it is the only way that distinct two objects for a

computer.

Color based image processing also plays in artificial intelligence more often it

boosts the accuracy of algorithms we also came to know that color can be helpful

in system automation and can have numerous applications in near future.

60 | P a g e

BIBILIOGRAPHY BOOKS

Practical Image and Video Processing Using MATLAB – by O. Marques

Digital Image Processing by Gonzalez & Woods

WEBSITES

http://www.ncbi.nlm.nih.gov

http://mkweb.bcgsc.ca

http://www.asysdefence.com

http://www.aishack.in

PAPERS

Boosting Color Feature Selection for Color Face Recognition Jae Young Choi, Student Member, IEEE, Yong Man Ro, Senior Member, IEEE, and

Konstantinos N. Plataniotis, Senior Member, IEEE

Color Feature Detection THEO GEVERS, JOOST VAN DE WEIJER

Hue, Saturation and Value

hummie

61 | P a g e

Kamal Pradhan Email: kamal.pradhan@suiit.ac.in

Phone: +918093850704