Embedded System Based Face Recognition

A.K.M Fazla MehrabRoll:0707021

Palash DebnathRoll:0707023

15 Novenmer,2011

Khulna University of Engineering & TechnologyDepartment of Computer Science and Engineering

Contents

1 Introduction: 2

2 Feasibility study: 32.1 OpenCV Haartraining: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1.1 Data Preparation: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.1.2 Training: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2 Viola Jones Face Detection Algorithm: . . . . . . . . . . . . . . . . . . . . . . . . 42.2.1 Using Several Libraries: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2.2 Extracting Unwanted functions from library: . . . . . . . . . . . . . . . . 42.2.3 Compiling the program: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.3 CMUcam3 Open Source Programmable Embedded Color Vision Platform: . . . 52.3.1 Cygwin: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3.2 GNU ARM GCC: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 Requirement analysis: 73.1 ARM7TDMI ARM processor: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 CMUcam: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.3 Compiler: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.4 GNU toolchain: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.5 Philips LPC210x FLASH Utility: . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Progress: 12

5 Conclusion: 13

6 Bibliography: 14

1

Chapter 1

Introduction:

Face recognition is challenging because it is a real world problem. The human face is a complex,natural object that tends not to have easily (automatically) identified edges and features. Becauseof this, it is difficult to develop a mathematical model of the face that can be used as prior knowl-edge when analyzing a particular image. Applications of face recognition is widespread. Perhapsthe most obvious is that of human computer interaction. One could make computers easier to useif when one simply sat down at a computer terminal, the computer could identify the user by nameand automatically load personal preferences. This identification could even be useful in enhancingother technologies such as speech recognition, since if the computer can identify the individual whois speaking, the voice patterns being observed can be more accurately classified against the knownindividual’s voice.

Human face recognition by computer systems has become a major field of interest. Face Recogni-tion algorithms are used in a wide range of applications, such as security control, video retrieving,biometric signal processing, human computer interface and image database management. However,it is difficult to develop a complete robust face recognizer due to various light conditions, face sizes,face orientations, background and skin colors. First of all we started working with face detectionusing OpenCV. Then we studied about Viola-Jones algorithm and tried to implement it. But wedidn’t succeed. Then we studied about CMUcam3 Open Source Programmable Embedded ColorVision Platform. We implemented the Viola-Jones algorithm here. In this report we have explainedour way of working methodology as well as the progress we have achieved so far.

2

Chapter 2

Feasibility study:

Face Recognition is a wide field of study. There are several ways to implement the idea. Amongthem we have studied a couple of ideas.

2.1 OpenCV Haartraining:

OpenCV haartraining is rapid object detection training with a cascade of boosted classifiers Basedon Haar like features. Several steps are followed for haartraining.

2.1.1 Data Preparation:

• Positive Images: Positive images means only objects of interest that means only faces. Wehad to collect positive images that contain only objects of interest. For our purpose the objectof interest is Faces. We have used 5000 positive frontal face patterns, and 5000 positive frontalface patterns were derived from 1000 original faces.

• Negative Images:: Negative images means only objects that doesn’t contain object of interest.We needed to collect negative images that do not contain objects of interest, e.g., faces totrain haarcascade classifier.

2.1.2 Training:

• Haartraining: Haartraining is used to train classifiers for their face detection system Thus;we can create our own object classifiers using the functions. For OpenCV haartraining 20stages were trained.

• Generate a XML File: The haartraing generates an xml file when the process is completelyfinished. This xml file contains the properties of faces.

3

2.2 Viola Jones Face Detection Algorithm:

2.2.1 Using Several Libraries:

Viola Jones Face Detection Algorithm uses several OpenCV libraries. These libraries are typicallylinked together. As the memory of our processor is not enough to store all the libraries and xmlfiles so we decided not to use all the libraries.

2.2.2 Extracting Unwanted functions from library:

As we have mentioned earlier that the main problem of using Viola Jones Face Detection Algorithmis their linked library. We have tried about a week just only to separate the library functions thatare not necessary for our program. But there are about 148 header files that use each other astheir header. we have tried to separate the functions but they are so linked that we are unable toextract them from each other and compile our program.

2.2.3 Compiling the program:

The main challenge of Viola Jones Face Detection Algorithm is to compile the program and gener-ating our required hex file. We had included all the libraries of the OpenCV at last to compile theprogram. But they are so linked together that we were unable to compile the program. So we hadto leave this path.

4

2.3 CMUcam3 Open Source Programmable Embedded Color Vision Platform:

The goal of the CMUcam project is to provide simple vision capabilities to small embedded systemsin the form of an intelligent sensor. The CMUcam3 extends upon this idea by providing a flexibleand easy to use open source development environment that complements a low cost hardwareplatform. The CMUcam3 is an ARM7TDMI based fully programmable embedded computer visionsensor. The main processor is the NXP LPC2106 connected to an Omnivision CMOS camera sensormodule. Custom C code can be developed for the CMUcam3 using a port of the GNU toolchainalong with a set of open source libraries and example programs. Executables can be link buildingflashed onto the board using the serial port with no external downloading hardware required.In order to compile and download code to the CMUcam3 in Windows you will need the followingsoftware:

• Cygwin

Make

Subversion

• ARM gcc

• LPC210x FLASH Utility

• CMUcam3 Frame Grab Utility (using .NET runtime v2.0 or greater)

5

2.3.1 Cygwin:

Cygwin is a Linux-like environment for Windows. It provides you with a shell and many familiarLinux tools like ”make” which you will use to compile CMUcam3 code. So we had to downloadCygwin from the site http://www.cygwin.com/ The installer is there. We downloaded the Cygwinfrom there and installed. We installed the following extra packages:

• Make

• Subversion

• Open-ssl

These packages are essential for compiling the project. So if any of the commands don’t workthen we need to find out which is not working and then install it manually.

2.3.2 GNU ARM GCC:

The compiler choice is left up to the hal. This allows different hardware platforms to use differentcompilers. Currently, the only supported hal is ”lpc2106-cmucam3”, which uses a version of GCCoptimized for the ARM processor. To build using this hal, we needed the arm-none-eabi-* styleGCC tools from CodeSourcery. But there is a problem with the latest version of GNU ARM GCC.The hall couldn’t be built using the latest version of the GNU ARM GCC. We had to use theearlier version ”arm-2010q1-188-arm-none-eabi” for compiling the hall.

• Building the project:

We have the compiler and the CMUcam3 source; you need to build the system now. Fromthe top level, invoking ”make” will build all the hal libraries, and will build a subset of itemsin the ”projects” directory.

Open a Cygwin shell and type ”cd c:” to move to our C drive. Next navigate to thesource directory using the ”cd mydirectory”command.Onceinthe”cc3”directorydothefollowing.

First clean the hal before trying to build any projects by the command ”makeclean”.

The build the hal by ”make” command.

Then to build the project go to the project directory by ”cd” command

Type ”make clean” to clean the previously build elements.

Type ”make” to build the project.

If all this commands perfectly then a hex file will be generated for that particularproject.

6

Chapter 3

Requirement analysis:

3.1 ARM7TDMI ARM processor:

The ARM7 family is the world’s most widely used 32-bit embedded processor family, with morethan 170 silicon licensees and over 10 Billion units shipped since its introduction in 1994. While theARM7 processor family continues to be used today for simple 32-bit devices, newer digital designsare increasingly making use of the newer, more powerful and feature-rich ARM processors whichoffer significant technical enhancements over the ARM7 family.

Figure 3.1:

7

3.2 CMUcam:

CMUcam is a new low-cost, low-power sensor for mobile robots. We can use CMUcam visionsystem to do many different kinds of on-board, real-time vision processing. Because CMUcam usesa serial port, it can be directly interfaced to other low-power processors such as PIC chips. At 17frames per second, CMUcam can do the following:

• ge Track the position and size of a colorful or bright object

• Measure the RGB or YUV statistics of an image region

• Automatically acquire and track the first object it sees

• Physically track using a directly connected servo

• Dump a complete image over the serial port

• Dump a bitmap showing the shape of the tracked object

Figure 3.2:

8

3.3 Compiler:

We need a compiler to build our projects. For this purpose we need a tool called CYGWIN. Itis a collection of tools which provide a Linux look and feel environment for Windows. A DLL(cygwin1.dll) which acts as a Linux API layer providing substantial Linux API functionality. TheCygwin DLL currently works with all recent, commercially released x86 32 bit and 64 bit versionsof Windows, with the exception of Windows CE and Windows NT4.

Figure 3.3:

9

3.4 GNU toolchain:

The toolchain consists of the GNU binutils, compiler set (GCC) and debugger (Insight for Windowsand Linux, GDB only for MacOS). Newlib is used for the C library. The toolchain includes the Cand C++ compilers.

Figure 3.4:

10

3.5 Philips LPC210x FLASH Utility:

The LPC2000 Flash ISP Utility allows us to download programs to the on-chip Flash using theon-chip Flash boot loader of the NXP (Philips) LPC2000 devices. This utility works as a commu-nicating tool for loading our hex file into memory from computer.

Figure 3.5:

11

Chapter 4

Progress:

There were several algorithms for face recognition. Our main challenge was to find out an efficientway to implement the algorithm. After working hard and reading so many blogs and mail someexperienced persons we have now reached a way to implement the algorithm. We have created anenvironment for implementing our code for face recognition. We have got the camera in hand thatwill be used for effectiveness of the programs. We have the processor ARM7TDMI but now weneed another IC which will be used as a buffer. This IC is not available in our country so we haveordered it and when we got it in our hand then we will be able to implement our research so far.

12

Chapter 5

Conclusion:

The primary conclusion that can be drawn from the results of our experiments is that we are nowable to create a new project and by building the project we are able to generate our required hexfile. We will load this hex file in the memory and we are now waiting for getting the IC requiredto do this function. We have ordered this IC but it is not available in our country. As soon as wewill get the IC chip we will be able to implement our research that we have done so far.

13

Chapter 6

Bibliography:

1. www.cmucam3.org2. www.cygwin.com3. www.codesourcery.com4. www.semiconductors.philips.com5. http://en.wikipedia.org/wiki/GNUtoolchain

14