Embed Size (px)
Citation preview
TRIBHUVAN UNIVERSITY
INSTITUTE OF ENGINEERING
PULCHOWK CAMPUS
DEPARTMENT OF ELECTRONICS AND COMPUTER ENGINEERING
A
FINAL YEAR PROJECT REPORT ON
FPGA BASED MOTION DETECTION AND ALARM SYSTEM USING GSM
TECHNOLOGY
By:
Mallika Shree Pokharel (066/BEX/419)
Nikita Agrawal (066/BEX/424)
Priyanka Tiwari (066/BEX/428)
Upama Nakarmi (066/BEX/447)
A PROJECT WAS SUBMITTED TO THE DEPARTMENT OF ELECTR ONICS AND
COMPUTER ENGINEERING IN PARTIAL FULLFILLMENT OF THE
REQUIREMENT FOR THE BACHELOR’S DEGREE IN ELECTRONIC S &
COMMUNICATION ENGINEERING
DEPARTMENT OF ELECTRONICS AND COMPUTER ENGINEERING
LALITPUR, NEPAL
October, 2013
i
PAGE OF APPROVAL
TRIBHUVAN UNIVERSITY
INSTITUTE OF ENGINEERING
PULCHOWK CAMPUS
DEPARTMENT OF ELECTRONICS AND COMPUTER ENGINEERING
The undersigned certify that they have read, and recommended to the Institute of
Engineering for acceptance, a project report entitled "FPGA based motion detection and
Alarm system using GSM technology" submitted by Mallika Shree Pokharel, Nikita
Agrawal, Priyanka Tiwari and Upama Nakarmi in partial fulfilment of the requirements for
the Bachelor’s degree in Electronics & Communication / Computer Engineering.
________________________________ __________________________________
Supervisor, Internal Examiner,
Mr. Daya Sagar Baral, Lecturer Surendra Shrestha, PhD.
Department of Electronics and Department of Electronics and
Computer Engineering Computer Engineering
Pulchowk Campus, IOE.
________________________________ ___________________________________
External Examiner, Coordinator,
Mr. Saroj Shakya Dr. Aman Shakya
Associate Professor Deputy Head
NCIT Department of Electronics and
Computer Engineering
DATE OF APPROVAL: 1st October, 2013
ii
COPYRIGHT
The author has agreed that the Library, Department of Electronics and Computer
Engineering, Pulchowk Campus, Institute of Engineering may make this report freely
available for inspection. Moreover, the author has agreed that permission for extensive
copying of this project report for scholarly purpose may be granted by the supervisors who
supervised the project work recorded herein or, in their absence, by the Head of the
Department wherein the project report was done. It is understood that the recognition will
be given to the author of this report and to the Department of Electronics and Computer
Engineering, Pulchowk Campus, Institute of Engineering in any use of the material of this
project report. Copying or publication or the other use of this report for financial gain
without approval of to the Department of Electronics and Computer Engineering,
Pulchowk Campus, Institute of Engineering and author’s written permission is prohibited.
Request for permission to copy or to make any other use of the material in this report in
whole or in part should be addressed to:
Arun Kumar Timalsina, PhD,
Head of Department
Department of Electronics and Computer Engineering
Pulchowk Campus, Institute of Engineering
Lalitpur, Kathmandu
Nepal
iii
ACKNOWLEDGEMENT
The progress and success of any project require a lot of guidance and assistance from many
people and we are extremely fortunate to have got this along this project. Whatever we
have done is only due to such guidance and support.
We respect and thank our Department of Electronics and Computer Engineering for their
unlisted encouragement and moreover their motivation for this project. We are extremely
grateful to our Supervisor Sir Daya Sagar Baral for his support and guidance in this project
in spite of his hectic schedule. It is an honor to thank our project coordinator Dr. Aman
Shakya for encouraging us to do the project. We are indebted to Mr. Dinesh Baniya for
providing us the insight knowledge of handling the project well and for his support.
Last but not the least we would like to thank our parents for their belief and support.
Mallika Shree Pokharel (066/BEX/419)
Nikita Agrawal (066/BEX/424)
Priyanka Tiwari (066/BEX/428)
Upama Nakarmi (066/BEX/447)
iv
ABSTRACT
As the technology leaps forward for the betterment of mankind we have seen that various
improvements have been made in the way human has interfaced with the system.
Similarly, present Human Computer Interface (HCI) aspires for such media which are easy
to use, have fewer devices and are more user-friendly. This project entitled “FPGA Based
Motion Detection and Alarm System using GSM technology” aims at developing a system
in which the motion is detected and authenticated user is notified about the detection of
motion by a message.
The project consists of two parts, the motion is detected by the background subtraction
algorithm implemented in FPGA, and after the detection of motion, a message is sent to
the owner via the GSM modem according to the programs stored in PIC16F877A
microcontroller.
The system analyses sixtieth frame of the environment, i.e a frame in every single second
captured by the image sensor mounted on the PCB board. Whenever motion is detected,
change occurs in the pixels value for the frames taken. If the change is greater than a
certain specified threshold, a SMS will be send to the owner of the system. Thus, the
project was controlled by FPGA which is indeed a fast and accurate process as all the
operations run in parallel, and hence the system is an efficient way to overcome the
ongoing theft and robbery in the present world considering the security perspective.
v
TABLE OF CONTENTS
PAGE OF APPROVAL ...................................................................................................................................... i
COPYRIGHT .................................................................................................................................................... ii
ACKNOWLEDGEMENT ................................................................................................................................ iii
ABSTRACT ..................................................................................................................................................... iv
LIST OF FIGURES ......................................................................................................................................... vii
LIST OF TABLES ............................................................................................................................................ ix
LIST OF ABBREVIATIONS ............................................................................................................................ x
1. INTRODUCTION ........................................................................................................................................ 1
1.1. Background ...................................................................................................................................... 1
1.2. Motivation ........................................................................................................................................ 3
1.3. Objectives ......................................................................................................................................... 4
2. LITERATURE REVIEW ............................................................................................................................. 5
3. SOFTWARE AND TOOLS USED .............................................................................................................. 7
3.1. For Initial Simulation ............................................................................................................................. 7
3.1.1. MATLAB....................................................................................................................................... 7
3.2. For GSM Interfacing .............................................................................................................................. 8
3.2.1. MPLAB IDE ................................................................................................................................... 8
3.2.2. Proteus VSM ................................................................................................................................... 8
3.3. For FPGA Based Design ........................................................................................................................ 9
3.3.1. ISE .................................................................................................................................................. 9
3.3.2. Verilog ............................................................................................................................................ 9
3.3.3. Modelsim ...................................................................................................................................... 10
3.4. Design Tools ........................................................................................................................................ 10
3.4.1. PCB Wizard .................................................................................................................................. 10
4. BACKGROUND THEORY ....................................................................................................................... 11
4.1. FPGA Board ........................................................................................................................................ 11
4.1.1. Architecture .................................................................................................................................. 12
4.1.2. FPGA design and programming ................................................................................................... 13
4.2. C3088 Camera Interfacing ................................................................................................................... 16
4.2.1. Specifications and Features ........................................................................................................... 16
4.2.2. Video Signal ................................................................................................................................. 18
4.3. Memory Usage ..................................................................................................................................... 19
4.3.1. Configurable Logic Blocks (CLBs) .............................................................................................. 20
4.3.2. Strata Flash: .................................................................................................................................. 23
4.3.3. Block RAM ................................................................................................................................... 27
4.4. GSM Module ....................................................................................................................................... 31
4.4.1. General Description ...................................................................................................................... 31
vi
4.4.2. Technical Description of GSM ..................................................................................................... 32
4.5. PIC16F877A ........................................................................................................................................ 34
5. SYSTEM DESIGN ..................................................................................................................................... 38
5.1. System Block Diagram ........................................................................................................................ 38
5.2. Diagram Description ............................................................................................................................ 39
6. SIMULATION AND ANALYSIS ............................................................................................................. 41
6.1. Algorithm used in MATLAB ............................................................................................................... 41
6.2. Programming the FPGA, CPLD or Platform Flash PROM ................................................................ 43
6.2.1. Connecting the USB Cable ........................................................................................................... 43
6.2.2. Programming via iMPACT ........................................................................................................... 44
6.3. Clock Divider ....................................................................................................................................... 46
6.3.1. VGA video signal generation ........................................................................................................ 50
6.4. FPGA Simulation Part ......................................................................................................................... 52
6.4.1. Frame Grabber working mechanism ............................................................................................. 52
6.4.2. Flowchart of Frame Grabber Module ........................................................................................... 55
7. PROBLEMS FACED ................................................................................................................................. 57
7.1. New to FPGA and Verilog ................................................................................................................... 57
7.2. Clock Synchronizing Problem ............................................................................................................. 57
7.3. Camera Light ....................................................................................................................................... 57
8. CONCLUSION AND FUTURE ENHANCEMENT ................................................................................. 58
8.1. Conclusion ........................................................................................................................................... 58
8.2. Future Enhancement ............................................................................................................................ 59
9. REFERENCES ........................................................................................................................................... 61
APPENDIX A: XILINX SPARTAN 3E FPGA BOARD ............................................................................... 64
APPENDIX B: C3088 1/4” COLOR CAMERA MODULE WITH DIGITAL OUTPUT .............................. 66
APPENDIX C: PIC16F877A ........................................................................................................................... 67
APPENDIX E: PCB DESIGNS ....................................................................................................................... 71
APPENDIX F: VGA MONITOR .................................................................................................................... 72
vii
LIST OF FIGURES
Figure 4.1: SPARTAN 3E FPGA Board ............................................................................. 15
Figure 4.2 : Cmos 3088 module ........................................................................................... 16
Figure 4.3 :Pin description of cmos 3088 ............................................................................ 17
Figure 4.4 :Circuit configuration of camera ........................................................................ 18
Figure 4.5 :Signals href, pclk, vsync timing diagram .......................................................... 19
Figure 4.6 :Generic FPGA architecture ............................................................................... 20
Figure 4.7: FPGA configuration logic block (CLB) ............................................................ 21
Figure 4.8: FPGA configuration I/O block .......................................................................... 22
Figure 4.9: FPGA programmable interconnect .................................................................... 23
Figure 4.10: Block diagram of Intel strata flash .................................................................. 25
Figure 4.11: Memory map 3 volt strata flash memory ........................................................ 26
Figure 4.12: Connection of strata flash memory to spartan 3E FPGA ................................ 26
Figure 4.13: Block RAM and surrounding neighbourhood ................................................. 28
Figure 4.14: Dual Port and Single port Block RAM memory diagram ............................... 29
Figure 4. 15: Port A and Port B of block RAM ................................................................... 30
Figure 4.16: Structure of GSM network .............................................................................. 32
Figure 4.17: GSM SIM 300 module .................................................................................... 33
Figure 4.18: PIC16F877A pin description ........................................................................... 35
Figure 5.1: Overall block diagram of the system ................................................................. 38
Figure 5.2: Motion Detected and No motion Message Display .......................................... 40
Figure 6.1: Flowchart of motion detection algorithm used in MATLAB ........................... 42
Figure 6.2: Standard USB Type A/ Type B cable ............................................................... 43
Figure 6.3: USB connection of FPGA to laptop .................................................................. 44
Figure 6.4: Double click to Invoke iMPACT ...................................................................... 44
Figure 6.5: Assign New Configuration File......................................................................... 45
Figure 6.6: Program the Xilinx project ................................................................................ 45
Figure 6.7: Program succeeded mesaage ............................................................................. 46
Figure 6.8: Clock output for divide by 2 frequency divider ................................................ 47
Figure 6.9: Signals for different duty cycle ......................................................................... 48
viii
Figure 6.10: Clock output for 50% duty cycle ..................................................................... 48
Figure 6.11: Clock output for 90% duty cycle ..................................................................... 49
Figure 6.12: VGA display range .......................................................................................... 49
Figure 6.13: Clock generation ............................................................................................. 50
Figure 6.14: VGA timing diagrams ..................................................................................... 51
Figure 6.15: VGA connector ............................................................................................... 51
Figure 6.16: Total timing diagram in test bench .................................................................. 53
Figure 6.17: RTL schematic of the motion detection part with VGA ................................. 54
Figure 6.18: Flowchart of frame grabber module ................................................................ 56
Figure 8.1: Overall module of motion detection part .......................................................... 59
Figure A.1 :Xilinx Spartan 3E FPGA board ........................................................................ 64
Figure B.1: Cmos 3088 module ........................................................................................... 66
Figure C.1: PIC16F877a pin description ............................................................................. 67
Figure E.1: PCB design for camera interfacing circuit ........................................................ 71
Figure E.2: PCB design for SIM 300 ................................................................................... 71
Figure E.3: HCM 582 VGA monitor ................................................................................... 72
ix
LIST OF TABLES
Table 4.1: Control pins used in strata flash ......................................................................... 27 Table 4.2: Block RAM available in Spartan 3E device ....................................................... 28 Table 4.3: Block RAM interface signals ............................................................................. 29 Table D.1: Connection of C3088 to FPGA board ............................................................... 69 Table D.2: Hardware connection of C3088 sensor in PCB board ....................................... 70
Table D.3: Connection of LCD in FPGA board .................................................................. 70
x
LIST OF ABBREVIATIONS
A/D Analog to Digital
ADC Analog to Digital Converter
API Application Program Interface
ASIC Application Specific Integrated Circuit
AT Commands Attention Commands
CCD Charge Coupled Device
CLB Configurable Logic Block
CLK Clock
CMOS Complementary MOSFET
CPLD Complex Programmable Logic Device
CRT Cathode Ray Tube
EEPROM Electrically Erasable Programmable Read Only Memory
FPGA Field Programmable Gate Array
FSM Finite State Machine
GND Ground
GSM Global System for mobile communication
xi
HS Horizontal Synchronization
HSYNC Horizontal SYNChronization
I2C Inter IC Communication
IC Integrated Circuit
IDE Integrated Development Environment
IO Input Output
IOE Institute Of Engineering
ISE Integrated Software Environment
LAB Logic Array Block
LCD Liquid Crystal Display
LED Light Emitting Diode
LUTs Lookup tables
MATLAB MATrix LABoratory
MCU MicroController Unit
MPLAB IDE MPLAB Integrated Development Environment
OS Operating System
xii
PC Personal Computer
PCB Printed Circuit Board
PCLK Pixel CLK
PWM Pulse Width Modulation
RAM Random Access Memory
RGB Red Green Blue
RPM Revolution Per Minute
RX Receiver
TX Transmitter
SIM Subscriber Identity Module
SMS Short Messaging Service
UART Universal Asynchronous Receiver / Transmitter
USB Universal Serial Bus
Verilog HDL Verilog Hardware Description Language
VGA Video Graphic Adapter
VHDL VHSIC Hardware DescriptionLanguage
VS Vertical Synchronization
xiii
VSYNC Vertical SYNChronization
VSM Virtual system modeling
1
1. INTRODUCTION
1.1. Background
The use of technologies has been rapidly increasing from the past few decades.
Technology has brought a disastrous change in the world today. Keeping the fact in mind,
this project has been brought in existence. Since the world has changed a lot, their interest
in the technology has also gradually increased. The use of mobile is the best example we
can get. Everyone or the other in this era uses mobile technology.
A system is designed to detect intrusion, unauthorized access to the valuable and protected
item. Security alarms are used in residential, commercial, industrial, and military
properties for protection against burglary (theft) or property damage, as well as personal
protection against intruders. Car alarms likewise protect vehicles and their
contents. Prisons also use security systems for control of inmates. Some systems serve a
single purpose of burglary protection; combination systems provide both fire and intrusion
protection. Systems range from small, self-contained noisemakers, to complicated, multi-
area systems with computer monitoring and control.
A plan to develop a motion technology controlled by FPGA (field-programmable gate
array) using a GSM (Global System for Mobile Communications technology) has been
focused hereby in this project. Motion detection is a technology that can extract moving
objects from a sequence of frames. A simple algorithm for motion detection by a fixed
camera compares the current image with a reference image and simply counts the number
of different pixels. Since images will naturally differ due to factors such as varying
lighting, camera flicker and CCD dark currents, pre-processing is useful to reduce the
number of false positive alarms. More complex algorithms are necessary to detect motion
when the camera itself is moving, or when the motion of a specific object must be detected
in a field containing other movement which can be ignored. An example might be a
painting surrounded by visitors in an art gallery. But here in this project we are considering
only the simple algorithm. This is the enabling component for many important
applications, such as security monitor, vehicle detection and human activity analysis.
2
Through this technology, the motion is detected using the camera module controlled by the
Spartan-3E FPGA module and the message of motion detection, if occurred, is send to the
authorized person through GSM technology in his cell phone.
A FPGA is an integrated circuit designed to be configured by a customer or a designer
after manufacturing. This is the main brain of the project. Any type of change in the
motion is controlled by this FPGA. The FPGA used in this project, firstly controls the
detected motion and via the GSM module, the information is reported to the respected user
into his cell phone in the form of SMS. The most commonly used technology i.e. GSM
technology is the other important part of this project. The GSM standard is the most widely
accepted standard and is implemented globally. A GSM is an open, digital cellular
technology used for transmitting mobile voice and data services. A GSM digitizes and
compresses data, then sends it down through a channel with two other streams of user data,
each in its own time slot. It operates at either the 900 MHz or 1,800 MHz frequency band.
A GSM module assembles a GSM modem with standard communication interfaces like
RS-232 (Serial Port), USB etc., so that it can be easily interfaced with a computer or a
microprocessor / microcontroller based system. In this project, the GSM module sends the
data through SMS to the users’ cell phone.
A specific application is planned to be implemented in this project. Imagine a place where
there are artifacts and historical items worth thousands and lakhs which have to be
protected almost every moment from potential thieves. Hence the areas of utmost
importance are to be monitored with a camera and zone is defined over which motion or
movement is detected. After that an alarm as SMS is sent to the cell phone of the
authorized personnel to inform him/her of possible threats in the specified place.
3
1.2. Motivation
At the core, security systems are primarily designed to detect intrusions and break-ins.
Starting as a basic, security has now developed into a large industry with the addition of
total home protection, including a variety of detectors, sensors, and surveillance cameras
that can be installed around the specified area. Advances in technology have turned the
best security systems into fully automated systems, enabling owners to control lighting,
appliances, locks, and cameras remotely. Seeing the necessity of the security system all
over and learning about the advancement of the technology lead to this project.
The ability to reliably detect human motion is a useful tool for higher–level applications
that rely on visual input. An algorithm for human motion detection digests high-bandwidth
video into a compact description of the human presence in that scene. This high-level
description can then be put to use in other applications.
Some examples of applications that could be realized with reliable human motion detection
are:
• Automated surveillance for security-conscious venues such as airports, casinos,
museums, and government installations: Intelligent software could monitor security
cameras and detect suspicious behaviour. Furthermore, human operators could
search archived video for classes of activity that they specify without requiring
manual viewing of each sequence.
• Safety devices for pedestrian detection on motor vehicles: Intelligent software on a
camera-equipped car could detect pedestrians and warn the driver.
4
1.3. Objectives
The purpose of this project is to help new researchers learn and further research on their
topic of interest, which in this case is the human motion detection system. The question to
be addressed is, given a sequence of images, how is motion detected? The project is to
mainly answer this particular question addressed by providing a prototype to emulate or
prove the algorithms or techniques that are available to perform motion detection by an
input of images in a number of frames.
The main objective of this project is to design a complete framework able to automatically
detect motion in video sequences and to develop a motion technology controlled by FPGA
(Field-programmable gate array). A real-time method using CMOS camera is to be
designed which is able to deal with any surrounding environment, disregarding the
individuals’ appearance (clothes). To do so, cameras were used to take real time images
and the output resulting from background subtraction algorithm were analysed.
The problem of background subtraction can be efficiently handled with a simple algorithm
that reduces the amount of required memory and computations to the absolute minimum.
The secondary objective was to implement the system on an FPGA board exploiting the
available feature of hardware level parallelism.
5
2. LITERATURE REVIEW
Ying-Hao Yu [28] proposed the implementation of an FPGA–Based Formation Control of
Multiple Ubiquitous Indoor Robots. It explored the feasibility of using FPGA technology
for formation control of multiple indoor robots in a ubiquitous computing environment.
The computers and embedded systems, indoor robots will receive commands from users
and surrounding sensors and execute tasks such as home and office chores in a cooperative
manner. With the use of infrared sensors and a global high-resolution digital camera for
environment sensing, all computation required for data acquisition, image processing, and
closed-loop servo control was then performed on an FPGA chip.
Tianguang Zhang, Haiyan Wu and Alexander Borst [29] proposed the formulation of an
array of biologically inspired elementary motion detectors (EMDs) on an FPGA platform.
The Reichardt detector is Na well-known model which describes the process of local
motion detection in the fly. In a structure of the fly brain called ‘lobula plate’, large
neurons are found which integrate the local motion signals and additionally form extensive
connections amongst themselves. These neurons have large receptive fields and respond
best to particular flow-fields such as occurring during certain maneuvers of the fly in free
flight. In engineering applications such as robotics, driver assistance systems or
surveillance systems, a camera system is usually used as a sensor to gather information
about the environment. Motion perception based on the fly’s vision system is
computationally cheap and, thus, particularly suited for real-time applications.
Yu Shi et. at al [10] have proposed a smart camera that can not only see but also think and
act. A smart camera is an embedded vision system which captures and processes image to
extract application-specific information in real time. This FPGA-based smart camera built
from scratch that can recognize simple hand gestures. They have implement object
segmentation, feature extraction, and gesture classification. For object segmentation, they
have applied skin color detection and contour tracing techniques to segment hand from
background. For feature extraction, they calculated the center of mass of the segmented
hand. For gesture classification, they applied a simple non-trainable neural network (with
hard-coded weightings) to classify gestures.
6
Antonio de Sousa[9] again proposed a "smart camera" which is basically a video camera
coupled to a computer vision system in a tiny package. A static thresholding algorithm is
presented which demonstrates the ability to track non-uniformity in the inspection target. A
multi camera inspection system application is presented where a maximum of twenty smart
cameras may be networked together to a single host computer.
Hence, based upon the above ideas, a system was planned to be build that detect the
motion and notify the authenticated user at the same time by a SMS via the GSM modem.
7
3. SOFTWARE AND TOOLS USED
This project for motion detection is controlled by FPGA and the information of the
intrusion detected is sent to mobile via GSM technology. For programming in FPGA,
VERILOG programming was used in Xilinx ISE Software. Before implementing in FPGA,
all the simulation related to motion detection is done in MATLAB for simplicity. Since
GSM interfacing is done with PIC16F877a, MPLAB and PROTEUS were also used. The
designing of related pcb board of camera and GSM interfacing is done with the help of pcb
wizard. Hence, both hardware and software components is included in this project.
The hardware modules which were used are listed below.
• FPGA board
• Camera module interfaced with FPGA
• GSM module
• A mobile phone
• PIC 16F877a
Similarly, the software used in implementing our project is listed as:
• VERILOG
• MATLAB
• XILINX ISE
• PROTEUS
• MODELSIM
• PCB Wizard
3.1. For Initial Simulation
3.1.1. MATLAB
The motion detection simulation process was first accomplished using MATLAB. MATLAB
is a high level technical computing language and interactive environment for algorithm
8
development, data visualization, data analysis, and numeric computation. It can be used in a
wide range of applications, including signal and image processing, communications, control
design, test and measurement, financial modeling and analysis, and computational biology.
MATLAB was used to test the algorithms and then to model the system. The input video in
real time was taken via the web camera of the laptop. The average of 10 frames were taken as
the background image. The camera was triggered and the background subtraction algorithm
was implemented to calculate the absolute difference.
3.2. For GSM Interfacing
3.2.1. MPLAB IDE
MPLAB IDE is a program that is used to burn the codes to run on the corresponding
microcontroller (PIC16F877A in our project). This software runs on a PC to develop
applications for Microchip microcontrollers and digital signal controllers. The IDE
provides a single integrated environment to develop code for embedded microcontroller.
The compiler associated with the IDE compiles the high level C code and corresponding
hex code is generated. The compiler can optimize high level C code, eliminate code that
will never be executed, share common code fragments among multiple functions and can
identify data and registers that are used inefficiently optimizing their access. The MPLAB
graphical user interface serves as a single environment to write, compile and debug for
embedded applications.
3.2.2. Proteus VSM
After the generation of hex code to be written into microcontroller, a program is required
that simulates the required operation to be performed along with physical components like
switches, virtual terminal, resistors, capacitors, voltage dividers etc. Proteus VSM is the
desired program in which the code is loaded in the microcontroller along with the required
circuit configuration and the simulation is carried on. It tests whether the software and
hardware configuration meets the desired requirement in the project.
9
Proteus Virtual system modeling (VSM) combines animated components, mixed mode
SPICE circuit simulation, and microcontroller models to facilitate co-simulation of
complete microcontroller based design. It is possible to develop and test such designs
before a physical prototype is constructed. This is possible because we can interact with
the design using circuit indicators like LED, display panels, actuators etc. It also provides
extensive debugging facility by employing break-points, single stepping and variable
display of both assembly code and high level language source code.
3.3. For FPGA Based Design
3.3.1. ISE
The Integrated Software Environment (ISETM) is the Xilinx design software suite for a
complete embedded system design, allowing the user to take the design from design entry
through Xilinx devices (FPGAs and CPLDs) programming. Various utilities such as design
entry, constraints entry, simulation, I/O assignments, synthesis and implementation, device
configuration, power analysis, timing analysis, logic placement and routing, device
programming and in-system design debugging along with the compiler and various
libraries required for hardware programming are all integrated into ISE. The edition
(Xilinx 13.2) fully supports Spartan 3 (Spartan 3A and 3E) family of FPGAs , Virtex series
(Virtex 4, Virtex 5, Virtex )as well as the family of CPLDs.
3.3.2. Verilog
Verilog is a hardware description language used in electronic design automation to
describe digital and mixed-signal systems such as field-programmable gate arrays and
integrated circuits. It allows the behaviour of the required system to be described and
verified before synthesis tools translate the design into real hardware. Unlike procedural
computing languages verilog is a dataflow language thus allowing the concurrent system
description.
10
3.3.3. Modelsim
ModelSim is a popular hardware simulation and debug environment for ASIC and FPGA
designs. It is an easy-to-use yet versatile VHDL/ Verilog/ SystemC simulator by Mentor
Graphics. It supports behavioral, register transfer level, and gate level modelling in all
platforms. It combines high performance and high capacity with the code coverage and
debugging capabilities required to simulate larger blocks and systems.
3.4. Design Tools
3.4.1. PCB Wizard
The PCB wizard is a powerful and a highly innovative package for designing single-sided
and double-sided printed circuit boards (PCBs). It provides a comprehensive range of tools
covering all the traditional steps in PCB production, including schematic drawing,
schematic capture, component placement, automatic routing. Circuit diagrams are drawn
using circuit symbol components connected with wires. Components are simply dragged
and dropped and arranged in the working window and necessary connections are made
between the component terminals.
11
4. BACKGROUND THEORY
4.1. FPGA Board
A field-programmable gate array (FPGA) is an integrated circuit designed to be configured
by a customer or a designer after manufacturing, hence "field-programmable". The FPGA
configuration is generally specified using a hardware description language (HDL), similar
to that used for an application-specific integrated circuit (ASIC). Contemporary FPGAs
have large resources of logic gates and RAM blocks to implement complex digital
computations. As FPGA designs employ very fast IOs and bidirectional data buses it
becomes a challenge to verify correct timing of valid data within setup time and hold
time. Floor planning enables resources allocation within FPGA to meet these time
constraints. FPGAs can be used to implement any logical function that an ASIC could
perform. The ability to update the functionality after shipping, partial re-configuration of a
portion of the design and the low non-recurring engineering costs relative to an ASIC
design (not withstanding the generally higher unit cost), offer advantages for many
applications.
FPGAs contain programmable logic components called "logic blocks", and a hierarchy of
reconfigurable interconnects that allow the blocks to be "wired together"—somewhat like
many (changeable) logic gates that can be inter-wired in (many) different configurations.
Logic blocks can be configured to perform complex combinational functions, or merely
simple logic gates like AND and XOR. In most FPGAs, the logic blocks also include
memory elements, which may be simple flip-flops or more complete blocks of memory.
Historically, FPGAs have been slower, less energy efficient and generally achieved less
functionality than their fixed ASIC counterparts. An older study had shown that designs
implemented on FPGAs need on average 40 times as much area, draw 12 times as much
dynamic power, and run at one third the speed of corresponding ASIC implementations.
More recently, FPGAs such as the Xilinx Virtex-7 or the Altera Stratix 5 have come to
rival corresponding ASIC and ASSP solutions by providing significantly reduced power,
increased speed, lower materials cost, minimal implementation real-estate, and increased
12
possibilities for re-configuration 'on-the-fly'. Where previously a design may have included
6 to 10 ASICs, the same design can now be achieved using only one FPGA.
Advantages include the ability to re-program in the field to fix bugs, and may include a
shorter time to market and lower non-recurring engineering costs. Vendors can also take a
middle road by developing their hardware on ordinary FPGAs, but manufacture their final
version as an ASIC so that it can no longer be modified after the design has been
committed.
4.1.1. Architecture
The most common FPGA architecture consists of an array of logic blocks (called
Configurable Logic Block, CLB, or Logic Array Block, LAB, depending on vendor), I/O
pads, and routing channels. Generally, all the routing channels have the same width
(number of wires). Multiple I/O pads may fit into the height of one row or the width of one
column in the array.
An application circuit must be mapped into an FPGA with adequate resources. While the
number of CLBs/LABs and I/Os required is easily determined from the design, the number
of routing tracks needed may vary considerably even among designs with the same amount
of logic. For example, a crossbar switch requires much more routing than asystolic
array with the same gate count. Since unused routing tracks increase the cost (and decrease
the performance) of the part without providing any benefit, FPGA manufacturers try to
provide just enough tracks so that most designs that will fit in terms of Lookup
tables (LUTs) and IOs can be routed. This is determined by estimates such as those derived
from Rent's rule or by experiments with existing designs.
In general, a logic block (CLB or LAB) consists of a few logical cells (called ALM, LE,
Slice etc.). A typical cell consists of a 4-input LUT, a Full adder (FA) and a D-type flip-
flop, as shown below. The LUTs are in this figure split into two 3-input LUTs. In normal
mode those are combined into a 4-input LUT through the left mux. In arithmetic mode,
their outputs are fed to the FA. The selection of mode is programmed into the middle
multiplexer. The output can be either synchronous or asynchronous, depending on the
13
programming of the mux to the right, in the figure example. In practice, entire or parts of
the FA are put as functions into the LUTs in order to save space.
4.1.2. FPGA design and programming
To define the behavior of the FPGA, the user provides a hardware description
language (HDL) or a schematic design. The HDL form is more suited to work with large
structures because it’s possible to just specify them numerically rather than having to draw
every piece by hand. However, schematic entry can allow for easier visualization of a
design.
Then, using an electronic design automation tool, a technology-mapped net list is
generated. The net list can then be fitted to the actual FPGA architecture using a process
called place-and-route, usually performed by the FPGA Company’s proprietary place-and-
route software. The user will validate the map, place and route results via timing
analysis, simulation, and other verification methodologies. Once the design and validation
process is complete, the binary file generated (also using the FPGA company’s proprietary
software) is used to (re)configure the FPGA. This file is transferred to the FPGA/CPLD via
a serial interface (JTAG) or to an external memory device like an EEPROM.
The most common HDLs are VHDL and Verilog, although in an attempt to reduce the
complexity of designing in HDLs, which have been compared to the equivalent of
assembly languages, there are moves to raise the abstraction level through the introduction
of alternative languages. National Instrument’s Lab VIEW graphical programming
language (sometimes referred to as “G”) has an FPGA add-in module available to target
and program FPGA hardware.
To simplify the design of complex systems in FPGAs, there exist libraries of predefined
complex functions and circuits that have been tested and optimized to speed up the design
process. These predefined circuits are commonly called IP cores, and are available from
FPGA vendors and third-party IP suppliers (rarely free and typically released under
proprietary licenses). Other predefined circuits are available from developer communities
14
such as Open Cores (typically released under free and open source licenses such as
the GPL, BSD or similar license), and other sources.
In a typical design flow, an FPGA application developer will simulate the design at
multiple stages throughout the design process. Initially the RTL description in VHDL or
Verilog is simulated by creating test benches to simulate the system and observe results.
Then, after the synthesis engine has mapped the design to a net list, the net list is translated
to a gate level description where simulation is repeated to confirm the synthesis proceeded
without errors. Finally the design is laid out in the FPGA at which point propagation
delays can be added and the simulation run again with these values back-annotated onto
the net list.
Image processing is difficult to achieve on a serial processor. This is due to the large data
set required to represent the image and the complex operations that need to be performed
on the image. Many image processing applications require that several operations be
performed on each pixel in the image resulting in an even large number of operations per
second. Thus the perfect alternative is to make use of an FPGA.
Why FPGA?
� The biggest advantage of FPGA’s is usually speeding: problems can be solved in
‘hardware’ rather than in software, make use of parallelization.
� Own peripherals, CPU cores, etc can be created.
� FPGA don’t run on a core, but runs multithreaded cores.
� To learn FPGA and Verilog programming.
15
Figure 4.1: SPARTAN 3E FPGA Board
16
4.2. C3088 Camera Interfacing
4.2.1. Specifications and Features
C3088 1/4" Color Camera Module With Digital Output.
The camera is a 1/4" color camera module with digital
output. It uses OmniVision's CMOS image sensor
OV6620. Combining CMOS technology together with
an easy to use digital interface makes C3088 a low cost
solution for higher quality video image application. The
digital video port supplies a continuous 8/16 bit-wide
image data stream. All camera functions, such as
exposure, gamma, gain, white balance, color matrix,
windowing, are programmable through I2C interface.
The camera is supplied as standard with an f4.9mm F2.8
lens on a 12mm screw mount allowing focal point adjustment.
Many high level imaging projects can be made like Realtime Object Tracking, Color
recognition, Image processing, Face Recognition and such.In combine with OV511+, USB
controller chip, it will easily form a USB camera for PC application.
Camera CIF Camera Module Features
• OV6620 CMOS color image sensor
• 101,376 pixels, 1/4" lens, CIF/QCIF format
• Array size: 356 × 292 pixels CIF, 176 × 144 pixels QCIF
• Pixel size: 9.0 × 8.2 µm
• Effective image area: 3.1 × 2.5 mm
• Board dimensions: 40 × 28 mm
• Lens: f4.9mm, F2.8
• Progressive Scanning
• 8/16-bit video data: CCIR601, CCIR656, ZV port
• Data format: YCrCb 4:2:2, GRB 4:2:2, RGB raw
Figure 4.2 : Cmos 3088 module
17
• I2C interface
• Electronic Exposure: 500:1 for selected FPS
• Gain / white balance control
• Gamma Correction: 0.45/0.55/1.0
• Image enhancement: brightness, contrast, gamma, saturation, sharpness, window,
etc.
• SCCB programmable (400 kb/s)
• S/N Ratio: >48dB (AGC off; Gamma=1)
• Internal / external synchronization scheme
• Frame exposure / line exposure option
• Min. Illumination: <3 lux @ f1.2
• Dynamic range >72dB
• Operation Voltage: 5V DC ±5%
• Operation Current: <80mW Active; <30µW Standby
The pin description (32 pins) is mentioned below:
Figure 4.3 :Pin description of cmos 3088
18
In order to operate the camera effectively, a PCB circuit was designed so that the camera
could be mounted in an efficient manner to capture images. The circuit configuration for
camera is shown in Figure 4.4.
4.2.2. Video Signal
The OV6620 image sensor outputs an 8-bit parallel gray-scale digital video signal on pins
Y7-Y0. It also outputs PCLK, VSYNC, and HREF for timing. The timing diagram is
shown in Figure 4.5. A 4 millisecond (ms) pulse on VSYNC indicates the start of a new
frame. After this, data is output row by row. Every row, HREF goes high and data is
subsequently sent out on Y7-Y0, one pixel per rising edge of PCLK. HREF is high for
about 0.6 ms, during which time 352 pixels are clocked out. HREF is then low for about
1.4 ms before starting the next row.
Figure 4.4 :Circuit configuration of camera
19
Figure 4.5 :Signals href, pclk, vsync timing diagram
The array size of C3088 camera is 352*288 pixels. It takes 60 frames per one second. If
the total pixel size for one frame is taken, then the total size occupying in memory is
352*288*8 bits=811008 bits=792 Kbits=99Kbytes. Hence, while taking the total pixel of
the array of camera, the frame can’t be stored in block memory as the memory of block
memory is limited to 16Kbytes. So, for the purpose of storing two frames, one as
background image, and the other one as the real time frame whose motion( if any) is to be
detected, the process of interleaving was followed.
In interleaving, each consecutive fourth pixel and fourth line is taken. It means, 352 pixels
are divided by 4, and similarly, 288 lines are also divided by 4. Thus,
Total memory required for each frame = (352/4) * (288/4) * 8
= 50688 bits
= 49.5 Kbits
= 6.1875Kbyte.
4.3. Memory Usage
FPGA memory architectures
Each FPGA vendor has its own FPGA architecture, but in general terms they are all a
variation of that shown in Figure 4.6. The architecture consists of configurable logic
blocks, configurable I/O blocks, and programmable interconnect. Also, there will be clock
20
circuitry for driving the clock signals to each logic block. Additional logic resources such
as ALUs, memory, and decoders may also be available. The three basic types of
programmable elements for an FPGA are static RAM, anti-fuses, and flash EPROM.
Figure 4.6 :Generic FPGA architecture
4.3.1. Configurable Logic Blocks (CLBs)
These blocks contain the logic for the FPGA. In the large-grain architecture used by all
FPGA vendors today, these CLBs contain enough logic to create a small state machine as
illustrated in Figure 4.7. The block contains RAM for creating arbitrary combinatorial
logic functions, also known as lookup tables (LUTs). It also contains flip-flops for clocked
storage elements, along with multiplexers in order to route the logic within the block and
to and from external resources. The multiplexers also allow polarity selection and reset and
clear input selection.
21
Figure 4.7: FPGA configuration logic block (CLB)
Configurable I/O Blocks:
A Configurable input/output (I/O) Block, as shown in Figure 4.8, is used to bring signals
onto the chip and send them back off again. It consists of an input buffer and an output
buffer with three-state and open collector output controls. Typically there are pull up
resistors on the outputs and sometimes pull down resistors that can be used to terminate
signals and buses without requiring discrete resistors external to the chip.
The polarity of the output can usually be programmed for active high or active low output,
and often the slew rate of the output can be programmed for fast or slow rise and fall times.
There are typically flip-flops on outputs so that clocked signals can be output directly to
the pins without encountering significant delay, more easily meeting the setup time
requirement for external devices. Similarly, flip-flops on the inputs reduce delay on a
signal before reaching a flip-flop, thus reducing the hold time requirement of the FPGA.
22
Figure 4.8: FPGA configuration I/O block
Programmable Interconnect: In Figure 4.9, a hierarchy of interconnect resources can be
seen. There are long lines that can be used to connect critical CLBs that are physically far
from each other on the chip without inducing much delay. Theses long lines can also be
used as buses within the chip.
There are also short lines that are used to connect individual CLBs that are located
physically close to each other. Transistors are used to turn on or off connections between
different lines. There are also several programmable switch matrices in the FPGA to
connect these long and short lines together in specific, flexible combinations.
Three-state buffers are used to connect many CLBs to a long line, creating a bus. Special
long lines, called global clock lines, are specially designed for low impedance and thus fast
propagation times. These are connected to the clock buffers and to each clocked element in
each CLB. This is how the clocks are distributed throughout the FPGA, ensuring minimal
skew between clock signals arriving at different flip-flops within the chip.
In an ASIC, the majority of the delay comes from the logic in the design, because logic is
connected with metal lines that exhibit little delay. In an FGPA, however, most of the
delay in the chip comes from the interconnect, because the interconnect – like the logic – is
23
fixed on the chip. In order to connect one CLB to another CLB in a different part of the
chip often requires a connection through many transistors and switch matrices, each of
which introduces extra delay.
Figure 4.9: FPGA programmable interconnect
Xilinx Spartan 3E FPGA has large number of configurable logic blocks. So, 101376 pixels
values (352*288 pixels) were directly stored in the blocks. But, this value was so large that
the program was not synthesized .CLB is not actually a memory but the array which uses
logic blocks present in the FPGA. Since the memory required to store the whole image is
so large that it requires a huge amount of memory for storage of whole pixels of data.
Hence, Block Ram was used for memory storage.
4.3.2. Strata Flash:
The Spartan-3E FPGA Starter Kit boards include a 128 M-bit (16Mbyte) Intel Strata Flash
parallel NOR Flash PROM. Although the XC3S500E FPGA only requires just slightly
over 2 M-bits per configuration image, the FPGA-to-Strata Flash interface on the board
support up to a 256 M-bit Strata Flash. The Spartan-3E FPGA Starter Kit board is shipped
24
with a 128 M-bit device.
In general, the Strata Flash device connects to the XC3S500E to support Byte Peripheral
Interface (BPI) configuration. The upper four address bits from the FPGA, A [23:19] do
not connect directly to the Strata Flash device. Instead, the XC2C64 CPLD controls the
pins during configuration.
Product Features Performance
—110/120/150 ns Initial Access Speed for 32/64/128 Mbit Densities
—25 ns Asynchronous Page-Mode Reads
—32-Byte Write Buffer
—6.8 µs per Byte Effective Programming Time Software
—Program and Erase suspend support
—Flash Data Integrator (FDI), Common Flash Interface (CFI) Compatible Security
—128-bit Protection Register
—64-bit Unique Device Identifier
—64-bit User Programmable OTP Cells
—Absolute Protection with VPEN =GND
—Individual Block Locking
—Block Erase/Program Lockout during Power Transitions Architecture
—Multi-Level Cell Technology: High Density at Low Cost
—High-Density Symmetrical 128-Kbyte Blocks
—128 Mbit (128 Blocks)
—64 Mbit (64 Blocks)
—32 Mbit (32 Blocks)
Quality and Reliability
—Operating Temperature: -40 °C to +85 °C
—100K Minimum Erase Cycles per Block
—0.25 µm ETOX™ VI Process
25
Packaging and Voltage
—56-Lead TSOP Package
—64-Ball Intel
Easy BGA Package
—48-Ball Intel
VF BGA Package (32 M) (x16 only)
—VCC = 2.7 V – 3.6 V
—VCCQ = 2.7 V – 3.6 V
Figure 4.10: Block diagram of Intel strata flash
26
Figure 4.11: Memory map 3 volt strata flash memory
The connection of Strata Flash memory to Spartan 3E FPGA is as shown:
Figure 4.12: Connection of strata flash memory to spartan 3E FPGA
27
In order to carry any operation in Strata Flash memory, a number of functions like Erase
Setup, Erase confirm, Write setup, Write data, Status Register Setup, Status Register
Confirm, Read Setup, Read data etc were performed. Some of the control pins used in
Strata flash memory is as shown in the following table:
Table 4.1: Control pins used in strata flash
Since the memory of block RAM is very low, and doesn’t have the capacity to hold all the
frames in memory, Strata Flash memory was tried to accomplish the project by storing
images. To write a whole pixel data of an image, a number of control functions were
needed to be carried on at the same pclk. So, on using this memory, many synchronization
problems were encountered.
4.3.3. Block RAM
The total amount of block RAM memory depends on the size of the Spartan-3 Generation
FPGA. Xilinx Spartan 3E FPGA board has a total of 360 K bits (45 Kbytes) as Block
RAM. Each block RAM contains 18K bytes of fast static RAM, 16K bytes of which is
allocated to data storage and, in some memory configurations, an additional 2K bytes
allocated to parity or additional data bits.
28
Table 4.2: Block RAM available in Spartan 3E device
Block RAM location and surrounding neighbourhood:
Block RAM is organized in
columns. Figure 4.13 shows the
Block RAM column arrangement
for the XC3S200. The XC3S50
has a single column of block
RAM, located two CLB columns
from the left edge of the device.
Spartan-3 devices larger than the
XC3S50 have two columns of
block RAM, adjacent to the left
and right edges of the die, located two columns of CLBs from the I/Os at the edge. In
addition to the block RAM columns at the edge, the XC3S4000 and XC3S5000 have two
additional columns—a total of four columns—nearly equally distributed between the two
edge columns.
Block RAM data flow:
1. Port A behaves as an independent single-port RAM supporting simultaneous read and
write operations using a single set of address lines.
2. Port B behaves as an independent single-port RAM supporting simultaneous read and
write operations using a single set of address lines.
3. Port A is the write port with a separate write address and Port B is the read port with a
separate read address. The data widths for Port A and Port B can be different also.
Figure 4.13: Block RAM and surrounding neighbourhood
29
4. Port B is the write port with a separate write address and Port A is the read port with a
separate read address. The data widths for Port B and Port A can be different also.
Single port or simple dual port configuration can be used according to the program to be
processed on. For the purpose of interfacing camera to FPGA, two clocks were used; clock
‘a’ (to take the pixel clock from camera working at 8.86MHZ) and system clock i.e. clock
‘b’ (to run the operations in FPGA board working at 50MHz). Due to this, a simple dual
port configuration was used so that the data from camera can be written to the address at
certain clock rate and read from the respective address at another clock rate. The Block
RAM interface signals are described in Table 4.3.
Table 4.3: Block RAM interface signals
Figure 4.14: Dual Port and Single port Block RAM memory diagram
30
In dual port configuration, the block RAM
memory has two completely independent
access ports, labelled Port A and Port B. The
structure is fully symmetrical, and both ports
are interchangeable and both ports support
data read and write operations. Each
memory port is synchronous, with its own
clock, clock enable, and write enable. Read
operations are also synchronous and require a clock edge and clock enable.
The CORE Generator system creates a wide variety of memories with very flexible aspect
ratios. Unlike the actual block RAM primitive, the CORE generator system does not
differentiate between data and parity bits and considers all bits data bits. For dual-port
memories, each port can have different organizations or aspect ratios. Within the CORE
Generator system, locate the Memory Size group and enter the desired memory
organization.
As dual-port RAM, both ports operate independently while accessing the same set of 18K-
byte memory cells.
Address Bus — ADDR[#:0] (ADDRA[#:0], ADDRB[#:0])
The address bus selects the memory cells for read or writes operations. The width of the
address bus input determines the required address bus width.
Control Inputs:
Clock — CLK (CLKA, CLKB):
Each port is fully synchronous with independent clock pins. All port input pins have setup
time referenced to the port CLK pin. The data bus has a clock-to-out time referenced to the
CLK pin.
Clock polarity is configurable and is rising edge triggered by default. With default polarity,
a Low-to-High transition on the clock (CLK) input controls read, write, and reset
operations.
Figure 4. 15: Port A and Port B of block RAM
31
Write Enable — WE (WEA):
The write enable input, WE, controls when data is written to RAM. When both EN and
WE are asserted at the rising clock edge, the value on the data and parity input buses is
written to memory location selected by the address bus. The data output latches are loaded
or not loaded according to the WRITE_MODE attribute. The polarity of WE is
configurable and is active High by default. If data is to be written to the address WEA is
set as ‘1’ and if the data is to be read from consecutive address, WEA is set as ‘0’.
4.4. GSM Module
4.4.1. General Description
GSM stands for Global System for Mobile Communications. It is regarded as the world’s
most widely used cell phone technology. Cell phones use a cell phone service carrier’s
GSM network by searching for cell phone towers in the nearby area. The origins of GSM
can be traced back to 1982 when the Groupe Spécial Mobile (GSM) was created by the
European Conference of Postal and Telecommunications Administrations (CEPT) for the
purpose of designing a pan-European mobile technology.
It is approximated that 80 percent of the world uses GSM technology when placing
wireless calls, according to the GSM Association (GSMA), which represents the interests
of the worldwide mobile communications industry. This amounts to nearly 3 billion global
people. It is a standard set developed by the European Telecommunications Standards
Institute (ETSI) to describe protocols for second generation (2G) digital cellular networks
used by mobile phones.
The GSM standard was developed as a replacement for first generation (1G) analog
cellular networks, and originally described a digital, circuit switched network optimized
for full duplex voice telephony. This was expanded over time to include data
communications, first by circuit switched transport, then packet data transport via GPRS
(General Packet Radio Services) and EDGE (Enhanced Data rates for GSM Evolution or
EGPRS). Further improvements were made when the 3GPP developed third generation
(3G) UMTS standards followed by fourth generation (4G) LTE Advanced standards.
32
The structure of a GSM network is given in Figure 4.16.
Figure 4.16: Structure of GSM network
The GSM network can be divided into four main parts:
• The Mobile Station (MS).
• The Base Station Subsystem (BSS).
• The Network and Switching Subsystem (NSS).
• The Operation and Support Subsystem (OSS).
4.4.2. Technical Description of GSM
The Short Message Service – Point to Point (SMS-PP) was originally defined in GSM
recommendation 03.40, which is now maintained in 3GPP as TS 23.040. GSM 03.41 (now
3GPP TS 23.041) defines the Short Message Service – Cell Broadcast (SMS-CB), which
allows messages (advertising, public information, etc.) to be broadcast to all mobile users
in a specified geographical area.
Messages are sent to a Short message service center (SMSC) which provides a "store and
forward" mechanism. It attempts to send messages to the SMSC's recipients. If a recipient
is not reachable, the SMSC queues the message for later retry. Some SMSCs also provide a
"forward and forget" option where transmission is tried only once.
33
AT commands
Many mobile and satellite transceiver units support the sending and receiving of SMS
using an extended version of the Hayes command set, a specific command language
originally developed for the Hayes Smartmodem 300-baud modem in 1977.
The connection between the terminal equipment and the transceiver can be realized with a
serial cable (e.g. USB), a Bluetooth link, an infrared link, etc. Common AT commands
include AT+CMGS (send message), AT+CMSS (send message from storage), AT+CMGL
(list messages) and AT+CMGR (read message).
However, not all modern devices support receiving of messages if the message storage (for
instance the device's internal memory) is not accessible using AT commands.
GSM SIM 300 module:
Figure 4.17: GSM SIM 300 module
This GSM Modem can accept any GSM network operator SIM card and act just like a
mobile phone with its own unique phone number. RS232 port can be used to communicate
and develop embedded applications. USB-CDC was used for the communication between
GSM modem and laptop without using the RS232 port.
The modem can either be connected to PC serial port directly or to any microcontroller. In
this project, it was first connected to PC via USB CDC to carry the operation of sending
the message. Later, it was connected to the microcontroller PIC16F877A with the code
34
written to perform the same operation. It can be used make/receive voice calls. Likewise, it
can also be used in GPRS mode to connect to internet and do many applications for data
logging and control.
Real Term was used as hyper terminal to connect the GSM SIM 300 modem with the PC
via USB CDC. In this process, the following AT commands were used to send message
from GSM modem to the authorized mobile number used in the respective AT commands.
The following commands were used:
1. AT
2. AT+CGMF=1 , press enter(Instruct the GSM modem to operate in SMS text mode)
3. AT+CMGS=”mobile number” ( The number is written where the text message is to
be sent)
4. Type the message to send via SMS.
5. Press CTRL+Z , which indicates the end of message.
6. If the SMS sending is successful, “ok” will be displayed along with the message
number.
Hence, in this way, the message “MOTION DETECTED” was sent successfully to mobile
phone via GSM.
4.5. PIC16F877A
Microcontroller PIC16F877A is one of the PIC Micro Family microcontroller which is
popular at this moment, start from beginner until all professionals. PIC16F877A have 40
pin by 33 path of I/O. PIC16F877A perfectly fits many uses, from automotive industries
and controlling home appliances to industrial instruments, remote sensors, electrical door
locks and safety devices. It is also ideal for smart cards as well as for battery supplied
devices because of its low consumption. EEPROM memory makes it easier to apply
microcontrollers to devices where permanent storage of various parameters is needed
(codes for transmitters, motor speed, receiver frequencies, etc.). Low cost, low
consumption, easy handling and flexibility make PIC16F877A applicable even in areas
where microcontrollers had not previously been considered (example: timer functions,
35
interface replacement in larger systems, coprocessor applications, etc.)
Figure 4.18: PIC16F877A pin description
PORTC is an 8-bit wide, bidirectional port. The corresponding data direction register is
TRISC. Setting a TRISC bit (= 1) will make the corresponding PORTC pin an input (i.e.,
put the corresponding output driver in a High-Impedance mode). Clearing a TRISC bit (=
0) will make the corresponding PORTC pin an output (i.e., put the contents of the output
latch on the selected pin).PORTC is multiplexed with several peripheral functions. PORTC
pins have Schmitt Trigger input buffers.
When enabling peripheral functions, care should be taken in defining TRIS bits for each
PORTC pin. Some peripherals override the TRIS bit to make a pin an output, while other
peripherals override the TRIS bit to make a pin an input. Since the TRIS bit override is in
effect while the peripheral is enabled, read-modify- write instructions (BSF, BCF,
XORWF) with TRISC as the destination, should be avoided. The user should refer to the
corresponding peripheral section for the correct TRIS bit settings.
PORTD is an 8-bit port with Schmitt Trigger input buffers. Each pin is individually
configurable as an input or output. PORTD can be configured as an 8-bit wide
microprocessor port (Parallel Slave Port) by setting control bit, PSPMODE (TRISE<4>).
In this mode, the input buffers are TTL.
36
Peripheral Features:
• Timer0: 8-bit timer/counter with 8-bit pre scalar
• Timer1: 16-bit timer/counter with pre scalar, can be incremented during Sleep via
external crystal/clock
• Timer2: 8-bit timer/counter with 8-bit period register, pre scaler and post scaler
• Two Capture, Compare, PWM modules
• Synchronous Serial Port (SSP) with SP (Master mode) and I2C(Master/Slave)
• Universal Synchronous Asynchronous Receiver
• Transmitter (USART/SCI) with 9-bit address detection
• Parallel Slave Port (PSP) – 8 bits wide with external RD, WR and CS controls
(40/44-pin only)
• Brown-out detection circuitry for Brown-out Reset (BOR)
Analog Features:
• 10-bit, up to 8-channel Analog-to-Digital Converter (A/D)
• Brown-out Reset (BOR)
• Analog Comparator module (Two analog comparators , Programmable on-chip
voltage reference (VREF) module , Programmable input multiplexing from device
inputs and internal voltage reference , Comparator outputs are externally
accessible)
Special Microcontroller Features:
• 100,000 erase/write cycle Enhanced Flash program memory typical
• 1,000,000 erase/write cycle Data EEPROM memory typical
• Data EEPROM Retention > 40 years
• Self-reprogrammable under software control
• In-Circuit Serial Programming(ICS) via two pins
• Single-supply 5V In-Circuit Serial Programming
• Watchdog Timer (WDT) with its own on-chip RC oscillator for reliable operation
• Programmable code protection
• Power saving Sleep mode
• Selectable oscillator options
• In-Circuit Debug (ICD) via two pins
37
CMOS Technology:
• Low-power, high-speed Flash/EEPROM technology
• Fully static design
• Wide operating voltage range (2.0V to 5.5V)
• Commercial and Industrial temperature ranges
• Low-power consumption
38
5. SYSTEM DESIGN
5.1. System Block Diagram
Figure 5.1: Overall block diagram of the system
CMOS 3088
SPARTAN 3E FPGA BOARD
PIC 16F877a Microcontroller
CELL PHONE
MOTION DETECTED
39
5.2. Diagram Description
As per the title of the project, “FPGA Based Motion Detection and Alarm System using
GSM Technology”, it has a lot of smaller subparts to deal with. Firstly, the core and the
major part is to deal with the FPGA board. Second part is the interfacing of camera with
SPARTAN 3E FPGA board. It is done to capture the image continuously for the secured
area. All the image reading, writing and background subtraction processes are done in
FPGA board with the help of Xilinx software using verilog programming. Thirdly, the
work is to interface the PIC 16F877a with our FPGA board so that some necessary signals
could be transmitted from FPGA board to the PIC when and as required. The other part is
interfacing of GSM with the PIC 16F877a microcontroller. This is done for the transfer of
the motion detected message to the cell phone of the authorized person when the FPGA
board with the help of camera detects the occurrence of the motion.
Block diagram shown in figure 5.1 shows how all the subparts are connected to make the
overall system of the project and how it functions. Cmos 3088 camera is interfaced with
the SPARTAN 3E FPGA board to continuously scan the area for the security purpose. The
capturing of the image using camera and the writing of the pixel value of each image to the
memory is done using frame grabber module which uses Finite state machine (FSM)
technique. Block Ram is used as a memory to store the pixel values and to retrieve it when
needed. For finding whether the motion is detected or not background subtraction
algorithm was used, which calculates the difference between the set up background frame
and the captured current frames. If number of differences exceeds the threshold value, it
was considered as motion detection. Threshold value is specified because of the amount of
error rate occurring due to noise. The signal from the FPGA is to be sent to the PIC
16F877a microcontroller, which then controls the GSM module. If the signal sent to the
PIC is such that it signifies that a motion has detected, then the GSM module gets activated
and the message is sent to the cell phone of the authorized person as an alarm for him
saying that the motion has detected. If no motion is encountered, GSM module does not
get any signal from PIC so that it can get activated and the scanning process is continued
continuously.
40
Figure 5.2: Motion Detected and No motion Message Display
41
6. SIMULATION AND ANALYSIS
6.1. Algorithm used in MATLAB
The algorithm used for the motion detection was a simple background subtraction
algorithm where the background frame was subtracted from the continuously scanned
current frames. While simulating in MATLAB, the percentage difference of the two
frames was calculated using the formula. If the difference was beyond the specified
threshold, the motion detected output was observed and if the difference was very less, the
little difference could be due to noise too, that part of difference was neglected and was not
considered as the detection of motion. In matlab programming, an average of 10 initial
frames was taken as a background frame and the average of the next consecutive 10 frames
as a current frame. The formula used for the percentage difference calculation was;
percentDiff = 200 * mean(imDiff(:)) / mean(imSum(:))
Where,
percentDiff is the calculated percentage difference of the two images
imDiff is the subtraction of the two images and
imSum is the addition of the two images.
Step wise procedure to detect the motion in MATLAB is as follows;
Step 1: Connect camera to matlab using videoinput.
Step 2: Take video frames of the monitored environment and find the consequent
background frame by averaging the first few frames.
Step 3: Start taking frames that are the current frames.
Step 4: Subtract the current frame from background frame.
Step 5: Set a threshold.
Step 6: Compare the value of the subtracted image with a threshold. If the threshold
exceeds a certain predefined value then display motion detected else no motion detected.
42
Flowchart used for this system:
Y
N
Figure 6.1: Flowchart of motion detection algorithm used in MATLAB
Start
Start camera.
Take 10 frames.
Backgroundframe=average of the 10 frames.
Set a threshold.
Diffframe= background frame - current frame.
Take another frame.
If diffframe< threshold Display ‘ No motion’
Display ‘Motion Detected’
STOP Buzz the alarm as an output
43
6.2. Programming the FPGA, CPLD or Platform Flash PROM
The Spartan-3E Starter Kit includes embedded USB-based programming logic and an USB
endpoint with a Type B connector. Via a USB cable connection with the host PC, the
iMPACT programming software directly programs the FPGA, the Platform Flash PROM,
or the on-board CPLD. Direct programming of the parallel or serial Flash PROMs is not
presently supported.
6.2.1. Connecting the USB Cable
The kit includes a standard USB Type A/Type B cable, similar to the one shown in Figure
6.2.
Figure 6.2: Standard USB Type A/ Type B cable
The wider and narrower Type A connector fits the USB connector at the back of the
computer. After installing the Xilinx software, connect the square Type B connector to the
Spartan-3E FPGA Starter Kit board, as shown in Figure 6.3. The USB connector is on the
left side of the board, immediately next to the Ethernet connector. When the board is
powered on, the Windows operating system should recognize and install the associated
driver software.
44
Figure 6.3: USB connection of FPGA to laptop
When the USB cable driver is successfully installed and the board is correctly connected to
the PC, a green LED lights up, indicating a good connection.
6.2.2. Programming via iMPACT
After successfully compiling an FPGA design using the Xilinx development software, the
design can be downloaded using the iMPACT programming software and the USB cable.
To begin programming, connect the USB cable to the starter kit board and apply power to
the board. Then, double-click Configure Device (iMPACT) from within Project Navigator,
as shown in Figure 6.4.
Figure 6.4: Double click to Invoke iMPACT
If the board is connected properly, the iMPACT programming software automatically
recognizes the three devices in the JTAG programming file, as shown in Figure 6.5. If not
already prompted, first device in the chain is clicked, the Spartan-3E FPGA, to highlight it.
45
Right-click the FPGA and select Assign New Configuration File. Select the desired FPGA
configuration file and click OK.
Figure 6.5: Assign New Configuration File
If the original FPGA configuration file used the default StartUp clock source, CCLK,
iMPACT issues the warning message. The message can be safely ignored. When
downloading via JTAG, the iMPACT software must change the StartUP clock source to
use the TCK JTAG clock source.
To start programming the FPGA, right-click was done and Program was selected. The
iMPACT software reports status during programming process. Direct programming to the
FPGA takes a few seconds to less than a minute, depending on the speed of the PC’s USB
port and the iMPACT settings.
Figure 6.6: Program the Xilinx project
46
When the FPGA successfully programs, the iMPACT software indicates success, as shown
in Figure 6.7. The FPGA application is now executing on the board and the DONE pin
LED lights up.
Figure 6.7: Program succeeded mesaage
6.3. Clock Divider
A clock divider also called a frequency divider or scaler or prescaler, is a circuit that takes
an input signal of a frequency, fin, and generates an output signal of a frequency:
fout=fin/n;
Where ‘n’ is an integer.
Clock dividers provide an output clock signal that is a divided frequency of the input. They
can also be used to provide signal buffering and make multiple copies of the output
frequency. Clock divider devices, when used in divide-by-1 mode, can also function as a
fan out buffer.
The VGA monitor operates at a frequency of 25MHz, whereas the input source clock of
FPGA is 50MHz.So, a divide by 2 frequency divider in needed in which the input clock
source is divided by 2 to yield the desired clock frequency of 25 MHz to synchronize
signals on the VGA monitor, i.e.
Frequency at VGA monitor=Frequency (input clock source)/2;
=50MHz/2;
=25MHz
47
Figure 6.8: Clock output for divide by 2 frequency divider
The output clock of 25MHz is observed as shown in Figure 6.8.
Duty cycles play an important role for the generating a clock divider. A duty cycle is the
percent of time that an entity spends in an active state as a fraction of the total time under
consideration. The term is often used pertaining to electrical devices, e.g., switching power
supplies in an electrical device, a 60% duty cycle means the power is on 60% of the time
and off 40% of the time. The "on time" for a 60% duty cycle could be a fraction of a
second depending on how long the device's period is. Here one period is the length of time
it takes for the device to go through a complete on/off cycle.
In a periodic event, duty cycle is the ratio of the duration of the event to the total period of
a signal.
duty cycle=t/T;
Where
‘t’ is the duration that the function is active.
‘T’ is the period of the function.
48
Figure 6.9: Signals for different duty cycle
Simulation process was carried on to yield the clock output from the input clock of
50MHz, at both positive and negative duty cycles.
Figure 6.10: Clock output for 50% duty cycle
Here, the input signal is high for 10 ns and low for 10 ns. The input frequency (50MHz) is
divided by 1000, so that the output signal is yielded which is high for 1us and low for 1us
as shown in the Figure 6.10.
49
Figure 6.11: Clock output for 90% duty cycle
In the Figure 6.11, the input clock is high for 10ns and low for 10 ns, and the output clock
is high for 200ns and low for 20ns with a total of 90% duty cycle.
While interfacing C3088 camera with Xilinx Spartan 3E FPGA board, the fourth pixel
values of four consecutive lines were
respectively displayed on the LEDs at the
board. But the clock frequency of the
board (50MHz) is very high, that the pixel
values when displayed on the board can’t
be visualized as the process is very fast.
So, a clock divider was used such that
clock frequency of 50MHz was divided to
yield the frequency of 1 Hz, and the pixel
values were observed at a duration of 1
second.
Figure 6.12: VGA display range
50
6.3.1. VGA video signal generation
A VGA video signal contains 5 active signals:
• horizontal sync: digital signal, used for synchronisation of the video.
• vertical sync: digital signal, used for synchronisation of the video.
• red (R): analog signal (0-0.7 v), used to control the color.
• green (G): analog signal (0-0.7 v), used to control the color.
• blue (B): analog signal (0-0.7 v), used to control the color.
By changing the analog levels of the three RGB signals all other colors are produced. The
electron beam must be scanned over the viewing screen in a sequence of horizontal lines to
generate an image. The RGB color information in the video signal is used to control the
strength of the electron beam.
The screen refresh process
begins in the top left corner and
paints 1 pixel at a time from
left to right. At the end of the
first row, the row increments
and the column address is reset
to the first column. Once the
entire screen has been painted,
the refresh process begins again. The video signal must redraw the entire screen 60 times
per second to provide for motion in the image and to reduce flicker: this period is called
the refresh rate. Refresh rates higher than 60 Hz are used in PC monitors. In 640 by 480-
pixel mode, with a 60 Hz refresh rate, this is approximately 40 ns per pixel. A 25 MHz
clock has a period of 40 ns.
The vertical sync signal tells the monitor to start displaying a new image or frame, and the
monitor starts in the upper left corner with pixel (0,0). The horizontal sync signal tells the
monitor to refresh another row of 640 pixels. After 480 rows of pixels are refreshed with
480 horizontal sync signals, a vertical sync signal resets the monitor to the upper left
corner and the process continues.
Figure 6.13: Clock generation
51
During the time when pixel data is not being displayed and the beam is returning to the left
column to start another horizontal scan, the RGB signals should all be set to black color
(all zero). In a PC graphics card, a dedicated memory location is used to store the color
value of every pixel in the display. This memory is read out as the beam scans across the
screen to produce the RGB signals.
Figure 6.14: VGA timing diagrams
Figure 6.15: VGA connector
52
6.4. FPGA Simulation Part
6.4.1. Frame Grabber working mechanism
The flowchart for the frame grabber module used to capture the image from the cmos
camera is shown in Figure 6.18. Here, href, pclk and vsync are three main signals coming
out from the camera when it is powered up. Every state of the frame grabber module is
dependent on one or more of these signals. The operation of this frame grabber module is
based on the concept of Finite State Machine (FSM), i.e. the machine is in only at one state
at a time. The brief description of these states and the signals and variables used is
described below.
Vsync is the signal which when high signifies that the frame is ready to come from the
camera. When the vsync gets low after the previous high state, data (in the form of pixels)
starts to appear as the camera clicks the image. Href signifies the start of the new line. In a
frame, there are altogether 288 lines. This means that in low to high transition of the vsync,
total of 288 href transition is seen. In each href high signal, there appears the valid data in
the form of pixels. In each high to low transition of the href signal, total of 352 pclk
transitions is seen, which means that in a href signal, there exist 352 pixels data each of 8
bit.
At first, all the necessary variables were initialized with the proper value before the frame
grabber gets into the start up phase. After initialization was completed, the FSM processes
started. In this process, the important part was to find out when the frame was coming.
Hence the vsync signal was checked for a high, and then it is checked for a low value of
signal. This indicates incoming data. In the actual program, 60th frame was taken every
time as the camera being used captures 60 frames per second. So, until the frame counter
got equal to 60, the states to check vsync signal high and low continued. When this counter
value got equal to 60, then the next stage was to check for href signal. Since, href signal
specifies the start of a new line, unless and until it got high, no new line was ready and no
data could be received. Since the memory used, i.e. BRAM had limited amount of memory
space, it was impossible to use all the lines and pixel data for storing the data and further
processing of the data. Hence, the algorithm was so designed that it took each 4th line and
each 4th pixel of those lines. So to ch
register type of variables named, numLine and write_counter respectively
which was made zero after each 4
checking continued. This got
line of the frame was encountered, then it wa
language, when the line cou
by 4, then the pixel appearing at the output pins wa
When for a particular line, the pixel counter value wa
was particularly taken and stored and if it beca
line. If the total of 288 lines was reached, it wa
frame the dimension of the image being stored and used
Figure 6.
pixel of those lines. So to check whether the line and pixel were divisible by 4
register type of variables named, numLine and write_counter respectively
made zero after each 4th line and 4th pixel was encountered so
checking continued. This got incremented at every href high and every pclk high. If the 4
line of the frame was encountered, then it was checked for every 4
anguage, when the line counter and the pclk counter both were such that it wa
l appearing at the output pins was stored in the block Ram memory.
line, the pixel counter value was less than 352, the 4
taken and stored and if it became equal to 352, it meant the start of a
ine. If the total of 288 lines was reached, it was the end of one frame. So, in total for a
me the dimension of the image being stored and used was 72X88.
Figure 6.16: Total timing diagram in test bench
53
eck whether the line and pixel were divisible by 4 two
register type of variables named, numLine and write_counter respectively were introduced,
s encountered so that further
incremented at every href high and every pclk high. If the 4th
s checked for every 4th pixel. In simple
e such that it was divisible
s stored in the block Ram memory.
s less than 352, the 4th pixel for a line
e equal to 352, it meant the start of a new
s the end of one frame. So, in total for a
: Total timing diagram in test bench
Figure 6.17: RTL schematic of the motion detection part with VGA
: RTL schematic of the motion detection part with VGA
54
: RTL schematic of the motion detection part with VGA
55
6.4.2. Flowchart of Frame Grabber Module
vsync=0
vsync!=0 vsync=0
no
yes
href=0 href=1
no
yes
href!=0 href=0
pclk=0
pclk=1
Start
write_Counter=0, numLine=0 pixel_counter=0, line_counter=0
frame_counter=0, address_write=0
wait_vsync_high
wait_vsync_low
frame_counter=frame_counter+1
frame_check
If frame_check==60
wait_href_high
numLine<3
line_counter=line_counter+1numLine=numLine+1
wait_href_low
line_counter=line_counter +1 numLine=0
wait_pclk_high
B C A
56
no
yes
pclk!=0
yes
no
yes
no
Figure 6.18: Flowchart of frame grabber module
A B C
write_Counter<3
pixel_counter=pixel_counter+1 write_Counter=write_Counter+1
pixel_counter=pixel_counter+1 write_Counter=0
wait_pclk_low
PixelsWrite
count_pixelsof_line
addr_wr=addr_wr+1 y_out=y_in
pixel_counter<352
pixel_counter=0
count_line
line_counter<288
frame_complete
57
7. PROBLEMS FACED
7.1. New to FPGA and Verilog
The initial experience with the FPGA took a lot of time for acquaintance with the design
conditions. Also programming in Verilog was the most difficult task since everything had
to be started from scratch. Hence, it could be said that learning and making mistakes and
trying everything with a new concept every time was the part that took most of the time.
7.2. Clock Synchronizing Problem
The system implemented using FPGA used three different clocks: Pixel Clock of Camera
module (8.9285 MHz), FPGA Clock (50 MHz) and VGA Clock (25 MHz). This made the
synchronization between the signals from different blocks in the system a problem.
The solution to the synchronization problem was to use FSM approach in the
implementation of frame grabber module and other related modules that required
synchronization between the signals.
7.3. Camera Light
The camera used, i.e, cmos 3088 camera module after lot of inspection was found to
capture the image more properly and accurate when there was enough light present in the
room containing camera. The problem occurs when there was insufficient light in the
room. Since the project was based on calculating the difference, unless and until the
difference calculation was accurate, the project did not provide the expected output. For
the proper motion calculation which was dependent totally on the difference between two
frames, the requirement of light was the most needed as a flash.
The solution to this was to use the external light if there was in sufficient light in the room.
58
8. CONCLUSION AND FUTURE ENHANCEMENT
8.1. Conclusion
The FPGA based motion detection system was capable of detecting the small changes in
the motion of the valuable piece within the protected area using the frames captured by the
CMOS camera in every one second. The image capturing and the checking of motion
detection task was proper when sufficient amount of light was present in the room where
the camera was kept for protection or the light was provided externally for proper
operation of the system. Though the system was mainly focused keeping in mind the
protection valuable piece present in the museum, the system application could be extended
to even other area of security. In case of museum, since there may be some such items
which need to be protected every time from even the slight movement of it, this system
was mainly designed for those areas. If any valuable object was moved from its position,
movement was detected and the message was transferred to the authorised person. Until
then, the scanning of the protected area was done continuously for the purpose of security.
59
Figure 8.1: Overall module of motion detection part
8.2. Future Enhancement
The project, “FPGA Based Motion Detection and Alarm System using GSM Technology”
is completed for now but some minor changes can be added as an enhancement . The
major part that can be added is the addition of some filters while capturing the image by
the CMOS camera. This is because the data which comes out from the camera output pins
is not very accurate and since the algorithm was based on background subtraction, unless
and until a filter is included, the noise or the error prone data could not be removed.
Currently, a threshold is set for noise elimination. But since the threshold used was based
on hit and trial, some errors are present. Such errors are like: detection of motion even for
no change or no detection even for change. Such conflicts can be avoided using filtering
techniques.
The project now is based on whether the motion is detected or not, if detected send the
message to the authorized person and if not, the scanning of the protected area is
continuously done for security. The future work that can be added is storing the changes
occurred in the background while the motion is detected. This can help to trace out what
60
kind of motion was encountered within the protected area and if any intruder was present,
then to trace out some of its physical appearance so that the intruder can also be caught
easily.
Unlike in MATLAB simulation, images in FPGA cannot be seen directly. If the image
captured by the camera is to be displayed, extra programming would be required for
interfacing VGA with the FPGA board. Some attempts were made to do this but successful
output wasn’t obtained. Due to time limitations , further attempts could not be done .
CMOS 3088 camera worked only for sufficient light present in the room and if there was
insufficient light, then the camera did not capture the image properly. This light sensitive
nature of the camera demanded external light so that the camera captured images properly
and the movement was clear.
61
9. REFERENCES
[1] Z. Wei, D. Lee, N. Brent, J. Archibald, and B. Edwards, “FPGA-based embedded
motion estimation sensor,” International Journal of Reconfigurable Computing, vol.
2008, Article ID 636145, 9 pages, 2008.
[2] K. Shimizu and S. Hirai, “CMOS+FPGA vision system for visual feedback of
mechanical systems,” in Proceedings of the IEEE International Conference on
Robotics and Automation (ICRA '06), pp. 2060–2065, May 2006.
[3] Chen-Chiung Hsieh and Dung-Hua Liou David Lee “A Real Time Hand Gesture
Recognition System Using Motion History Image”,2010 2nd International Conference
on Signal Processing Systems (ICSPS) .
[4] A real time hand gesture recognition system using motion history image ,Chen-
Chiung Hsieh; Dung-Hua Liou; Lee, D.;Signal Processing Systems (ICSPS), 2010
2nd International Conference on Volume:2,Digital Object Identifier:
10.1109/ICSPS.2010.5555462 ,Publication Year: 2010 , Page(s): V2-394 - V2-398
[5] S. Mitra and T. Acharya, “Gesture recognition: a survey,” IEEE Trans. on Systems,
Man, and Cybernetics—Part C: Applications and Reviews, vol. 37, no. 3, May 2007,
pp. 311-324.
[6] Robert Pless, Tomas Brodsky, and Yiannis Aloimonos, “Detecting Independent
Motion: The Statistics of Temporal Continuity” IEEE transactions on pattern analysis
and machine intelligence, vol. 22, no. 8, August 2000.
[7] S. Kang, J. Paik, A. Koschan, B. Abidi, and M. A. Abidi, "Real-time video tracking
using PTZ cameras," Proc. of SPIE 6th International Conference on Quality Control
by Artificial Vision, Vol. 5132, pp. 103-111, Gatlinburg, TN, May 2003.
[8] Marco Krips , Thomas Lammert , Anton Kummert, “FPGA Implementation of a
Neural Network for a Real-Time Hand Tracking System”, Proceedings of the The
First IEEE International Workshop on Electronic Design, Test and Applications
(DELTA '02), p.313, January 29-31, 2002.
[9] António de Sousa, ” Smart Cameras as Embedded Systems ”, ICCA‟03, Page 105.
[10] Yu Shi and Timothy Tsui,” An FPGA-Based Smart Camera for Gesture
Recognition in HCI Applications”, Springer-Verlag Berlin Heidelberg , Part I, LNCS
4843, pp. 718–727, 2007.
62
[11] Jos´e Antonio Montero1, and L. Enrique Sucar,” Context-Based Gesture
Recognition”, Springer-Verlag Berlin Heidelberg 2006, CIARP, LNCS 4225, pp.
764–773, 2006.
[12] Jahnvi Vaidya, “ Real Time Video Capture and Image Processing System using
FPGA”, 12/09/2008.
[13] Kirillov, Andrew. “Motion Detection Algorithms”, United Kingdom, 2007.
[14] Derhgawen , Ashish. “Laser Gesture Recognition”, India, 2007.
[15] Shapiro, Adam M. “FPGA-BASED REAL-TIME GPS RECEIVER”,2010.
[16] Shin-Hyun Ka. “Digital Image and Video Compression Techniques”, 2003.
[17] Datasheet, “C3088 1/4” Color Camera Module with Digital Output”.
[18] Datasheet, “PIC16F87X Data Sheet”, Microchip Technology Inc.
[19] “Spartan-3E FPGA Starter Kit Board User Guide”,
www.xilinx.com/support/documentation/boards_and_kits/ug230.pdf
(last accessed : 10th December, 2012)
[20] Hong Shan Neoh, Asher Hazanchuk, “Adaptive Edge Detection for Real-Time
Video Processing using FPGAs”, Altera Corporation.
[21] F Javier Toledo, J. Javier Martlnez and J. Manuel Ferrandez, “FPGA based
platform for image and video processing embedded systems”, Universidad
Politecnica, de Cartagena.
[22] Muthukumar Venkatesan and Daggu Venkateshwar Rao, “Hardware Acceleration
of Edge Detection Algorithm on FPGAs,” Department of Electrical and Computer
Engineering, University of Nevada Las Vegas. Las Vegas, NV 89154.
[23] Rafael C. Gonzalez and Richard E. Woods, “Digital Image Processing”, Prentice
Hall, pp 719.
[24] “Displaying Graphics through the VGA Interface”,
http://www.es.isy.liu.se/courses/TSTE12/download/TSTE12_lab_vga_080902.pdf.
(last accessed : 18th June, 2013)
[25] David L. Jones, “PCB Design Tutorial”, www.alternatezone.com. (last accessed :
12th July, 2013)
[26] Todd H. Hubing, “Printed Circuit Board Decoupling”, Clemson University,
October 2006.
[27] “Etching Your Own PC Boards”, Jaycar Electronics,
www.jaycar.com.au/images_uploaded/pcboards.pdf, 2002. (last accessed : 25th July,
63
2013)
[28] Ying-Hao Yu, “FPGA–Based Formation Control of Multiple Ubiquitous Indoor
Robots ”, 19/05/2011.
[29] Tianguang Zhang, Haiyan Wu, Alexander Borst, “An FPGA Implementation of
Insect-Inspired Motion Detector for High-Speed Vision Systems”, 23/06/2012
[30] T. Kryjak and M. Gorgoń, “Real-time implementation of moving object detection
in video surveillance systems using FPGA,” Computer Science, vol. 12, pp. 149–162,
2011.
[31] R. Rodriguez-Gomez, E. Fernandez-Sanchez, J. Diaz, and E. Ros, “FPGA
implementation for real-time background subtraction based on horprasert model,”
Sensors, vol. 12, pp. 585–611, 2012.
[32] Chi-Jeng Chang, Pei-Yung Hsiao, Zen-Yi Huang (2006). ‘Integrated Operation of
Image Capturing and Processing in FPGA’, IJCSNS International Journal of
Computer Science and Network Security,VOL.6 No.1A, pp 173-179.
[33] Christopher T. Johnston, Kim T Gribbon, Donald G. Bailey (2005) ‘FPGA based
Remote Object Tracking for Real-time Control’, 1st International Conference on
Sensing Technology November 21-23, 2005 Palmerston North, New Zealand.
[34] Crookes D., Benkrid K., Bouridane A., Alotaibi K., and Benkrid A.(2000), ‘Design
and implementation of a high level programming environment for FPGA-based image
processing’, Vision, Image and Signal Processing, IEE Proceedings, vol. 147, Issue: 4
, Aug, 2000, pp. 377 -384.
[35] L. Kilmartin, M. O Conghaile (1999), ‘Real Time Image Processing Object
Detection and Tracking Algorithms’, Proceedings of the Irish Signals and Systems
Conference, NUI, Galway, June 1999, pp. 207- 214.
64
APPENDIX A: XILINX SPARTAN 3E FPGA BOARD
Figure A.1 :Xilinx Spartan 3E FPGA board
The key features of the Spartan-3E board are:
• Xilinx XC3S500E Spartan-3E FPGA
• Up to 232 user-I/O pins
• 320-pin FBGA package
• Over 10,000 logic cells
• Xilinx 4 Mbit Platform Flash configuration PROM
• Xilinx 64-macrocell XC2C64A CoolRunner™ CPLD
• 64 MByte (512 Mbit) of DDR SDRAM, x16 data interface, 100+ MHz
• 16 MByte (128 Mbit) of parallel NOR Flash (Intel StrataFlash)
• FPGA configuration storage
• MicroBlaze code storage/shadowing
• 16 Mbits of SPI serial Flash (STMicro)
• FPGA configuration storage
• MicroBlaze code shadowing
• 2-line, 16-character LCD screen
• PS/2 mouse or keyboard port
65
• VGA display port
• 10/100 Ethernet PHY (requires Ethernet MAC in FPGA)
• Two 9-pin RS-232 ports (DTE- and DCE-style)
• On-board USB-based FPGA/CPLD download/debug interface
• 50 MHz clock oscillator
• SHA-1 1-wire serial EEPROM for bitstream copy protection
• Hirose FX2 expansion connector
• Three Digilent 6-pin expansion connectors
• Four-output, SPI-based Digital-to-Analog Converter (DAC)
• Two-input, SPI-based Analog-to-Digital Converter (ADC) with programmable-gain
pre-amplifier
• ChipScope SoftTouch debugging port
• Rotary-encoder with push-button shaft
• Eight discrete LEDs
• Four slide switches
• Four push-button switches
• SMA clock input
• 8-pin DIP socket for auxiliary clock oscillator
66
APPENDIX B: C3088 1/4” COLOR CAMERA MODULE WITH
DIGITAL OUTPUT
The C3088 is a 1/4” colour camera module with digital output. It
uses Omni Vision’s CMOS image sensor OV6620. Combining
CMOS technology together with an easy to use digital interface
makes C3088 a low cost solution for higher quality video image
application.
The digital video port supplies a continuous 8/16 bit-wide
image data stream. All camera functions, such as exposure, gamma, gain, white balance,
colour matrix, windowing, are programmable through I2C interface.
In combine with OV511+, USB controller chip, it will be easily form a USB camera for
PC application.
Features:
� 101,376 pixel, CIF/QCIF format
� Small size : 40 x 28 mm
� Lens: f=4.9mm (Optional)
� 8/16 bit video data : CCIR601, CCIR656, ZV port
� Read out – progressive
� Data format -YCrCb 4:2:2, GRB 4:2:2, RGB
� I2C interface
� Wide dynamic range, anti blooming, zero smearing
� Electronic exposure / Gain / White balance control
� Image enhancement - brightness, contrast, gamma,saturation, sharpness, window,
etc
� Internal / external synchronization scheme
� Frame exposure / line exposure option
� Single 5V operation
� Low power consumption (<100mW)
� Monochrome composite video signal output(50 Hz)
Figure B.1: Cmos 3088 module
67
APPENDIX C: PIC16F877A
Figure C.1: PIC16F877a pin description
Microcontroller PIC16F877A is one of the PIC Micro Family microcontroller which is
popular at this moment, start from beginner until all professionals. The use of FLASH
memory technology enables it to write and erase until thousand times. It has total of 40
pins where 33 pins are specified as input/output pins.
Peripheral Features:
• Timer0: 8-bit timer/counter with 8-bit pre scalar
• Timer1: 16-bit timer/counter with pre scalar, can be incremented during Sleep via
external crystal/clock
• Timer2: 8-bit timer/counter with 8-bit period register, pre scaler and post scaler
• Two Capture, Compare, PWM modules
• Synchronous Serial Port (SSP) with SP (Master mode) and I2C(Master/Slave)
• Universal Synchronous Asynchronous Receiver
• Transmitter (USART/SCI) with 9-bit address detection
68
• Parallel Slave Port (PSP) – 8 bits wide with external RD, WR and CS controls
(40/44-pin only)
• Brown-out detection circuitry for Brown-out Reset (BOR)
Analog Features:
• 10-bit, up to 8-channel Analog-to-Digital Converter (A/D)
• Brown-out Reset (BOR)
• Analog Comparator module (Two analog comparators , Programmable on-chip
voltage reference (VREF) module , Programmable input multiplexing from device
inputs and internal voltage reference , Comparator outputs are externally
accessible)
Special Microcontroller Features:
• 100,000 erase/write cycle Enhanced Flash program memory typical
• 1,000,000 erase/write cycle Data EEPROM memory typical
• Data EEPROM Retention > 40 years
• Self-reprogrammable under software control
• In-Circuit Serial Programming(ICS) via two pins
• Single-supply 5V In-Circuit Serial Programming
• Watchdog Timer (WDT) with its own on-chip RC oscillator for reliable operation
• Programmable code protection
• Power saving Sleep mode
• Selectable oscillator options
• In-Circuit Debug (ICD) via two pins
69
APPENDIX D: TABLE OF HARDWARE CONNECTIONS
Pin
name
C3088
pin no.
Jumper Protoboard pin
name
JX1 pin no. FPGA Pin
No.
Y0 1 J2 FX2_IO<8> 8 F7
Y1 2 J2 FX2_IO<7> 7 E7
Y2 3 J2 FX2_IO<6> 6 B6
Y3 4 J2 FX2_IO<5> 5 A6
Y4 5 J1 FX2_IO<4> 4 C5
Y5 6 J1 FX2_IO<3> 3 D5
Y6 7 J1 FX2_IO<2> 2 A4
Y7 8 J1 FX2_IO<1> 1 B4
HREF 14 J4 FX2_IO<9> 9 D7
VSYNC 16 J4 FX2_IO<10> 10 C7
PCLK 18 J4 FX2_IO<11> 11 F8
Table D. 1: Connection of C3088 to FPGA board
Pin name C3088 pin no. Description
PWDN 9 GND
RST 10 Pulled down thorough 10 K resistor. Can be reset
using a switch.
SDA 11 Pulled up through 10 K resistor
FODD 12 Not connected
SCL 13 Pulled up through 10 K resistor
AGND 15 GND
AGND 17 GND
EXCLK 19 Not connected
VCC 20 +3.3V
AGND 21 GND
VCC 22 +3.3V
70
UV0 23 Not connected
UV1 24 Not connected
UV2 25 Not connected
UV3 26 Not connected
UV4 27 Not connected
UV5 28 Not connected
UV6 29 Not connected
UV7 30 Not connected
GND 31 GND
VTO 32 Not connected
Table D. 2: Hardware connection of C3088 sensor in PCB board
Table D. 3: Connection of LCD in FPGA board
71
APPENDIX E: PCB DESIGNS
Figure E.1: PCB design for camera interfacing circuit
Figure E.2: PCB design for SIM 300
72
APPENDIX F: VGA MONITOR
Figure E.3: HCM 582 VGA monitor
HCM 582 is a 15” Digital SVGA Color Monitor, with Antiglare, High resolution
(0.28mm) CRT. This model supports High Resolution mode of 1024X768 @60Hz max,
and FLICKERFREE resolution of 800 X 600 @85 Hz. HCM 582 supports DPMS and
makes an ideal choice for Windows and Business applications.
KEY FEATURES
� Microprocessor based Digital controls for Precise Control of Display parameters.
� High Contrast and High Definition Picture Quality – 0.28mm Dot Pitch.
� Flicker Free Operation at 800 X 600 @ 85 Hz
� Maximum Resolution of 1024X768 @60HZ.
� MULTI-SCAN Horizontal Frequency 30~54 kHz (AUTO SCANNING)
� DDC ½B Plug and Play compliant.
� Universal Power Supply operation at 90V ~ 264V @ 50Hz/60Hz
