April 9, 2023
1
GESTURE CONTROL ROBOTS
Guided By: Presented By:Vibin C Thomas Jibin PouloseAsst.Professor S7 EEEASIET No:-29
Date of Presentaion : 27/08/2014
Contents•Robot and Types•Controlling Methods•Gesture Control•Transmitting
▫Flex Sensor▫Accelerometer▫Robotic Hands▫Image Processing
•Receiving•Comparison of different systems•References
April 9, 2023
3
WHAT IS A ROBOT
ROBOT
Electro-mechanical device
It finds it’s uses in all as-pects of our life.
Performs Various tasks
May be human controlled or automated
April 9, 2023
4
TYPES OF ROBOTS (BASED ON CONTROL)ROBOTS
MANUAL
WIREDWIREL
ESS
SEMI-AUTONOMOUS
AUTONOMOUS
PRE-PROGRAM
MED
SELF LEARNIN
G
[Figure 1]
April 9, 2023
5
CONTROLLING OF ROBOTS
•Gestures
•Bluetooth enable devices
•Voice control
•Text
•Commands
April 9, 2023
6
WHY GESTURE CONTROL?
• Human Machine Interaction (HMI )[3]
• Interact naturally without any mechanical devices.
• Interface to both human and computer.
• Substitute the common interface devices
• Can also be exploited to extend their functionality.
• Avoid to program which required extensive hard work
• minimized the need for text interfaces and GUIs (Graphical
User Interface).
April 9, 2023
7
WHAT IS A GESTURE
• Motion originate from face or hand• An action seen by someone else • Convey some piece of information. • Movement of part of the body, especially a hand or the head, to
express an idea or meaning.
[Figure 2]
April 9, 2023
8
TRANSMITTER SIDE
TransducerSensor
Signal Conditioning
Micro Controller
EncoderTransmitter
Flex Action
Control Signals
[Figure 3]
April 9, 2023
9
SENSORS AND SIGNAL CONDITIONING
• Physical movements cannot be processed and transmitted
• Sensors converts physical movements into electrical signals.
• Converts different gesture movements of hands into corresponding electrical signals.
• Commonly used transducers are Flex sensors, Accelerometer and Gyroscopes.
• They are placed in a glove and used to wear.
April 9, 2023
10
FLEX SENSORS• Analog resistors work as variable analog voltage divider.• Carbon resistive elements with thin flexible substrate.• More carbon means less resistance. • Resistance increase with bending• output a stream of data that varies with degree of bend
[Figure 4]
April 9, 2023
11
VARIATION OF RESISTANCS
Voltage reduces with increase increase in deflection of flex sensor [2]
Resistance increases with increase increase in deflection of flex sensor [2]
[Figure 5]
[Figure 6]
April 9, 2023
12
Vout=5Vx Rsensor Rsensor+Rpullup
Hand glove with flex sensors in each fingers
[Figure 7]
[Figure 8]
April 9, 2023
13
ACCELEROMETER
• Gives an analog data while moving in X,Y,Z direction• Piezoelectric material sandwiched between base and resonant
mass• Motion causes resonant mass to move• Force to move mass is transmitted through piezoelectric
material (F = ma)• Compression/Tension or Shear of piezoelectric material causes
charge buildup proportional to acceleration• Equivalent electric circuit measures the capacitance
April 9, 2023
14
ICP ACCELEROMETER
• To Make Accelerometer Use Simple, Incorporate Electronics Package Into Transducer
• Equivalent Circuit => No Dependence on Cable• Output Sensitivity Constant
[Figure 9]
April 9, 2023
15
ROBOTIC HANDS
• Have five fingers each having three joints
• Each joint requires each sensors
• Readings from each sensor is taken for signal conditioning
[Figure 10] [4]
[Figure 11] [4]
April 9, 2023
16
CONTROL OF ROBOTIC HAND
•I2c communication•SDA ,SCl•Unique ID number•Pwm control of servo
[Figure 12]
April 9, 2023
17
IMAGE PROCESSING
• A Vision Based Gesture Recognition system• Image: array, or a matrix, of square pixels (picture elements)
arranged in columns and rows.• each picture element has an intensity ranges from 0 to 255 (for
8bit)• Common steps : image scanning, storing, enhancing and
interpretation.• image will be converted to digital form using a
scanner/digitizer and then process it. • It is defined as the subjecting numerical representations of
objects to a series of operations
April 9, 2023
18
DIFFERENT TYPES OF VISION BASED GESTURE RECOGINATON [3]
Mono Camera Based System
Multi Camera Based System
Infra red Camera Based System[Figure 14]
[Figure 15
[Figure 13]
April 9, 2023
19
• First Model the motion and posture and infer the gesture [3]
3D Textured volumetric model
3D wireframe volumetric model
3D Skeleton Model
Binary Silhouette
Contour
[Figure 16]
April 9, 2023
20
CONTROLLER AND TRANSMITTER
• Sensors outputs a stream of data that varies with degree of bend
and movement of hand.
• Analog voltage is given to controller.
• Mapped into digital values.
• Compare with the programmed values.
• Generate the corresponding signal of hand movement.
• Signal is transmitted to robot through RF Transmitter.
• Transmitter transmits the signal to robot.
April 9, 2023
21
RECEIVER SIDE
Transducer
Reciver Decoder
Micro Controller
Driver circuitMotor
Signal
Movments
Receiver receives the signal
Converts into electrical signal and give to controller
Controller process the signal and drives the motors corresponding to the signal.
[Figure 17]
COMPARISON
Methodologiess Advantages Disadvantages
Flex Sensor •Simple Programming•Simple Construction•Simple Calibration
Expensive
Accelerometer Simple Construction ExpensiveCalibration is difficult
Image Processing Less CostRequires only a camera
Programming is very difficult
April 9, 2023
23
REFERENCES[1] Ruize Xu, Shengli Zhou, and Wen J. Li, Fellow, IEEE
“MEMS Accelerometer Based Nonspecific-User Hand Gesture Recognition”
[2]Abidhusain Syed, Zamrrud Taj H. Agasbal, Thimmannagouday Melligeri, Bheemesh Gudur
“Flex Sensor Based Robotic Arm Controller Using MicroController”
[3]Miss. Kawade Sonam P, Prof. V. S. Ubale
IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) ISSN: 2278-2834, ISBN: 2278-8735, PP: 19-26
[4] “Gesture Recognition-A Review”
Dr. Shantanu K. Dixit, Mr. Nitin S. Shingi
“Implementation of Flex sensor and Electronic Compass for Hand Gesture Based Wireless
Automation of Material Handling Robot”
[5] Umesh Yadav, Ankur Tripathi , Sonali Dubey ,S.K.Dubey
“Gesture Controll Robot”
Questions??