KINESICS ARTICULATION

ANJALY GOPINATH1, KARTHIKA VIJAY2, KARTHIK RAJESH3, MEENAKSHI K.S4,

PRIYANKA P.S5

Abstract

One of the most precious gifts to a Human being is an ability to see, listen and respond according to the

situations. But here are some unfortunate ones who are deprived of this. Communication between normal

and dumb people have always been a challenging task. Our project deals with removing this

communication barrier between the mute community and normal people. The proposed system mainly

consisted of a glove attached with five flex sensors on each finger connected ta controller [Panella 2019]

Bends of the flex sensors are trained to produce corresponding text as well as speech output.

1. Introduction

Around 1% of the Indian population are deaf and dumb. The only means of communication for them

is through sign language. The existing implementation of the project involves either image processing or

the usage of flex sensors to produce output in a laptop. In this project it produces the output in an Android

phone making it more comfortable and easier to handle. In this paper we proposed a system to convert

hand gestures to text as well as speech output. The system mainly uses flex sensors and android technology.

Here the hand gesture is being converted into auditory speech. [Sun 2018]There are two modules for the

system. The first module consists of the glove with flex sensors connected to Arduino nano board which

produces digital output. The second module consist of the Android store. The system is based on American

sign language.

2. Proposed System

The proposed system supports two-way communication between disabled and normal people. System

consists of two modules:

Module I: Flex sensors connected to Arduino

This is a wearable module which consist of five flex sensors connected on each finger in the glove. All the

flex sensors are connected to an Arduino nano board. [Hussain 2017]

Module II: Android for converting speech to text

This module is an android app working on android studio which converts a speech to text and displays the

output on the mobile screen. [Kaur 2015]

3. Component Familiarisation

a) Arduino Nano: The Arduino nano is miniaturized and compatible micro controller board. It has an

operating voltage of 5V and the input voltage can range from 7 to 12V. It involves 14 digital pins, 8

analog Pins, 2 Reset Pins & 6 Power Pins.

b) Flex sensor: Otherwise known as bend sensor is a type of sensor that estimates the amount of

deflection or bending. By bending the fingers, the resistance of the sensor element can be changed.

This device has two terminals and does not have inverse terminals.

c) Button: It is a quiet switch apparatus to curb some facet of a machine or a process. A click on the

button indicates that it is pressed and there by capturing the corresponding gesture.

4. Working

The flex sensors connected to the glove and Arduino has the five flex sensors are bend, the

corresponding is recoginised. The trained data set will be entered using the SVM algorithm which will be

mentioned to the corresponding class. As the flex sensors are bend the patterns get recoginised and the data

is averaged after which the flex is send to the Arduino [Du 2018].For an SVM process, it requires a training

data, a label and a testing data. The python programming code, here the data is split and send to the predict

after converting to float. Thus, the corresponding flex is converted to text and speech output.

1,2,3,4,5Department of ECE, Adi Shankara Institute of Engineering & Technology, Kalady.

International Journal of Applied Engineering Research ISSN 0973-4562 Volume 15, Number 1, 2020 (Special Issue)© Research India Publications. http://www.ripublication.com

140

Figure 1: Pinout Diagram of Arduino Nano

Figure 2: Schematic Diagram of Flex Sensor

Figure 3: Button

International Journal of Applied Engineering Research ISSN 0973-4562 Volume 15, Number 11, 2020 (Special Issue)© Research India Publications. http://www.ripublication.com

141

5. Block Diagram

6. Conclusion

The system provides an efficient and effective method for the mute community for their interactions

with the normal people. Output in the Android phone makes it more convenient and easier to take care.

References

[Panella2019] Panella, M., & Altilio, R. (2019). A Smartphone-Based Application Using Machine

Learning for Gesture Recognition: Using Feature Extraction and Template Matching via Hu Image

Moments to Recognize Gestures. IEEE Electronics Magazine, 8(1), 25–29.

[Sun 2018] Sun, J.-H., Ji, T.-T., Zhang, S.-B., Yang, J.-K., & Ji, G.-R. (2018). Research on the Hand

Gesture Recognition Based on Deep Learning. 2018 12th International Symposium on

Antennas,Propagation and EM Theory (ISAPE).

[Hussain2017] Hussain, S., Saxena, R., Han, X., Khan, J. A., & Shin, H. (2017). Hand gesture recognition

using deep learning. 2017 International SoC Design Conference (ISOCC).

[Kaur 2015] Mandeep Kaur, Ahuja, Dr. Amardeep Singh, “Hand Gesture Recognition Using CAM”,

IJCSET, July 2015 Vol 5, Issue 7, 267-271.

[Du 2018] Tong Du, Xuemei Ren And Huichao Li, “Gesture Recoginition Method Based on Deep

Learning”, 2018.

International Journal of Applied Engineering Research ISSN 0973-4562 Volume 15, Number 11, 2020 (Special Issue)© Research India Publications. http://www.ripublication.com

142