Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
CMR Journal of Engineering and Technology Vol.2 Issue.3 December, 2018
www.cmrjet.org 1
Communication Assistance System for Speech Impaired
1 M.Sudhakar 2 B. Shruthi
1 Professor 2 PG Scholar
Dept of ECE, CMR College of Engineering & Technology, Hyderabad, Telangana-India.
Email: 1 [email protected]
Email: 2 [email protected]
Abstract-Sign Language is the only approach for communication of speech impaired persons. However normal
persons cannot perceive the linguistic communication, therefore it becomes difficult for speech impaired person
to communicate with others. In this paper an electronic speaking system has been developed to ease
communication of speech impaired persons with others. An electronic speaking system consists of 3 system
frameworks namely scene capturing, data processing and audio output. Camera as input device captures the
gestures and feed into raspberrypi3 processor. The processor identifies the gesture image from its data base
images and activates the corresponding recorded audio file from SD card to play to speaker. Simultaneously the
corresponding text command is displayed on the LCD. In this project ten most important gestures have been
captured and the data base for the same has been created. This device also helps in communicating between two
speech impaired persons using the texting feature.
Keywords- Communication of speech impaired, Raspberry Pi3, Open CV Python, Webcam, Speakers
I. INTRODUCTION Sign Language is the solely method for communication of speech impaired persons. However normal persons
cannot follow the sign communication therefore it becomes difficult for speech impaired to communicate with a
normal person. This necessitates development of a communication assistance system for the speech impaired.
Considering such necessity, an electronic speaking system has been developed in our paper to ease
communication of speech impaired persons. The electronic speaking system developed consists of 3 system
frame works namely scene capturing, data processing and audio output [1]. The camera captures the gestures of
speech impaired person and feed the same into raspberry pi3. The raspberry pi3 processes this information and
tries to match with the data in data base. A code is generated by the processor and accordingly the audio file is
picked and played. Simultaneously the processor displays the text on LCD, corresponding to the code. A data
base of ten commonly used gestures have been captured and placed in the data base. Accordingly the audio and
text files are created.
II. EXISTING METHOD
In existing framework a glove containing five flex sensors is worn by the speech impaired person. The gestures
of the hand are sensed by the flex sensors [2] [3] 4] [5] and the same are processed by a micro processor. The
disadvantage in this method is that the sensors are connected to the processor through wires [8].
III. SYSTEM PROPOSED
Our paper adopts a computerized talk me framework. There are no wearable sensors and connecting wires. An
electronic talking contraption ends up cutting edge which incorporates a 3 framework systems viz Scene
capturing, Data handling, Audio yield. The efficiency of communication between a normal person and speech
impaired individual is improved by texting the gesture made by speech impaired and then playing a
corresponding audio file. A camera and raspberry pi3 processor has been used to capture the gesture of the
speech impaired person. The image processing program has been written using OpenCV. The image is compared
with images available in the data based and a code is generated on matching. The generated code is used to
display corresponding text on the LCD. The same is also used to select the corresponding audio file and play it
[3].
CMR Journal of Engineering and Technology Vol.2 Issue.3 December, 2018
www.cmrjet.org 2
IV. SYSTEM HARDWARE
Fig1: Block diagram
V. RASPBERRY Pi 3 PROCESSOR The Raspberry Pi 3 demonstrate B has particularly worked with the Broadcom BCM2837 System-On-
Chip(SoC) incorporates four superior ARM Cortex-A53 process centers running at 1.2 GHz with 32 kB Level
one and 512 kB Level a couple of reserve memory, a Video Core IV illustrations processor, and is associated
with a 1GB LPDDR2 memory module on the back of the board. It also alternatives 40-pins broadly useful info
yields (GPIO) and enhanced property with Bluetooth Low Energy (BLE) and BCM43143 Wi-Fi on board. It
likewise has an updated control administration wellspring of 5V USB control supply up to 2.5 Amps. As of
now, Raspberry Pi 3 Model B is best of Raspberry Pi PCs. The framework handling is colossal with 1.2 GHz
clock speed and 1GB RAM Raspberry Pi can play out every single propelled process. As per the association
shrewd, the board ought to be equipped for sending information to and from the board quickly. Another double
band Wi-Fi underpins for 2.4 GHz and 5 GHz 802.11b/g/n/air conditioning which is likewise guarantees
twofold all through the 802.11b/g/n/air conditioning Wi-Fi on the Raspberry Pi 3 Model B. With the expansion
of Gigabit Ethernet over USB 2.0, the wired Ethernet execution is additionally supported, with an extraordinary
throughput of around 300MB.
VI. WEB CAM Webcams are video capturing objects identified with PCs or pc frameworks, more often than not making
utilization of USB or, if they interface with frameworks, Ethernet or Wi-Fi. They are fabulous for low gathering
charges and versatile applications.
Webcams typically contain a central factor, a preview sensor, and a couple of assistance equipment. Exceptional
central components are available, basically the most extensively perceived being a plastic point of convergence
that might be settled and out to set the digicam's center interest. Settled mindfulness central angles, which have
no relationship for trade, are in addition open. Preview sensors can be CMOS or CCD, the past being
overpowering for insignificant exertion cameras, yet CCD cameras don't generally beat CMOS-based cameras
inside the straightforwardness esteem run. Client webcams are for the most part VGA choice with an aspect cost
of 30 plots for each second. Bigger goals, in uber pixels, are available and more prominent packaging charges
are building up to appear. The video get framework includes various making prepared advances [6]. To with the
simple video signal is digitized through an easy to-automated converter to make an unrefined, developed know-
how stream. As an outcome of composite video, the luminance and chrominance are then disengaged.
Consequent, the chrominance is demodulated to give shading qualification video know-how. Finally, the
aptitude is altered with the guide of a shading house converter to make know-how in conformance with any of a
couple of shading space standards, for instance, RGB and YCbCr. By and large, these methods constituted video
unraveling, on the grounds that they "translate" a basic video foundation, for instance, NTSC or PAL. Support
equipment is close by to scrutinize the photograph from the sensor and transmit it to the host pc. The advanced
camera imagined to 1 angle, for example, uses a Sonix SN9C101 to transmit its photo over USB. A few cameras
- , for instance, cell phone cameras - make utilization of a CMOS sensor with helping objects.
CMR Journal of Engineering and Technology Vol.2 Issue.3 December, 2018
www.cmrjet.org 3
CMR Journal of Engineering and Technology Vol.2 Issue.3 December, 2018
www.cmrjet.org 4
Features:
Smallest remote video and sound advanced camera.
Wi-Fi transmission and social affair
High affectability
Effortless foundation and movement
Effortless to vanity
Mild weight
Low control usage
Small measure
Particulars:
Output repeat: 900 MHz 1200 MHz
Output control: 50 mW -200 mW
Power convey: DC 6-12 V
Partition anchored: 10 m
VII. SPEAKERS Add some extra impact to your sound undertaking with these USB controlled enhancers. We tried a huge bit of a
tremendous measure of different models to find ones with a better than average repeat response in this way,
you'll get quality sound yield for music playback. There's even a volume control wheel on it so you can set it up
splendidly. Essentially, interface the standard 3.5 mm stereo associate with the Raspberry Pi, Wave shield, et
cetera. For control, interface the USB connector to anything that can give USB control. We can normally keep
the volume about halfway (which is still to a great degree riotous), where the current drawn is 200-400 mA. At
max volume, you can end up with up to 1Amp.
USB Powered Speakers Features:
Completely Compatible with the Raspberry Pi 100 Hz - 18 KHZ recurrence reaction
Power necessities: 5V DC 1A crest (at max volume yield) 4 Ohm impedance speakers, 3W each
Speaker Dimensions: (Height x Width x Depth) 80mm x 70mm x 70mm
Link Length: 1m
VIII. FLOWCHART
Compare sign?
Start
Yes
No
Display of Corresponding sign
Play voice of corresponding sign
Draw contours for image
Thresholding of image
Converting Blur image
Capture frames and Convert to grayscale
CMR Journal of Engineering and Technology Vol.2 Issue.3 December, 2018
www.cmrjet.org 5
IX. SOFTWARE USED
This project is implemented using following software:
Open CV
Python
Raspbian OS
X. RESULTS
The images given below are various hand gestures made and corresponding American code that is generated.
Hand Gesture 1: “I am happy”
Hand Gesture 2: “Don‟t disturb me”
Hand Gesture 3: “I need your help”
Hand Gesture 4: as “please”
Hand gesture 5: “Hello” Hand Gesture 6: “Good morning”
Hand Gesture 7: “I am sorry”
Hand Gesture 8: “please can you help me with this
CMR Journal of Engineering and Technology Vol.2 Issue.3 December, 2018
www.cmrjet.org 6
address”
Hand Gesture 9: “Shall we go”
Hand Gesture 10: “I need some water”
XII. CONCLUSION
In this paper a „Communication Assistance System‟ has been developed to ease communication of speech
impaired persons with normal people. An electronic speaking system has been developed which consists of 3
system framework was scene capturing, data processing, audio output. Camera as input device captures the
gestures and feed into raspberrypi3 processor. The processor identifies the gesture from its data base and
activates the corresponding recorded audio file from SD card to play to speaker .Simultaneously the
corresponding text command is displayed on the LCD. In this project ten most important gestures have been
captured and the data base for the same has been created. This device also helps in communicating between two
speech impaired persons using the texting feature.
XIII. FUTURE SCOPE
In this project the gesture of the speech impaired person is captured by the camera and the frames are
processed by the raspberry pi processor. This required faster processing speeds for enhancing the
response of the system. Therefore processors with higher speeds can be adopted.
Only ten commonly used gestures have considered for implementation in this project. The number of
gestures can be increased so as to expand the communication spectrum.
The system working is based on analyzing the captured frames where as if it extended to motion based
analysis; it can be become a more useful tool. The camera used in this project is of 5 MP, whereas the
resolution can be increased.
The audio system used in this project has low power output. The power output may be increased for
better audibility.
XV. REFERENCES
[1] A. Y. Satpute, A. D. Bhoi, and T. Designing, "Electronic Speaking System for Dumb, "VO! 6, no.3, pp.
1132-1139, 2013.
[2] S. F. Ahmed, S. Muhammad, B. Ali, S. Saqib, and M. Qureshi, "Electronic Speaking Glove for moronic
Patients a Tongue to," no. November, pp.56-60, 2010
[3] M. Wald, "Inscribing for Deaf and debilitating of Hearing people by redaction Automatic Speech
Recognition in Real Time", Proceedings of tenth International Conference on Computers serving to people with
Special wants ICCHP 2006, LNCS 4061, pp.683-690.
[4] R. R. Itkarkar and A. V. Nandi, "Hand motion to discourse change the use of Mat research facility," in 2013
Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT),
2013, pp.1-4
[5]. Jingdong Zhao, Li Jiang, Shicai Shi, Hegao Cai, Hong Liu, G. Hirzinger, "A Five-fingered underneath
spurred Prosthetic Hand System", Proceedings of the 2006 IEEE International Conference on Mechatronics and
Automation, June 2006, pp.1453-1458.
[6] S. U. N. Praveen Kumar S Havalagi, “THE magnificent DIGITAL GLOVES THAT lucid TO THE
VOICELESS," vol.6, no.1, pp.471-480, 2013.
[7] M. Sudhakar, Vandana Khare, D. Vijaya Krishna kanth, “Speech to text conversion & display using
CMR Journal of Engineering and Technology Vol.2 Issue.3 December, 2018
www.cmrjet.org 7
Raspberry Pi” in „IOSR Journal of Computer Engineering’ (IOSR-JCE) e-ISSN: 2278-0661,p-ISSN: 2278-
8727, Volume 19, Issue 4, Ver. III (Jul.-Aug. 2017), PP 14-18.
[8] M. Mahesh, M. Sudhakar, “eFACE Recognition System” in International Journal of Engineering Science &
Advanced Technology, Volume-3, Issue-4, 194-196, ISSN: 2250-3676
AUTHORS‟ BIOGRAPHY
Dr. M. Sudhakar: Graduated from JNTU College of Engineering, Hyderabad in 1979, with
specialization in ECE. He completed his M.Tech from Indian Institute of Technology
Madras in 1986 with the specialization in Instrumentation, Control & Guidance. Obtained
doctoral degree from Annamalai University. Successfully headed R&D Project assigned by
IAF on “Mathematical Modeling & Simulation of Aero Engine Control System‟ at
Aeronautical Development Establishment, Bangalore and Gas Turbine Research Establishment, Bangalore.
He is presently working as a Professor in the department of ECE and Dean (Planning & Development) at
CMR College of Engineering & Technology, Hyderabad.
B. Shruthi is a PG scholar with specialization in „Embedded System‟ in the department of
Electronics & Communications Engineering at CMR College of Engineering & Technology,
Hyderabad.