Conversion of sign language to speech using kinect

By,

Rajaganapathy S

VIT UNIVERSITY

conversion of Sign Language to speech using Kinect

Aim and Objective:• The aim of the project is to build a tool for the people

who is unable to hear or speak. It should be modest, easy to use and recognize a users gestures with a highest degree of accuracy, so that users do not have to repeat themselves often, which can be annoying.

• The system should run on readily available commercial platforms, and must be cost effective. So that the poor background people may also use the app effectively.

Introduction• Sign language is a system of communication using

visual gestures and signs, as used Hearing impaired people.

• But none of the sign languages are universal or international. A person should know the sign language to understand the language;

• This becomes complicated when a person who has inability to speak or hear wants to convey something to a person or group of persons, since most of them are not familiar with the sign language.

Techniques● Locate specific joint from all 20 joints● Gather X,Y, Z coordinates● Compare coordinates with predefined dataset values

Detailed Design:

Module Description

• human gestures input stream

Kinect Receiving Gestures

Application Match Gestures

Convert Text into Speech

Results:

Conclusion:• The project is aimed to minimize the major

complexity in the system for further extensions, the sensor comes with the feature of face recognition and voice recognition and therefore the next phase of the project would be adding face recognition for capturing the expressions which in turn increases the productivity of the application by adding a little more accuracy

References • [1] Jungong Han, Enhanced Computer Vision with

Microsoft Kinect Sensor, A• Review, IEEE TRANSACTIONS ON CYBERNETICS.• [2] Microsoft Kinect SDK,http://www.microsoft.com/en-

us/kinectforwindows/.• [3] ] Gunasekaran. K, Manikandan. R, International

Journal of Engineering and• Technology (IJET): Sign Language to Speech

Translation System Using PIC• Microcontroller.• [4] RiniAkmeliawatil, Melanie PO-LeenOoi et al,Real-

Time Malaysian Sign.