Bengali Sign Language

Khulna University of Engineering & TechnologyDepartment of Electrical and Electronic Engineering

Seminar On EE-4130

Presented by–S.M.Kamrul HasanRoll No. 1003079

Department ofElectrical & Electronic

Engineering1

American Sign Language word recognition with a sensory glove using artificial neural networks

Published in: Elsevier Journal of Engineering Application of

Artificial Intelligence, Volume 24, Issue 7. Date of Publication: October 2011

Pages: 1204-1213

Authors: Cemil Oz & Ming C. Leu

2

Outline :-

Problem definition Motivation What is Sign Language Why Data glove System Structure Data Collection Feature Extraction Artificial Neural Network(ANN) Training Algorithm Test Results Conclusion

3

Problem definition:

Humans have been endowed by nature with the voice & hearing capability...

Kids speaking

4 / 40

Problem definition:-

4

Problem definition:

Humans have been endowed by nature with the voice & hearing capability... ...but not everybody possesses this capability←deaf people

Deaf kid

4 / 40

Problem definition:-

5

Motivation:-

A Real time Sign Language Recognition System

Combines Adaptive Filtering & Artificial Neural Network

Interpret Sign language into English word.

Fully Flexible Intelligent online learning Training time so faster Better performance Adaptable

6

What is Sign Language ?

Visual gestural communicating language used by deaf.

Movement not just with hands

Varies from country to country, language to language

English alphabet in Sign Language7

System Structure:-

8System Architecture

Why Data Glove?

Cyber-Glove measure hand shape accurately light in weight high resolution data

data acquiring is more difficult complicated data processing slower recognition rate

Vision based system

Flock of Birds(3D Motion tracker) tracks hand orientation & position

Cyber glove

1.Data Collection:

10

Data collection Block

Indicate hand moving or static

Cyber-glove & motion Tracker

Velocity Network

XYZ

Data store

Velocity Network

Vel

ocit

y

2.Feature Extraction:-

to determine exactly which features are Important

Part of the data reduction process

11

Seven feature vectors

Artificial Neural Network (ANN):

A Genetic Algorithm, resembles human brain

12

acquires knowledge through learning. it involves human like thinking. they handle noisy or missing data.

BackPropagation

Training Algorithm

Successful approach to construct ANN A supervised learning

Predicted output != actual outputWeight is adjusted until no error

13

Error Adjust

N.NCompare

Actual output

Desired output

Input output

Weight

14

Test Results

50 different American signs each with 6 samples total of 50x6=300 samples for training.

Successfully recognize sign language to English Word

Sign Recognition by system

Test Results

15ANN Test results for Known words

16

Output

Output Decoding

Does it cross threshold?

YesTrainingby N.N

NoUnknown

word

Do you want to add the word?

Yes

Unknown Words Recognition

Test Results

Test Results

Levenberg-Marquardt Vs. Backpropagation

Backpropagation gives Better performance with less train time

17Number of hidden layer nodes

18

Trained by 3 users Trained by 7 users

Algorithm Same users New users Same users New users

LM 87.96% 72.22% 92.09% 82.17%

BP 93.51% 80.09% 95.72% 85.41%

Levenberg-Marquardt Vs. Backpropagation

Better accuracy

Backpropagation gives Better performance with

Test Results

Conclusion

An advanced initiative to recognize American Sign Language with faster training, better accuracy & better recognition performance.

The ultimate goal of this paper is to further improve the proposed sign language recognition system that can use sentence recognition and eliminate the limitations and use it successfully for Human to Machine Interface for disable people.

19

THANK YOU ALL

20