Sign Recognition Technology for the Learning of Hearing Impaired

People

José Oramas M.

19-10-2009

Information Technology CenterEscuela Superior Politécnica del Litoral

Outline Background

− Problem Statement

− Previous Approaches Proposed Solution

Results

− Methodology

− Discussion Conclusion

Future Work

Background

Hands as Means of Interaction− Hearing Impaired People

Ecuadorian Hearing Impaired Community− ESL vs. Oralism ===> ESL

− Wide disparity in teacher:student ratio

− No technology-supported learning methodology

Previous Work Accessible Learning Environments

− eLearning system: Drigas et al. (2005)

− CMS: Drigas et al. (2005)

Sign Language Games− Henderson et al. (2005)

Playware− Especialized toys: Yarosh et al. (2008)

− Augmented Teddy bear: Huang et al. (2008)

Proposed Solution

Promote the practice of sign language. Easy to use/understand interface. Employ a pointer gesture recognition algorithm

Testing Prototype

Testing Methodology Two categories / chapters Controlled test / open test / questionnaire Measured Factors

● Accuracy

● Comfortability

Test users● 12 teachers (age: ~35 )● 10 students (age: ~ 8 , 14 years)

Results

Accuracy: from 65% to 85%− Different signs and hand sizes

Comfortability− “The glove was too rigid, produced too much pressure

that made it difficult to move the fingers”

Scale Invariant = Users of different heights

Motivation / Acceptance

Results (cont.)

Group / Self Learning

Conclusions

Hand gestures recognized as pointer gestures.− Recognition rate of 85% = potential application

Impact on Teaching/Learning− Increases motivation

− Portable sign language classrooms

− Group / Self learning.

Weakness: dataglove

Future Work Experienced Users (teachers / instructors)

− Study & testing of computer vision based pseudo-posture recognition.

Future Work (cont.)

Beginners− Alternative datagloves

− GUI enhancements

Promptthe User to sign

ListCategories

AdditionalFeedback

Sign Recognition Technology for the Learning of Hearing Impaired

People

José Oramas M.

19-10-2009

Information Technology CenterEscuela Superior Politécnica del Litoral