Show & Tell Sign Language Translator Glove

SHOW & TELLSIGN LANGUAGE TRANSLATOR GLOVESaron Paz Zvika Markfeld Tomer Daniel Oleg Imanilov

WHAT IS IT?An open source software & hardwareCombining: Arduino Sensors Android ADK Android

WHAT IT DOES? A glove that translates a predefined

vocabulary of the deaf and mute sign language into text

HOW DOES IT WORK?

Lily-Pad Arduino samples sensors attached to glove

ADK board relays data from the Lily-Pad to Android

Android uses a back propagation Neural Network to analyze data

WHAT IS LILY-PAD?

Arduino compatible microcontroller board designed for wearables and e-textiles

GLOVE SENSORS

Flex Sensors measure finger bending

Wii Nunchuck = Accelerometer + Gyros

Conductive tape detects fingers touching

FLEX SENSOR Flex Sensor changes

resistance when bent

Using a voltage divider the voltage is read by Arduino ADC (10 bits)

WII NUNCHUCK

WII NUNCHUCK SENSORS

2 Buttons

JoystickAccelerome

ter(X,Y,Z)Gyros

(Roll, Pitch)

I2C & Power

WHAT IS ANDROID ADK?

WHAT IS A NEURAL NETWORK? Interconnected group

of artificial neurons Good at

pattern recognition

SOFTWARE VIEW

ArtificialNeural

Network

GUI

Relay

ADK library ADK library

SensorControl

Serial Serial

HOW WE BUILT IT?

Click icon to add picture

PROTOTYPING FLEX SENSORS

PROTOTYPING FLEX SENSORS

FLEX SENSOR OUTPUT

ADDING NUNCHUCK SENSORS

THE PROBLEM WITH ASL

THE FINAL GLOVE

CHALLENGES How to position flex sensors?Fix tip to top and pass through sewn

loops How to distinguish between

R,U,V,S,E,T,N?Conductive tape detects fingers touching How to distinguish between each letter?Time based How to record movement?Freeze in the middle

DEMO

WHO ARE WE?

Saron Pazis an

experience designer

& a hopeless idealist

Zvika Markfel

dis a

design freshman

and an aspiring inventor

Tomer Daniel

is a software architect by day & a maker by night

Oleg Imanilov

is a software engineer

& a brilliant ideas

generator

REFERENCE Show & Tell Google code project

http://code.google.com/p/adk-sign-language-translator-glove/

Sensors Flex Sensors

http://www.sparkfun.com/products/8606 http://www.sparkfun.com/products/10264

Nunchuck http://www.dealextreme.com/p/designer-s-nunchuck-c

ontroller-for-wii-24529 Arduino

Wiichuck class http://www.arduino.cc/playground/Main/WiiChuckClass

Android ADK http://developer.android.com/guide/topics/usb/adk.ht

ml