Embed Size (px)
Citation preview
Department of Electrical and Computer Engineering
He ZhouHui ZhengWilliam MaiXiang Guo
Advisor: Professor Patrick Kelly
ASLLENGE
Department of Electrical and Computer Engineering
Outline
Background System Requirements Design Challenges System Overview Hardware and Software Team Roles Timeline Q/A
2
Department of Electrical and Computer Engineering
Background
Leverage Kinect technology to recognize and offer feedback on American Sign Language
ASL is the one of most commonly used language in US(.5 to 2 million population use ASL as their primary form of communication)
Educational learning tool for ASL
3
Department of Electrical and Computer Engineering
Motivation
Helping hearing impaired people to learn ASL. Facilitate communication between ASL speaker
and non-speaker
4
Department of Electrical and Computer Engineering
System Requirements
Create a user-friendly interface that recognize sign language and offer feedback• Software must be easy to use
Display translated word or phase being demonstrated by user• Provide quality feedback to user concerning
the accuracy of ASL Cost should be within the budget
5
Department of Electrical and Computer Engineering
Design Challenges Human Factors Background Interferences Finger Gestures Hardware limitation• Kinect only has 320x240 resolution and 16 bit
depth
6
Department of Electrical and Computer Engineering
Kinect
Has a depth and regular Camera Associated with software development kit
• Able to track a human being.• Use computer generated algorithms
to form human skeleton data points Track upper body movements
7
Department of Electrical and Computer Engineering
System Overview
8
Department of Electrical and Computer Engineering
Video demonstration(Hand Rotation)
9
Department of Electrical and Computer Engineering
Video demonstration(Hand Crossing)
10
Department of Electrical and Computer Engineering
Video demonstration(Hand behind Head)
11
Department of Electrical and Computer Engineering
Video demonstration(Hand behind Back)
12
Department of Electrical and Computer Engineering
Hardware Implementation 1 Increasing the flex sensor
output voltage with bend Voltage output goes into
an AD converter board AD converter digitizes the
output signal PC captures the signal via
USB or Bluetooth
13
Department of Electrical and Computer Engineering
Hardware Implementation 2 Kinect Camera captures the raw image and the
depth image data The data send to PC to be processed into skeleton
tracker Combining the signal from flex sensor and search
in database for matching motions
14
Department of Electrical and Computer Engineering
Software Design Flow
15
Department of Electrical and Computer Engineering
Development Model
16
Department of Electrical and Computer Engineering
Cost of Major Components
Name of Component
Flex Sensor
A/D Converter
Board
A Pair of Gloves
Kinect Sensor
Bluetooth Module
Quantities 10 1 1 1 1
Unit Price $12 $50 $10 $150 $20
Total Price $350
17
Department of Electrical and Computer Engineering
Team Roles
He - test and integrate software prototype
Hui- image processing and programming
William - build microcontroller to detect finger and palm movements
Xiang- website, overall testing and building ASL database
18
Department of Electrical and Computer Engineering
MDR deliverables
The goal for our MDR is to be able to detect and track arm and hand positions, possibly be able to translate simple hand gestures
19
Department of Electrical and Computer Engineering
Conclusion
Our project is good for SDP:• Contains both hardware and software• Expandable: database, audio recognition, braile
Our project is a good product:• Aid the disabled people• Personal/commercial use• Relatively inexpensive
20
Department of Electrical and Computer Engineering
Timeline
PDR
Purchase Parts
Planning
GUI
Interface with the Sensors
Build Finger Sensors
Build Test Database
Prototype 1
MileStone MDR Deliverables
System Integration
Winter Break
Expand Database
Prototype 2
MileStone CDR Deliverables
Alpha Testing
Update
Beta Testing
Spring Break
Finalize Project
MileStone FRP Deliverables
10/16/2011 12/5/2011 1/24/2012 3/14/2012 5/3/2012
21
Department of Electrical and Computer Engineering
Q&A
22
