View
38
Download
0
Category
Preview:
DESCRIPTION
JugEm : Software to help the world juggle. Level 4 Project 2013 Presentation David McBrierty. Juggling. The act of continuously tossing into the air and catching (a number of objects) so as to keep at least one in the air while handling the others. The idea…. - PowerPoint PPT Presentation
Citation preview
JugEm: Software to help the world juggle
Level 4 Project 2013 PresentationDavid McBrierty
Juggling
The act of continuously tossing into the air and catching (a number of objects) so as to keep at least one in the air while handling the others.
The idea…
• Build a piece of software that can be used to teach people how to juggle.
Existing Software
• Lots of existing tutorial help out there already….
Existing Software
More Existing Software
Theres something missing….
• Existing tutorials are not interactive!• Learn by doing!
What is JugEm?
• JugEm is an interactive application to help teach people how to juggle.
Initial ideas….
• Use mobile devices to get sensor data• Make users juggle actual devices• Strap devices to users hands
• Wii Controllers• Existing project has done this!
Kinect
• Kinect Sensor provides an interesting interface
How good is a good juggle?
• Lots of lovely physics behind what happens• Claude Shannon’s Uniform Juggle Theorem
Third Party Libraries
• Kinect for Windows SDK• Gives access to raw Kinect Data
• EmguCV Image Processing• .NET Wrapper around OpenCV library
• Microsoft Speech Platform 11• Allows Kinect Sensor to use voice commands
• Built around Microsoft XNA Framework• Framework for making Xbox and Windows software
System RequirementsMUST HAVE SHOULD HAVE
COULD HAVE WOULD LIKE TO HAVE
Detect user throwsDetect user catchesDetect peaksTrack users hand positionsProvide report on techniqueDetect 1,2 and 3 Ball patterns
Calculate Shannon ScoreSuggestions on better techniqueDetect dropsDetect 4 Ball patterns
Tasks to improve timingTasks to improve throwing
Define patterns using Siteswap
The good stuff….
• The system goes through 5 stages:
Kinect Data Input
• Color Image represented as byte[]
• byte[] can easily be converted to an image.
Kinect Data Input
• Depth Data provided as DepthImagePixel[]
• Each pixel from Color Image has DepthImagePixel for its distance from sensor
• Depth data has to be processed before it is useful!
The good stuff….
• The system goes through 5 stages:
Depth Image Filtering
• Raw depth data is very noisy, has to be filtered• 76800 pixels filtered (30 times a second)• Image is converted to gray scale (0-255)• All non player pixels are removed• EmguCV Binary threshold applied to results
• Anything behind players wrists is removed!
• Kinect thinks the balls are part of the player
The good stuff….
• The system goes through 5 stages:
Ball Detection
• EmguCV runs Canny Edge Detection on image• Finds edges with specific radius• Need to detect circles and capped ellipses
Screenshot
Ball Color Detection
• Uses Kinect Color Image• RGB Color Detection proved to be inaccurate
• Hue, Saturation, Value Color Detection instead!
• Extra work, but necessary
Frames
• A Frame is used to store all the data collected from processing
• All the Juggling Balls seen with color, direction and position• Positions of Users hands• Distance from each ball to each hand• Time Frame was seen
• Each new Frame is compared to the previous one to find out directions and detect Events.
The good stuff….
• The system goes through 5 stages:
Detecting Events
• Two areas around users hands• Hand• Throw
Detecting Throws and Catches
• THROW: Ball seen inside Throw Zone and then outside
• CATCH: Ball seen outside Throw Zone and then inside
The good stuff….
• The system goes through 5 stages:
• System collects the Events generated and processes them once the user is finished
• Each type of Juggle has a pattern
• System must be able to cope with missed events
Processing Events
Missed Events
• Inferring missed Events can be tricky…
• Pattern = P, RT, RC, P, LT, LC.
Missed Events
• Inferring missed Events can be tricky…
• Pattern = P, RT, RC, P, LT, LC.
• Inferring missed Events can be tricky…
• Pattern = P, RT, RC, P, LT, LC.
Missed Events
• Inferring missed Events can be tricky…
• Pattern = P, RT, RC, P, LT, LC.
Missed Events
• Inferring missed Events can be tricky…• The times of some inferred Events are slightly
out• Just sometimes a problem…
• Overall effect on time is small.
Missed Events
Juggle Report
• After the Events have been analysed and processed
• System has all the information it needs for Shannon Score
• Display report to the user
Personal Reflection
• Learned how to juggle!• Got to work in a new language• Working with Kinect• Working with all the frameworks• Image processing• Easily the biggest bit of work completed to
date….
Personal Reflection
fin
Recommended