View
88
Download
0
Category
Tags:
Preview:
DESCRIPTION
One-Shot Learning Gesture Recognition. Students: Itay Hubara Amit Nishry Supervisor: Maayan Harel Gal-On. Outline. Background. Gesture recognition is a strong upcoming field in computer vision - PowerPoint PPT Presentation
Citation preview
PowerPoint Presentation
One-Shot LearningGesture RecognitionStudents:Itay HubaraAmit Nishry
Supervisor:Maayan Harel Gal-On
Outline2BackgroundGesture recognition is a strong upcoming field in computer visionGesture recognition can be seen as a way for computers to begin to understand human body language3
3MotivationExisting Gesture recognition demand a long configuration and trainingDifferent Gestures are been solved using different approaches
44GoalsLearn and understand existing Gesture recognition algorithms.Compare different approachesDesign Gesture recognition algorithm which reduces training time
55DataThe Data is compose from several set each contains:
Gesture vocabulary (learning set) which contain only one sample per gesture.Test set which contain one or more gestures.
Each of the sets has different vocabulary features such as large/small gesture hand/legs movement etc.
6Data Train7Base gesture
Data Test8
Multiple base gesturesLarge movementsData - Test9
Multiple base gesturesSmall movementsChallengesOne shot learning - only one learning sample (unlike the common approach of multi class classification)Tests videos segmentationSame gesture can have different number of framesEach set has different features (small/big gestures)1010Outline11Reduced ProblemAssume that each of the test movies has only one gesture
Goal:finding features space and distance function which have good separation of the features space
1212Problem ApproachClassic machine learning problemSelect FeatureOne Shot Match using similarity function
13FeaturesMotion Energysubtracting consecutive framesSpace Quantization14
14FeaturesHarris Corner DetectorFind interest point in the difference image based on corner detectionSpace Time Interest PointsExtend Harris to the time domain
15
15FeaturesHarris Corner DetectorFind interest point in the image based on corner detection
16
16FeaturesSpace Time Interest Points17
17Features18
STIPHarris18FeaturesHead Relative Interest Points19
19Features20
Interest pointsHead Histogram20Distance FunctionsGood features space is defined not only by the features but also by the distance (similarity) function
Different features need different distance functions
2121Principal Motion Using PCAUsing principal component analysis (PCA), to find the main motion vectors.For test set - project feature onto each of train principals and evaluate similarity22
22Earth Moving Distance Given two sets of distribution, EMD will measure the minimum cost to shift dirt from one distribution to the other.23
23Perturbed Variations24
Given two sets of distribution and predefined value of permitted variations optimally perturbs the distribution to best fit each other.
Transportationproblem underpermitted variations constrain
24Perturbed Variations25
25Levenshtein DistanceMeasure the difference between two sequences.Consider lengths and classification.26
26Results27Results28Top 10Top 2028Results29Outline30Complete ProblemSeparate problems Basic Segmentation (equal/movement)Whole problem solving approachMoving WindowDynamic Time Warping (DTW)
3131Problem ApproachThree different method to solve the problem:Basic Segmentation (equal/movement)Moving WindowDynamic Time Warping (DTW)
3232Moving WindowMove a window along the test video.Assume each window frames has only one gesture Preform basic analysis as did before to and build the distance matrix33
33Moving Window34
34Moving WindowAfter Sorting the Distance matrix we extract labels and cuts35
Several other operation (such as smoothing) are done before extracting the final resultDynamic Time Warping Create a state machine from train data:Module standing positionForm standing position can move to start of base gesturesAssume we can move forward, or stay in the same sate.For a given gesture find the best path along the sate machine3636Dynamic Time Warping 37
Results3838Results3939Results4040Results41Top 10Top 2041Outline42ConclusionsEach approach receive better results in different feature and similarity function
Different algorithms has different strengths (segmentation\recognition)
Segmentation require standing position model.
43Conclusions44Pre-processing unsupervised algorithms help better representing the data.
There is still allot left to do on the field
Future WorkTry different models for the standing position to improve segmentation results
Try combing DTW for segmentation and PCA for recognition.
Use different unsupervised algorithms to better represent the data.
45ReferencesIvan Laptev, "On Space-Time Interest Points, 2005Hugo Jair Escalantea and Isabelle Guyonb, "Principal motion: PCA-based reconstruction of motion histograms
M.Harel, S.Manor, "The Perturbed Variation, NIPS 2012Elizaveta Levina, Peter Bickel Department of Statistics, The EarthMovers Distance is the Mallows Distance: Some Insights from Statistics.Ofir Pele,Michael Werman, Fast and Robust Earth Movers Distances.2008
4646
Recommended