View
50
Download
0
Category
Tags:
Preview:
DESCRIPTION
Perceptually Consistent Example-based Human Motion Retrieval. Zhigang Deng*, Qin Gu, Qing Li University of Houston. Introduction. Popularization of human motion capture data in animation and gaming applications Efficient retrieval of similar motions from a large data repository - PowerPoint PPT Presentation
Citation preview
Perceptually Consistent Example-based Human Motion Retrieval
Zhigang Deng*, Qin Gu, Qing Li University of Houston
Introduction
Popularization of human motion capture data in animation and gaming applications
Efficient retrieval of similar motions from a large data repository Fundamental basis for many motion data based
applications
e.g. CMU motion capture library. http://mocap.cs.cmu.edu.2605 trials in 6 categories and 23 subcategories.
Related Work – Motion retrieval Transform original high-dimensional human motion data
to a reduced representation [Agrawal et al. 1993; Faloutsos et al. 1994; Chan and Fu 1999; Liu et al. 2003; Chiu et al. 2004; Baciu 2006; Lin 2006] .
Match webs [Kovar and Gleicher 2004]
Describe potential subsequence matches between any pair of motion sequences.
Semantics-based motion retrieval [Muller et al. 2005; Muller and Roder 2006] Users provide a query motion as a set of time-varying geometric
feature relationships.
Human hierarchy construction
Motion segmentation and normalization
Motion pattern detection and indexing
Hierarchical pattern matching
Search result ranking
Our Approach PipelineMotion Data Preprocessing
Runtime Motion Query
Data Preprocessing - Motion Hierarchy Construction Decompose human motion into a hierarchical
structure [Gu et al. 08]
Local control granularity Correlations among different human parts are embedded in
different layers 4 layers, 18 parts are used in this work.
Data Preprocessing - Motion Segmentation and Normalization Existing human motion segmentation techniques
Angular acceleration [Zhao 01, Fod et al. 02, Kim et al.03], SVM
classifier [Li et al. 07], weighted sum of marker velocities [Gu et al. 08], PCA/PPCA [Barbic et al.04].
Probabilistic PCA [Barbic et al. 04] is used to segment motion into short motion segments for each body part in the hierarchy.
Parts Head LHand LArm RArm
Ave Frm + Var
18.32 + 2.32
8.43 + 6.43
11.39 + 6.54
12.75 + 7.34
Parts Torso RLeg LFoot RFoot
Ave Frm + Var
13.43 + 5.65
11.24 + 5.12
6.75 + 5.35
6.05 + 5.88
Average Frame Information of segments
Data Preprocessing - Clustering
Motion Pattern for each body part A representative motion segment for a node (i.e.,a body
part) in the constructed human hierarchy Normalization of motion segments
Adaptive K-Means clustering Increase K when the clustering error metric is larger than a
threshold Resulting data structures
(1) Motion Pattern Library, (2) Pattern Index Lists, (3) Pattern Dissimilarity Maps.
Review of Motion Preprocessing
Runtime Motion Query Query motion transformation
Map the query motion into a motion pattern index list for each hierarchy node
Fast (no clustering, just database matching) Motion similarity score computing
Local motion similarity between two index lists Extended Knuth-Morris-Pratt (KMP) string matching
algorithm [Knuth et al. 77] Global motion similarity computing and ranking
Hierarchical propagation
Local Motion Similarity
Similarity between two pattern index lists Different length of index lists Matching of two integer lists
Extended KMP String match algorithm Introducing “Quasi-Match” based on the pre-constructed
pattern dissimilarity maps Large numbers of different motion segments Distance is less than a threshold
Update matching score If the number of consecutive quasi-matches is larger than a
threshold, otherwise decrease.. Score normalization based on the length of index lists
Global Motion Similarity
Hierarchical Score Propagation High local motion similarity does not mean global
motion similarity Nodes in the upper levels encode more global motion
information From bottom to top
Ranking of the final scores at the root node
Review of Runtime Motion Retrieval
Results and Evaluation
Time and Storage Search Accuracy Search Quality Perceptual Consistency Experiment
Results and Evaluations – Time and Storage
We tested our method on four datasets with different sizes
The test computer with a Intel Duo Core 2GHz CPU and 2GB memory.
The average duration of used query motions is 10 seconds.
56MB, 170 motions,68,293 frames456MB, 396 motions, 556,097 frames976MB, 542 motions, 1,190,243 frames1452MB, 941 motions, 1,770,731 frames
Results and Evaluations – Search Accuracy Accuracy evaluation scheme [Kovar and Gleicher 04]
Two different types of datasets: single-type motion datasets (pre-labeled dataset with the same semantic category, walking) – Ground truth, mixed motion dataset (unlabeled, mixed of various types).
True-positive accuracy ratio is defined top N (=20) results from mixed motion datasets are in the correct/expected single-type motion dataset.
56M test dataset: 170 sequences, 68,293 frames, five categories – walking, running, jumping, kicking, basket-playing.
Results and Evaluation – Comparative User Studies
Compare our approach with match-webs approach [Kovar and Gleicher 04], piecewise linear space [Liu et al. 05], weighted PCA [Forbes and Fiume 05]. Semantic-based motion retrieval [Muller et al. 05] was not
chosen, because of significant differences in input requirements.
Two usability questions (a) Perceptual Consistency: Retrieved results (motions) are
ranked in a perceptually consistent order? (b) Search Quality: Motion similarities of retrieved results?
Results and Evaluation – Comparative User Studies Perceptual-consistency
Computer algorithms rank motions in a certain order, C. Humans rank these (the same) motions in another order, H. Relationship/consistency between C and H?
Study Experiments 3 query motions (walking, running, basketball-playing),Top-ranked
N (=6) results for query, 4 approaches, total 72 = 3*6*4 results. Side-by-side comparison and user rating (one is a searched
motion, the other is the query motion), in a random order. Rating is from 1 (“completely different”) to 10 (“identical”). 24 experiment participants
Results and Evaluation – Comparative User Studies
Quality of searched motions Compute average similar ratings and standard
deviation Higher the average similar rating is, the better
quality of search it achieves.
Results and Evaluation – Comparative User Studies
Perceptual-Consistency Plot human-rankings vs computer-
rankings in a 2D space. Ideal consistency is shown as a straight
line.
Canonical Correlation Analysis Scale-invariant optimum linear
framework
CCA Coefficient results
Walking
Running
Basketball-playing
Review of User Studies
Conclusions An efficient, example-based human motion retrieval
technique Major distinctions of our approach
Efficiency Linear to the size of query motion and database size
Flexible search query A human motion subsequence, or a hybrid of multiple motion sequences
Perceptually consistent search outcomes Comparative user studies to find out the correlations between the result-
ranking by computer algorithms and the result-ranking by humans
Discussion and Limitations
Current approach does not consider the path/motion trajectory of the root of the human in the retrieval algorithm. The search results may enclose different paths/trajectories.
Current approach can only search for single-character motion sequences.
Future Work
A number of empirical parameters of current approach may critically affect the search accuracy and outcomes. Establish quantitative correlations between
“parameter setting” and “search accuracy and outcomes”.
Graphics hardware accelerated, motion query processing.
Thank You!
Questions?
Recommended