Terry Taewoong Um (terry.t.um@gmail.com)

University of Waterloo

Department of Electrical & Computer Engineering

Terry Taewoong Um

AN UNSUPERVISED APPROACH

TO DETECTING AND ISOLATING

ATHLETIC MOVEMENTS

Terry T. Um and Dana Kulić

Motivation

B. Tekin. et., “Direct Prediction of 3D Body Poses from Motion Compensated Sequences” (2016)

CMU motion capture dataset(http://www.cs.cmu.edu/~jkh/uobio/bio.html)

We want to analyze athletic movements

from a long sequence of skeleton data

Scenario

Procedure

Data collection

Segmentation(Extracting athletic movements)

Movement analysis

1 / 12

What are athletic movements?

• There can be various ways to define athletic movements

2 / 12

• In this research, we attempt to detect stretch-shortening cycles from a long sequence of skeleton data

(R. Bartlett, Introduction to sports biomechanics: Analysing human movement patterns. Routledge, 2007)

- Use of the stretch–shortening cycle of muscle contraction.

- Minimisation of energy used to perform the task.

- Control of redundant degrees of freedom in the segmental chain.

• Universal principles that apply to all sports tasks

What are athletic movements?

pre-stretch

stretch

http://goo.gl/Gc5Gej

pre-stretch stretch

https://en.wikipedia.org/wiki/Pitcher

Pre-Stretch

• Storing potential energy• (Usually) flexing body limbs

Stretch• Stretching body limbs

explosively & coherently

• We will detect flexed poses followed by a explosive and coherent movement

3 / 12

0 T

Stretch

Pre-Stretch

proposedmeasure Stretch followed

by pre-stretch

detect!

Detection of Athletic Movements

9 / 12

Stretch• I will measure the extent of & , and blend them

for detecting athletic movement

Pre-Stretch

Detection of Pre-Stretch Phase

• Manipulability

The robot arm’s ability to change the position or orientation of its endpoint in each direction

pre-stretch stretch

• From the Jacobian 𝑱 of the limb,

pre-stretch close to 1

stretch close to 0

4 / 12

(T. Yoshikawa, 1985)

Detection of Stretching Phase

• In the stretching phase, all joints should move coherently

• Different joint movements make a coherent kinematic synergy

at the limb’s endpoint.

• Prob. 1) How can we represent kinematic synergy of joints?

Forward Kinematics

joint axis

joint angle

in products of exponentials (POE) formula (by Lie group formulation)

KinematicSynergy

An approximate abstraction of all joints’ instantaneous movements

5 / 12

problem1problem2

Detection of Stretching Phase

• Baker–Campbell–Hausdorff (BCH) formula

If A and B is enough small,

• If we keep merging POEs one after another,

KinematicDimensionality

Reduction (KDR)

• Note that 𝜔𝑖 𝑞𝑖 is small enough if we set 𝑞𝑖 as instantaneous changes of joint angles

𝜔𝐵𝐶𝐻 ∈ ℝ3 for rotations, ∈ ℝ6 rotations & translations

6 / 12

Detection of Stretching Phase

7 / 12

• Prob. 2) How can we detect coherency from the kinematic synergy, 𝜔𝐵𝐶𝐻?

𝜔𝐵𝐶𝐻

(Right leg)

𝜔𝐵𝐶𝐻

(Left leg)

Displaying the trajectory of kinematic synergy values (𝜔𝐵𝐶𝐻 ∈ ℝ3)

Detection of Stretching Phase

stretching part • In the stretching part, 𝜔𝐵𝐶𝐻 travels long distance toward a certain direction.

• Or we can say 𝜔𝐵𝐶𝐻 momentarily forms a submanifold (lower dimensional manifold).

[Scaling factor for measuring distance][eigenvalue ratio for detecting submanifold]

Coherency

8 / 12

• Prob. 2) How can we detect coherency from the kinematic synergy, 𝜔𝐵𝐶𝐻?

Detection of Athletic Movements

1. Use fixed size (e.g. 0.25sec) sliding window(Let’s assume that the window contains 2m+1 data from 𝑥𝑖−𝑚 to 𝑥𝑖+𝑚)

2. Calculate manipulability at (𝑖 − 𝑚) for detecting pre-stretches

3. Calculate scaled coherency from 𝑥(𝑖−𝑚 ∶ 𝑖+𝑚) for detecting stretches

4. Blend them with a ratio 𝛽

5. Report athletic movements when 𝜐𝑖 > 𝜐𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑

𝜇(𝑖−𝑚)

9 / 12

Experiments

9 / 12

• The proposed method is evaluated on the CMU motion capture dataset

• We experimented with 4 athletic motions (which has about 60 trials in total), which are jumping, soccer kicking, baseball pitching, and golf, and 2 long sequences of random motions (walking, hand waving, squat, etc.)

All codes are available from

http://terryum.io/publications/#EMBC2016

Results

340

10 / 12

Pre-stretch StretchMeasure

Results

(a) (b)

11 / 12

: Test with long sequences of activities

Remarks

• No prior knowledge for movements is required

• No machine learning techniques have been applied, that is, no training data or training time is required.

• You can enhance the detection performance with the combination of machine learning techniques.

• Or, you can use the proposed representation (kinematic synergy) for machine learning tasks, e.g., human activity classification

• In the future work, we will verified the proposed concepts in machine learning tasks

12 / 12

“An Unsupervised approach to Detecting and isolating athletic movements,”

T. T. Um and D. Kulić, EMBC2016

Summary

• An approach to detect athletic movements of body parts is proposed

• Pre-stretching motions are captured by using manipulability

• We proposed kinematic dimensionality reduction (KDR) method for representing kinematic synergy of joint movements

• Stretching motions are captured by detecting submanifold in the kinematic synergy

• By detecting sequential pre-stretching and stretching motions, we can detect athletic movements of the body parts

• The proposed approach has been verified with CMU mocap dataset(The Matlab code is available from http://terryum.io/publications/#EMBC2016)

12 / 12

“An Unsupervised approach to Detecting and isolating athletic movements,”

T. T. Um and D. Kulić, EMBC2016