Human Motion Analysis: methodologies and applications

HUMAN MOTION ANALYSIS:

METHODOLOGIES AND APPLICATIONS

Maria João M. Vasconcelos1 and João Manuel R.S. Tavares

2

1. ABSTRACT

The study of motion is one of the most interesting and active areas in Computational

Vision, particularly considering the human motion. Human motion analysis usually

follows a general framework: feature extraction, where the identification of the objects

characteristics to be analyzed in the image frames is made; feature correspondence,

where the problem of matching features between consecutive frames is approached; and

finally high level processing can be considered, for instance, in the recognition of

human movements, activities or poses.

In this paper, we present a review about the leading computational techniques used in

human motion analysis and some of their main applications.

2. INTRODUCTION

During the last decades several surveys have been made regarding the subject of human

motion analysis. The first significant review about human motion analysis was probably

due to Aggarwal et al [1]. In this paper, the authors reported the developments on non-

rigid motion analysis regarding the articulated and the elastic motion, and discuss both:

motion recovery methods using no a priori shape models; and model based approaches.

Aggarwal and Cai presented another overview of the tasks involved in human motion

analysis [2] which covered the work prior to 1998. The paper focuses on three major

areas related to interpreting human motion: motion analysis involving human body

parts; tracking of human motion using single or multiple cameras; and recognizing

human activities from image sequences.

In the same year, Gavrila [3] published a survey about the analysis of human movement.

The work was limited on whole-body or hand motion and does not include the work on

human faces. Here the author grouped the methodologies in 2D approaches, with or

without explicit shape models, and 3D approaches.

Later, Moeslund and Granum [4] presented a survey about computer vision based

human motion capture from the last two decades. The focus was on a general overview

1 PhD student, Laboratory of Optics and Experimental Mechanics, Institute of Mechanical Engineering

and Industrial Management, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, s/n,

4200-465 Porto, Portugal 2 Professor, Laboratory of Optics and Experimental Mechanics, Institute of Mechanical Engineering and

Industrial Management, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, s/n, 4200-

465 Porto, Portugal

Proceedings of the Intl. Symposium CMBBE2008: Published by Arup and MediTech

based on the taxonomy of system functionalities: initialization, tracking, pose estimation

and recognition. Throughout the paper a number of general assumptions used in this

field were identified and suggestions for future research directions were presented.

Recently, a survey of the various studies related to the human tracking and body parts

was presented by Wang and Singh [5]. Approaches related with modeling behavior

using motion analysis are also presented. In the same year, Wang et al. [6] presented a

review of research on computer-vision-based human motion analysis, giving special

emphasis on three major issues involved in this area, namely human detection, tracking

and activity understanding. The authors also discussed some research challenges and

future directions.

In this paper we intend to present a review about the key computational methodologies

used in human motion and some of their main applications. Usually, the study of human

motion in image sequences starts with feature extraction, where the identification of the

objects characteristics to be analyzed in the image frames is made. The second step

regards to feature correspondence, where the problem of matching features between two

consecutive frames is addressed. And, finally, after the features are extracted and

correctly matched over an image sequence, high level processing is taken, for instance,

in the recognition of human movements, activities or poses.

3. HUMAN MOTION METHODOLOGIES

Most methods developed for human motion analysis use models to fit human body parts

to the given images. The geometric structure of human body can be represented as stick

figures, 2D contours or volumetric models.

In [7], the authors used a stick figure model which learns the 3D variability of human

posture using a set of training sequences. They developed a matching algorithm based

on Dynamic Programming to establish a mapping between postures from different

motion cycles. Then, the model is trained, a mean walking performance is automatically

learnt and the statistics about the observed variability of the postures and motion

direction are also computed.

2D contours are often used to detect humans in image sequences; for example, in [8] an

algorithm was presented that consists in three main steps: detecting human candidates,

validating the model of a human and tracking of the model in consequent frames. The

model adopted is a six-link model with an articulated head that can cope with a frontal

view of a person. It starts using simple means to find a human candidate within a region

of interest and afterward validates it using an extended biped human model.

In [9] the authors presented an integrated system for automatic acquisition of the human

body model and motion tracking using input data acquired from multiple synchronized

video streams. The system performs the tracking on the 3D voxel reconstructions

computed from the 2D foreground silhouettes, the human body model used consists of

ellipsoids and cylinders and is described using the twists framework resulting.

Other type of methodology consists in using the appearance to construct the human

model. In [10] moving people are modeled with the assumption that, while

configuration can vary substantially from frame to frame, appearance does not. Thus,

the authors present an algorithm that first builds a model of the appearance of the body

of each individual by clustering candidate body segments and then uses this model to

find all individual in each frame.

A different possibility is to use a motion model to accomplish human tracking. For

example, in [11] a motion model was built from the semi-automatically acquired

training data and motion constraints were explored by analyzing the dependency of


joints. Both of them were then integrated into a dynamic model in order to reduce the

size of the sample set.

In [12] the authors combine the last two methodologies by integrating information from

appearance with motion information. They use a detection style algorithm and train it to

take advantage of both motion and appearance information to detect a walking person.

In [13] the authors presented a robust feature-based tracking method of human motion.

The approach presented enables to track motions of different body parts without

articulated body models and their initialization by using a standard point-wise tracker

modified for robustness and grouping image points undergoing the same rigid motions.

An approach that combines the prior knowledge regarding a person’s motion with

human body kinematics constraints was presented in [14]. The former technique uses an

efficient feature point selection and tracking approach to compute feature points’

trajectories and then 3D motion models associated with each joint are locally tined by

using the key frames, meaning frames where both legs are in contact in the floor. In

[15], the authors propose a marker less system for analyzing and classifying human gait

by computer vision techniques. The gait figure is extracted from the body contour by

determining the body points using linear regression and motion tracking with

topological analysis. Then, the detection of the gait cycle was done by symmetry

analysis and the extraction of the gait figure was made using 2D stick figures; finally,

kinematic analysis and feature extraction was done to classify the gait patterns. In [16]

another approach for model-free marker less model and motion capture is presented. In

their approach, a kinematic model and joint angle motion are extracted from volume

sequences of subjects with arbitrary tree-structured kinematics. The motion capture

method uses a skeleton curve, found in each frame of a volume sequence, to

automatically determine kinematic postures and latter these postures will be aligned to

determine a common kinematic model for the volume sequence. The motion sequence

suited to this model in found through the reapplication of the kinematic model to each

frame.

In [17] the authors described a method for automatic person recognition from body

silhouette and gait. It combines a background subtraction procedure with a simple

correspondence method to segment and track spatial silhouettes of a walking figure. In

order to reduce the computational cost during training and recognition, simple feature

selection and parametric eigenspace representation are used.

4. APPLICATIONS

The improvement of the interaction between men and machines is essential for the

growth of human motion analysis. A wide variety of disciplines has been interested in

human motion. For instance, in surveillance systems the human motion analysis can be

used to identify suspicious movements of persons in a parking lot or to monitor the

actions of individuals and classifying its nature in a commercial space. These types of

activity can require a considerable effort from the human operators, since it is common

to have several cameras in a parking lot or a shopping area that should be analysed

simultaneously. In [18] the authors proposed an algorithm to model, segment and

classify human activities in a constrained environment by using switched dynamical

models. In [19] the authors analyse human behaviours by classifying the posture of the

monitored person and consequently detecting corresponding events and alarm

situations, like a fall. The former approach can be applied to monitor people at home,

especially elders with limited autonomy, and define potential alarm situations.


In sports, the biomechanical analysis of the movements of athletes can help them to

understand and improve their performances or even facilitate the recovery process after

injuries. In [20] the authors present a model-based image matching technique to extract

kinematic characteristics of three typical anterior cruciate ligament (ACL) injury

situations, which can provide valuable information on the mechanisms for ACL injuries

in sports. Other example, is the Football Interaction and Process Model system (FIPM)

that can acquire action models, infer action-selection criteria and determine player and

team strengths and weaknesses, [21].

Another application area where human motion analysis plays an important role is Gait

Analysis. In [7] the authors propose an action specific model which automatically learns

the variability of 3D human postures observed in a set of training sequences. Dynamic

Programming techniques are used to synchronize the training sequences and, as a result,

they obtain an action model with a representative manifold for the action; namely, the

mean performance, the standard deviation from the mean performance and the mean

observed direction vectors from each motion subsequence of a given length. The

resulted model can be used for gait recognition applications such as in the identification

of a subject when performing an action by observing only a very reduced motion

portion of it. In [22] the authors show results that support vector machines are able to

automatically recognize gait patterns of old and young people. Both histogram and

Poincaré plot diagrams features are effective in discriminating the two age groups,

which can indicate that such plots might be useful in detecting movement abnormalities

or for monitoring improvements in walking performances because of treatment or

intervention in a clinical/rehabilitation procedure.

In medicine, the study of human motion can be also extremely valuable. In [23] the

authors described a clinical gait analysis system used at the Newington Children’s

Hospital, the clinical testing protocol and the algorithms used are also presented. Over

ten years later, in [24], motion analysis was used in the study of spondylolisthesis and in

[25] is also presented a motion study in patients with Parkinson’s Disease. In [26] the

authors present tests of an extensive range of dimensionality reduction and robust

classification techniques for linking pathological plantar hyperkeratosis and functional

biomechanical foot data.

Other area of application of human motion analysis is Computer Graphics. In [27] the

authors present a framework for the modelling and animation of human characters from

monocular videos. In [28] is described a real-time system for capturing humans in 3D

and placing them into a mixed reality environment, where the images of the subject are

constructed using a robust and fast shape-from-silhouette algorithm.

5. CONCLUSIONS

The analysis of the human motion has been a subject of large research in the last

decades. The detection, tracking and identification of humans have attracted great

interests from computer vision researchers due to its promising and important

applications in many key areas.

The majority of the methodologies used for human motion analysis are based on

models, for example, shape models like stick figures, 2D contours or volumetric

models. Other researchers use models based on appearance to build the human model.

The motion information can also be used in human motion analysis and some

researchers combine both the appearance with motion information. Other examples of

methodologies are feature-based or take into account the human body kinematic

constraints.


So, this paper aims to provide a comprehensive survey of the most recent developments

in human motion analysis, particularly in the tracking issue and its main applications,

covering the latest research ranging mainly from 2003 to 2007.

6. ACKNOWLEDGEMENTS

The first author would like to thank the support of the PhD grant SFRH/BD/28817/2006

from FCT – Fundação para a Ciência e Tecnologia from Portugal. This work was

partially done in the scope of the project “Segmentation, Tracking and Motion Analysis

of Deformable (2D/3D) Objects using Physical Principles”, reference POSC/EEA-

SRI/55386/2004, financially supported by FCT.

7. REFERENCES

1. Aggarwal, J. K., et al., Articulated and elastic non-rigid motion: a review, in

Proceedings of the IEEE Workshop on Motion of Non-Rigid and Articulated

Objects, 1994 Austin, Texas, USA, 2-14.

2. Aggarwal, J. K. and Cai Q., Human Motion Analysis: A Review, Computer

Vision and Image Understanding, 1999, Vol. 73, No. 3, 428-440.

3. Gavrila, D. M., The Visual Analysis of Human Movement: A Survey, Computer

Vision and Image Understanding, 1999, Vol. 73, No. 1, 82-98.

4. Moeslund, T. B. and Granum E., A Survey of Computer Vision-Based Human

Motion Capture, Computer Vision and Image Understanding, 2001, Vol. 81, No.

3, 231-268.

5. Wang, J. W. and Singh S., Video Analysis of Human Dynamics - A Survey,

Real-time Imaging Journal, 2003, Vol. 9, No. 5, 320-345.

6. Wang, L., Hu W., and Tan T., Recent developments in human motion analysis,

Pattern Recognition, 2003, Vol. 36, 585-601.

7. Rius, I., et al., Automatic Learning of 3D Pose Variability in Walking

Performances for Gait Analysis, International Journal for Computational Vision

and Biomechanics, 2007, Vol. 1, No. 1, 33-43.

8. Korc, F. and Hlavac V., Detection and Tracking of Humans in Single View

Sequences Using 2D Articulated Models, in Human Motion: Understanding,

Modelling, Capture and Animation, B. Rosenhaln, R. Klette, and D. Metaxas,

Editors, 2007, Springer.

9. Mikic, I., et al., Human Body Model Acquisition and Tracking Using Voxel

Data, International Journal for Computater Vision, 2003, Vol. 53, No. 3, 199-

223.

10. Ramanan, D. and Forsyth D. A., Finding and Tracking People from the Bottom

Up, in IEEE Computer Society Conference on Computer Vision and Pattern

Recognition, 2003, Wisconsin, USA.

11. Ning, H., et al., People tracking based on motion model and motion constraints

with automatic initialization, Pattern Recognition, 2004, Vol. 37, 1423-1440.

12. Viola, P., Jones M. J., and Snow D., Detecting Pedestrians Using Patterns of

Motion and Appearance, International Journal for Computer Vision, 2005, Vol.

63, No. 2, 153-161.

13. Gonzalez, J. J., et al., Robust tracking and segmentation of human motion in an

image sequence, in International Conference on Acoustics, Speech and Signal

Processing 2003 - ICASSP03, 2003, Hong Kong, China.


14. Sappa, A. D., et al., Prior Knowledge Based Motion Model Representation,

Electronic Letters on Computer Vision and Image Analysis, 2005, Vol. 5, No. 3,

55-67.

15. Yoo, J. H. and Nixon M., Markerless Human Gait Analysis via Image

Sequences, in International Society of Biomechanics XIXth Congress, 2003,

Dunedin NZ.

16. Chu, C. W., Jenkins O. C., and Mataric M. J., Markerless Kinematic Model and

Motion Capture from Volume Sequences, in Proceedings of IEEE Computer

Vision and Pattern Recognition, 2003, Wisconsin, USA.

17. Wang, L., et al., Silhouette Analysis-Based Gair Recognition for Human

Identification, IEEE Transactions on Pattern Analysis and Machine Intelligence,

2003, Vol. 25, No. 12, 1505-1516.

18. Nascimento, J. C., Figueiredo M. A. T., and Marques J. S., Segmentation and

Classification of Human Activities, in HAREM 2005 - International Workshop

on Human Activity Recognition and Modelling, 2005 Oxford, UK.

19. Cucchiara, R., et al., Probabilistic Posture Classification for Human-Behavior

Analysis, IEEE Transactions on Systems, Man, and Cybernetics - Part A:

Systems and Humans, 2005, Vol. 35, No. 1, 42-54.

20. Krosshaug, T., et al., Biomechanical analysis of anterior cruciate ligament injury

mechanisms: three-dimensional motion reconstruction from video sequences,

Scandinavian Journal of Medicine and Science in Sports, 2007, Vol. 17, 508-

519.

21. Beetz, M., Kirchlechner B., and Lames M., Computerized Real-Time Analysis

of Football Games, PERVASIVE computing, 2005, Vol., 33-39.

22. Begg, R. K., Palaniswami M., and Owen B., Support Vector Machines for

Automated Gait Classification, IEEE Transactions on Biomedical Engineering,

2005, Vol. 52, No. 5, 828-838.

23. Davis, R. B., Õunpuu S., and Tybursky D., A gait analysis data collection and

reduction technique, Human Movement Science, 1991, Vol. 10, 575-587.

24. Simsik, D., et al., Study of spondylolisthesis using videomotion analysis,

Computer Methods in Biomechanics and Biomedical Engineering, 2005, Vol.

Supplement 1, 293-294.

25. Schubert, M., et al., Visual Kinesthesia and Locomotion in Parkison's Disease,

Movement Disorders, 2005, Vol. 20, No. 2, 141-150.

26. Goulermas, J. Y., et al., Automated Design of Robust Discriminant Analysi

Classifier for Foot Pressure Lesions Using Kinematic Data, IEEE Transactions

on Biomedical Engineering, 2005, Vol. 52, No. 9, 1549-1562.

27. Remondino, F. and Roditakis A., Human motion reconstruction and animation

from video sequences, in 17th International Conference on Computer Animation

and Social Agents, 2004, Computer Graphics Society (CGS) Geneva,

Switzerland.

28. Nguyen, T. H. D., et al., Real-Time 3D Human Capture System for Mixed-

Reality Art and Entertainment, IEEE Transactions on Visualization and

Computer Graphics, 2005, Vol. 11, No. 6, 706-721.


Documents

Human Motion Analysis: methodologies and applications