Human motion analilysis: methodologies applicationstavares/downloads/publications/...Human motion analilysis: methodologies and applications Maria João M. Vasconcelos, João Manuel

H i l iHuman motion analysis: methodologies and applications

Maria João M. Vasconcelos, João Manuel R. S. Tavares

i l @f [email protected]

CMBBE 2008 ‐ 8th International Symposium on Computer Methods in Biomechanics and Biomedical Engineering27th February ‐ 1st March, Porto, Portugal

Contents:

• Introduction;

M th d l i• Methodologies;

• Applications;Applications;

• Conclusions;

• Future work.

Maria Vasconcelos, João Tavares 2Human motion analysis: methodologies and applications

Introduction:

The main goal of this work is to present a review about the

leading computational techniques used in human motionleading computational techniques used in human motion

analysis and some of their applications.

This review is the first step of a project which will focus in the

study development and implementation of computationalstudy, development and implementation of computational

techniques capable of segment, track and analyse the

movement of objects in image sequences, particularly for

human motion analysis.y


Introduction:The study of human motion in image sequences usually follows aThe study of human motion in image sequences usually follows a

general framework:

1. Feature extractionIdentification of the objects characteristics to be analysed in the image framesIdentification of the objects characteristics to be analysed in the image frames

2. Feature correspondenceWhere the problem of matching features between two consecutive image

frames is approached

3. High level processing Such as recognition of human movements activities or poses


Such as recognition of human movements, activities or poses

• There are two typical approaches to the motion analysis of

Feature extraction:• There are two typical approaches to the motion analysis of

human body depending on whether a priori shape models are

employed or not;

• In each approaches varying models of increasing complexityIn each approaches, varying models of increasing complexity

can be considered;

• It is possible to use simple models, such as stick figures or

more complex models involving 2D contours or even 3D p g

volumes.


•Aggarwal, J. and Q. Cai. Human Motion Analysis: A Review. In Computer Vision and Image Understanding, vol. 73 (3), p. 428-440, 1999.

Articulated motion without a priori shape models:•Stick figures•Stick figures- Consists of line segments linked by joints where its motion provides the key for

motion estimation.

- Johansson (1975) first showed that the human eyes can interpret a moving

human-like structure with moving light displays (MLD), their relative movement

creates a vivid impression of a person walking, running, dancing.

- So attempts have been made to recover a connected human structure with a

projected MLD by assuming that points belonging to the same object have higher

correlations in projected positions and velocities.•Johansson, G.. Visual motion perception. In Sci American, vol. 232(6), p. 76-88, 1975.R hid R F T d f h i i f i li h di l I IEEE T O PAMI


•Rashid, R.F.. Towards a system for the interpretation of moving light display. In IEEE Trans. On PAMI, vol. 2(6), p. 574-581, 1980.

Articulated motion without a priori shape models:•2D contours•2D contoursIn this representation, the human body segments are analogous to 2D ribbons or

blobs.

Kurakake (1992) attempted to obtain the joint locations in images of walking

humans by establishing correspondence between extracted ribbons. The joints

were identified as the centre of the area where two ribbons overlaps.

•Kurakake S Description and tracking of moving articulated objects In 11th Intl Conf On Pattern


Kurakake, S. Description and tracking of moving articulated objects. In 11 Intl. Conf. On Pattern Recognition, p. 491-495, 1992.

Articulated motion with a priori shape models:•Stick figures•Stick figuresIs based on the observation that human motion is

essentially the movement of the human skeleton

brought about by the attached muscles.

Is often used to recover the 3D configuration of a

moving object according to its projected 2D

image.

•Wang, J. and S. Singh. Video Analysis of Human Dynamics - A Survey. In Real-time Imaging Journal, 2003.


g g y y y g g•Rius, I. et al. Automatic Learning of 3D Pose Variability in Walking Performances for Gait Analysis. In International Journal for Computational Vision and Biomechanics, 2007.

•2D contours

Articulated motion with a priori shape models:•2D contoursThese models are directly associated with the projection of the human figure in

images.

2D ribbons are the most commonly used models and

are comprised of two components:

-the basic human body model (outlines the structural and

shape relationships between the body parts);

-the extended body model (the support posture model, the

side view kneeling model and side horse motion model).•Aggarwal, J. and Q. Cai. Human Motion Analysis: A review. In Computer Vision and Image Understanding, 1999


1999.•Leung, M.K. and Y.H. Yang. First sight: A human body outline labelling system. In IEEE Trans on PAMI, 1995.

Articulated motion with a priori shape models:•Volumetric models (3D)•Volumetric models (3D)- Elliptical cylinders

These models can also be used to model

articulated and self-occluding objects such as

fingers

•Mixture of models-Stick models with volumetric models

Where the first provides the basic shape and the p p

second define the outer appearance of a person•Aggarwal, J. and Q. Cai. Human Motion Analysis: A review. In Computer Vision and Image Understanding,


gg y p g g1999.•Wang, J. and S. Singh. Video Analysis of Human Dynamics - A Survey. In Real-time Imaging Journal, 2003.

• Methods which do not use a priori shape models

Feature correspondence:• Methods which do not use a priori shape models The correspondence between successive frames is based upon prediction or

estimation of features related to position, velocity, shape, texture and colour.

• Methods which assume a priori shape modelsFeature correspondence is automatically achieved once the matching between

the images and the model data is established.


•Aggarwal, J. and Q. Cai. Human Motion Analysis: A Review. In Computer Vision and Image Understanding, vol. 73 (3), p. 428-440, 1999.

Applications:

• Clinical studiesStudy of Spondylolisthesis or Parkinson disease, for example

• Gait classificationHuman motion analysis to differentiate between male and female walkersHuman motion analysis to differentiate between male and female walkers

and their performances; or recognize gait patterns of old and young people

G it l i• Gait analysisRecognition of human movements, activities or poses

• Biomechanical study of athletesIn order to improve their performances


Human motion applications:

Example of motion tracking using markers (point data to be analysed)Example of motion tracking using markers (point data to be analysed)

•Pinho et al. Human Movement Tracking and Analysis with Kalman Filtering and Global Optimization Techniques ICCB 2005 Lisbon Portugal 2005


Techniques. ICCB 2005, Lisbon, Portugal, 2005.•Pinho et al. Correspondência entre Pontos no Seguimento de Movimento em Imagens. 6APAET 2005, Azores, Portugal, 2005.


Example of pedestrian tracking


•Baumberg, A. and D. Hogg. Generating spatiotemporal models from examples. In Image and Vision Computing, vol. 14, p. 525-532, 1996.


i h i l d f hlBiomechanical study of athletes


•Jesus, R.M. et al. Tracking of Human Motion using Multiple Predictors. In International Workshop on Articulated Motion and Deformable Objects, Palma de Mallorca, Spain, 2002.

Conclusions:• A review about the computational techniques used for human motionA review about the computational techniques used for human motion

analysis and some of their applications was presented;

• Human motion analysis is a complex non linear and time variant subject;• Human motion analysis is a complex, non-linear and time variant subject;

• The study of human motion in image sequences starts with feature

t ti th f t d i t bli h d d d ith hi h l lextraction, then feature correspondence is established and ends with high level

processing task;

• There are several human motion applications such as human recognition in

surveillance systems, motion analysis in clinical studies or gait classification.


Future work:• Develop methodologies for human motion analysis in 2D/3D:Develop methodologies for human motion analysis in 2D/3D:

• Develop methodologies to segment persons in image frames using

modelization and update of images backgroundmodelization and update of images background

• Develop methodologies to track humans without using marks, treating

i b t bl f l i bl b i d littiin a robust away problems of occlusion, blobs merging and splitting

• Using non-linear filters for tracking, like particle filters or unscented

Kalman filter

• Develop methodologies to automatically build models for the motion

and shape involved

• Develop mechanisms to automatically switch between the most


appropriate models to be used

Acknowledgements:• The first author would like to thank the support of the PhD grantThe first author would like to thank the support of the PhD grant

SFRH/BD/28817/2006 from FCT – Fundação para a Ciência e Tecnologia from

Portugal;Portugal;

• This work was partially done in the scope of the project “Segmentation,

T ki d M ti A l i f D f bl (2D/3D) Obj t i Ph i lTracking and Motion Analysis of Deformable (2D/3D) Objects using Physical

Principles”, reference POSC/EEA-SRI/55386/2004, financially supported by

FCT.


H i l iHuman motion analysis: methodologies and applications

Maria João M. Vasconcelos, João Manuel R. S. Tavares

i l @f [email protected]

CMBBE 2008 ‐ 8th International Symposium on Computer Methods in Biomechanics and Biomedical Engineering27th February ‐ 1st March, Porto, Portugal

Documents

Human motion analilysis: methodologies applicationstavares/downloads/publications/...Human motion analilysis: methodologies and applications Maria João M. Vasconcelos, João Manuel