52
Vision-based Recognition of Human Behaviour for Intelligent Environments Alexandros Andre Chaaraoui Departamento de Tecnolog´ ıaInform´aticayComputaci´on Universidad de Alicante [email protected] Supervisor: Dr. Francisco Fl´ orez-Revuelta January 20, 2014 Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 1 / 45

Vision-based Recognition of Human Behaviour for Intelligent Environments

Embed Size (px)

DESCRIPTION

Human behaviour analysis is being of great interest in the field of artificial intelligence. Specifically, human action recognition deals with the lowest level of semantic interpretation of meaningful human behaviours, as walking, sitting or falling. In the field of ambient-assisted living, the recognition of such actions at home can support several safety and health care services for the independent living of elderly or impaired people at home. In this sense, this thesis aims to provide valuable advances in vision-based human action recognition for ambient-assisted living scenarios. Initially, a taxonomy is proposed in order to classify the different levels of human behaviour analysis and join existing definitions. Then, a human action recognition method is presented, that is based on fusion of multiple cameras and key pose sequence recognition, and performs in real time. By relying on fusion of multiple views, sufficient correlated data can be obtained despite possible occlusions, noise and unfavourable viewing angles. A visual feature is proposed that only relies on the boundary points of the human silhouette, and does not need the actual RGB colour image. Furthermore, several optimisations and extensions of this method are proposed. In this regard, evolutionary algorithms are employed for the selection of scenario-specific configurations. As a result, the robustness and accuracy of the classification are significantly improved.\linebreak In order to support online learning of such parameters, an adaptive and incremental learning technique is introduced. Last but not least, the presented method is also extended to support the recognition of human actions in continuous video streams. Outstanding results have been obtained on several publicly available datasets achieving the desired robustness required by real-world applications. Therefore, this thesis may pave the way for more advanced human behaviour analysis techniques, such as the recognition of complex activities, personal routines and abnormal behaviour detection.

Citation preview

Page 1: Vision-based Recognition of Human Behaviour for Intelligent Environments

Vision-based Recognition of HumanBehaviour for Intelligent Environments

Alexandros Andre Chaaraoui

Departamento de Tecnologıa Informatica y ComputacionUniversidad de Alicante

[email protected]

Supervisor: Dr. Francisco Florez-Revuelta

January 20, 2014

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 1 / 45

Page 2: Vision-based Recognition of Human Behaviour for Intelligent Environments

Overview

1 Introduction

2 Research framework

3 Contributions

4 Results

5 Concluding remarks

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 2 / 45

Page 3: Vision-based Recognition of Human Behaviour for Intelligent Environments

Introduction

Introduction

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 3 / 45

Page 4: Vision-based Recognition of Human Behaviour for Intelligent Environments

Introduction Motivation

Introduction (I)

Motivation

Demographic ageing - Ambient-assisted living (AAL)

Intelligent environments

Human behaviour analysis

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 4 / 45

Page 5: Vision-based Recognition of Human Behaviour for Intelligent Environments

Introduction Objectives

Introduction (II)

In this thesis, our goal is to support the development of AAL servicesfor smart homes with advances in human behaviour analysis.

Main objectives

1 Establish the research framework

2 Propose a method for the recognition of human behaviour

3 Satisfy specific demands of AAL services

4 Reach robustness for different scenarios

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 5 / 45

Page 6: Vision-based Recognition of Human Behaviour for Intelligent Environments

Research framework

Research framework

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 6 / 45

Page 7: Vision-based Recognition of Human Behaviour for Intelligent Environments

Research framework Related work

Research framework

Proposed taxonomy

Figure 1: Human Behaviour Analysis levels — Classification proposedin Chaaraoui et al. (2012).

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 7 / 45

Page 8: Vision-based Recognition of Human Behaviour for Intelligent Environments

Research framework Proposal

Conclusions of the analysis

Motion

Motion, pose and gaze estimation is the most resolved level.

Action

Action recognition is currently receiving the greatest interest both fromresearch and industry.

Activity-Behaviour

Activity and long-term behaviour recognition is performed directlybased on low-level sensor data, instead of using action recognition.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 8 / 45

Page 9: Vision-based Recognition of Human Behaviour for Intelligent Environments

Research framework Proposal

Camera 1

Motion Detection

Human Behaviour Analysis

Multi-view Human Behaviour Analysis

Privacy

Reasoning System

Alarm Actuators

Camera 2

Motion Detection

Human Behaviour Analysis

Camera N

Motion Detection

Human Behaviour Analysis

...

Setup (activities,

inhabitants, objects…),

Profiles andLog

Event

Long-term Analysis

...

...

Environmental Sensor Information

Caregiver

Figure 2: Architecture of the intelligent monitoring system to promoteindependent living at home and support AAL services.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 9 / 45

Page 10: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions

Contributions

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 10 / 45

Page 11: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Outline

Contributions

Proposals have been made at different processing stages:

1 Pose representation

2 Fusion of multiple views

3 Action classification

4 Evolutionary-based optimisation

5 Continuous recognition

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 11 / 45

Page 12: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Pose representation

Pose representation

Pose

Representations

Silhouettes

Images

Figure 3: Outline of the pose representation process. Based on the recordedvideo frames, foreground segmentations are obtained. Holistic features canthen be extracted relying on the shape of the human silhouettes.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 12 / 45

Page 13: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Pose representation

Radial Summary (I)

Figure 4: Overview of the feature extraction process: 1) All the contour points areassigned to the corresponding radial bin; 2) for each bin, a summary representationis obtained (example with 18 bins).

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 13 / 45

Page 14: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Pose representation

Radial Summary (II)

Figure 5: Graphical explanation of the statistical range frange of a sampleradial bin.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 14 / 45

Page 15: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Fusion of multiple views

Fusion of multiple views

Multiple views of the same field of view provide

Additional characteristic data

Advantages with respect to occlusions

Advantages with respect to unfavourable viewing angles(ambiguous actions)

However, difficulties have been observed

The recognition does not necessarily improve

Performance issues (temporal and spatial)

Burdensome, highly-restricted systems (3D pose estimation,calibrated and synchronised camera networks, ...)

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 15 / 45

Page 16: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Fusion of multiple views

Weighted feature fusion scheme

Weights are learnt for each view and action:

Figure 6: Overview of the feature fusion process of the multi-view poserepresentation. This example shows five different views of a specific pose takenfrom the walk action class from the IXMAS dataset (Weinland et al., 2006).

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 16 / 45

Page 17: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Action classification

Action classification

Key Poses

K Key Poses...

...

...K Key Poses

Bag of Key

Poses

Sequences of

Key Poses

Figure 7: Outline of the learning stage. Using the pose representations, keyposes are obtained for each action. In this way, a bag-of-key-poses model islearnt. The temporal relation between key poses is modelled using sequencesof key poses.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 17 / 45

Page 18: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Action classification

Bag of key poses

K Key Poses

Bag of Key Poses

Action1

ActionA

... ...

...

...

K Key PosesAction2

K Key Poses

Figure 8: Learning scheme of the bag-of-key-poses model. For each actionclass, K key poses are obtained separately and then joined together.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 18 / 45

Page 19: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Action classification

Recognition

Sequences

of Key Poses

Sequence

Matching

Action

Recognition

DTW

Figure 9: Outline of the recognition stage. The unknown sequence of keyposes is obtained and compared to the known sequences. Through sequencematching, the action of the video sequence can be recognised.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 19 / 45

Page 20: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Evolutionary-based optimisation

Evolutionary-based optimisation

Genetic feature subset selection

By means of a genetic algorithm for binary feature selection, theinteresting body parts can be selected, and redundant or noisy bodyparts can be ignored.

Figure 10: Example of a result provided by genetic feature selection (dismissed

radial bins are shaded in grey).

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 20 / 45

Page 21: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Evolutionary-based optimisation

Evolutionary-based optimisation

Genetic feature subset selection

Coevolutionary optimisation

Simultaneous selection of training instances, features andparameter values

Coevolution enables to split the problem in subproblems ofoptimisation with a common goal (Wiegand, 2004)

Cooperative coevolution allows to consider intrinsic dependenciesamong optimisation goals (Derrac et al., 2012)

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 21 / 45

Page 22: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Evolutionary-based optimisation

Evolutionary-based optimisation

Genetic feature subset selection

Coevolutionary optimisation

Adaptive learning

Evolving bag of key poses

Supports incremental and adaptive learning of new data

Applies selection of training instances, features and parametervalues

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 22 / 45

Page 23: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Continuous recognition

Continuous recognition

In AAL, human action recognition has to be applied to continuousvideo streams.

This requires:

To detect meaningful human actions online

And to recognise the appropriate action in real-time

We propose:

To learn action zones, i.e. the most discriminative parts ofaction performances

The usage of a sliding and growing window technique forrecognition

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 23 / 45

Page 24: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Continuous recognition

Action zones

Figure 11: Sample silhouettes of a waving sequence of the DAI RGBDdataset. The action zone that should be extracted is highlighted.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 24 / 45

Page 25: Vision-based Recognition of Human Behaviour for Intelligent Environments

Contributions Continuous recognition

Action zones

Figure 12: Evidence values H(t) of each action class and the detected actionzones are shown for a scratch head sequence of the IXMAS dataset.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 25 / 45

Page 26: Vision-based Recognition of Human Behaviour for Intelligent Environments

Results

Results

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 26 / 45

Page 27: Vision-based Recognition of Human Behaviour for Intelligent Environments

Results Evaluation methodology

Results

Experimentation has been performed on:

Single- and multi-view datasets (up to five views)

Manually- and automatically-extracted silhouettes (includingdepth-based segmentation)

Using the following cross validations:

Leave one sequence out (LOSO)

Leave one actor out (LOAO)

Leave one view out (LOVO)

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 27 / 45

Page 28: Vision-based Recognition of Human Behaviour for Intelligent Environments

Results Benchmarks

Weizmann dataset

Table 1: Comparison of recognition rates and speeds obtained on theWeizmann dataset (Gorelick et al., 2007) with other state-of-the-art

approaches.

Approach Actions Test Rate fps

Ikizler and Duygulu (2007) 9 LOSO 100% N/ATran and Sorokin (2008) 10 LOSO 100% N/AFathi and Mori (2008) 10 LOSO 100% N/A

Hernandez et al. (2011) 10 LOAO 90.3% 98Cheema et al. (2011) 9 LOSO 91.6% 56Sadek et al. (2012) 10 LOAO 97.8% 18

Our approach 10 LOSO 93.5% 188Our approach 10 LOAO 97.8% 188

Optimised approach 10 LOAO 100% 210

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 28 / 45

Page 29: Vision-based Recognition of Human Behaviour for Intelligent Environments

Results Benchmarks

MuHAVi-14 dataset

Table 2: Comparison of recognition rates and speeds obtained on theMuHAVi-14 dataset (Singh et al., 2010) with other state-of-the-art

approaches.

Approach LOSO LOAO LOVO fps

Singh et al. (2010) 82.4% 61.8% 42.6% N/AEweiwi et al. (2011) 91.9% 77.9% 55.8% N/A

Cheema et al. (2011) 86.0% 73.5% 50.0% 56

Our approach 98.5% 94.1% 59.6% 99

Optimised approach 100% 100% - -

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 29 / 45

Page 30: Vision-based Recognition of Human Behaviour for Intelligent Environments

Results Benchmarks

MuHAVi-8 dataset

Table 3: Comparison of recognition rates and speeds obtained on theMuHAVi-8 dataset (Singh et al., 2010) with other state-of-the-art approaches.

Metodo LOSO LOAO LOVO fps

Singh et al. (2010) 97.8% 76.4% 50.0% N/AMartınez-Contreras et al. (2009) 98.4% - - N/AEweiwi et al. (2011) 98.5% 85.3% 38.2% N/A

Cheema et al. (2011) 95.6% 83.1% 57.4% 56

Our approach 100% 100% 82.4% 98

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 30 / 45

Page 31: Vision-based Recognition of Human Behaviour for Intelligent Environments

Results Benchmarks

IXMAS dataset

Table 4: Comparison with other multi-view human action recognitionapproaches of the state of the art. The rates obtained in the LOAO cross

validation performed on the IXMAS dataset are shown.

Approach Actions Actors Views Rate fps

Yan et al. (2008) 11 12 4 78% N/AWu et al. (2011) 12 12 4 89.4% N/ACilla et al. (2012) 11 12 5 91.3% N/AWeinland et al. (2006) 11 10 5 93.3% N/ACilla et al. (2013) 11 10 5 94.0% N/AHolte et al. (2012) 13 12 5 100% N/A

Cherla et al. (2008) 13 N/A 4 80.1% 20Weinland et al. (2010) 11 10 5 83.5% ∼500

Our approach 11 12 5 91.4% 207

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 31 / 45

Page 32: Vision-based Recognition of Human Behaviour for Intelligent Environments

Results Benchmarks

RGBD datasets

DAI RGBD dataset

Table 5: Cross validation results obtained on our multi-view depth dataset.

Approach LOSO LOAO fps

Our approach 94.4% 100% 80

DHA dataset

Table 6: LOSO cross validation results obtained on the DHA dataset (Linet al., 2012) (10 Weizmann actions).

Approach LOSO fps

Lin et al. (2012) 90.8% N/A

Our approach 95.2% 99

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 32 / 45

Page 33: Vision-based Recognition of Human Behaviour for Intelligent Environments

Results Benchmarks

Continuous recognition

Table 7: Obtained results applying CHAR and segment analysis evaluation(LOAO). Results are detailed using the segmented sequences or the proposed

action zones.

Dataset Approach F1-measure

IXMAS Segmented sequences 0.504IXMAS Action zones 0.705

Weizmann Segmented sequences 0.693Weizmann Action zones 0.928

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 33 / 45

Page 34: Vision-based Recognition of Human Behaviour for Intelligent Environments

Concluding remarks

Concluding remarks

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 34 / 45

Page 35: Vision-based Recognition of Human Behaviour for Intelligent Environments

Concluding remarks Discussion

Discussion

Silhouettes

2D silhouettes can be difficult to obtain and they are view dependant

Privacy

Privacy concerns of indoor monitoring

Intelligent monitoring system with privacy protection

The method only relies on the boundary of the silhouette

Validation of the proposed method

The classification method based on the bag of key poses has also beenvalidated for gaming and NUI (Chaaraoui et al., 2014, 2013;Climent-Perez et al., 2013)

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 35 / 45

Page 36: Vision-based Recognition of Human Behaviour for Intelligent Environments

Concluding remarks Conclusions

Conclusions

1 Proposal of a 2D template-based non-parametric method forhuman action recognition & optimisations and extensions

2 Specific demands of AAL services have been satisfied: relaxedcamera setup requirements, adaptive learning, continuousrecognition and real-time execution

3 The HAR method based on a bag-of-key-poses model handlessingle- and multi-view recognition proficiently.

4 State-of-the-art recognition rates have been achieved,outperforming the best known rates in several benchmarks.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 36 / 45

Page 37: Vision-based Recognition of Human Behaviour for Intelligent Environments

Concluding remarks Future work

Future work

Future directions of this work:

Pose representation: Other local and global features

Key poses: Generation algorithms

Bag-of-key-poses model: Applications

Distance metrics: Key poses and sequences of key poses

Evaluation and deployment

However, two main future lines stand out:

Recognition of complex activities based on action sequencesand multi-modal data

Feature fusion techniques, e.g. for recognition of subtle movementsor gestures

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 37 / 45

Page 38: Vision-based Recognition of Human Behaviour for Intelligent Environments

Other information and details

Other information and details

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 38 / 45

Page 39: Vision-based Recognition of Human Behaviour for Intelligent Environments

Other information and details Research projects and grants

Research projects and grants

Intelligent system for follow-up and promotion of personalautonomy: Computer vision system for the monitoring ofactivities of daily living at home – Sistema de vision para la

monitorizacion de la actividad diaria en el hogar. Spanish Ministry of Science and

Innovation and Valencian Ministry of Education, Culture and Sport (TALISMAN+,

Technical University of Madrid, University of Deusto, University of Castile-La

Mancha and University of Alicante)

PhD. Research Fellowship – Programa VALi+d para investigadores en

formacion. Valencian Ministry of Education, Culture and Sport (ACIF/2011/160)

Research Collaboration Stay – Digital Imaging Research Centre, Faculty

of Science, Engineering and Computing, Kingston University. Kingston upon

Thames, UK. (Funded by VALi+d, BEFPI/2013/015)

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 39 / 45

Page 40: Vision-based Recognition of Human Behaviour for Intelligent Environments

Other information and details Other activities

Other activities

Teaching collaboration – Information Technology. First year core subject of

Degree in Sound and Image in Telecommunication Engineering

Reviewer of international journals – Neurocomputing (Elsevier),

Pervasive Computing (IEEE), EURASIP Journal on Image and Video Processing

(Springer), Expert Systems With Applications (Elsevier)

Conference session chair – IEEE/RSJ Intelligent Robots and Systems

(IROS 2012), Genetic and Evolutionary Computation Conference (GECCO 2013)

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 40 / 45

Page 41: Vision-based Recognition of Human Behaviour for Intelligent Environments

Other information and details Intl. review

International review

This thesis has been approvedby the following reviewers for the

International PhD Honourable Mention

Dr. Jean-Christophe Nebel(Kingston University, UK)

Dr. Jesus Martınez del Rincon(Queen’s University of Belfast, UK)

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 41 / 45

Page 42: Vision-based Recognition of Human Behaviour for Intelligent Environments

Other information and details Publications

Publications

Journals

I Chaaraoui, A.A., Climent-Perez, P., Florez-Revuelta, F., 2012. A Review on

Vision Techniques Applied to Human Behaviour Analysis for Ambient-Assisted

Living. Expert Systems with Applications. Citations: 10

II Chaaraoui, A.A., Climent-Perez, P., Florez-Revuelta, F., 2013.

Silhouette-based Human Action Recognition Using Sequences of Key Poses.

Pattern Recognition Letters. Citations: 6

III Chaaraoui, A.A., Florez-Revuelta, F., 2013. Optimizing Human Action

Recognition Based on a Cooperative Coevolutionary Algorithm. Engineering

Applications of Artificial Intelligence.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 42 / 45

Page 43: Vision-based Recognition of Human Behaviour for Intelligent Environments

Other information and details Publications

Publications

IV Chaaraoui, A.A., Padilla-Lopez, J.R., Climent-Perez, P., Florez-Revuelta, F.,

2014. Evolutionary Joint Selection to Improve Human Action Recognition

with RGB-D Devices. Expert Systems with Applications.

V Chaaraoui, A.A., Florez-Revuelta, F., 2014. A Low-Dimensional Radial

Silhouette-based Feature for Fast Human Action Recognition Fusing Multiple

Views. Information Fusion. Under review

VI Chaaraoui, A.A., Florez-Revuelta, F., 2014. Adaptive Human Action

Recognition With an Evolving Bag of Key Poses. IEEE Transactions on

Autonomous Mental Development. Under review

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 43 / 45

Page 44: Vision-based Recognition of Human Behaviour for Intelligent Environments

Other information and details Publications

Publications

Conferences and workshops

I Intl. Symposium on Ambient Intelligence (ISAmI 2012)

II 3rd Intl. Workshop on Human Behavior Understanding (HBU 2012),

IEEE/RSJ Intl. Conference on Intelligent Robots and Systems (IROS 2012)

III 11th Mexican Intl. Conference on Artificial Intelligence (MICAI 2012)

IV Genetic and Evolutionary Computation Conference (GECCO 2013)

V 5th Intl. Work-conference on Ambient Assisted Living (IWAAL 2013)

VI 3rd Workshop on Consumer Depth Cameras for Computer Vision

(CDC4CV13), IEEE Intl. Conference on Computer Vision (ICCV 2013)

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 44 / 45

Page 45: Vision-based Recognition of Human Behaviour for Intelligent Environments

Other information and details Publications

“We can only see a short distance ahead, but we can seeplenty there that needs to be done.” (Turing, 1950)

Vision-based Recognition of Human Behaviourfor Intelligent Environments

PhD Thesis

Alexandros Andre Chaaraoui

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 45 / 45

Page 46: Vision-based Recognition of Human Behaviour for Intelligent Environments

References

References

Chaaraoui, A. A., P. Climent-Perez, and F. Florez-Revuelta (2012). A review on visiontechniques applied to human behaviour analysis for ambient-assisted living. ExpertSystems with Applications 39 (12), 10873 – 10888.

Chaaraoui, A. A., J. R. Padilla-Lopez, P. Climent-Perez, and F. Florez-Revuelta (2014).Evolutionary joint selection to improve human action recognition with RGB-D devices.Expert Systems with Applications 41 (3), 786 – 794. Methods and Applications ofArtificial and Computational Intelligence.

Chaaraoui, A. A., J. R. Padilla-Lopez, and F. Florez-Revuelta (2013). Fusion of skeletaland silhouette-based features for human action recognition with RGB-D devices. InIEEE 14th International Conference on Computer Vision Workshops, 2013. ICCVWorkshops 2013. To be presented in 3rd Workshop on Consumer Depth Cameras forComputer Vision (CDC4CV13).

Cheema, S., A. Eweiwi, C. Thurau, and C. Bauckhage (2011). Action recognition bylearning discriminative key poses. In IEEE 13th International Conference on ComputerVision Workshops, 2011. ICCV Workshops 2011, pp. 1302 –1309.

Cherla, S., K. Kulkarni, A. Kale, and V. Ramasubramanian (2008). Towards fast,view-invariant human action recognition. In IEEE Conference on Computer Vision andPattern Recognition Workshops, 2008. CVPRW 2008, pp. 1 – 8.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 46 / 45

Page 47: Vision-based Recognition of Human Behaviour for Intelligent Environments

References

References

Cilla, R., M. A. Patricio, A. Berlanga, and J. M. Molina (2012). A probabilistic,discriminative and distributed system for the recognition of human actions from multipleviews. Neurocomputing 75 (1), 78 – 87. Brazilian Symposium on Neural Networks (SBRN2010), International Conference on Hybrid Artificial Intelligence Systems (HAIS 2010).

Cilla, R., M. A. Patricio, A. Berlanga, and J. M. Molina (2013). Human action recognitionwith sparse classification and multiple-view learning. Expert Systems. DOI10.1111/exsy.12040.

Climent-Perez, P., A. A. Chaaraoui, J. R. Padilla-Lopez, and F. Florez-Revuelta (2013).Optimal joint selection for skeletal data from RGB-D devices using a genetic algorithm.In I. Batyrshin and M. Mendoza (Eds.), Advances in Computational Intelligence, Volume7630 of Lecture Notes in Computer Science, pp. 163 – 174. Springer Berlin / Heidelberg.

Derrac, J., I. Triguero, S. Garcıa, and F. Herrera (2012). A co-evolutionary framework fornearest neighbor enhancement: Combining instance and feature weighting with instanceselection. In E. Corchado, V. Snasel, A. Abraham, M. Wozniak, M. Grana, and S.-B.Cho (Eds.), Hybrid Artificial Intelligent Systems, Volume 7209 of Lecture Notes inComputer Science, pp. 176 – 187. Springer Berlin / Heidelberg.

Eweiwi, A., S. Cheema, C. Thurau, and C. Bauckhage (2011). Temporal key poses forhuman action recognition. In IEEE 13th International Conference on Computer VisionWorkshops, 2011. ICCV Workshops 2011, pp. 1310 –1317.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 47 / 45

Page 48: Vision-based Recognition of Human Behaviour for Intelligent Environments

References

References

Fathi, A. and G. Mori (2008). Action recognition by learning mid-level motion features. InIEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pp.1 – 8.

Gorelick, L., M. Blank, E. Shechtman, M. Irani, and R. Basri (2007). Actions as space-timeshapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 29 (12), 2247– 2253.

Hernandez, J., A. Montemayor, J. Jose Pantrigo, and A. Sanchez (2011). Human action

recognition based on tracking features. In J. Ferrandez, J. Alvarez Sanchez, F. de la Paz,and F. Toledo (Eds.), Foundations on Natural and Artificial Computation, Volume 6686of Lecture Notes in Computer Science, pp. 471 – 480. Springer Berlin / Heidelberg.

Holte, M., B. Chakraborty, J. Gonzalez, and T. Moeslund (2012). A local 3-D motiondescriptor for multi-view human action recognition from 4-D spatio-temporal interestpoints. IEEE Journal of Selected Topics in Signal Processing 6 (5), 553 – 565.

Ikizler, N. and P. Duygulu (2007). Human action recognition using distribution of orientedrectangular patches. In A. Elgammal, B. Rosenhahn, and R. Klette (Eds.), HumanMotion - Understanding, Modeling, Capture and Animation, Volume 4814 of LectureNotes in Computer Science, pp. 271 – 284. Springer Berlin / Heidelberg.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 48 / 45

Page 49: Vision-based Recognition of Human Behaviour for Intelligent Environments

References

References

Lin, Y.-C., M.-C. Hu, W.-H. Cheng, Y.-H. Hsieh, and H.-M. Chen (2012). Human actionrecognition and retrieval using sole depth information. In Proceedings of the 20th ACMinternational conference on Multimedia, MM ’12, New York, NY, USA, pp. 1053 – 1056.ACM.

Martınez-Contreras, F., C. Orrite-Urunuela, E. Herrero-Jaraba, H. Ragheb, and S. Velastin(2009). Recognizing human actions using silhouette-based HMM. In IEEE Int.Conference on Advanced Video and Signal Based Surveillance, 2009. AVSS 2009, pp. 43– 48.

Sadek, S., A. Al-Hamadi, B. Michaelis, and U. Sayed (2012). A fast statistical approach forhuman activity recognition. International Journal of Intelligence Science 2 (1), 9 – 15.

Singh, S., S. Velastin, and H. Ragheb (2010). MuHAVi: A multicamera human action videodataset for the evaluation of action recognition methods. In IEEE Int. Conference onAdvanced Video and Signal Based Surveillance, 2010. AVSS 2010, pp. 48 – 55. IEEE.

Tran, D. and A. Sorokin (2008). Human activity recognition with metric learning. InD. Forsyth, P. Torr, and A. Zisserman (Eds.), European Conference on ComputerVision. ECCV 2008, Volume 5302 of Lecture Notes in Computer Science, pp. 548 – 561.Springer Berlin / Heidelberg.

Turing, A. M. (1950). Computing machinery and intelligence. Mind 59 (236), 433 – 460.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 49 / 45

Page 50: Vision-based Recognition of Human Behaviour for Intelligent Environments

References

References

Weinland, D., M. Ozuysal, and P. Fua (2010). Making action recognition robust toocclusions and viewpoint changes. In K. Daniilidis, P. Maragos, and N. Paragios (Eds.),European Conference on Computer Vision. ECCV 2010, Volume 6313 of Lecture Notesin Computer Science, pp. 635 – 648. Springer Berlin / Heidelberg.

Weinland, D., R. Ronfard, and E. Boyer (2006). Free viewpoint action recognition usingmotion history volumes. Computer Vision and Image Understanding 104 (2-3), 249 –257.

Wiegand, R. P. (2004). An analysis of cooperative coevolutionary algorithms. Ph. D. thesis,George Mason University, Fairfax, VA, USA.

Wu, X., D. Xu, L. Duan, and J. Luo (2011). Action recognition using context andappearance distribution features. In IEEE Conference on Computer Vision and PatternRecognition, 2011. CVPR 2011, pp. 489 – 496.

Yan, P., S. Khan, and M. Shah (2008). Learning 4D action feature models for arbitraryview action recognition. In IEEE Conference on Computer Vision and PatternRecognition, 2008. CVPR 2008, pp. 1 – 7.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 50 / 45

Page 51: Vision-based Recognition of Human Behaviour for Intelligent Environments

Copyright

Copyright

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0Unported License. You are free to copy, distribute and transmit the work under the following conditions:1) you must attribute the work in the manner specified by the author or licensor (but not in any way thatsuggests that they endorse you or your use of the work); 2) you may not use this work for commercialpurposes; and 3) you may not alter, transform, or build upon this work. With the understanding that:1) any of the above conditions can be waived if you get permission from the copyright holder; 2) wherethe work or any of its elements is in the public domain under applicable law, that status is in no wayaffected by the license; and 3) in no way are any of the following rights affected by the license: your fairdealing or fair use rights, other applicable copyright exceptions and limitations and rights other personsmay have either in the work itself or in how the work is used, such as publicity or privacy rights. Pleasesee http://creativecommons.org/licenses/by-nc-nd/3.0/ for greater detail.

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 51 / 45

Page 52: Vision-based Recognition of Human Behaviour for Intelligent Environments

Copyright

Author’s contact details

Alexandros Andre [email protected]

www.alexandrosandre.com

Departamento de Tecnologıa Informatica y ComputacionUniversidad de AlicanteCarretera San Vicente del Raspeig s/nE-03690 San Vicente del Raspeig (Alicante) - Spain

Alexandros Andre Chaaraoui (UA) PhD Thesis January 20, 2014 52 / 45