Institute of Systems and Robotics ISR – Coimbra Mobile Robotics Laboratory PROMETHEUS WP5 – Behavior Modeling Kamrad Khoshhal Roudposhti Version: 2.5

Institute of Systems and RoboticsISR – Coimbra

Mobile Robotics Laboratory

PROMETHEUS

WP5 – Behavior Modeling

Kamrad Khoshhal Roudposhti

Version: 2.5



Objectives



Summary of WP5START: T0+12 (January 2009)

Deliverables

D5.1 (T0+24) Progress on Behaviour Modelling (TEIC)

D5.2 (T0+30) Learning and short-term prediction (FCTUC)

Tasks

Task 5.1 : Particle filtering techniques applied to the learning process of Bayesian network

structures Task 5.2 :

Learning/Recognition of Human Action/ Interaction patterns

Task 5.3 : Short time Prediction of Human Intention

Involved partners

Person - months

FOI, UOP, TUM, ISR-FCTUC, PROBAYES, TEIC

2 11 15 20 10 3



Objectives

The scope of WP5 :•analysis and recognition of motion patterns and the production of high-level description of actions and interactions.

•Understanding of behaviors

Specifically, this WP must conclude on –a) represent semantic concepts of behavior, –b) map motion characteristics -mainly velocities and feature trajectories- to semantic concepts

–c) choose efficient representations to interpret the scene meanings.

The detection is based on matching observed behavior with the learned patterns.



Behavior Samples



Indoor Part



Place down bag



Pick up bag



Robbery



Falling down



Faint & robbery



Faint (SmartHome)



Fighting



Greeting



Panic situation



Lift and move a heavy box and...

The person falls down The normal action



Outdoor Part



Fighting & Pushing



2 Persons Fighting and one of them escape



An angry person An normal person



Loitering Normal



Other projects in around of the our subjects



The COBOL project

Communication with Emotional Body Language

•COmmunication with Emotional BOdy Language (COBOL) was launched in 2006 by the the European Commission as project, in the 6th EU framework programme. The Commission will be supporting this Specific Targeted Research Project for three years, to the tune of €1.8 million.

The project consists of 5 workpackages, each of which is described belowWorkpackage 1:Description and analysis of the kinematic and dynamical structure of EBLWorkpackage 2:Development of EBL avatars and measurement of EBL perception and recognitionWorkpackage 3:The cognitive basis of EBLWorkpackage 4:Coordinating social interactionsWorkpackage 5:Cross-cultural EBL



The AMI project

Augmented Multi-party Interaction

•The AMI Consortium formed in January 2004 to conduct basic and applied research, with the aim of developing technologies that help people have more productive meetings.

•Their technologies rely on basic research in disciplines ranging from speech recognition, language processing, computer vision, human-human communication modeling, and multimedia indexing and retrieval. The AMI Consortium brings together scientists from these fields as well as technologists, interface specialists, and social psychologists in order to achieve its vision.



Netcarity

Ambient technology to support older people at home

Netcarity was launched in 2007 and for 4 years, €13 million European project researching and testing technologies which will help older people improve their:•Wellbeing •Independence •Safety •Health



Reference from Partners• [16] A. Lazaridis, T. Kostoulas, I. Mporas, T. Ganchev, N. Katsaounos, S. Ntalampiras, and N. Fakotakis. Human evaluation of the logos

multimodal dialogue system. Athens, Greece, July 2008. 1st International Conference on PErvasive Technologies Related to Assistive Environments July 16 - 19.

• [17] Anna Linderhed, Stefan Sjökvist, Sten Nyberg, Magnus Uppsäll, Christina Grönwall, Pierre Andersson, and Dietmar Letalick. Temporal analysis for land mine detection. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis (2005), 2005.

• [18] I. Potamitis, N. Fakotakis, and G. Kokkinakis. Robust automatic speech recognition in the presence of impulsive noise. ELECTRONICS LETTERS 7th June 2007, 37, June 2007.

• [19] Ilyas Potamitis. Estimation of speech presence probability in the eld of microphone array. IEEE SIGNAL PROCESSING LETTERS, 11, DECEMBER 2004.

• [20] Ilyas Potamitis, Huimin Chen, and George Tremoulis. Tracking of multiple moving speakers with multiple microphone arrays. IEEE TRANS- ACTIONS ON SPEECH AND AUDIO PROCESSING, 12, SEPTEMBER 2004.

• [21] Ilyas Potamitis and George Kokkinakis. Speech separation of multiple moving speakers using multisensor multitarget techniques. IEEE TRANS- ACTIONS ON SYSTEMS, MAN, AND CYBERNETICS-PART A: SYS- TEMS AND HUMANS, 37, JANUARY 2007.

• [22] Stephan Reiter, Bj²orn Schuller, and Gerhard Rigoll. Segmentation and recognition of meeting events using a two-layered hmm and a combined mlp-hmm approach. pages 953 956. Multimedia and Expo, 2006 IEEE International Conference on 9-12 July 2006, July 2006.

• [23] J. Rett and J. Dias. Human robot interaction based on bayesian analysis of human movements. Proceedings of EPIA 07, Lecture Notes in AI, Springer Verlag, Berlin., 2007.

• [24] Joerg Rett, Jorge Dias, and Juan-Manuel Ahuactzin. Laban Movement Analysis using a Bayesian model and perspective projections. Brain, Vision and AI, 2008. ISBN: 978-953-7619-04-6.

• [25] Jörg Rett. ROBOT-HUMAN Interface using LABAN Movement Analysis Inside a Bayesian framework. PhD thesis, University of Coimbra, 2008.

• [26] Gerhard Rigoll, Stefan Eickeler, and Stefan M²uller. Person tracking in realworld scenarios using statistical methods. pages 398 402. Automatic Face and Gesture Recognition, 2000. Proceedings. Fourth IEEE International Conference on 28-30 March 2000, March 2000.

• [27] Sascha Schreiber, Andre Stormer, and Gerhard Rigoll. A hierarchical asm/aam approach in a stochastic framework for fully automatic tracking and recognition. pages 1773 1776. Image Processing, 2006 IEEE International Conference on 8-11 Oct. 2006, Oct. 2006.

• [28] Frank Wallho, Martin RuB, Gerhard Rigoll, Johann Gobel, and Hermann Diehl. Surveillance and activity recognition with depth information. Pages 1103 1106. Multimedia and Expo, 2007 IEEE International Conference on 2-5 July 2007, July 2007.

• [29] N. Xiong and P. Svensson. Multi-sensor management for information fusion: issues and approaches. Information Fusion 3, pages 163186, 2002.

• [30] P. Zervas, N. Fakotakis, and G. Kokkinakis. Development and evaluation of a prosodic database for greek speech synthesis and research. Journal of Quantitative Linguistics, 15(2):154184, 2008.



Planning for WP5 (Tomorrow)



Thank you for your attention!

Documents

Institute of Systems and Robotics ISR – Coimbra Mobile Robotics Laboratory PROMETHEUS WP5 – Behavior Modeling Kamrad Khoshhal Roudposhti Version: 2.5