Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Christian LAUGIER – e-Motion project-team 1
EE--Motion Motion TeamTeam--ProjectProject““Geometry and Probability for motion and actionGeometry and Probability for motion and action””
INRIA Grenoble Rhône-Alpes
&
Laboratory of Informatics of Grenoble (LIG)
Scientific leader : Christian LAUGIER (DR1 INRIA)
http://emotion.inrialpes.fr
Christian LAUGIER – e-Motion project-team 2
Context & Scientific challengeContext & Scientific challenge
• Overall challengeOverall challenge Human-Centered Robotics
ITS for improving safety & comfort & efficiency Personal Assistant & House Keeping & Rehabilitation
• Main MotivationsMain Motivations
� Important socio-economic perspectives => Transport, Aging society, Medical care &
Rehabilitation, Human assistance, Intelligent home …
� Increasing interest of industry => Automotive industry, Robots, Health sector, Services …
� Challenging research topics => Dynamic world, Robust perception, Safety, Human Aware
Motion, Complex Human-Robot interactions …
� Robotics state-of-the-art & Progress in ICT Technologies (computers, sensors, micro-
nano technologies, energy …) make this challenge potentially reachable
Robotics Technologies Robotics Technologies –– Current LimitationsCurrent Limitations
=> DARPA Grand Challenge 2004
� Significant step towards Motion Autonomy
� But still some “Uncontrolled Behaviors” !!!!
Current Autonomous robots are able to exhibit quite impressive skills …. BUT
they are NOT adapted to human environments and they are often UNSAFE !
Some technologies are almost ready for use in some restricted anSome technologies are almost ready for use in some restricted and/or d/or
protected public areas protected public areas …….. BUT.. BUT
�� Open environments are still beyond the state of the artOpen environments are still beyond the state of the art
�� Safety is still not guaranteedSafety is still not guaranteed
�� Too many costly sensors are still requiredToo many costly sensors are still required
=> URBAN Challenge 2007
� A large step towards urban road environments
� But still some accidents, even at low speed !!!
New challenges to be addressed New challenges to be addressed
� Dynamicity & Uncertainty
� Interpreting ambiguities
� Prediction of future states
=> Avoiding future collisions !!!
e.g. Traffic scene understanding
�� Human environment understandingHuman environment understanding
�� Share control & Safe Interaction with humanShare control & Safe Interaction with human
Human beings are unbeatable in taking decisions in complex situations
Technology is better for “simple” but “fast” control decisions (ABS, ESP …)
Human – Robot interaction issue has to be addressed
Driver monitoring
Patient – Wheelchair interaction
Christian LAUGIER – e-Motion project-team 5
Robotics Experimental PlatformRobotics Experimental Platform
Parkview Cycab & Simulator
KoalaAutonomous Wheelchairs Industrial Experimental Vehicles
Commercialized by
Robosoft
Commercialized by
Bluebotics
Equipped Toyota Lexus
Christian LAUGIER – e-Motion project-team 6
Main Topics & Achievements Main Topics & Achievements (2005(2005--09)09)11 PhD, 2 books, 20 journal papers, 3 patents & technological transfers
Observation
Prediction
Risk-based navigation
Partial Motion Planning&
Inevitable Collision States
Autonomous navigation & Safety (3 theses)
Probabilistic Risk Assessment(coop. Probayes & Toyota)
Learning & World change prediction(coop. ETH Zurich)
Prediction & Risk assessment (2 theses)
Efficient 3D Multi-resolution Mapping & Localization using “Tensor maps”
(coop. Perception, IBEO)
Dense Mapping & Localisation (1 thesis)
Vision based Detection & TTC(Coop. Prima)
Perception & Situation awareness (5 theses)
Robust Perception (BOF) & DATMO(Coop. Probayes & Toyota)
Parameters estimation from noisy sensor data
(2 on going theses)
• Continuous symmetry (appli. to calibration)• Fusion of IMU & Vision
(Coop. ETH Zurich & Bluebotics)
Christian LAUGIER – e-Motion project-team 7
Main Topics & Achievements Main Topics & Achievements (Theme 3, 2005(Theme 3, 2005--09)09)
Action selection & Attention focusing[Koike 06]
Bayesian learning[LeHy 07] [Dangauthier 08]
Bayesian models of Superior Colliculus[Colas et al. 09] (coop. LPPA)
Human Perception of Shape from Motion [Colas 06] (coop. LPPA)
Sensory-motor systems & Handwriting[E. Gillet PhD Thesis] (coop. LPN)
Robots
Living systems
Prey & Predator scenario
6 PhD, 1 Book, 9 journal papers (Robotics & Neurosciences), 1 start-up (Probayes)
Brain controlled wheelchair [Rebsamen 09](coop. NUS Singapore)
Christian LAUGIER – e-Motion project-team 8
Key perception concepts for safe navigation: Key perception concepts for safe navigation: Situation Awareness & Risk AssessmentSituation Awareness & Risk Assessment
Bayesian Perception
Probabilistic Risk Assessment
Equipped Toyota Lexus
Ibeo Lux
Stereo camera
IMU + GPS+ Odometry
ADAS & Autonomous Driving=> Cooperation Probayes, Toyota, Renault …
Human-Centered Navigation=> INRIA PAL & ICT-Asia PAMM
Bayesian Perception Bayesian Perception –– Processing Uncertainty & DynamicityProcessing Uncertainty & Dynamicity
Bayesian Occupation Filter paradigm (BOF)Patented by INRIA & Probayes, Commercialized by ProbayesPatented by INRIA & Probayes, Commercialized by Probayes
Prediction
EstimationOccupied space
Freespace
Unobservable space
Concealed space
(“shadow” of the obstacle)
Sensed moving obstacle P( [Oc=occ] | z c)
c = [x, y, 0, 0] and z=(5, 2, 0, 0)
Occupancy grid
� Continuous Dynamic environment modelling using one or
several sensors
� Grid approach based on Bayesian Filtering
� Estimates at each time step the Occupation & Velocity
probabilities of each cell in a “space-velocity” grid
� Uses Probabilistic Sensor & Dynamic models
=> More robust to Sensing errors & Temporary occultation
=> Designed for Sensor Fusion & Parallel processing
BOFBOF
Christian LAUGIER – e-Motion project-team
[Coué & al IJRR 05]
Application to Detection & Tracking
(coop Toyota & Denso)
Driving Experimentations Driving Experimentations –– INRIA Lexus PlatformINRIA Lexus Platform
Toyota Lexus LS600h
2 Lidars IBEO Lux
Stereo camera TYZX
Inertial sensor / GPS Xsens MTi-G
Dell computer + GPU + SSD memory
GPS track example(Using Open Street Map)
Sensor Fusion experiment: Sensor Fusion experiment: Stereo + 2 LidarsStereo + 2 Lidars
Front view from left camera Fusion result using BOF
OG from left Lidar OG from right Lidar OG from Stereo
[Perrollaz et al 10] [Paromtchik et al 10]Movie
Conservative Collision Anticipation using the BOFConservative Collision Anticipation using the BOFTracking + Conservative hypotheses
Autonomous Vehicle Parked Vehicle (occultation)
Thanks to the prediction capability of the BOF technology, the Autonomous Vehicle “anticipates” the
behavior of the pedestrian and brakes (even if the pedestrian is temporarily hidden by the parked vehicle)
Collision Risk Assessment Collision Risk Assessment –– Problem statementProblem statementBehavior Prediction + Probabilistic Risk Assessment
Consistent Prediction & Risk Assessment requires to reason about :
� History of obstacles Positions & Velocities (perception or communications)
� Obstacles expected Behaviors e.g. turning, overtaking, crossing ...
� Road geometry e.g. lanes, curves, intersections … using GIS
TTC-based crash warning is not sufficient !
False alarm !
Conservative hypotheses
Previous observations
Collision Risk Assessment Collision Risk Assessment –– Functional ArchitectureFunctional Architecture
Estimate the probability of
the feasible driving behaviors
Probabilistic representation of
a possible evolution of a car
motion for a given behavior
Probabilistic Collision Risk:
Calculated for a few seconds
ahead from the probability
distributions over Behaviors
Recognition & Realization
Patent INRIA & Toyota 2009 [Tay 09] [Laugier et al 11]
Motion Prediction: Learn & Predict paradigmMotion Prediction: Learn & Predict paradigm
• Observe & Learn “typical motions”
• Continuously “Learn & Predict”� Learn => GHMM & Topological maps (SON)
� Predict => Exact inference, linear complexity
Experiments using Leeds parking data
Euron PhD Thesis Award 07 (Dizan Vasquez)
Christian LAUGIER – Keynote FSR’09, Boston
Probabilistic Collision Risk AssessmentProbabilistic Collision Risk Assessment
•• Behaviors :Behaviors : Hierarchical HMM (learned)
BehaviorPrediction
e.g. Overtaking => Lane change, Accelerate …
GP: Gaussian distribution over functions
Prediction: Probability distribution (GP) using mapped
past n position observation
• Motion Execution & Prediction :Motion Execution & Prediction : Gaussian Process
[[Tay & Laugier & Mekhnacha 11]PhD Thesis Tay Meng Keat + Patent Toyota & Inria & Probayes (2010)
High-level Behavior predictionfor other vehicles
(Observations + HMM)
Ego vehicleRisk estimation
(Gaussian Process)
Behavior Prediction
(HMM)Observations Behavior models
+Prediction 0
0,1
0,2
0,3
0,4
0,5
0,6
Overtaking TurningLeft TurningRight ContinuingStraightAhead
Behaviour Probability
Behavior belief table
RiskAssessment
(GP)Road geometry (GIS) + Ego vehicle trajectory to evaluate
Collision probability for ego vehicle
Behavior belief table for each vehicle in the scene
0
0,1
0,2
0,3
0,4
0,5
0,6
Ov ertaking TurningLeft TurningRight ContinuingStraightAhead
Behaviour Probability +Evaluation
Experimental validation:
Toyota Simulator + Driving device
Ego vehicle
An other vehicle
Collision Risk Assessment Collision Risk Assessment –– Simulation resultsSimulation results
Equipped Toyota Lexus
Ibeo Lux
Stereo camera
IMU + GPS+ Odometry
Collision Risk Assessment Collision Risk Assessment –– Experimental results (Real data)Experimental results (Real data)
Behaviors prediction on a highway (Real time)
Cooperation Toyota & Probayes
Performance summary (statistics)
Maneuvers Maneuvers prediction at roads intersectionsprediction at roads intersectionsCooperation Stanford & Renault
� Scenario� A vehicle is approaching, then crossing an intersection
� Available information
=> perception, previous mapping, communication ...
� Digital map of the road network
� State of the vehicle: position, orientation, turn signal
� Associated uncertainty
� Objective� At any t, estimate the manoeuvre intention of the driver of the approaching vehicle
(e.g. turn left), using states information and extracted information from the digital
map
[Lefevre & Laugier & Guzman IV’11]
Maneuvers Prediction at Road IntersectionsManeuvers Prediction at Road IntersectionsCooperation Stanford & RenaultCooperation Stanford & Renault
– Digital map obtained using Google Map, an annotated using the RNDF format
– Typical paths are obtained with a 3D laser (velodyne), by observing real traffic
– 40 recorded trajectories have been manually annotated
– 2 datasets have been constructed with these trajectories, by automatically annotating
the turn signal
• 40 trajectories with consistent turn signal
• 40 trajectories with inconsistent turn signal
Stanford’s Junior Vehicle (parked)
Intersection 1 Intersection 2
Experimental evaluation : Qualitative ResultsExperimental evaluation : Qualitative Results
� Consistent turn signal � Inconsistent turn signal
– Definitions
• mA, mB = most probable manoeuvre and second most probable manoeuvre
• Undecidable prediction: P(mA) - P(mB) ≤ 0.2
• Incorrect prediction: P(mA) - P(mB) > 0.2 and mA is incorrect
• Correct prediction: P(mA) - P(mB) > 0.2 and mA is correct
– Results on 2 datasets (40 trajectories each)
� Consistent turn signal � Inconsistent turn signal
Entrance Exit
Experimental evaluation : Quantitative ResultsExperimental evaluation : Quantitative Results
e-Motion contributions
on Mobility Assistance
Anne Spalanzani
Arturo Escobedo
Jorge Rios Martinez
Christian Laugier
INRIA Rhône-Alpes
Navigation of a wheelchair taking into account the
context of use : main challenges
• Study of the needs : who might benefit from an Autonomous Wheelchair?
• The wheelchair is a robot : autonomous navigation
– Uncertain and incomplete knowledge of the environment
– Ability to predict the behavior of the obstacles (which can be humans)
• The wheelchair transports a person
– Person/wheelchair communication
– Integration of social conventions in the navigation decision
– Autonomous/Semi-autonomous navigation
• Validation of the proposed system
Scientific challenges• The static environment is unknown
⇒ Construction of maps of the environment
• Mobile obstacles are not known, but they follow typical patterns
⇒ Detection & Tracking + Prediction on-line
⇒ Learning of typical patterns
• Deal with dynamic and uncertain environments
⇒ Navigation decisions based on a risk criteria (Risk-RRT Fulgenzi 08)
⇒ Social conventions with proxemics constraints (Personal Space, Interaction) (Rios 2011)
Fulgenzi C., Tay C., Spalanzani A., Laugier C. “Probabilistic navigation in dynamic environment using Rapidly-exploring Random Trees and Gaussian
Processes”, IEEE/RSJ 2008 International Conference on Intelligent RObots and Systems, 2008
Rios-Martinez, J., Spalanzani, A., Laugier, C.: Probabilistic autonomous navigation using risk-rrt approach and models of human interaction. In: Proceedings of
the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (2011)
Risk-RRT (C. Fulgenzi)
26
Human aware Navigation (J. Rios Martinez)
� Personal Space
� Space of interaction
� Navigation among humans based on risk and comfort
Pconf (q) = Pcoll (q) ° Ppers ° Pinter
Not tak
ing
into
account
interactio
ns
Tak
ing in
to
account
interactio
ns
Mobility Assistance (A. Escobedo)
• Navigation system adapted to the person (elder people,
disabled, poly disabled…)
– Autonomy-semi autonomy
– Interacting with a wheelchair
– From following to accompanying …
28