Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Prof Trivedi, UCSD Fujitsu World Tour 2017 1
Mohan M. Trivedi LISA: Laboratory for Intelligent and Safe Automobiles
University of California at San Diego http://cvrr.ucsd.edu/LISA
A Quest for Human Robot Cohabitation
Connecting AI Technologies to Real-World Needs Panel
Prof Trivedi, UCSD Fujitsu World Tour 2017 2
Safe, Stress-free, Efficient, Enjoyable Driving
Learning by Li Lo: Looking-in and Looking-out
Understanding human behavior is essential for humans It may very well be essential for robotic vehicles as well Make humans and vehicles form a distributed cognitive system to accomplish mutually beneficial goals
How to make two intelligent systems, one robotic and another human
understand each other and collaborate?
LISA Publications http://cvrr.ucsd.edu/publications/index.html
Key Points: Humanizing Automated Vehicles
• �Driving� = Environment + Vehicle + Driver • Autonomous Driving in the “Real World “ => Humanized Robotic Systems • Should be Proactive: highly reliable, safety-time critical operation
• Situational Awareness and Intent Predication – needs Holistic Perception • Holistic Perception:
- Looking-Out: Lanes, Vehicles, Pedestrians and “Surround” - Looking-In: Occupant and “Driver” - Looking In and Out: Attention, Intentions, Activities, “Awareness”
• Key Challenges: - Multilevel Signal-> Semantics Representation and Analysis - Multisensory and Distributed Systems and Software - Communication – speed, reliability, fail safe - Human-Centric (Distributed) System Architectures - Robustness - Performance and Evaluation Metrics, Protocols, Standards
LISA Publications http://cvrr.ucsd.edu/publications/index.html
Prof Trivedi, UCSD Fujitsu World Tour 2017 3
Humans in
vehicle cabin
Humans
in surround vehicles
Humans
around vehicle
Safe to deploy airbag? Distracted driver?
Noticed pedestrian?
Pedestrian intent?
Pedestrian trajectory? New traffic rules?
Distracted pedestrian?
My neighbor’s intent?
Acknowledge right of way?
Distracted neighbor?
Ready to take over? Hands on wheel?
“Humanizing” Intelligent Vehicles: LISA Research Agenda
Eshed Ohn-Bar, Mohan Trivedi, "Looking at Humans in the Age of Self-Driving and Highly Automated Vehicles", IEEE Transactions on Intelligent Vehicles, 2016.
Vehicle Environment Driver
Vision for Intelligent Vehicles: LISA-A Test bed 2013
Prof Trivedi, UCSD Fujitsu World Tour 2017 4
Aim: The early detection of an intended maneuver using driver, vehicle, and surround information.
Looking In and Out: Lane Change Intent Prediction [Discovery Channel, 2013]
Doshi, Morris, Trivedi, IEEE Pervasive Computing 2011
• When should the driver merge? • Should he accelerate or decelerate? How much? • Is it ok to change lanes? If yes Left/right? If so, how? • Can an “Intelligent” system assist with this?
Predictive Driver Assistance
Prof Trivedi, UCSD Fujitsu World Tour 2017 5
Robust, Reliable, Generalizable, Practical, Market ready in 2017!
Intelligent Merge- Break Assist, Attention Monitoring CES Week, San Francisco Media Event, Jan 2014
LISA-Audi Urban Intelligent Assist, 2011-2014
Vehicle Trajectory Learning Flowchart
B. Morris, M. Trivedi, "Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011
Prof Trivedi, UCSD Fujitsu World Tour 2017 6
Panoramic Trajectory Analysis
M Kristoffersen, J. Dueholm, R. Satzoda, M. Trivedi, T. Moeslund, "Trajectories and Maneuvers of Surrounding Vehicles with Panoramic Camera Arrays,” IEEE Transactions Intelligent Vehicles, 2016
Proposed Approach – Integration of Regions Looking at Humans in Surrounding Vehicles ).
Surround Vehicle Behavior Analysis
Intent prediction
M Kristoffersen, J. Dueholm, R. Satzoda, M. Trivedi, T. Moeslund, "Trajectories and Maneuvers of Surrounding Vehicles with Panoramic Camera Arrays,” IEEE Transactions Intelligent Vehicles, 2016
Prof Trivedi, UCSD Fujitsu World Tour 2017 7
Proposed Approach – Integration of Regions Looking at Humans Around the Vehicle ).
Pedestrian path prediction Pedestrian body pose and fine-grained classification
T. Gandhi, M. Trivedi, "Image Based Estimation of Pedestrian Orientation for Improving Path Prediction", IEEE Intelligent Vehicles Symposium, June 2008
Proposed Approach – Integration of Regions Looking at Humans Around the Vehicle: Detections ).
Akshay Rangesh , Mohan Trivedi, “Pedestrians and their Phones", IEEE ITSC 2016, IEEE T-IV 2017
Prof Trivedi, UCSD Fujitsu World Tour 2017 8
Proposed Approach – Integration of Regions Looking at Humans inside the Vehicle ).
Head and hand cues integration Gaze zone classification
Secondary tasks Interactions and Activities
S. Martin, K. Yuen, M. Trivedi, "Vision for Intelligent Vehicles and Applications (VIVA): Face Detection and Head Pose Challenge," IEEE Intelligent Vehicles Symposium, 2016. N. Das, E. Ohn-Bar, M. Trivedi, "On Performance Evaluation of Driver Hand Detection Algorithms: Challenges, Dataset, and Metrics,” IEEE ITSC 2015.
Proposed Approach – Integration of Regions ). Privacy and Safety: Balancing the Scale
Sujitha Martin, Ashish Tawari and Mohan M. Trivedi, “Towards Privacy Protecting Safety Systems for Naturalistic Driving Videos,” IEEE Transactions Intelligent Transportation Systems, 2014.
Prof Trivedi, UCSD Fujitsu World Tour 2017 9
LISA @ CES, Jan 2016 LISA SafeShield @ CES 2016
R. Satzoda, Sean Lee, Frankie Lu, M. Trivedi, "Snap-DAS: A Vision-based Driver Assistance System on a Snapdragon Embedded Platform,” IEEE Intelligent Vehicles Symposium, 2015.
Holistic Distributed Cognitive Systems Perspective: Learning from Naturalistic Driving Studies, Predictive, Attentive Systems
Long-Term Goals: Human cohabitation with intelligent robots
Open Issues: Fail-safe, Control transitions, Performance Metrics, standards, evaluations, multi-agents, cooperation, etc. etc.
Big Picture: Safe, Stress-free, Efficient, Enjoyable Driving
Four Point Summary
Thanks ! more information: [email protected] cvrr.ucsd.edu/lisa