39
The Road to Autonomous Vehicles Bob Feldmaier Director of the Center for Advanced Automotive Technology Macomb Community College

The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Embed Size (px)

Citation preview

Page 1: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

The Road to Autonomous Vehicles

Bob Feldmaier Director of the Center for Advanced

Automotive Technology Macomb Community College

Page 2: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Main Questions

• Why the interest in autonomous vehicles?

• How does the technology work?

• What are the remaining challenges?

Page 3: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

GM’s Road to Automated Driving

Page 4: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Some of Today’s Advanced Driver Assistance Technologies

Page 5: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

NHTSA Automated Driving Levels (0-4)

Today 2020 2025+

Driver only Assisted Partial Conditional Full

Active high beam

Traffic jam assist

Collision avoidance

Valet self-parking

Collision imminent

braking

Adaptive cruise & lane keeping

Automated highway

Highway point-to-point

Cruise control Self-parking (with driver)

Automated urban

Urban point-to-point

Forward Radar Forward Radar Lidar & 360⁰ Radar

Lidar & 360⁰ Radar

Forward Vision Forward Vision High accuracy GPS

High accuracy GPS

Multi-domain controller

Multi-domain controller

Multi-domain controller

Driver State Sensor (DSS)

Forward Vision cameras, DSS

Forward Vision cameras, DSS

V2X V2X V2X

IMU IMU

Feat

ure

Te

chn

olo

gy

Level 0 Level 1 Level 2 Level 3 Level 4

Page 6: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Potential Benefits of Vehicle Automation

Page 7: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

The Impact of Car Crashes on the Economy beyond 34,000 Deaths per Year in the US Alone

Page 9: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

US Consumers Rate Safety and Advanced Driver Assistance Technologies Most Important

Page 10: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Dr. Lawrence D. Burns, 2015

Page 11: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Integrated Systems Approach to Vehicle Automation

Coming application: 2017 Cadillac “Super Cruise”

Page 12: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

ADAS and Automated Vehicle Sensors

Multi Domain

Controller

Side Radars

Side Radars

V2V/V2X

Rear View Cam

Side View Cam

Side View Cam

Driver State

Sensing Side Radars

Front View Cam

Front Radars

Front Cameras

Lidar

Side Radars

Page 13: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Vision/Radar/Lidar Operation and Fusion

Page 14: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information
Page 15: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Sensor Fusion Improves Performance

Vision Lane Information

Vision Vehicle Information Radar Object Information

Radar Barrier Detection

Left Right

Adjacent Left

Adjacent Right

Page 16: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

• Probability of correct detection and classification1

– Increases with additional sensors and redundancy

– Utilize sensors with highest signal to noise ratio (S/N) under the ambient conditions

– Disregard sensors that have low S/N under the ambient conditions

– Marginal gains decrease for more than 5 sensors

• Reliability of systems2

– Adding more sensors increases the reliability of the overall system

– Mean time to failure of a system with more sensors is increased

Advantages of Redundant Sensor Fusion

References: 1Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information Warfare Library, February 26, 2004 2Deyst, John, “Real Time Systems for Aerospace Vehicles”, MIT 16.840 Aeronautics & Astronautics Course Notes, Spring 1999

Page 17: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Multi-domain Controller

17

Enables future system optimization/upgradability

Multi-domain controller

Front Radar

controller Camera

controller Airbag Controller

Rear and Side Radar controller

ESC controlle

r

Scalable software platform

Reduced architecture complexity

Faster communication/interconnection

Multi-processor configuration

Production launch in 2017

Centralized Sensor Fusion/Control Active Safety Multi-domain controller

Page 18: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Typical Software Applications: Lines of Code 24 million lines of code

F-35 fighter jet

61 million lines of code Facebook

12 million lines of code Android Operating System

Premium vehicle

44 million lines of code Microsoft Office 2013

50+ computers To deliver a world-class user

experience, active safety and high performance drivability

*Software lines of code information courtesy of informationisbeautiful.net

Premium vehicles today operate with over 100 million+ lines of code

Page 19: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Automated Driving: Enabling and Supporting Technology

19

Source: Texas Instruments ADAS Solutions Guide

HIGH DEFINITION MAPS V2X COMMUNICATIONS

Page 20: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Introducing the Concept of “Connected” Vehicles

Page 21: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

26B connected devices 250M connected vehicles

by 2020…

Page 22: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

The Connected Car Evolves…

Page 23: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Cars talking with surrounding infrastructure…

Page 24: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information
Page 25: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

The Process of Delivering Real-Time Maps

Page 26: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Once a Vehicle is Connected, Many More Features Become Available

Page 27: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

The Vehicle Becomes Integrated with the Web of Everything

Page 28: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information
Page 29: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information
Page 30: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

With Connectivity, Data Becomes “Bigger”

Page 31: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

With More Data and Connectivity Comes More Vulnerability of Cybersecurity

Page 32: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Toyota’s Assessment of Automated Vehicle Technology

Page 33: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Toyota’s Assessment (Continued)

Page 35: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Summary of Major Advantages

• Fewer traffic collisions

• Increased roadway capacity and reduced congestion

• Relief for occupants from driving and navigation

• Removal of constraints on occupants’ state or handicaps

• Lighter more fuel efficient vehicles

• Reduced insurance costs

• Higher speed limits

• Increased productivity

Page 36: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Summary of Major Concerns

• Assignment of liability for errors

• Resistance to loss of vehicle control

• Hardware function in bad weather

• Software decision protocols

• Software reliability

• Cybersecurity

• Loss of privacy

• Managing the transition from automated control to driver control

Page 37: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Typical Technician Skills Required in the Field of Automated Vehicles

• Basic automotive and prototype shop knowledge (teardown vehicles, build harnesses, basic fabrication skills, troubleshoot auto systems without manuals)

• Electronics skills (ECMs, sensors and sensor fusion, antennas, CAN and cable protocols, displays, soldering, shielding, troubleshooting)

• Software Skills (embedded systems, basic programming, networks, security systems, user interfaces)

• Understanding of Communication protocols (Satellite, LTE/cellular, WiFi, DSRC, Bluetooth)

• Lab testing, data acquisition and analysis

37

Page 38: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Stay Connected with the CAAT

• Visit our website at www.autocaat.org

• Sign up for our monthly newsletter

• Follow us on social media

• Contact us with your seed funding project ideas!

– Connected/Automated Vehicles

– Lightweighting

– Testing

– http://autocaat.org/Educators/Seed_Funding/

38

Page 39: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information

Thank You!

Questions?