How to unlock and monetize data for real-time use cases in series production cars
27th May 2020
1) see Garnter disclaimer on slide 23
Contents
1. Market Trends & Challenges
2. Value of Intelligent Edge Processing for AI-accuracy
3. The car IT-stack and the AI-model "data loop"
4. Telematic (1D) use cases:
I. PHEV Blending, eco-Routing + eco ACC
5. Video (2D) use cases:
I. Driving safety (incl. Driver Monitoring & Lane Departure)
II. Remote Driving
6. 3D point cloud (3D) use cases:
I. Sensor Fusion
II. SLAM
7. Automatizing data ingestion for AI-model training
8. Summary
New or better products and services (and new revenue streams) are based on data-driven AI-models.
Market trends enabled by AI-models
Big structural changes in automotive and mobility
Digital
connectivity
Autonomous
driving
Advanced
safety systemsAlgorithmic
insurance
New mobility
Flexible ownership
Electrical
cars
The ‘Engine’:
The above trends are - fully or partially - enabled by AI-models.
The ‘Fuel':
These AI-models are fed by ever-growing sensor data.
Automotive AI is set to be a key driver in the evolution of the industry expecting to reach $215bn by 2025(1)
(1) McKinsey Jan-2018: “Artificial intelligence as auto companies’ new engine of value”
Teraki achieves highest accuracy for AI-models with intelligent and efficient edge processing.
New challenges caused by these trends
How to build reliable AI-models and deal with the explosion of sensor data?
AI
Achieve highest AI-model accuracy:• Scalable data ingestion.
• High frequent signals.
• Selection of "events".
• Continuous updating.
Challenges in building AI-model: Teraki's contribution to building AI-models:
Hardware LatencyCost
Challenges in AI-model creation (and edge processing) Teraki helps customers to overcome these challenges
AI
Continuous, intelligent training of AI-models:• Efficient, deterministic processing.
• Embedded on constrained hardware.
• Automation of high-scale data ingestion.
• ROI and ToI to increase accuracy.
x10 higher
efficiency
x10 faster
application
speed
x50 less
communication
& storage
ToI & RoI
"event"
selection
AI-accuracyHardware LatencyCost AI-accuracy
The time to get to
99% accuracy
Achieve real-time
performance
Network and
storage costs
CPU/RAM
too low
Data processing is important part of the automotive IT-stack: responsible for 60% of application processing
Machine learning in car IT-stack
Automotive Edge Hardware (AI) Chips
Edge Pre-processing SoftwareAI-compatible Pre-processing
Edge SoftwareNeural Network Applications
Cloud-based Analytics
Automotive AI Stack Industry Participants
60% processing
40% processing
Proven
integrations with
NXP/Infineon/
Qualcomm.
OEM-cloud
OEM-models
Telemetry Signal
Intelligent Edge Processing
Up to 98% efficiency Teraki decoding
Reconstructed
signals
Training time up
to x20 to x40
smaller
User controlled
accuracy
Compressed
representations
Lidar Point Cloud
Up to 75% efficiency
Camera
Up to 96% efficiency
Sensor fusion
Edge
(car)
Cloud
Teraki: intelligent, fast and AI-focused data selection at the edge
How? Smart, AI-geared software at low CPU & RAM footprint for production-scaling.
Reduced data
collection time
More than 6X times smaller
models (RAM/ROM) than NNs
More than 10X shorter
inference times
List of features
Features from
reduced dataLightweight
algorithm✓ Driver Score
✓ Event detection
✓ Object detection
✓ SLAM
✓ etc.
1
2
Train Models
The continuous learning loop of DevCenter enables increasing accuracy of customer's AI-models
The "data loop": automatizing AI-model training via DevCenter
1. Extract 2. Train 3. Deploy
Train Decode StoreAnalyzeIngest
DevCenter
Quick training
CLOUD
RES
T A
PIs
Customer ML Pipeline
Export
Seamless model
adaptation
Data
Export
Export
Teraki Intelligent
Edge Processing
Customer
AI-Model
✓ 10x-50x faster actualization of AI model.
✓ Lower costs for chip
✓ 10%-30% increased accuracies
EdgeExtract
DevCenter
Export Machine Learning
PipelineDeploy
AI model adaptation:
Training of ML schemes
in real-time
Teraki
Client
SDK
ECU 1
ECU 2
ECU 3
Gateway
Teraki
Client
SDK
Gateway
ECU 1
ECU 2
ECU 3Select
Continuous improvement of
customers' AI-models
via DevCenter
Embedded with low CPU/RAM footprint. Deterministic. Configurable accuracy.
Telematics 1D – Intelligent selection with <10Kb RAM, 100MHz CPU
Extracting high resolution data at very low file sizesTime
Raw vs. Decoded example
Sensor Signals Edge EncodingReduced
representation
Production ready
RPMVelocity
AccelerationGyroscope
GPSetc.
Car
Decoded data at 2% of raw data
Teraki delivers 10% - 30% higher accuracy outcomes than alternative methods.
Edge processing done in an AI-compatible way
Crash Detection
Only Teraki delivers high accuracy (<1%
deviation) while reducing with factor 20X
Teraki. Intelligent Edge Processing vs. alternatives
Alternatives start failing in preserving accuracy already at 5X
reduction rates.
With Teraki, the accuracy of the AI-model's result does
not degrade - and can be even improved - vs. raw data
Best reduction
with lowest error
Detection accuracies 30% higher compared
with other processing techniques
Application Scope Value
• PHEV Blending
• Eco-Adaptive
Cruise Control
• Eco-routing
Sensors 1D. Speed, torque, acaccelerometer, GPS,
current, voltage, temp, etc. • 18 - 25 % energy consumption
improvement.
• Translastes to saving of up to $2,125 per
year per car in USA. (3)Better "mile per power" of EV & ICE engines.
USE CASE 1D: PHEV Blending, eco ACC & eco-Routing
Longer reach Fuel savingsCO2 savings
Access to high resolution and TOI processed data leads to more than $ 2,000 in savings per year.
High frequency signals essential
for 10-30% higher accuracies ….
…. in personalized
machine learning....
Signals (100Hz-1kH)
+ Historical data
+ Traffic & weather
+ On-board telemetry
Optimal blending of battery and fuelProcess
Benefits
(3) newsroom.aaa.com/tag/gas-cost/
+
Minimum-energy speed profile
Real-time energy optimized route
…. leading to $ 2,125 in yearly savings.
• Region dependent compression
• Customer defined importance
• No quality loss in important regions
Focusing on relevant objects and/or events lowers latency, hardware, power. Without affecting model accuracy
Video 2D: Intelligent RoI & ToI based selections
Achieve maximum compression by a user-specified information reduction method
Second step – Regularization:First step – Identification of RoI parameters:
• Vehicle, car, person, bicycle, etc.
• Depth estimation
• Relative motion detection
• Customer defined RoI e.g.:
✓ Car, Faces, Pedestrian, Moving objects etc.
• Regions of Low Interest, e.g.:
✓ Sky detection, Streets, vegetation, Non-moving
objects, Far away objects etc.
Non RoIRoI
Non - RoI
These 2 frames have the same (Kb) size:
but blue hasless information...
...and black has more information.
Video 2D: Introduction Video
Teraki codec: Accuracy preservation for ML vs. standard codecs
Negligible loss by Teraki vs. raw.
Jpeg/H.264 has 2% (AlexNet) to 10% (mobilNet) loss.
Only Teraki provides controllable deviation
and has 40% better accuracy at 80% reduction
Video reduction performanceMachine Learning performance (AlexNet,
MobileNet) when training on pre-processed data
Teraki better preserves the raw data leading to 40% per pixel accuracy, 10% better ML based detection
Driver score
based on
performance
Application Scope Value
Contextualized
driver safety
(combining
sensors)
Sensors 1D. Accelerometer, gyroscope, GPS, speed.
2D. Interior and exterior cameras. • $38K average cost of car accident in USA(5)
• 71% of collisions due to distracted driver
(mostly rear-end and side swipe).
• Current h/w costs too high; accuracy too low.
• 20% decrease of insurance premium costs(6)
Reliable and relevant by combining for example:
Hard maneuvers + drowsiness + smartphone usage +
traffic situation + lane departure
USE CASE 2D: Driving safety in L2+
10X more TOI models to store/process
lead to improved driver safety
Lower costs of accidents
and insurance premiumLow impact on (OEM)
car architecture
Benefits
Continuous model update leads to increased detection rates beyond 99%. At no degradation vs. raw data
Process
Real-time alert
Drive
monitoring
Rating / Analytics
of critical eventsExport to cloud
(5). www.asirt.org/safe-travel/road-safety-facts
Teraki Lane departure
(6). www.nerdwallet.com/blog/insurance/auto/car-insuracne-discounts-driving-data-worth-risk
10X less training per
ToI and per RoI
USE CASE 2D: Remote Driving as for fail-safe L2+
Application Scope Value
Remote Driving Sensors 2D. Cameras• Returning rental cars from low- to
high demand locations.
• Safety: x4 lower critical latency for safe,
real-time operation.
• Bandwidth saving up to $6 per hour.
Low latency, streaming of 2-5 camera to
remote operator.
Increasing safety (and costs) for Remote Driving for higher fleet operation efficiency.
4 real-time video streams
RoI relevance Real time transfer
4 X
• Latency of 47ms
• 75% additional
efficiency to
H.264
• Latency on par
with ffmpeg
Benefits
Process
Low Latency of
operation (real-time)
Extend operation in low
bandwidth situationsSafety when Remote
driving
0 0.1 1.0 2.0
2x 6x 9x 12x
0.7 2
Intelligent 3D edge processing with less hardware, less power and in less time.
3D Point Cloud: Smart segmentation & AI-compatible data reduction
I. Configurable, state-of-the-art data reduction of 3D point cloud data
Max. allowed deviation: (cm)
Reduction factor: (x times)
Latency: (ms)
Easily wraps around existing code
Accuracy preserved with
96.5% IoU at 500 points
per object.
Benefits
Light weight
segmentation.
1. Point cloud 2. Detect object 3. Label object
Light weight: Runs
real-time on limited
edge-CPU.
Powerful: 10X -15X times faster than any open source (e.g. PCL + NN).
Easy integration
II. Object segmentation /
ROI application
USE CASE 3D: Real-time Sensor Fusion on one single Arm-core.
Application Scope Value
Sensor Fusion (L4)
Lidar and camera
Sensors 2D. Camera
3D. Lidar, radar, ToF.• Accurate real-time integration of 3D point
cloud data with HD camera data done on
one single Arm core.
• Making Sensor Fusion scalable and sell
more cars with more ADAS functionalities sooner.
Lower CPU and energy requirements.
Lower latencies.
3D RoI's for further camera processing Use 2D to recognise objectsUse 3D to detect (moving) objects
Accelerating the sales of new ADAS functions with high precision, low latency on series production hardware.
Process
BenefitsPowerful combination of
complementary sensors
+
Real-time, local processing
of large data streams
Regular production-scale
automotive hardware (single core)
40 ms latency for 100K ppf point cloud and 30 fps HD camera
Sensor Fusion – in real-time on single CPU-core
Application Scope Value
SLAM Sensors 3D. RGB(D) camera, lidar • Significantly increase market-size for
inexpensive SLAM-devices and sell more
SLAM-based products as they run faster.
• Fast SLAM with low powered hardware and without degrading accuracy.
Fast SLAM enabled on low powered
hardware
USE CASE 3D: Fast SLAM on low-powered hardware
Process
Benefits
Input:
2D/3D
Update map and
estimated position
Extract
feature set
Map
creation
Low processing
capacityQuicker processing
hence low latencyLower power
consumption
Tracking of vehicle success-%
< 100% success
100%
tracking success
aacc100% tracking
with only 3% of data
Depth (3D) data reduction
Teraki processes the original path at 10X - 20X speed-up and without degrading SLAM.
Lowers embedded SLAM-based localization latency by more than 5X.
RG
B (
2D
) d
ata
red
ucti
on
Automation via Teraki Platform delivers easy scaling
TERAKI
PLATFORM
REST APIs
File Service
Telematic Service
Model Service
Teraki DevCenter3rd Parties
solutions
Telematics. Followed by Video & 3D
Decoder Service
Easy to integrate, scalable and agnostic
File Service
File upload and management
Telematic Service
Data reduction model training
Model testing and simulation
Model Service
Query and manage trained
models
Decoder Service
Decode binary payloads from
Teraki's Encoder SDKs
Teraki’s Platform offers REST APIs; can be easily called using any scripting language
Easy integration of the above services into applications.
Provides flexibility for the customers to easily implement their own AI-models
Enables - in a fully automated way - 10X lower training and inference time for model training
Supports Telematics, 3D point cloud and video processing in addition to AI-Enabling Services.
Summary
AI-models driver for new products and services
EDGE: Edge processing for >10X lower training/inference time for accurately updated AI-models.
USE CASES: Energy efficient driving, Driver Safety, Lane Departure, Remote Driving, Fusion & SLAM.
ESSENTIAL: Intelligent edge processing and "data loop“ are essential for high model accuracy.
EASY: Teraki Platform easy and scalable tool to manage and automatize ingestion of edge data.
VALUE: Improved model accuracy directly delivers increased business value.
READY: Teraki is Autosar compliant and proven integrations done on automotive hardware.
DATA: Sensor data is the foundation for creating and improving AI-models.
Thank you for listening
Contact: [email protected]
1) The GARTNER COOL VENDOR badge is a trademark and service mark of Gartner, Inc., and/or its affiliates, and is used herein with permission. All rights reserved. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
Q & A Session
with:
Daniel Richart – CEO
Geert van Nunen - CCO