How to unlock and monetize data for real-time use cases in series production cars

27th May 2020

1) see Garnter disclaimer on slide 23

Contents

1. Market Trends & Challenges

2. Value of Intelligent Edge Processing for AI-accuracy

3. The car IT-stack and the AI-model "data loop"

4. Telematic (1D) use cases:

I. PHEV Blending, eco-Routing + eco ACC

5. Video (2D) use cases:

I. Driving safety (incl. Driver Monitoring & Lane Departure)

II. Remote Driving

6. 3D point cloud (3D) use cases:

I. Sensor Fusion

II. SLAM

7. Automatizing data ingestion for AI-model training

8. Summary

New or better products and services (and new revenue streams) are based on data-driven AI-models.

Market trends enabled by AI-models

Big structural changes in automotive and mobility

Digital

connectivity

Autonomous

driving

Advanced

safety systemsAlgorithmic

insurance

New mobility

Flexible ownership

Electrical

cars

The ‘Engine’:

The above trends are - fully or partially - enabled by AI-models.

The ‘Fuel':

These AI-models are fed by ever-growing sensor data.

Automotive AI is set to be a key driver in the evolution of the industry expecting to reach $215bn by 2025(1)

(1) McKinsey Jan-2018: “Artificial intelligence as auto companies’ new engine of value”

Teraki achieves highest accuracy for AI-models with intelligent and efficient edge processing.

New challenges caused by these trends

How to build reliable AI-models and deal with the explosion of sensor data?

AI

Achieve highest AI-model accuracy:• Scalable data ingestion.

• High frequent signals.

• Selection of "events".

• Continuous updating.

Challenges in building AI-model: Teraki's contribution to building AI-models:

Hardware LatencyCost

Challenges in AI-model creation (and edge processing) Teraki helps customers to overcome these challenges

AI

Continuous, intelligent training of AI-models:• Efficient, deterministic processing.

• Embedded on constrained hardware.

• Automation of high-scale data ingestion.

• ROI and ToI to increase accuracy.

x10 higher

efficiency

x10 faster

application

speed

x50 less

communication

& storage

ToI & RoI

"event"

selection

AI-accuracyHardware LatencyCost AI-accuracy

The time to get to

99% accuracy

Achieve real-time

performance

Network and

storage costs

CPU/RAM

too low

Data processing is important part of the automotive IT-stack: responsible for 60% of application processing

Machine learning in car IT-stack

Automotive Edge Hardware (AI) Chips

Edge Pre-processing SoftwareAI-compatible Pre-processing

Edge SoftwareNeural Network Applications

Cloud-based Analytics

Automotive AI Stack Industry Participants

60% processing

40% processing

Proven

integrations with

NXP/Infineon/

Qualcomm.

OEM-cloud

OEM-models

Telemetry Signal

Intelligent Edge Processing

Up to 98% efficiency Teraki decoding

Reconstructed

signals

Training time up

to x20 to x40

smaller

User controlled

accuracy

Compressed

representations

Lidar Point Cloud

Up to 75% efficiency

Camera

Up to 96% efficiency

Sensor fusion

Edge

(car)

Cloud

Teraki: intelligent, fast and AI-focused data selection at the edge

How? Smart, AI-geared software at low CPU & RAM footprint for production-scaling.

Reduced data

collection time

More than 6X times smaller

models (RAM/ROM) than NNs

More than 10X shorter

inference times

List of features

Features from

reduced dataLightweight

algorithm✓ Driver Score

✓ Event detection

✓ Object detection

✓ SLAM

✓ etc.

1

2

Train Models

The continuous learning loop of DevCenter enables increasing accuracy of customer's AI-models

The "data loop": automatizing AI-model training via DevCenter

1. Extract 2. Train 3. Deploy

Train Decode StoreAnalyzeIngest

DevCenter

Quick training

CLOUD

RES

T A

PIs

Customer ML Pipeline

Export

Seamless model

adaptation

Data

Export

Export

Teraki Intelligent

Edge Processing

Customer

AI-Model

✓ 10x-50x faster actualization of AI model.

✓ Lower costs for chip

✓ 10%-30% increased accuracies

EdgeExtract

DevCenter

Export Machine Learning

PipelineDeploy

AI model adaptation:

Training of ML schemes

in real-time

Teraki

Client

SDK

ECU 1

ECU 2

ECU 3

Gateway

Teraki

Client

SDK

Gateway

ECU 1

ECU 2

ECU 3Select

Continuous improvement of

customers' AI-models

via DevCenter

Embedded with low CPU/RAM footprint. Deterministic. Configurable accuracy.

Telematics 1D – Intelligent selection with <10Kb RAM, 100MHz CPU

Extracting high resolution data at very low file sizesTime

Raw vs. Decoded example

Sensor Signals Edge EncodingReduced

representation

Production ready

RPMVelocity

AccelerationGyroscope

GPSetc.

Car

Decoded data at 2% of raw data

Teraki delivers 10% - 30% higher accuracy outcomes than alternative methods.

Edge processing done in an AI-compatible way

Crash Detection

Only Teraki delivers high accuracy (<1%

deviation) while reducing with factor 20X

Teraki. Intelligent Edge Processing vs. alternatives

Alternatives start failing in preserving accuracy already at 5X

reduction rates.

With Teraki, the accuracy of the AI-model's result does

not degrade - and can be even improved - vs. raw data

Best reduction

with lowest error

Detection accuracies 30% higher compared

with other processing techniques

Application Scope Value

• PHEV Blending

• Eco-Adaptive

Cruise Control

• Eco-routing

Sensors 1D. Speed, torque, acaccelerometer, GPS,

current, voltage, temp, etc. • 18 - 25 % energy consumption

improvement.

• Translastes to saving of up to $2,125 per

year per car in USA. (3)Better "mile per power" of EV & ICE engines.

USE CASE 1D: PHEV Blending, eco ACC & eco-Routing

Longer reach Fuel savingsCO2 savings

Access to high resolution and TOI processed data leads to more than $ 2,000 in savings per year.

High frequency signals essential

for 10-30% higher accuracies ….

…. in personalized

machine learning....

Signals (100Hz-1kH)

+ Historical data

+ Traffic & weather

+ On-board telemetry

Optimal blending of battery and fuelProcess

Benefits

(3) newsroom.aaa.com/tag/gas-cost/

+

Minimum-energy speed profile

Real-time energy optimized route

…. leading to $ 2,125 in yearly savings.

• Region dependent compression

• Customer defined importance

• No quality loss in important regions

Focusing on relevant objects and/or events lowers latency, hardware, power. Without affecting model accuracy

Video 2D: Intelligent RoI & ToI based selections

Achieve maximum compression by a user-specified information reduction method

Second step – Regularization:First step – Identification of RoI parameters:

• Vehicle, car, person, bicycle, etc.

• Depth estimation

• Relative motion detection

• Customer defined RoI e.g.:

✓ Car, Faces, Pedestrian, Moving objects etc.

• Regions of Low Interest, e.g.:

✓ Sky detection, Streets, vegetation, Non-moving

objects, Far away objects etc.

Non RoIRoI

Non - RoI

These 2 frames have the same (Kb) size:

but blue hasless information...

...and black has more information.

Video 2D: Introduction Video

Teraki codec: Accuracy preservation for ML vs. standard codecs

Negligible loss by Teraki vs. raw.

Jpeg/H.264 has 2% (AlexNet) to 10% (mobilNet) loss.

Only Teraki provides controllable deviation

and has 40% better accuracy at 80% reduction

Video reduction performanceMachine Learning performance (AlexNet,

MobileNet) when training on pre-processed data

Teraki better preserves the raw data leading to 40% per pixel accuracy, 10% better ML based detection

Driver score

based on

performance

Application Scope Value

Contextualized

driver safety

(combining

sensors)

Sensors 1D. Accelerometer, gyroscope, GPS, speed.

2D. Interior and exterior cameras. • $38K average cost of car accident in USA(5)

• 71% of collisions due to distracted driver

(mostly rear-end and side swipe).

• Current h/w costs too high; accuracy too low.

• 20% decrease of insurance premium costs(6)

Reliable and relevant by combining for example:

Hard maneuvers + drowsiness + smartphone usage +

traffic situation + lane departure

USE CASE 2D: Driving safety in L2+

10X more TOI models to store/process

lead to improved driver safety

Lower costs of accidents

and insurance premiumLow impact on (OEM)

car architecture

Benefits

Continuous model update leads to increased detection rates beyond 99%. At no degradation vs. raw data

Process

Real-time alert

Drive

monitoring

Rating / Analytics

of critical eventsExport to cloud

(5). www.asirt.org/safe-travel/road-safety-facts

Teraki Lane departure

(6). www.nerdwallet.com/blog/insurance/auto/car-insuracne-discounts-driving-data-worth-risk

10X less training per

ToI and per RoI

USE CASE 2D: Remote Driving as for fail-safe L2+

Application Scope Value

Remote Driving Sensors 2D. Cameras• Returning rental cars from low- to

high demand locations.

• Safety: x4 lower critical latency for safe,

real-time operation.

• Bandwidth saving up to $6 per hour.

Low latency, streaming of 2-5 camera to

remote operator.

Increasing safety (and costs) for Remote Driving for higher fleet operation efficiency.

4 real-time video streams

RoI relevance Real time transfer

4 X

• Latency of 47ms

• 75% additional

efficiency to

H.264

• Latency on par

with ffmpeg

Benefits

Process

Low Latency of

operation (real-time)

Extend operation in low

bandwidth situationsSafety when Remote

driving

0 0.1 1.0 2.0

2x 6x 9x 12x

0.7 2

Intelligent 3D edge processing with less hardware, less power and in less time.

3D Point Cloud: Smart segmentation & AI-compatible data reduction

I. Configurable, state-of-the-art data reduction of 3D point cloud data

Max. allowed deviation: (cm)

Reduction factor: (x times)

Latency: (ms)

Easily wraps around existing code

Accuracy preserved with

96.5% IoU at 500 points

per object.

Benefits

Light weight

segmentation.

1. Point cloud 2. Detect object 3. Label object

Light weight: Runs

real-time on limited

edge-CPU.

Powerful: 10X -15X times faster than any open source (e.g. PCL + NN).

Easy integration

II. Object segmentation /

ROI application

USE CASE 3D: Real-time Sensor Fusion on one single Arm-core.

Application Scope Value

Sensor Fusion (L4)

Lidar and camera

Sensors 2D. Camera

3D. Lidar, radar, ToF.• Accurate real-time integration of 3D point

cloud data with HD camera data done on

one single Arm core.

• Making Sensor Fusion scalable and sell

more cars with more ADAS functionalities sooner.

Lower CPU and energy requirements.

Lower latencies.

3D RoI's for further camera processing Use 2D to recognise objectsUse 3D to detect (moving) objects

Accelerating the sales of new ADAS functions with high precision, low latency on series production hardware.

Process

BenefitsPowerful combination of

complementary sensors

+

Real-time, local processing

of large data streams

Regular production-scale

automotive hardware (single core)

40 ms latency for 100K ppf point cloud and 30 fps HD camera

Sensor Fusion – in real-time on single CPU-core

Application Scope Value

SLAM Sensors 3D. RGB(D) camera, lidar • Significantly increase market-size for

inexpensive SLAM-devices and sell more

SLAM-based products as they run faster.

• Fast SLAM with low powered hardware and without degrading accuracy.

Fast SLAM enabled on low powered

hardware

USE CASE 3D: Fast SLAM on low-powered hardware

Process

Benefits

Input:

2D/3D

Update map and

estimated position

Extract

feature set

Map

creation

Low processing

capacityQuicker processing

hence low latencyLower power

consumption

Tracking of vehicle success-%

< 100% success

100%

tracking success

aacc100% tracking

with only 3% of data

Depth (3D) data reduction

Teraki processes the original path at 10X - 20X speed-up and without degrading SLAM.

Lowers embedded SLAM-based localization latency by more than 5X.

RG

B (

2D

) d

ata

red

ucti

on

Automation via Teraki Platform delivers easy scaling

TERAKI

PLATFORM

REST APIs

File Service

Telematic Service

Model Service

Teraki DevCenter3rd Parties

solutions

Telematics. Followed by Video & 3D

Decoder Service

Easy to integrate, scalable and agnostic

File Service

File upload and management

Telematic Service

Data reduction model training

Model testing and simulation

Model Service

Query and manage trained

models

Decoder Service

Decode binary payloads from

Teraki's Encoder SDKs

Teraki’s Platform offers REST APIs; can be easily called using any scripting language

Easy integration of the above services into applications.

Provides flexibility for the customers to easily implement their own AI-models

Enables - in a fully automated way - 10X lower training and inference time for model training

Supports Telematics, 3D point cloud and video processing in addition to AI-Enabling Services.

Summary

AI-models driver for new products and services

EDGE: Edge processing for >10X lower training/inference time for accurately updated AI-models.

USE CASES: Energy efficient driving, Driver Safety, Lane Departure, Remote Driving, Fusion & SLAM.

ESSENTIAL: Intelligent edge processing and "data loop“ are essential for high model accuracy.

EASY: Teraki Platform easy and scalable tool to manage and automatize ingestion of edge data.

VALUE: Improved model accuracy directly delivers increased business value.

READY: Teraki is Autosar compliant and proven integrations done on automotive hardware.

DATA: Sensor data is the foundation for creating and improving AI-models.

Thank you for listening

Contact: info@teraki.com

1) The GARTNER COOL VENDOR badge is a trademark and service mark of Gartner, Inc., and/or its affiliates, and is used herein with permission. All rights reserved. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Q & A Session

with:

Daniel Richart – CEO

Geert van Nunen - CCO