Building Beautiful High Performance Connected Car Applications

Building Beautiful High Performance Connected Car Applications

Jason WienerCEO - Hyperdrive

@jasonwiener


Serial CTO since 1996

● Part of the founding team or a first-hire● Assist or lead fundraising (VC/Angel/Revenue)● Build the POC & technical team● Launch ● Grow to steady state● Handoff● What’s next?

Open wheel racer

Grand Prize Winner of 2015 Toyota Onramp Smart Mobility Challenge

CEO - Hyperdrive

About Me


HyperdriveThink: part activity tracker, part driving coach, part social game.

Hyperdrive makes you a better driver by making driving fun

Lowest time with the highest MPG owns the section of road

Profitable & privately-held company


● MPG race @ Sonoma Raceway - March 3, 2017

● 20 Teams of 3-5 members

● Build strategies using a web-based telemetry editor

● Test strategies using simulations

● Race w/ real-time in-car feedback

● Team with the best avg MPG over 9 laps wins

SIGN UP at http://toyotaonramp.com

ToyotaResearchInstitute

Toyota OnrampDriver Strategy Editor

● Use “clips” of captured vehicle data

● Edit like iMovie or Adobe Premiere to build your race strategy

● Playback on Race Day against real-time performance using an onboard iPad

What we’ll cover

1. Getting car data2. Storing & retrieving data

by use case3. Visualizing data


Data Capture


Data Capture> Where do we start?

Adapters or APIs?

Workflow

Sample code

First thing’s first…

What do you want to get your hands on?

● Telemetry

● Error/Trouble codes

● Location

● YES!



Adapters or APIs?

Workflow

Sample code

Telemetry

● Get the current operating values for the car

● Core to data-driven connected car apps

● PIDs

A “go to” reference is the OBD-II PIDs page on Wikipediahttps://en.wikipedia.org/wiki/OBD-II_PIDs


https://en.wikipedia.org/wiki/OBD-II_PIDs

https://en.wikipedia.org/wiki/OBD-II_PIDs


Adapters or APIs?

Workflow

Sample code

Error/Trouble Codes

● Get warning indicators

● Useful for notifications

● Drive commerce



Adapters or APIs?

Workflow

Sample code

Location

● Get where the car is right now

● Logging activity

● Routing/Directions

● Traffic

● Needs some thinking ahead of time


Data Capture Where do we start?

> Adapters or APIs?

Workflow

Sample code

Now the how

● Rely on hardware or software?

● Adapters

○ High-end vs low-end

○ Connectivity options

○ Documentation

● APIs

○ Platforms

○ Availability on vehicles


Adapters

● High-end/Best of Breed

○ Automatic

○ GoPoint

○ Vinli


> Adapters or APIs?

Workflow

Sample code



> Adapters or APIs?

Workflow

Sample code

Adapters: High-end/Best of Breed

● Pros

○ MFi + Android

○ Location (kinda)

○ Security

○ Higher refresh rates

○ Awesome documentation

○ Example source

○ Great technical support



> Adapters or APIs?

Workflow

Sample code

Adapters: High-end/Best of Breed

● Cons

○ Price

○ Pairing

○ Tied to an account elsewhere

○ Adapter-specific integrations

○ If you use it, will they come?



> Adapters or APIs?

Workflow

Sample code

Adapters

● Low-end

○ Amazon

○ eBay

○ Alibaba



> Adapters or APIs?

Workflow

Sample code

Adapters: Low-End

● Pros

○ Plug and play

○ Cheap (as little as $3)

○ Good adapter for $15

○ Low POC costs

○ Standard chipset (ELM327)

○ Open source community



> Adapters or APIs?

Workflow

Sample code

Adapters: Low-End

● Cons

○ Less secure

○ Lower refresh rates

○ No device documentation

○ No technical support

○ Spotty Quality Control

○ BLE 5-10x 2.1 adpater costs



> Adapters or APIs?

Workflow

Sample code

APIs

● Vehicle:

○ Smart Device Link

● Cloud:

○ Automatic

○ Vinli

○ Mojio



> Adapters or APIs?

Workflow

Sample code

APIs: Vehicle (SmartDeviceLink)

● Pros

○ No adapter necessary

○ Excellent APIs

○ Location

○ Backed by Ford & Toyota

○ Supported by Mazda, Suzuki & more

○ Great documentation

○ Awesome support via Slack



> Adapters or APIs?

Workflow

Sample code

APIs

● Cons

○ Only Ford & Toyota right now

○ Presentation limitations

○ App store review process



> Adapters or APIs?

Workflow

Sample code

APIs: Cloud

● Pros

○ No hardware integration

○ REST APIs

○ Speed to market

○ Minimal costs

○ Great documentation



> Adapters or APIs?

Workflow

Sample code

APIs: Cloud

● Cons

○ Only Vinli has a telemetry API

○ Refresh rate is low

○ Reliance on adapter users



> Adapters or APIs?

Workflow

Sample code

Location

● How do we get it?

○ Mobile device

○ Laptop

○ Some adapters

○ SDL

○ Why you have to consider your use case



Adapters or APIs?

> Workflow

Sample code

How to approach it

● Start small

● Get something easy working

● Then add complexity

● Mercedes-Benz Hackathon 2015

○ MacBook Pro + Garmin 510 + ieGeek WIFI

● Moved to iOS + WIFI (POC)

● Moved to BLE, Automatic, Vinli, GoPoint



Adapters or APIs?

Workflow

> Sample code

Sample code

● Python 2.7 + gevent

● Data:

○ Mass Airflow Sensor (MAF)

○ RPM

○ Vehicle Speed (VSS)

● ieGeek OBD II WIFI adapter http://amzn.to/2jtHn5W

● code: /capture/


Data Storage


Data Storage> How will we use it?

Real-time

Trailing

Session

Snapshots

Now, we have data…

How will we be using it?

● Real-time

● Trailing

● Session

● Snapshots



Real-time

Trailing

Session

Snapshots

Do we need to know what’s going on?

● Will our app update based what’s

happening in the vehicle?

● Will our users consume the data as it

happens?

● What happens to the data after we’ve used

it or our users have seen it?



Real-time

Trailing

Session

Snapshots

Do we need to know what’s happened recently?

● Will we be providing summary data for

users?

○ Week, month, year i.e. Last week you drove 45 miles at an average of 37 MPG

● Will real-time data interact with recent

data?i.e. You’re currently driving at 37 MPG, +4 MPG over your last trailing

20-min average



Real-time

Trailing

Session

Snapshots

We’re saving everything once the user finishes driving, right?

● We don’t know what we don’t know, yet.

● Persist for future use

○ New features

○ Trends

○ Machine learning

○ #becausedata



Real-time

Trailing

Session

Snapshots

Will user need to browse and search their data?

● Trips

● Activities

● Cases when we don’t need to whole

session capture


Data Storage How will we use it?

> Real-time

Trailing

Session

Snapshots

Real-time

● Enter/exit geofences

● Major events

● Monitoring

● Diagnostics



> Real-time

Trailing

Session

Snapshots

Real-time: 2 approaches

● Polling

○ Make a request periodically

○ Old-School

● Long-polling (socket.io)

○ Open a connection, receive updates,

close on quit

○ Popularized by modern activity feeds

and chat clients



> Real-time

Trailing

Session

Snapshots

Real-time: Polling

● Pros

○ Existing tool kit

○ Easy to add

○ Just a timer firing a REST request

● Cons

○ Lots of overhead

○ Adds latency to your app

○ Can hammer your servers



> Real-time

Trailing

Session

Snapshots

Real-time: Socket.IO

● Pros

○ Better memory management

○ Can support more traffic

○ Built in lifecycle/presence

● Cons

○ Learning curve

○ Different usage pattern over traditional

network calls



> Real-time

Trailing

Session

Snapshots

Real-time: Sample codeuse with the Capture code

● Polling

○ Python 2.7 + gevent

● Socket.IO

○ Python 2.7

○ HTML+JS (Client)

○ PubNubhttps://www.pubnub.com/socket.io/

● code: /storage/real-time/



Real-time

> Trailing

Session

Snapshots

Trailing

● Time spans rollups

● Time series

● Can also power real-time stats

● Common uses:

○ Drives this week

○ MPH between 10am - 11am

○ Times you went through the Taco Bell

drive in this month


Trailing: Graphite

● Best in class

● Created at Orbitz in 2006

● Hardened/Bombproof

● Solid documentationhttp://graphite.readthedocs.org/

● #awesome


Real-time

> Trailing

Session

Snapshots



Real-time

> Trailing

Session

Snapshots

Trailing: Graphite

● Pros

○ Easy to PUT/GET data

○ Flexible ad hoc metrics

● Cons

○ Django

○ PITA to setup

■ Solution: Hosted Graphite #ftwhttps://www.hostedgraphite.com



Real-time

> Trailing

Session

Snapshots

Trailing: Sample code

● Python 2.7

● code: /storage/trailing/



Real-time

Trailing

> Session

Snapshots

Session Logs

● Text or BLOBs

● Dumping everything you’ve captured from

the user and vehicle during the session

● @ Hyperdrive:

○ Compartmentalized by time

○ Vehicle OFF & no updates for > 5 mins

○ Compressed JSON object


Session Logs

● Amazon S3 or Microsoft Azure Storage

● Quickest & easiest

● Write once/read forever

● Use it to create snapshots

● Don’t need SQL or even NoSQL

● Just need a performant key/value store


Real-time

Trailing

> Session

Snapshots



Real-time

Trailing

> Session

Snapshots

Session Logs: Sample code

● Python 2.7

● boto

● code: /storage/session/



Real-time

Trailing

Session

> Snapshots

Snapshots

● Indexed data for common use-cases

● You don’t always need the whole log

● You only need the META

○ Small, cacheable, easy-to-consume

● @ Hyperdrive:

○ Drive summary

■ Distance, MPG, time, etc.

■ Avg 20x smaller than the log


Snapshots: How to store them

● MySQL, PostgreSQL, MongoDB

● This way you can also query values

● The model is still write once/read forever,

but who cares


Real-time

Trailing

Session

> Snapshots



Real-time

Trailing

Session

> Snapshots

Snapshots: Sample code

● Python 2.7

● MySQL

● code: /storage/snapshots/


Data Visualization


Data Viz

> Making data beautiful

Maps

Real-time

History

Now, we’ve captured and stored the data…

Let’s make it beautiful!

● Maps

● Real-time

● History


Data Viz


Maps

Real-time

History

Applying the decisions that lead up to this point.

● Depending on

○ The data you’ve captured

○ What you want your users to see

○ How they’ll interact with it

● Defines

○ The metaphors you’ll leverage for data

visualization


Data Viz


Maps

Real-time

History

Maps

● They’re everywhere now

● All are simple to integrate for the most part

● Top picks:

○ Mapbox

○ Google Maps

○ Apple Maps

○ OpenStreetMap

○ Here


Data Viz


Maps

Real-time

History

Maps: deck.gl

● Gorgeous library

● Like d3.js for map data

● Designed for large data sets

● Maintained by Uberhttp://uber.github.io/deck.gl/#/


Data Viz

Making data beautiful

> Maps

Real-time

History

Maps

● @ Hyperdrive we use Mapbox

○ Vector maps

○ Killer theme editor

○ Turf.jshttps://github.com/turfjs

○ Ability to keep themes on static and

dynamic maps



Mapbox

Data Viz


> Maps

Real-time

History

Mapbox Example: Sonoma Raceway Map


Data Viz


> Maps

Real-time

History

Maps: Sample code

● HTML + JS

● Mapbox library

● code: /dataviz/maps/


Data Viz


Maps

> Real-time

History

Real-time

● Gauges

● Line graphs


Data Viz


Maps

> Real-time

History

Real-time: Gauges

● Awesome for expressing a single channel

of data

○ Speed, RPM, MPG, etc

● @ Hyperdrive we use:

○ Raphael.js (web)http://dmitrybaranovskiy.github.io/raphael/

○ RYO (native mobile)*Tons of OS libraries on GitHub


Data Viz


Maps

> Real-time

History

Real-time: Gauges

● How we wire them to work:

○ Shared method for polling, function or

sockets updates

○ Trigger an animated update of the

value

○ ease-in-out @ 0.25 sec = sweet spot for

update durations


Data Viz


Maps

> Real-time

History

Gauges Example


Data Viz


Maps

> Real-time

History

Real-time: Line Graphs

● Gives a sense of trends and history

● Updates as it happens

● @ Hyperdrive we use d3.js


Data Viz


Maps

> Real-time

History

Real-time: Sample code

● HTML + JS

● d3.js

● RaphaelJS

● code: /dataviz/real-time/


Data Viz


Maps

Real-time

> History

History Data

● Trailing

● Telemetry


Data Viz


Maps

Real-time

> History

History: Trailing

● Line charts

● Multiplex data together


○ d3.js (web)https://d3js.org/



Data Viz


Maps

Real-time

> History

History: Trailing - Approach

● Rolled-up queries from SQL, NoSQL

and/or Graphite

● A single JSON response

● Cacheable


Data Viz


Maps

Real-time

> History

History: Telemetry

● Line charts

● Stack channels to add context

● Ability to sync cursor interactions


○ d3.js (web)https://d3js.org/



Data Viz


Maps

Real-time

> History

History: Telemetry - Approach

● Index data into arrays on time or distance

depending on the use case

● On user interaction (ala mouseover):

○ Handle the cursor position (very simple

math) to get the array position

○ Lookup the data in each channel

○ Display & update readouts


Toyota OnrampDriver Strategy Editor

● Use “clips” of captured vehicle data

● Edit like iMovie or Adobe Premiere to build your race strategy

● Playback on Race Day against real-time performance using an onboard iPad

Data Viz


Maps

Real-time

> History

History: Sample code

● HTML + JS

● d3.js

● RaphaelJS

● code: /dataviz/history/


What we’ve covered

1. Getting car data2. Storing & retrieving data

by use case3. Visualizing data


https://github.com/hyperdriveme

repo: building-beautiful-connected-car-apps

Code


http://hyperdrive.me

Thank you!

Automotive

Building Beautiful High Performance Connected Car Applications