Bringing IoT Data to Life!

Date Dr. Joachim Schaper, VP Research

2

The Potential…and Challenges…of IoT Data

DATA

REAL TIME

VAST AMOUNTS

REAL WORLD HETEROGENEOUS

NOISY

sense How do we make sense of IoT data?

I T A

An IoT Analytics Platform

6

IoTA: Analytics in Action!

Mobility Pattern Analytics

Behavior Learning & Prediction

Crowd Analytics

Anomaly Detection Anomaly Detection

7

Anomaly Detection Answers Difficult Questions

What just happened that shouldn‘t have?

• What does something that shouldn‘t have happened look like?

How can I find it in time?

• Before there is serious damage

• Before supply chains, customers, competitors and VIPs are impacted

Why are you disturbing my sleep?!

• False alarms are costly

8

Anomaly Detection Addresses Multiple Problems

Anomaly Detection

Framework

Traffic Incidents

Electrical Grid

Smart Home

Social Media

Crowd

Dike Stability

9

Traffic Anomaly Detection

86%

14%

Detection performance

True

positives

False

positives

Customer problems addressed

• Ensure road network efficiency, safety

• Minimize impact of traffic incidents

• Real-time, automatic detection of abnormal traffic congestion based on sensor data

2.00

0.11 0.00

5.00

Alert rates per day and road segment

Rule-based detection

Anomaly DetectionAnomaly detection

10

Traffic Anomalies Captured Live

Different anomalies are identified depending on index threshold and event filtering.

Abnormal traffic congestion identified

Construction worker caused traffic changes

Taxi parked for >15 min caused traffic changes

28-Nov 13:15 22-Nov 04:15 20-Nov 21:26

11

Engine Configuration for Traffic Incident Detection

Use-Case Specific Plugin

Generic Plugin

Detection and Classification

API

Context Dependency Modeling

Preprocessing

Normality Model Learning

Robust Density Estimation

Support Vector Machines

Apache Thrift

Timestamp Discretization

Auto Partitioning

Discrete Context

Switching

Hypothesis & Persistence Test

Noise Reduction

Data Imputation

Nearest Neighbor

Python Storm Spark MapReduce

Principal Component Analysis

Clustering

Manifold Learning

Feature Extraction

Traffic Parameter Extraction

Enhanced HMM System

Identification

Random Forest

Event Filter

Normalization

Robust Density Estimation

Timestamp Discretization

Auto Partitioning

Discrete Context

Switching

Hypothesis & Persistence Test

Noise Reduction

Data Imputation

Storm MapReduce

12

Traffic Anomaly Detection – Data and Extracted Patterns

Raw Data: LPR, VA Characteristic Features Normal Pattern Model

• Speed and volume per lane

• High frequency noise (10% - 30% std. dev.)

• Full/partial sensor outages

• 4 feature vectors:

• Average speed

• Total volume

• Lane average speed

• Lane speed difference

• Aggregation to 1 min interval

• Data cleaning

• Noise filtering

Multi-dimensional model

• Traffic features (4 dim.)

• Context dependency

• Time of day

• Day of week; public holidays

• Covariance optimization for robustness against anomalies in training set

13

Traffic Anomaly Detection – Anomaly Index

Anomaly Example – Features and Model

Mahalanobis Distance

• Mahalanobis Distance indicates magnitude of deviation between model and measurements

• Index threshold (red line) determines detection sensitivity

• Anomalies affect multiple traffic characteristics

• Deviation vectors used to further classify the type of anomaly

14

Results from Large-scale Deployment

1,000 LPR cameras 16 million vehicle detections/day

230 road segments analyzed

Same configuration applied across highways, on-ramps, urban arterials and side streets Events validated

on CCTV Low false alert rate Recurring congestion

suppressed

15

Moonscape Ventures

• Corporate development and investment company

• Launched August 2014; operates in TLV, NYC, Silicon Valley

• Grows startups: IoT, smart cities, big data, news and media, other

• Invests in late-seed stage, Series A round

• Led by Tammy Mahn

16

Bringing IoT Data to Life!