Devising Your Data Movement Strategy for IoT

1confidential

Devising your Data Movement Strategy

for IoTShawn McAllisterSeptember, 2016

2confidential

All Aboot ShawnCTO of Solace SystemsMember of OASIS MQTTTechnical CommitteeEarly member of AMQPTechnical CommitteeHockey EnthusiastFrom Canada, eh!

3confidential

Data Movement Between Elements of IoT

Top level system considerations

Architectures for different types of data movement

Which standards to use;where and why

4confidential

Relevant System Requirements

Scale of Devices

Capabilities ofDevices

Message Exchange Patterns

Messaging QoS

TrafficCharacteristics

SecurityIT Application Architecture & Technologies

5confidential

ProblemStatement

Firewall

Firewall

LoadBalancer

? ? ? ?

6confidential

7confidential

8confidential

Message Bus

9confidential

Firewall

Connection Tier

LoadBalancer

EDGE

FOG

Typical IoT CloudDeploymentArchitecture

Enterprise BusFirewall

CORE

10confidential

MQTT Paho

Proprietary Proprietary/JMS/JCA

HTTP/REST In Runtime

AMQP 1.0 Qpid / JMS

CoAP Several OS

P R O T O C O L A P IRelevantProtocols& APIs

11confidential

o Device Constraintso Message Exchange

Patternso Messaging QoSo Traffic Characteristicso Security

Fog

12confidential

Fog

MQTT Paho

Proprietary Proprietary/JMS/JCA

HTTP/REST In Runtime

AMQP 1.0 Qpid / JMS

CoAP Several OS

P R O T O C O L A P I

13confidential

Edgeo # Concurrent

Connectionso “Always on”

Connections?o Burstinesso Congestion /

Prioritizationo Multi-protocol?o QoSo Security

14confidential

Edge

MQTT Paho

Proprietary Proprietary/JMS/JCA

HTTP/REST In Runtime

AMQP 1.0 Qpid / JMS

CoAP Several OS

P R O T O C O L A P I

15confidential

Coreo Less scale, more

concentrationo Need enterprise

featureso Enterprise grade,

popular APIso Support streaming &

microservices

16confidential

Core

MQTT Paho

Proprietary Proprietary/JMS/JCA

HTTP/REST In Runtime

AMQP 1.0 Qpid / JMS

CoAP Several OS

P R O T O C O L A P I

17confidential

BigDataLake

Big Data River BigDataLake

Big Data

18confidential

Enterprise Bus

Connection Tier

Addressing Scalabilityo Command & Control

sending message to a specific device

19confidential

Enterprise Bus

Connection Tier

Addressing Scalabilityo Command & Control

sending message to a specific device

o Device-initiated request/reply

o Among millions of deviceso Across 2 middleware tierso Need Integrated

addressing between Enterprise Bus & Connection Tier at massive scale

20confidential

Enterprise Bus

Connection Tier

Addressing Scalability

Load Balancer

21confidential

Enterprise Bus

Connection Tier

Addressing Scalabilityo Connection Tier

Addressing‐ MQTT Topics

o Enterprise Bus Addressing‐ Other topics & queues

o Need to scale ‐ In an integrated manner‐ Update in realtime‐ Handling bursts of changes

22confidential

Singapore Land Transport Authority – ERP2http://www.straitstimes.com/singapore/transport/ncs-mhi-to-build-islandwide-satellite-based-erp-for-556m

Routing Server

Payments

Analytics

0 0 0

• 200K Concurrent Connections

• MQTT Termination

• Request/reply forfirmware upgrades

• Push Notifications

• Persistent Messaging

• Shock Absorber

• High Throughput

• Fanout

• Slow Consumer Handling

MQTT JMS

MHI OBU

Next-Gen Electronic Road Pricingo Connect 1.5M vehicles

(200K at a time) in real-time for pay per use road tolls

o Suggest better routes, traffic management – real time and long term and value-added services/analytics

23confidential

Bus Monitoringo Collect real-time

location, loading information and more thousands of buses

o Optimize bus movement, fleet management, avoid bunching, forecast arrival time and more

REST/MQTT

Regulator’s Dashboard

Commuter’s Apps and

RIAs

Analytics

DWH

ESB (if needed)

CEP

JMS

Legacy REST

Legacy Gateway

REST/MQTT

24confidential

Can’t predict coming evolution of IoT,and changing infrastructure ain’t cheapNeed flexible data movement platform

25confidential

Strong Architecture• Decoupling of concerns

• Scales to meetneeds of clients

• Right featuresat each layer

Smart Use of Standards• Right standards• Features & exchange patterns

• Optimized for needs of users

• Security• Supported by key products andopen source projects

26confidential

Questions?Swing by booth # 309shawn.mcallister@solacesystems.com