Upload
angelo-corsaro
View
556
Download
2
Tags:
Embed Size (px)
Citation preview
Introducing VORTEX Lite
Chief Technology Officer PrismTech
Angelo Corsaro, PhD
Cop
yrig
ht P
rism
Tech
, 201
4
Specialised device implementations optimally addressing requirements of OT and IT platforms
VORTEX can readily deal with data ingestion seamlessly integrating with other protocols, e.g. MQTT, CoAP, etc.
VORTEX leverages the DDS standard for interoperability and uniquely extends it with support for Internet Scale systems, mobility and Web 2.0 applications
The VORTEX PlatformVortex Device
Tools
Integration
Vortex Cloud
MaaS
Cop
yrig
ht P
rism
Tech
, 201
4
All VORTEX Device implementation, i.e. OpenSplice, Café, Lite and Web are Fog ready
VORTEX Device implementations feature efficient peer-to-peer communication
VORTEX Device implementations are also cloud-enabled, in the sense that they can also communicate via VORTEX Cloud
VORTEX Device
Vortex Device
Cop
yrig
ht P
rism
Tech
, 201
4
VORTEX Architecture
IP
UDP TCP
DTLS TLS
WebSocketDDSI Wire Protocol
VORTEX Café
VORTEX Cloud
VORTEX Web
VORTEX Gateway
VORTEX Lite VORTEX OpenSplice
MQTT AMQP XMPP HTTP HBase DMBS TCP/IP, UDP/IP
…
VORTEX Café
DDS API
DDS Security
Cop
yrig
ht P
rism
Tech
, 201
4
VORTEX supports both the Cloud and the Fog Computing Paradigm
VORTEX natively supports:
- Device-to-Device Communication
- Device-to-Cloud Communication
Cloud and Fog/Edge Computing
Fog Computing
Cloud Computing
Fog Computing
Fog Computing
Device-to-Cloud Communication
Device-to-Device Communication
Fog-to-Cloud Communication
Cloud-to-Cloud Communication
Device-to-Device Communication
Collect | Store | Analyse | Share
Collect | Store | Analyse | Share
Cop
yrig
ht P
rism
Tech
, 201
4
Abstracting Connectivity
Fog Computing
Cloud Computing
Fog Computing
Fog Computing
Device-to-Cloud Communication
Device-to-Device Communication
Fog-to-Cloud Communication
Cloud-to-Cloud Communication
Device-to-Device Communication
Deployment Models
Cop
yrig
ht P
rism
Tech
, 201
4
Fog + Cloud
Fog Computing Fog Computing
Fog Computing
Device-to-Cloud Communication
Peer-to-Peer (Brokerless)
Device-to-Device Communication
Devices communicate peer-to-peer within a fog-domain and through Cloud across fog-domains
Some device concurrently communicate with peers and the cloud
Cop
yrig
ht P
rism
Tech
, 201
4
Fog + Cloud-Link + CloudDevices communicate peer-to-peer within a fog-domain
A Cloud-Link controls which data is exchanged with the could
Fog Computing Fog Computing
Fog Computing
Device-to-Cloud Communication
Peer-to-Peer (Brokerless)
Device-to-Device Communication
Cloud-LinkCloud-Link
Cop
yrig
ht P
rism
Tech
, 201
4
Federated Fog
Fog Computing Fog Computing
Fog Computing
Peer-to-Peer (Brokerless)
Device-to-Device Communication
Cloud-Link
Cloud-Link
Fog domain are federated by Cloud-Link instances
A Cloud-Link controls which data is exchanged with the could
VORTEX Abstractions
Cop
yrig
ht P
rism
Tech
, 201
4
VORTEX provides a Distributed Data Space abstraction where applications can autonomously and asynchronously read and write data
Its built-in dynamic discovery isolates applications from network topology and connectivity details
Global Data Space
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Cop
yrig
ht P
rism
Tech
, 201
4
A Topic defines a domain-wide information’s class
A Topic is defined by means of a (name, type, qos) tuple, where
• name: identifies the topic within the domain
• type: is the programming language type associated with the topic. Types are extensible and evolvable
• qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc.
Topic
TopicType
Name
QoS
DDS Global Data Space
...
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
QoS Policies
Cop
yrig
ht P
rism
Tech
, 201
4
DDS provides a rich set of QoS-Policies to control local as well as end-to-end properties of data sharing
Some QoS-Policies are matched based on a Request vs. Offered (RxO) Model
QoS Policies
HISTORY
LIFESPAN
DURABILITY
DEADLINE
LATENCY BUDGET
TRANSPORT PRIO
TIME-BASED FILTER
RESOURCE LIMITS
USER DATA
TOPIC DATA
GROUP DATA
OWENERSHIP
OWN. STRENGTH
LIVELINESS
ENTITY FACTORY
DW LIFECYCLE
DR LIFECYCLE
PRESENTATION
RELIABILITY
PARTITION
DEST. ORDER
RxO QoS Local QoS
Cop
yrig
ht P
rism
Tech
, 201
4
For data to flow from a DataWriter (DW) to one or many DataReader (DR) a few conditions have to apply:
The DR and DW domain participants have to be in the same domain
The partition expression of the DR’s Subscriber and the DW’s Publisher should match (in terms of regular expression match)
The QoS Policies offered by the DW should exceed or match those requested by the DR
QoS ModelDomain
Participant
DURABILITY
OWENERSHIP
DEADLINE
LATENCY BUDGET
LIVELINESS
RELIABILITY
DEST. ORDER
Publisher
DataWriter
PARTITION
DataReader
Subscriber
DomainParticipant
offered QoS
Topicwrites reads
Domain Idjoins joins
produces-in consumes-from
RxO QoS Policies
requested QoS
Cop
yrig
ht P
rism
Tech
, 201
4
Support for fine grained access control
Support for Symmetric and Asymmetric Authentication
Standard Authentication, Access Control, Crypto, and Logging plug-in API
Security
Arthur Dent
Arthur Dent
Ford Prerfect
Zaphod Beeblebrox
Marvin
Trillian
A(r,w), B(r)
A(r,w), B(r,w), X(r)
*(r,w)
*(r)
A(r,w), B(r,w), C(r,w)
Ford Prerfect
Zaphod Beeblebrox
Trillian
Marvin
A
B
A,BX
*
*
A,B,C
Identity Access RightsSessions are authenticated and communication is encrypted
Only the Topic included as part of the access rights are visible and accessible
Cop
yrig
ht P
rism
Tech
, 201
4
Anatomy of a DDS Application
Vortex Lite
Cop
yrig
ht P
rism
Tech
, 201
4
Bring software-based DDS implementation to a range of resource constrained devices
Allow for variability on functionalities, transport and support of underlying OS/BSP
Provides very low footprint and very high performance
v1.0 runtime footprint ~450 KB
v1.0 latency as low as ~30 usec (on Gigabit network)
Vortex Lite
DDSI$$
DDS$API C99 ISO&C++
OS/RTOS$$ BSP
Abstraction$OS
Abstraction$Transport
Architecture
Cop
yrig
ht P
rism
Tech
, 201
4
The Vortex Lite architecture aims at optimising performance on constrained environment.
By default, Lite performs reads and writes on a single thread.
Writes can be multi-threaded when using TCP/IP
Vortex Lite Architecture
DDSI
DCPS
Listeners/ WaitSets
Network Packets
DDSI Messages
Samples
Cache Changes
n threads (n=1 by default)
Cop
yrig
ht P
rism
Tech
, 201
4
By default, Lite uses the same thread to:
- reads data from the socket
- execute the DDSI protocol state machine
- de-serialise data
- invoke listeners (if any)
This ensures that no context switch happens on the dispatch path thus, reducing overhead on single-core procures, as well as providing very low and deterministic latency
Default Read Path
DDSI
DCPS
Listeners/ WaitSets
Network Packets
DDSI Messages
Samples
Cache Changes
n threads (n=1 by default)
Cop
yrig
ht P
rism
Tech
, 201
4
Lite can be configured to use a separate thread pool to notify listeners
In this case the bottom thread (pool) takes care of:
- reading data from the socket
- executing the DDSI protocol state machine
While the other thread (pool) takes care of:
- de-serialising data
- invoking listeners (if any)
This configuration improves throughput on multi-core processors
Async Read Path
DDSI
DCPS
Listeners/ WaitSets
Network Packets
DDSI Messages
Samples
Cache Changes
n threads (n=1 by default)
Cop
yrig
ht P
rism
Tech
, 201
4
For UDP/IP communication Lite uses a write through strategy leveraging the application thread for executing the entire stack
Write Path
DDSI
DCPS
Writer
Network Packets
DDSI Messages
Samples
Cache Changes
application thread
Cop
yrig
ht P
rism
Tech
, 201
4
When communicating over TCP/IP Lite can be configured to use a thread-pool to write over TCP/IP sockets
This helps with parallelising some of the fan-out
TCP/IP Write Path
DDSI
DCPS
Writer
Network Packets
DDSI Messages
Samples
Cache Changes
application thread
n threads (n=1 by default)
Lite in Numbers
Cop
yrig
ht P
rism
Tech
, 201
4
488 KBytes footprint for the dds library
4 KB for all the DDS entities required to publish data
- Domain Participant
- Topic
- Subscriber
- Data Reader
- Listener
Adding more Writer on the same participant has very little overhead
Subscriber Memory Usage (pmap -x)
Address Kbytes RSS Dirty Mode Mapping08048000 4 4 0 r-x-- subscriber08049000 4 4 4 r---- subscriber0804a000 4 4 4 rw--- subscriber748a000 1700 572 0 r-x-- libc-2.19.sob7633000 8 8 8 r---- libc-2.19.sob7635000 4 4 4 rw--- libc-2.19.sob763a000 96 72 0 r-x-- libpthread-2.19.sob7652000 4 4 4 r---- libpthread-2.19.sob7653000 4 4 4 rw--- libpthread-2.19.sob766d000 588 488 0 r-x-- libdds.so.0.9.2b7700000 12 12 12 r---- libdds.so.0.9.2b7703000 4 4 4 rw--- libdds.so.0.9.2b770c000 128 100 0 r-x-- ld-2.19.sob772c000 4 4 4 r---- ld-2.19.sob772d000 4 4 4 rw--- ld-2.19.so
Cop
yrig
ht P
rism
Tech
, 201
4
456 KBytes footprint for the dds library
4 KB for all the DDS entities required to publish data
- Domain Participant
- Topic
- Publisher
- Data Writer
Adding more Writer on the same participant has very little
Publisher Memory Usage (pmap -x)
Address Kbytes RSS Dirty Mode Mapping08048000 4 4 0 r-x-- publisher08049000 4 4 4 r---- publisher0804a000 4 4 4 rw--- publisherb74ae000 1700 564 0 r-x-- libc-2.19.sob7657000 8 8 8 r---- libc-2.19.sob7659000 4 4 4 rw--- libc-2.19.sob765e000 96 72 0 r-x-- libpthread-2.19.sob7676000 4 4 4 r---- libpthread-2.19.sob7677000 4 4 4 rw--- libpthread-2.19.sob7691000 588 456 0 r-x-- libdds.so.0.9.2b7724000 12 12 12 r---- libdds.so.0.9.2b7727000 4 4 4 rw--- libdds.so.0.9.2b7730000 128 100 0 r-x-- ld-2.19.sob7750000 4 4 4 r---- ld-2.19.sob7751000 4 4 4 rw--- ld-2.19.so
Cop
yrig
ht P
rism
Tech
, 201
4
Median Latency as low as 31.5 micro-secs for small data
Latency of 51 micro-secs for 1024 data size!
Lite is by far the fastest DDS implementation on the market!
LatencyMedian Latency
mic
ro-s
ecs
30
36
43
49
55
data size (bytes)0 300 600 900 1200
31,5
34,5
51,0Lite 0.9.2
Cop
yrig
ht P
rism
Tech
, 201
4
Latency nicely fit by the quadratic polynomial:
Latency
mic
ro-s
ecs
30
55
80
105
130
data size (bytes)0 2250 4500 6750 9000
y = -7,86E-7x2 + 0,0176x + 32,8
31,534,5
51,0
66,0
91,0
124,5
Lite 0.9.2
y = -7.86E-7x2+0.0176x + 32.8
NB: This quadratic fit should be used for data up to 8192 bytes. Otherwise the conservative linear fit y = 0,0113x + 37,425 can be used for bigger sizes
Cop
yrig
ht P
rism
Tech
, 201
4
Saturates a 1 Gbps network for data as small as 1024 bytes
Throughput
Mbp
s
0
250
500
750
1000
data-size (bytes)4 1253 2502 3751 500050
294
560
780
903928 948
Lite 0.9.2
Cop
yrig
ht P
rism
Tech
, 201
4
Throughput
Kmsg
s/se
c
1
10
100
1000
data-size (bytes)1 10 100 1000 10000 100000
523 506
11056
2914
7,33,7
1,8
Lite 0.9.2
Cloud Connectivity
Cop
yrig
ht P
rism
Tech
, 201
4
Vortex Lite can be easily configured to use cloud by enabling TCP/IP and adding the Vortex-Cloud address as one of the peers
If you want to enable only communication through cloud then you may disable support for multicast (see side example )
Vortex Cloud Connectivity<DDSI2EService> <General> <NetworkInterfaceAddress>AUTO</NetworkInterfaceAddress> <AllowMulticast>false</AllowMulticast> <EnableMulticastLoopback>false</EnableMulticastLoopback> <EnableLoopback>false</EnableLoopback> </General> <TCP> <Enable>True</Enable> </TCP> <Discovery> <Peers> <Peer>demo-eu.prismtech.com:7400</Peer> </Peers> </Discovery> <Compatibility> <StandardsConformance>lax</StandardsConformance> </Compatibility> <Tracing> <Verbosity>warning</Verbosity> </Tracing></DDSI2EService>
API
Cop
yrig
ht P
rism
Tech
, 201
4
Writing Data#include "generated/Test.h"
int main (int argc, char ** argv) { dds_entity_t ppant; dds_entity_t topic; dds_entity_t writer;
/* Initialize DDS */ dds_init (argc, argv);
/* Create participant */ dds_participant_create (&ppant, DDS_DOMAIN_DEFAULT, NULL, NULL);
/* Create topic */ dds_topic_create (ppant, &topic, &Test_Topic_desc, "TestTopic", NULL, NULL, 0);
/* Create writer with QoS */ dds_writer_create (ppant, &writer, topic, NULL, NULL, 0);
/* Topic instance */ Test_Topic sample = { 1, 2 };
/* Write topic */ dds_write (writer, &sample);
dds_sleepfor (DDS_SECS (10));
/* Clean up */ dds_entity_delete (ppant); dds_fini ();}
Cop
yrig
ht P
rism
Tech
, 201
4
Reading Data#include "generated/Test.h"
int main (int argc, char ** argv) { dds_entity_t ppant; dds_entity_t topic; dds_entity_t reader; dds_sample_info_t info; void * samples[1];
/* Initialize DDS */ dds_init (argc, argv);
/* Create participant */ dds_participant_create (&ppant, DDS_DOMAIN_DEFAULT, NULL, NULL);
/* Create topic */ dds_topic_create (ppant, &topic, &Test_Topic_desc, "TestTopic", NULL, NULL, 0);
/* Create writer with QoS */ dds_reader_create (ppant, &reader, topic, NULL, NULL, 0);
while (true) { /* Wait for valid sample then print and exit */ ret = dds_read (reader, samples, 1, &info, 0); if (ret > 0 && info.valid_data) { printf ("Test Sample (%d, %d)\n", sample.d1, sample.d2); break; } dds_sleepfor (DDS_SECS (1)); }
/* Clean up */ dds_entity_delete (ppant); dds_fini ();}
Cop
yrig
ht P
rism
Tech
, 201
4
Listener Declarationstatic void data_available_handler (dds_entity_t reader) { int ret; const uint32_t max = 1; dds_sample_info_t info[max]; void * samples[max]; Test_Topic sample; uint32_t mask = 0;
samples[0] = &sample;
/* Set type of samples required. */ mask = DDS_ALIVE_INSTANCE_STATE;
/* Read samples. */ dds_read (reader, samples, max, info, mask); printf ("Data Received ...\n"); printf ("Sample (%d, %d)\n", sample.d1, sample.d2);}
Cop
yrig
ht P
rism
Tech
, 201
4
Listener Registration
memset (&listener, 0, sizeof (listener)); listener.on_data_available = data_available_handler; listener.on_subscription_matched = subscription_matched_handler;
/* Create reader with DATA_AVAILABLE status condition enabled */
dds_reader_create (sub, &reader, topic, qos, &listener, DDS_DATA_AVAILABLE_STATUS);
Summing Up
Cop
yrig
ht P
rism
Tech
, 201
4
Lite expands Vortex connectivity to embedded devices
Vortex Lite is highly efficient in time and space and simple to use
Concluding Remarks
Fog Computing
Cloud Computing
Fog Computing
Fog Computing
Device-to-Cloud Communication
Device-to-Device Communication
Fog-to-Cloud Communication
Cloud-to-Cloud Communication
Device-to-Device Communication
Cop
yrig
ht P
rism
Tech
, 201
4