Upload
angelo-corsaro
View
738
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Interoperability, extensibility and efficiency are increasingly required to enable and effectively operate Smart Energy Grids, Smart Cities, Exploration and Production Systems in the Oil and Gas industry, and international Air Traffic Control and Management Systems. Yet, these key architectural attributes are often an after-thought as opposed to axioms upon which the entire architecture is designed – with the result that many systems are non-interoperable, hard to extend and inefficient. This presentation will (1) precisely define the meaning of interoperability, extensibility and efficiency, (2) propose metrics for their evaluation, and (3) explain these important properties can be “designed in” system architectures. We will introduce data-centricity as the paradigm and architectural pattern that fosters interoperability, extensibility and efficiency and will explain how existing standards such as the OMG DDS can be used to implement data-centric architectures.
Citation preview
Ope
nSpl
ice
DD
S
Angelo CORSARO, Ph.D.Chief Technology Officer OMG DDS Sig Co-Chair
Interoperable, Extensible and Efficient System Architectures
Ope
nSpl
ice
DD
S
Motivating Examples
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
SESAR: The Single Sky☐ Enable operational interoperability
across ATM/ATC in EU
☐ Allow information to flow seamlessly within the ATC/ATM ecosystem and across Europe
☐ Ensure the system is incrementally extensible and evolvable
☐ Ensure the system makes efficient use of resources such as network
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Smart City / Smart Grid
☐ Make relevant information available in real-time to an open ended number of consumers
☐ Ensure interoperability with third parties, efficient use of resources and extensibility
Ope
nSpl
ice
DD
S
Interoperability
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Defining Interoperability
The ability of two or more systems or components to exchange information and to use the information that has been exchanged.
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Syntactic Interoperability
☐ The ability of two or more systems to communicate and exchange data
☐ Requires specified data formats and communication protocols
☐ Syntactical Interoperability is a necessary condition for higher-level of interoperability
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Semantic Interoperability
☐ The ability to meaningfully and accurately interpret exchanged information
☐ Semantic interoperability requires communication parties to agree on a common information model
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Interoperability in Summary
☐ Interoperability requires agreements at a syntactic and semantic level
☐ Syntactic Interoperability is commonly achieved by agreeing a communication protocol and a data representation standard
☐ Semantic Interoperability is commonly achieved by agreeing on a Common Information Model
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Measuring Interoperability
Based on the previous definitions we can define 3 levels of interoperability:
☐ Level 0: No Interoperability (closed interfaces)
☐ Level 1: Syntactic Interoperability (open protocol/data format)
☐ Level 2: Semantic Interoperability (common information model)
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
☐ Suppose that we define legal messages as any word created over alphabet, more formally:☐ ∑ = {a, b, ..., z, ?, !}☐ words := { x ∈ ∑+}☐ sentence = words+
☐ Example:☐ Can Talk and hear w/o problems☐ Don’t necessarily “understand”
Example: Syntactic Interoperability
Che ora?
Sapresti dirmi che ore sono? What?
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Example: Semantic Interoperability
☐ Suppose now that we constrain our grammar to a subset of legal English phrases:
Thanks!
Excuse me, do you know the time?
Sure, it’s 11 o’clock.
Ope
nSpl
ice
DD
S
Extensibility
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Defining Extensibility
Extensibility is a systemic measure of the ability to extend a system and the level of effort required to implement the extension
Extensions can be through the addition of new functionality or the modification of an existing functionality
Modularity and loose coupling foster extensibility
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Forward Compatibility
☐ Forward compatibility (at times considered as part of extensibility) aims at the ability of a system to gracefully accept input intended for a later version of the system
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Measuring Extensibility☐ Extensibility can be measured w.r.t. the cost associated with
adding a new functionality to the system
☐ Where:☐ CDNF: Cost of developing the new feature☐ CINF: Cost of integration for the new feature
☐ For any system: 0 < Extensibility ≤ 1
Extensibility = CDNF/(CINF + CDNF)
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Measuring Forward Compatibility
☐ For simplicity we will assume that forward compatibility is binary, thus either is supported or not
☐ In general, many systems support forward compatibility under certain conditions. Yet for our discussion it is OK to consider it as a binary property.
Ope
nSpl
ice
DD
S
Efficiency
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Defining Efficiency
In general terms, efficiency describes the extent to which a valuable resource, such as time, space, network bandwidth, etc., is well used for the intended task of purpose
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
☐ Efficiency can be measured as follows:
☐ Where:☐ CO: Optimal Cost to perform the given task/operation ☐ CA: Actual Cost to perform the given task/action
☐ For any system: 0 < Efficiency ≤ 1
Measuring Efficiency
Efficiency = CO/CA
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Example: Data Encoding☐ As an example of efficiency, let’s consider different data
encoding for a relatively simple Temperature Sensor
☐ Let’s assume a very simple temperature sensor defined as follows:
struct TempSensor { short temp; short hum;};
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Example: Data Encoding
0 23 0 4B
TempSensor t = {35, 75};
{ “ t ebyte 0 1 2 30 1 2 3
m p “ :
3 5 , “
h u m “
: 7 5 }
byte
CDR JSON
4 bytes16 bytes
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Example: Data Encoding☐ Let’s compute now the efficiency. If we ignore variable length
encoding, then the cost of encoding two short integers is 4 bytes, thus CO = 4 bytes
☐ CDR Efficiency. The actual cost of encoding our temperature sensor in CDR was CA = 4 bytes this leads to EfficiencyCDR = 4/4 = 1
☐ JSON Efficiency. The actual cost of encoding our temperature sensor in JSON was CA = 16 bytes this leads to EfficiencyJSON = 4/16 = 0.25
☐ Note: We have taken a few short cuts here since a formally sound derivation of CO should have considered the entropy of the data source. Instead we have silently assumed a binary entropy function.
Ope
nSpl
ice
DD
S
Data Centric Architectures
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Data-Centric Architecture (DCA)Abstract DefinitionA Data Centric Architecture consists of:☐ Abstract Data Types: A, B, C, ... ☐ Arrows: f, g, h, ...where arrows satisfy the following properties:
Given f: A → B, g: B → C, and h: C → D
then the following holds true:
∃ g ∘ f: A → C [Composition]
∃ 1A : A → A [Identity Arrow]
h ∘ (g ∘ f) = (h ∘ g) ∘ f [Associativity]
f ∘ 1A = f = 1B ∘ f [Unit]
A Data Centric Architecture is a Category
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Data-Centric Architecture (DCA)
The abstract definition of a Data-Centric Architecture formally captures its key properties:
☐ The Abstraction Barrier is defined in terms of Abstract Data Types
☐ The System Behaviour is specified in terms of transformations (functions or arrows) over these Abstract Data Types
Note: From this emerges a natural relationship between DCA and Functional Programming Languages
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Example: Space Traveller☐ Suppose we want to create a
distributed game in which we have (some) UFOs that are trying to destroy our spaceship
☐ UFO can be our allies/enemies and can be played by other players or computers
☐ Some UFOs, might just be space travellers and should not be destroyed
☐ The goal of the game is to destroy the UFOs that have been classified as threats
[1/3]
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Example: Space Traveller
Data Types☐ FlyingObject☐ Dynamics☐ Classification☐ Collision
Arrows☐ animate: Dynamics → Dynamics☐ classify: FlyingObject x Dynamics → Classification☐ collide: [FlyingObject x Dynamics] → [Collision]☐ pilot: IO → Dynamics x FlyingObject☐ fire: IO → FlyingObject x Dynamics☐ display: FlyingObject x Dynamics x Classification x Collision
→ IO
☐ The DCA for this game could be described as follows (keeping it simple...)
[2/3]
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Example: Space Traveller
pilot
fire
animate classify
collide display
IODynamics
Dynamics DynamicsDynamics
Classification
Dynamics
Classification[Dynamics][Collision]
Collision
IO
FlyingObject
FlyingObject
IODynamics
FlyingObject
FlyingObject
[FlyingObject]
[3/3]
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Data-Centric vs. Service-CentricData-Centric and Service-Centric architectures, such as SOA, Distributed Objects, etc., differ in several dimensions:
☐ Abstraction Barrier:☐ Data-Centric architectures rely on data abstractions☐ Service-Centric architectures rely on service abstractions
☐ Level of Coupling☐ Data-Centric architectures rely solely on the shared knowledge of abstract data
types (agree on the “what”)☐ Service-Centric architectures rely on the shared knowledge of operational
interfaces along with the knowledge of the service provider and of its existence (agree on “what”, “who”, and “when”)
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DCA Benefits☐ Loose Coupling
☐ Anonymicity (e.g. only things to be known are the data types and not who produces/consumes them)
☐ Composability (especially when combined with functional languages)
☐ Extensibility
☐ Ease of Integration
☐ Performance (easier to parallelize)
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DCA Interoperability
☐ To enable Interoperability DCA should rely on middleware technology that provides a Level 2 Interoperability
☐ In other terms, middleware technologies that define☐ open/standard communication protocols☐ open/standard data representation format, and ☐ open/standard ways of describing a Common Information Model
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DCA Extensibility☐ Extensibility is enabled by DCA through modularity, composability
and loose coupling
☐ Modularity is tied to the granularity of the “arrows/functions” that defined in the system
☐ Composability follows from the properties (refer to definition) of DCA
☐ Loose coupling follows from the fact that the only “contract” between the different element of a system are the abstract data types and nothing else. Neither topological nor existential informations, e.g. it does not matter if there are consumer for a given type T nor how many of those are there and where they are located
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DCA Forward Compatibility
☐ Forward compatibility depends entirely from the technology used to implement the DCA
☐ However, the presence of a typed Common Information Model can facilitate the implementation of efficient and safe “forward compatibility” mechanism
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Efficiency
☐ The efficiency of a DCA depends on ☐ The definition of the Common Information Model (e.g. its level of
normalization), and☐ The efficiency of the infrastructure used to implement the DCA
Ope
nSpl
ice
DD
S
DDS and DCA
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DDS Core Standard
☐ DDS is a family of standards addressing the communication needs of Data Centric Architectures
☐ The two standards at the core are:☐ DDS v1.2 -- API☐ DDSI v2.1 -- Wire Protocol
Ownership Durability Content Subscription
Minimum Profile
Data Centric Publish Subscriber (DCPS)
DDS Interoperability Wire Protocol - DDSI-RTPS
Application
ApplicationUDP/IP
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DDS Standard EcosystemApplication
DDS RMI
Java-5ISO C++ ScalaANSI C
DDS
DDS RMI
X-Ty
pes
DDSI-RTPS
Secu
rity
Secu
rity
DDSI-RTPS
X-Ty
pes
network
2004
2006
2012 2012
20102012 2010 2012
2006
2004 2010 2010 201x
ApplicationAPI
Wire Protocol
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DCAs with DDS
☐ DDS provides first class support for DCAs
☐ DCAs Abstract Data Types are captured in DDS via Topics
☐ DDS does not provide primitive support for specifying DCAs Arrows. These are defined by the architect using her/his favorite formalism / programming language
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DDS Topics☐ A Topic defines a typed data
stream☐ A Topic has associated a data
type and QoS☐ The Topic name, type and QoS
defines the key functional and non-functional invariants
☐ Topics can be discovered or locally defined
DURABILITY,DEADLINE,PRIORITY,…
“net.gva.VehiclePosition”
struct Position2D { long vid; //@Key long x; long y;};
TopicTypeName
QoS
Ope
nSpl
ice
DD
S
DDS & Interoperability
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DDS: Level 2 Interoperability
DDSI Wire Interoperability
Data ModelCommon “Language”
+QoS Requirements
Interoperability
Semantic Interoperability
Syntactic Interoperability
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DDS: Portability
DDS API Standard
Application
Portability
Ope
nSpl
ice
DD
S
DDS: Extensibility & Forward Compatibility
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Extensibility
☐ DDS-based applications feature all the extensibility benefits deriving from DCA
☐ In addition, due to the built-in dynamic discovery, DDS-based Systems are completely decoupled from deployment details
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Forward Compatibility
☐ DDS supports Forward Compatibility, specifically, it allows for both monotonic as well as generalized type extensions
☐ Monotonic type extensions allow old systems to consume new types as far as changes consist of additions to the tail of the type
☐ Generalized type extensions allows attribute reordering and removal
Ope
nSpl
ice
DD
S
Efficiency
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
DDS Efficiency☐ The DDS standard was
designed in such a way to maximize time and space efficiency
☐ OpenSplice DDS has been designed to minimize resource usage while maximizing performance and scalability
Latency☐ 15-20 usec Inter-Core Latency
☐ 75 usec over GBps Ethernet
Throughput☐ Up to 10+M msg/sec inter-core
☐ Up to 5M msg/sec inter node
OpenSplice DDS Performance Metrics
Ope
nSpl
ice
DD
S
Back to the Space Traveller
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Example: Space Traveller
pilot
fire
animate classify
collide display
IODynamics
Dynamics DynamicsDynamics
Classification
Dynamics
Classification[Dynamics][Collision]
Collision
IO
FlyingObject
FlyingObject
IODynamics
FlyingObject
FlyingObject
[FlyingObject]
[3/3]
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Space Traveller Topic Types//@Nestedstruct Bounds { long width; long height;};
struct FlyingObject { long oid; //@Key long kind; Bounds bounds;};
//@Nestedstruct Vector { long x; long y;};
struct Dynamics { long oid; //@Key Vector pos; Vector speed;};
enum Classifier { UNKNOWN, FRIEND, THREAT};
struct Classification { long oid; // @key Classifier kind;};
struct Collision { long oid; long coid;};
Ope
nSpl
ice
DD
S
Real World Interoperability
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
SESAR: The Single Sky
☐ Adopted the DDS standard as the foundation of their DCA
☐ Defined and standardized a domain specific Common Information Model concerning flights and their management
City Service -‐ Architecture
Urbio2ca M
ESH network
Urbio&ca
City Message BUS – Opensplice DDS
Exis2ng system
s
Esper – Park Control
Data warehouse
APPLICATIONSUser, City agent
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
UK Generic Vehicle Architecture
☐ Adopted the DDS standard as the technology at the foundation of next generation military vehicles
☐ Defined a domain specific Common Information Model -- the GVA Information Model
Ope
nSpl
ice
DD
S
Summing Up
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
Concluding Remarks
☐ Interoperability, Extensibility and Efficiency is key for an increasing number of applications
☐ Data Centric Architectures (DCAs) provide an architectural framework that facilitates the design of systems that are interoperable, extensible and efficient
☐ DDS is the middleware infrastructure that most naturally supports DCAs
Ope
nSpl
ice
DD
S
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Ope
nSpl
ice
DD
S
References
¥Fastest growing JVM Language¥Open Source¥www.scala-lang.org
¥ #1 OMG DDS Implementation¥ Open Source¥ www.opensplice.org
OpenSplice | DDS¥Scala API for OpenSplice DDS¥Open Source¥github.com/kydos/escalier
Escalier
¥Simple C++ API for DDS¥Open Source¥github.com/kydos/simd-cxx
¥DDS-PSM-Java for OpenSplice DDS¥Open Source¥github.com/kydos/simd-java
¥ DDS-based Advanced Distributed Algorithms Toolkit
¥Open Source¥github.com/kydos/dada
Ope
nSpl
ice
DD
S ¥@prismtech
¥@acorsaro
¥youtube.com/opensplicetube ¥slideshare.net/angelo.corsaro
¥ [email protected]¥[email protected]
¥opensplice.com ¥forums.opensplice.org
¥opensplice.org ¥[email protected]
:: Connect with Us ::