How to Design Distributed Robotic Control Systems

Doug Barker, Senior Robotic Systems Engineer, Energid

Dr. Edwin de Jong, RTI

How to Design Distributed Robotic Control Systems

Moderator:

Curt Schwaderer, OpenSystems Media

Speakers:

Agenda

Housekeeping

Presentation

Questions and Answers

Wrap-up

Distributed Robotic Control using Actin and DDS

Douglas Barker, Ph.D.

Energid Technologieswww.energid.com

Copyright 2016 EnergidTechnologies

Overview

• Energid and Actin Overview

• Actin DDS Capabilities

• Project Examples

Copyright 2016 EnergidTechnologies

Energid develops software for simulation and control of any robotic system.

Actin software is at the core of our business:• Actin SDK• Actin Applications• Integration Services

Energid Technologies

Copyright 2016 EnergidTechnologies

Actin Software

• Multi-robot coordination

• Dynamic collision avoidance

• Singularity avoidance

• Kinematically redundant mechanisms

• Complex kinematic chains

• Global path planning

• Real-time dynamic simulation

• IO and sensor feedback

• Easy integration of new hardware components

• Integration with CAD

• Desktop applications for Windows, Linux, OS X

• Control on VxWorks, Real-Time Linux and RTOS32

• Distributed processing over DDS

Simulation and control software for any robotic system

Copyright 2016 EnergidTechnologies

Actin SoftwareActin is a high level robotics control framework

One column for each robot(executed on separate machine)

Task building blocks

Multi-robot tasks can be created on one machine and executed on many

Copyright 2016 EnergidTechnologies

Actin Use CasesSimulation for Design and Evaluation Control for Dynamic Environments

Copyright 2016 EnergidTechnologies

It was customer driven:

• Many government customers began emphasizing DDS

• Single-robot deployments

• Increasing demand for “remote” capability

• Multi-robot deployments

• One robot controller machine per robot

• Communication between the controllers consists of:• Commands, state synchronization, sensor feedback, hardware status

• Safety and mission critical systems

Learned first hand that developing and maintaining our own communications middleware was going to be cost prohibitive.

Why did Energid adopt DDS?

Copyright 2016 EnergidTechnologies

• There is a trend towards higher-level frameworks

• Joint-level control to end-effector control to path planning

• Traditional socket IPC to DDS

• Message contents and delivery are independent

• Powerful Quality-Of-Service (QOS) options

• Multiple communication channels (unicast, multicast, shared memory)

• Message is the “what”

• QOS determines “if”, “when”, “where”, and “how”

• Redundancy and fail safety

• Allow redundant processes running in parallel

• Respond to critical system events (liveliness, latency, etc)

• Simplify complex communication problems (Security)

• Reduce development time

Why use DDS for robotics?

Copyright 2016 EnergidTechnologies

Actin DDS Capabilities

• Common Architecture

• Distributed Robot Control Architecture

• Hierarchical Robot Control Architecture

Copyright 2016 EnergidTechnologies

Actin DDS Plugin Architecture

• Actin’s Plugin Architecture• Mechanism to add simulation capabilities at runtime

• Custom DDS plugins can convert local robotic control into distributed robotic control

• DDS Common Plugin• Shares DDS entities with custom DDS plugins

• Domain participant factory

• Domain(s)

• Participant

• Type registration

• Topics

• Partition(s)

• Publisher

• Subscriber

ddsCommonPlugin

Custom DDS Plugin 1

Custom DDS Plugin N

.

.

.

Shared DDS Entities

Copyright 2016 EnergidTechnologies

Distributed Robotic Control Architecture

actinViewer (Control)• ddsCommonPlugin• ddsSimSynchronizationPlugin• ddsSensorManagerPlugin• ddsHardwareControlRequesterGUIPlugin

actinRT (Robot 1)• ddsCommonPlugin• ddsSimSynchronizationPlugin• ddsSensorManagerPlugin• ddsHardwareControlProviderPlugin• hardwarePlugin

actinRT (Robot N)• ddsCommonPlugin• ddsSimSynchronizationPlugin• ddsSensorManagerPlugin• ddsHardwareControlProviderPlugin• hardwarePlugin

command primitives (joint frames, end-effector pose, …)

robot states

pub

sub

pub

sub

pub

sub

sub sub

Copyright 2016 EnergidTechnologies

Hierarchical Robotic Control Architecture

actinViewer (Control)• ddsCommonPlugin• ddsTaskPlayerPlugin• ddsDistributedTaskExecPlugin• ddsSimSynchronizationPlugin• ddsSensorManagerPlugin• ddsHardwareControlRequesterGUIPlugin

actinRT (Robot 1)• ddsCommonPlugin• ddsTaskExecPlugin• ddsSimSynchronizationPlugin• ddsSensorManagerPlugin• ddsHardwareControlProviderPlugin• hardwarePlugin

actinRT (Robot N)• ddsCommonPlugin• ddsTaskExecPlugin• ddsSimSynchronizationPlugin• ddsSensorManagerPlugin• ddsHardwareControlProviderPlugin• hardwarePlugin

higher level tasks

robot states and task status

pub

pub

pub

sub sub

sub

subsub

Copyright 2016 EnergidTechnologies

Project Examples and Demos

• Robot Drilling Systems• NASA Rover Proxy Simulation• Universal Robots

Copyright 2016 EnergidTechnologies

Creating a fully autonomous robotic drilling platform

Uses Hierarchical Distributed Tasking• High level tasking from a control machine

Publishes – task command

Subscribes – task status and robot state

• Robots run on separate machinesPublishes – task status and robot state

Subscribes – task command and robot state

Robotic Drilling Systems (RDS)

Copyright 2016 EnergidTechnologies

RDS Tasking Demo

Drill-Floor Robot

Elevator

Pipe Handler

Rough Neck

See videos at www.rds.no

Four robots move simultaneously with dynamic collision avoidance

Copyright 2016 EnergidTechnologies

NASA Rover Proxy Simulation

Network

Scientists

Controllers

Physical Vehicle

Proxy

Virtual

Switch

CORBATCP/IP DLLDirect Link

RTI DDS

Use simulation as a proxy for real hardware

Copyright 2016 EnergidTechnologies

Rover Proxy SimulationRover Tasking SoftwarePublish:

• Rover Commands

Subscribe:• Lidar Scan Data• Camera Data

• Two Hazard Cameras • Pan Tilt Cam• Depth camera

• Rover state

Actin SoftwarePublish:

• Lidar Scan Data• Camera Data

• Two Hazard Cameras • Pan Tilt Cam• Depth camera

• Rover state

Subscribe:• Rover Commands

Copyright 2016 EnergidTechnologies

Two robots and a motion capture system. Models a collaborative human-robot work environment.

• UR10 machine

• Publish: UR3 command and UR10 state

• Subscribe: UR3 state

• UR3 machine

• Publish: UR3 state

• Subscribe: UR3 command and UR10 state

• Optitrack Motion capture machine

• Sends rigid body poses over UDP

Universal Robots Demo

Copyright 2016 EnergidTechnologies

Universal Robots Collision Avoidance Demo

Robots dynamically avoid collisions with themselves and the environment.

Communication of robot joint state and status is time critical.

If one robot protective stops, then the others must know and respond.

Copyright 2016 EnergidTechnologies

Thanks!

Energid Technologies

One Mifflin Place Suite 400

Cambridge, MA 02138

888.547.4100

www.energid.com

How to Design Distributed Robotic Control Systems Using DDSEdwin de Jong, PhD – edwin@rti.com

RTI

©2015 Real-Time Innovations, Inc.

The smart machine era will be the most disruptive in the history of IT-- Gartner 2015

Robotics and IIoT Aligning

The real value is a common architecture that

connects sensor to cloud, interoperates

between vendors, and spans industries

Common technology that spans industries brings bold new approaches and enables fast change

©2015 Real-Time Innovations, Inc.

Space-Proven Data Link

©2015 Real-Time Innovations, Inc.

NASA’s Human-Robotic Systems prototypes robots for extraterrestrial surfaces

Surgical Systems

©2015 Real-Time Innovations, Inc.

• The Minimally Invasive Robotic Surgery (MIRS) system at DLR coordinates three robots to perform delicate heart surgery

• The system closes a distributed loopbetween the robots and the remote surgeon’s control at3kHz

200+ companies strong

Goal: build and prove a common architecture that spans sensor to cloud, interoperates between vendors, and works across industries

The DDS Standard for the IIoT

• The Data Distribution Service (DDS) is the Proven Data Connectivity Standard for the IoT

• OMG: world’s largest systems software standards org– UML, DDS, Industrial Internet

Consortium

• DDS: open & cross-vendor– Open Standard & Open Source– 12+ implementations

Interoperability between source written for different vendors

Interoperability between applications running on different implementations

DDS-RTPS ProtocolReal-Time Publish-Subscribe

Distribution Fabric

DDS API

Peer-To-Peer Plug & Play Databus

©2015 Real-Time Innovations, Inc.

OMG Data Distribution Service (DDS)

Control

Co

mm

and

s

Sensor

Sen

sor

Dat

a

ActuatorSensor

Sen

sor

Dat

a

Display

Data Centric Architecture

©2015 Real-Time Innovations, Inc.

• Data-centric middleware maintains state• Middleware manages the content• CRUD operations on distributed state

PersistenceService

RecordingService

Source(Key)

Pos Dir Accel

R1 (3.1, 7) (-1, 3.7) (0, 0)

R2 (8, 4.5) (0,0) (0,0)

R3 50.2 (5.1, 2) (0.2, 3)

More Robust Systems

©2015 Real-Time Innovations, Inc.

Messaging• Ann: Can you visit on 1/23?• John: Yes• A: 23rd is booked, how

about 2/20?• J: OK• A: March 6th is better…• J: OK• A: Can you stay longer?• J: No; start ½ hour earlier?• A: OK, confirmed!

Data-Centric Pub-Sub

• Add: 1/23 @ 11:30A

• Change: 2/20 @ 11:30A

• Change: 3/6 @ 11:30A

• Change: Add dial-in info

• Change: 3/6 @ 11:00A

J: 2/20

A:3/6

3/611:00

A J

DDS Real-Time Quality of Service

• Highly tunable reliabilityprotocol– Balancing throughput

and latency

– Across wide variety of interconnects

• Time aware– Deadline notifications

– Nanoseconds timestamps

• Historic data for late joiners

• Control over subscribed data– By time

– By content

Sensor-to-Cloud Data Bus

©2015 Real-Time Innovations, Inc.

Unit DataBusUnit DataBus

• Connect…– Fast– Seamless– Secure

• Across many platforms…

• Over any networking technology Intelligent

Machines

IntelligentSystems

IntelligentIndustrial Internet

Cloud DataBus

Site DataBus

IntelligentSystem of Systems

Unit DataBus

Sense Act

Think HMI

Machine DataBus

Think HMI

Machine DataBus

Sense Act

Think HMI

Machine DataBus

Hide

Complex

Topology

behind a

Single

logical

DataBus

Data Centricity Directly Controls Flow

• Global Data Space– Automatic discovery– Read & write data in any OS,

language, transport– Type Aware– Redundant sources/sinks/nets

• No Servers!• QoS control

– Timing, Reliability, Redundancy, Ordering, Filtering

Shared Global Data Space

DDS DataBus

Patient Hx

Device Identity

Surgicalrobot

Sup

ervi

sory

CD

S

Physiologic State

Op

erat

ing

Thea

ter

Cloud

Offer: publish this 3KHz

Reliable

Request: Read this 60 HzIf patient = “Joe”

Best-effort

©2015 Real-Time Innovations, Inc.

Applications of DDS Standard• Over 1,000 IIoT designs

– Robotics– Healthcare– Automotive– Communications– Energy– Industrial– Defense

• 15+ Standards & Consortia Efforts– Interoperability– Multi-vendor ecosystems

©2016 Real-Time Innovations, Inc.

©2015 Real-Time Innovations, Inc.

ROS 2 - Built on DDS

+ ROS usability

less time

spent heremeans

more time to

spent here

Data Security

©2015 Real-Time Innovations, Inc.

Security Example

39

Data Item Authentication Access Control

Integrity Non-repudiation

Confidentiality

Device diagnosticdata

X X

Remote commands

X X X X

Patient Data X X X X

Limitations of Transport Layer Security

TCP/IP Capable Network

NativeDDS App

DDS Library

NativeDDS APP

DDS Library

Secure Transport Secure Transport

SSL, TLS or DTLS

NativeDDS APP

DDS Library

Secure Transport

• No multicast

• Poor latency/jitter

• Robust networks only

• Reliable delivery only

• Data and headers always encrypted

• Gross level security

Practical Fine-Grain Security

• Per-Topic Security– Control r,w access for each function– Ensures proper dataflow operation

• Complete Protection– Discovery authentication– Data-centric access control– Cryptography– Tagging & logging– Non-repudiation– Secure multicast– 100% standards compliant

• No code changes!• Plugin architecture for advanced uses

© 2015 Real-Time Innovations, Inc.

CBM AnalysisPMU Control Operator

State Alarms SetPoint

Topic Security model:• PMU: State(w)• CBM: State(r); Alarms(w)• Control: State(r), SetPoint(w)• Operator: *(r), Setpoint(w)

New Secure DDS Standard

• Per topic security

• Complete protection

– Data-centric access control

– Cryptography

– Secure multicast

– Standards compliant

• No code changes!

Topic security model:

• PMU: State (w)

• CBM: State(r); Alarms(w)

• Control: State(r), SetPoint(w)

• Operator: *(r), SetPoint(w)

©2015 Real-Time Innovations, Inc.

The Network Is The Robot

For More Information

• DDS and RTI: www.rti.com

• Building robotics and haptic systems with DDS:www.rti.com/industries/robotics.html

• Get started with DDS for free: www.rti.com/downloads

Audience Q & ADoug Barker,

Senior Robotic Systems Engineer,

Energid

Dr. Edwin de Jong,

RTI

Thanks for joining us

Event archive available at:

http://ecast.opensystemsmedia.com/

E-mail us at: jgilmore@opensystemsmedia.com