Robotic System Simulation and Modeling · Folie 12 Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011 Motion RSS - Simulation Platform For Workflow Design

Robotic System Simulation and ModelingStefan JörgRobotic and Mechatronic Center

Folie 2Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011

Outline

IntroductionThe SAFROS Robotic System SimulatorRobotic System ModelingConclusions


DLR‘s Mirosurge: A versatile MIRS scenario

Slave Master


3 kg payload; 10 kg weight7 torque-controlled joints

DLR MIROTM


Various control concepts

Telepresent Semi-autonomous Autonomous

DLR MIROTM


Open surgeryMinimally invasive surgery

ButtonShaft

Plate

Magnet

DLR MIROTM


A10

A9A8

Manipulation forces up to 6 N; 0,85 kg weight3 actuated axes; Cardanic joint: ± 40°10 mm diameterIntegrated electronicsSensing of manipulation forces/torques

DLR MICA


DLR‘s Mirosurge: A versatile MIRS scenario


Robotic System Simulator (RSS)(for MIRS)

Motivation

Robot Simulator 1

Har

dwar

e A

bstra

ctio

nLa

yer

Inte

rface

to w

orld

sim

ulat

or

RobotDynamics

Model

Sensor/ActuatorModel

Har

dwar

e A

bstra

ctio

nLa

yer(

HA

L)

Robot Simulator 2

...

Application Control(MIRS)

• Workflow• Safety Checks• Control

Increase patient safety through the accurate simulation of the surgical robotic system.

Goal: Develop a surgical simulator perfectly interchangeable with the real surgical robot


Outline




The SAFROS Robotic System Simulator (RSS)

Robot Simulator 1

Har

dwar

e A

bstra

ctio

nLa

yer

Inte

rface

to w

orld

sim

ulat

or

RobotDynamics

Model


Har

dwar

e A

bstra

ctio

nLa

yer(

HA

L)

Robot Simulator 2

...



Goal: Develop a surgical simulator perfectly interchangeable with the real surgical robot

adaptable to various use casesfocus on real-time simulation withhardware-in-the-loop ( haptics ►1kHz )scaling of simulation effort (layers of detail)accurate dynamics modelsUse application control ofthe real system


Motion

RSS - Simulation Platform For Workflow Design

World Simulator (Patient)

Virt

ualR

ealit

y

VisualRendering

Robot Simulator 1

Har

dwar

e A

bstra

ctio

nLa

yer

Inte

rface

to w

orld

sim

ulat

or

WorldModel

HapticRendering

Forces/Torques

Inte

rface

to s

urge

onco

nsol

ean

d tra

inin

gRobot

DynamicsModel


Har

dwar

e A

bstra

ctio

nLa

yer(

HA

L)Robot Simulator 2

...



WORKFLOW DESIGN


SimulatorDesign Under Test


RSS - Simulation Platform ForWorkflow Design

Relevant to surgical robotic workflow: change the surgical tool


Parameterization

Motion

RSS - Simulation Platform For User Interface Design and Monitoring


Virt

ualR

ealit

y

VisualRendering

Robot Simulator 1

Har

dwar

e A

bstra

ctio

nLa

yer

Inte

rface

to w

orld

sim

ulat

or

WorldModel

HapticRendering

Forces/Torques

User Interface(Surgeon Console)

Real • Omegas • Display• Pedals• GUI• Planning

Forces/Torques

Positions

Monitoring

Inte

rface

to s

urge

onco

nsol

ean

d tra

inin

gRobot

DynamicsModel


Har

dwar

e A

bstra

ctio

nLa

yer(

HA

L)Robot Simulator 2

...



WORKFLOW DESIGN

USER INTERFACE DESIGN and MONITORING


SimulatorDesign Under Test


RSS - Simulation Platform ForUser Interface Design and Monitoring


RSS - Simulation Platform ForUser Interface Design and Monitoring

Compare simulation with reality during operation.

First tele-operated robot in space


Parameterization

Motion

RSS - Simulation Platform For Surgeon Training


Virt

ualR

ealit

y

VisualRendering

Robot Simulator 1

TrainingApplication

Har

dwar

e A

bstra

ctio

nLa

yer

Inte

rface

to w

orld

sim

ulat

or

WorldModel

HapticRendering

Forces/Torques



Forces/torques

Positions

Monitoring

Inte

rface

to s

urge

onco

nsol

ean

d tra

inin

gRobot

DynamicsModel


Har

dwar

e A

bstra

ctio

nLa

yer(

HA

L)Robot Simulator 2

...Surgeon



WORKFLOW DESIGN


TRAINING


SimulatorUser UnderTest



Virt

ualR

ealit

y

VisualRendering

WorldModel

HapticRenderingParameterization

Motion

RSS – Modular Simulation Platform For Various Use Cases

TrainingApplication

Forces/Torques



Forces/torques

Positions

Monitoring

Surgeon

TRAINING


Robot Simulator 1

Har

dwar

e A

bstra

ctio

nLa

yer

Inte

rface

to w

orld

sim

ulat

or

Inte

rface

to s

urge

onco

nsol

ean

d tra

inin

gRobot

DynamicsModel


Har

dwar

e A

bstra

ctio

nLa

yer(

HA

L)Robot Simulator 2

...



WORKFLOW DESIGN





Virt

ualR

ealit

y

VisualRendering

WorldModel

HapticRenderingParameterization

Motion

RSS - Simulation at Different Levels Of Modeling Detail

TrainingApplication

Forces/Torques



Forces/torques

Positions

Monitoring

Surgeon

TRAINING


Robot Simulator 1

Har

dwar

e A

bstra

ctio

nLa

yer

Inte

rface

to w

orld

sim

ulat

or

Inte

rface

to s

urge

onco

nsol

ean

d tra

inin

gRobot

DynamicsModel


Har

dwar

e A

bstra

ctio

nLa

yer(

HA

L)Robot Simulator 2

...



WORKFLOW DESIGN




Layers of modeling details

Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System

High abstraction

High detail

Scaling of effort (Implementation and Computation)Different use cases require different modeling detailsDefine component interfaces for each layer




High abstraction

High detail

models: kinematics, position, motion

abstracts: dynamics (masses and forces)




High abstraction

High detail

models: forces, torques w.r.t. motion, motor ripple, latency, friction, compliance

abstracts: implementation and computing hardwaredetails




High abstraction

High detail

models: implementation details (cycle-true)

abstracts: the real system




High abstraction

High detail

robotic hardware (e.g. MIRO, MICA,…)user interface (e.g. GUI, 3D Display, Omega Device)patient, animal model, phantomLayer 0 requires real-time simulation ( 1kHz )


Implementation Design:Main Components of a MIRS Simulator


db: GeometryDataBase

ibd [block] MIRS Simulator [Layer 3]

user interface:Surgeon Console

robot:Robotic System

Simulator

world:World Simulator

in:UiMotion

out:MotionList

in:MotionList

out:UiMotion

cmd:Commands

monitor:Monitor

monitor:Monitor

3dImage:DVI Link

:Objects

robotGeometry:Objects robotGeometry:Objects

Discrete Event communication shall be implemented asUDP sockets @ 1kHz rate

The two main interfaces are defined for each Layer of modeling detail (Example shows Layer 3:Kinematics)

Implementation Design:Interface Definition for each Layer


ibd [block] Surgeon Console [Adapter Layer 0 to Layer 2]

right:Omega Force

Feedback Device

:Omega DI

:Display:DVI

RealSystemToLayer2:Adapter

left:Omega Force

Feedback Device

:Omega DI

left:Omega DI

right:Omega DI

out:UiMotion

in:UiForcesTorques

cmds:Commands

monitor:Monitor

Example: Adapt Omega Input Devices (Layer 0) to Layer 2

Note: Real hardware binds the virtual simulation time to the physical real time.

Implementation Design:Adapters between Layers


Example: RSS with three MIRO modelsOne robot model for each robotApplication Control connects to the Surgeon ConsoleWorld Interface connects to the World Simulator

Implementation Design:Robotic System Simulator


Outline



Robotic System Modeling

Robot Dynamics Model (RDM)Sensor/Actuator Models

Together, RDMs, Sensor, and Actuator models simulate the robot‘s physical (Layer 2,3) and implemented (Layer 1) behavior.


Motion


Virt

ual R

ealit

y

VisualRendering

Robot Simulator 1H

ardw

are

Abs

tract

ion

Laye

r

Inte

rface

to w

orld

sim

ulat

or WorldModel

HapticRendering

Motion/Torques

Robot Dynamics Model

(kinematics, stiffness, damping, inertia)

Robot Simulator 2

...



Forces/Torques

HA

L Motion/Torques

(measured)

TorquesMotor

current Actuator Model

(friction, ripple)

Sensor Model

(gain, offset,latency)


Robotic System Modeling:Robot Dynamics Model (RDM)

Depending on the use-case different level of detail within a layer:

1. Find the right model2. Develop a calibration procedure to find

the paramters for a specific robot

Degree of Details

Computational Costs,

Implementation Time,

High

LowSingle Body (TCP)Layer 2Dynamics

Finite Elements (robot-structure)

Multi-Body (robot-segments)


Kinematics

Joint and structure stiffnesses/damping

Inertia

Scheme of the MIRO kinematics

Influence of inertia on forces and torques

Stiffness of a structure component

Robotic System Modeling:Robot Dynamics Model (RDM)


Robotic System Modeling:Calibration Methods

Static calibration is performed stepwise reduces complexity

In 3 steps, different subsets of parameters are optimizedStatic robot poses for measurements are plannedThe measurement setup and a calibration tool are planned Steps are repeated to account for their dependencies

[Klodmann et al., Static Calibration of the DLR Medical Robot MIRO, a Flexible Lightweight Robot with Integrated Torque Sensors,IROS 2011]


Robotic System Modeling:Calibration Methods


Conclusions

Dynamic Model of the Robotic System enables Accurate SimulationAccurate Simulation of the Robotic System allows better

Workflow Design User Interface Design Monitoring during operations Training

Increased Patient Saftey

Documents

Robotic System Simulation and Modeling · Folie 12 Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011 Motion RSS - Simulation Platform For Workflow Design