Active Motion Compensation in Robotic Cardiac Surgery

G. Moustris, A.I. Mantelos, C.S. Tzafestas

Intelligent Robotics & Automation LabNational Technical University of Athens

Theory

• Typical setup in Robotic Surgery

• Tele-operated System

• Surgeon Master

• Robot Slave

• Surgical field seen through endoscopic camera

Active Motion Compensation in Robotic Cardiac Surgery– G. Moustris, AI Mantelos, C. Tzafestas IRAL/NTUA

• Surgical Field Motion

• Some surgical sites contain inherent motion

• Prominent example: heart surgery

• Motion due to:

• cardiac pulsation

• respiration


• Coronary Artery Bypass Graft Surgery (CABGC)

• On-Pump: Heart is arrested. Patient undergoes Cardio-Pulmonary Bypass (heart-lung machine)

• Off-Pump: Heart is beating. Stabilizers (mechanical, vacuum) reduce motion (see picture)



• Restrict motion using cardiac stabilizers

• Can cause haemodynamicdisorders

• Further restricts space & movements

• Conversion to open hurt surgery

• Some patients excluded

• Residual motion


• Motion compensation refers to the apparent cancellation of organ motion

• Motion is captured by an imaging device in real-time, is rectified, and presented to the surgeon as still

• Robot maintains a steady pose with respect to the field, essentially tracking its motion and moving along with it

• Operation is transparent:Surgeon effectively operates on a static image


Mechanical

Synchroniza

tion

Shared

Control

Image

Stabilization

Motion

Compensation

• Consists of three main operations:

•Mechanical Synchronizationrobot tracking the motion of a reference in the surgical field

• Image Stabilizationcapture of an image of the surgical field and rectification

•Shared Controlcombination of the user input and the compensation signal


• Our team has recently presented a motion compensation framework(see “Shared Control for Motion Compensation in Robotic Beating Heart

Surgery”, G.Moustris, A.I. Mantelos, C.Tzafestas, ICRA’13 Karlsruhe Germany )

•Seamlessly integrates all three components

•Allows actual surgical motions to be performed

•Provides a platform to build more complex controls on-top


• We identify three spaces:1. Physical world space Wp

Cartesian space where the robot and surgical field reside

2. Physical image space Ipthe image space of the world space through the camera

3. Canonical image space Icthe rectified image space where the physical image is still

◦ It holds that P: Wp Ip, P in GPL(Wp),

◦Ψ: Ip Ic, Ψ is the image rectification map.


•Introduce the Canonical world space Wc

such that P: Wc Ic . Wc is a Cartesian 3D space containing the stabilized robot and reference manifold. The physical and canonical world spaces are related by the bijective map:

Φ: WcWp

•The image rectification map is then:

◦Ψ: P○ Φ-1○P-1

•Φ is the Strip-Wise Affine Map (SWAM).◦It takes a polygonal line from the physical space, and maps it to the x-axis in the canonical space in a strip-wise fashion

WpWc

Φ

Mp Mc


•Using the Canonical world spacethe control is transferred there, achieving the following:

1. The reference is always a straight line (the canonical x-axis)

2. Image stabilization and motion synchronization combine seamlessly

3. Shared control emerges naturally


•A control layer on-top of motion compensation

•Operates in the canonical (stabilized) space

•Enhances the surgeon’s motions

•(Shared Control)2 !


Physical

Space

•Actual Robot

Canonical

Space

•Motion

Compensation

Active

Assistance

•Tracking task (track the xc-axis)

•pqm: master position physical space

•cqm: master position canonical space

•pqs: slave position physical space

•pqr: slave reference physical space

•cqr: slave reference canonical space

•If cqr = cqm we get pure teleoperation

•If cqr ≠ cqm we can control how cqr

behaves and insert control


yc

pqs

pqr p

qm

Canonical space

xc

cqr

cqm

Wp Wm

Wc

Slave Master

•Make cqm and cq0 attractive only on the yc axis

•X-axis is controlled solely by human

•Apply spring-damper model on virtual mass m located at cqr

•cFT is the total force

•K, B spring, damping coefficients


yc

pqs

pqr p

qm

Canonical space

xc

cqr

cqm

Wp Wm

Wc

cq0

Slave MastercFm

cF0

0

0

( ) ( ),

,

c c c c c

m r m r m

c c c

r r

c c c c

r T m

F K y y B y y

F K y B y

m y F F F

•Drag effect will appear in entire field

•Insert zone ε about x-axis and inner zone δ, δ< ε

•Combine forces as:

•If cym> ε pure teleoperation

•If ε> cym> δ linear combination

•If δ> cym> 0 autonomous mode on yc


yc

pqs

pqr p

qm

Canonical space

ε

xc

cqr

cqm

Wp Wm

Wc

cq0

Slave Master

δ

0( ) 1 ( ) ,

1 , or

( ) / ( ) ,( )

0 ,

( ) / ( ) ,

c c c c c

T m m m

c c

m m

c cmc m

m c

m

c cm m

F y F y F

y y

y yy

y

y y

Dynamics are:

Substitute z1= cyr , z’1= cy’r , then state equations :

Transform the input as,

Final system is:


( )c c c c c c

r r r m m m

K B K By y y y y y

m m m m

1 1

2 2

0 1 0 0

( )

c

cm

mc

m

z z yyK B K B

z z ym m m m

1 2( , ) ( ) ( , )c c c

m m my y y u u

1 1 1

2 2 2

0 1 0 0z z u

K B K Bz z u

m m m m

System is 2nd order linear controllable

Solved iteratively in a control loop using Euler method

Implementation

z

x

y

Canonical Space

Physical Space


UDP

UDP

Camera

USB

Master Controller

Slave ControllerUser ConsoleActive Motion Compensation in Robotic Cardiac Surgery

– G. Moustris, AI Mantelos, C. Tzafestas IRAL/NTUA

•Three main loops running in parallel, distributed across three computers:

1. Servo LoopControls the Slave robot implementing a PID force controller for teleoperation on the first three joints. The loop operates at 1 KHz. PID update rate follows the update rate of the communication loop (100 Hz).

2. Graphics LoopImage acquisition/processing on the user console. Acquires the image from camera, processes it and presents it to the surgeon stabilized. Implemented in MATLAB and OpenCV at a refresh rate of about 30Hz. Resolution reduced to 320x240 grayscale for real-time performance.

3. Communications LoopUses UDP sockets for the communication between master, slave & user console. Update rate of 100 Hz. Transmission is fast, relatively immune to latency, but unreliable since UDP lacks error correction. Simple error correction algorithm was implemented by framing data packets with predefined header.



Experiments

Task: Track a line on a beating heart for 1.5 min

Experiments consist of two groups:


12bmp

No Comp.Simple

Comp.

Assist. with

DZ

Assist. w/o

DZ

15bmp

Assist. with

DZNo Comp.


Statistical results for 12 bmp

aAssistance with Dead Zone, bAssistance W/O Dead Zone, cSimpleCompensation, dNo Compensation, eRelative Difference of consecutive columns


AWDZA AWODZb SCc NCd

mean

(mm)4.487 5.131 6.031 7.171

rel. diff.e -12.57% -14.91% -15.90% -

12 BPM 15 bpm

AWDZ NC AWDZ NC

mean

(mm)4.487 7.171 4.400 8.022

rel. diff. -37.43% - -45.15% -

Aggregate results for the two groups


•Consistent decrease of mean error across the two frequencies

•Residual error of approximately 4.5mm can be attributed to the specific h/w implementation

•Assistance with Dead Zone seems to provide better results

•Better and more complex controllers and behaviors can be implemented

•Initial results are indeed encouraging!

Image www.cepolina.com

Documents

Active Motion Compensation in Robotic Cardiac Surgery