10
ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics and haptics M Hayashibe, N Suzuki, M Hashizume, Y Kakeji, K Konishi, S Suzuki, A Hattori M Hayashibe, N Suzuki, S Suzuki and A Hattori Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, Japan M Hashizume, Y Kakeji and K Konishi Center for Integration of Advanced Medicine and Innovative Technology, Kyushu University Hospital, Japan Correspondence to: M Hayashibe, E-mail: [email protected] Abstract Recently, some useful robotic surgical systems have been developed and applied in many surgical situations. Systems such as the da Vinci TM surgical system of Intuitive Surgical Inc., which facilitates minimally invasive surgery with increased dexterity, are commercially available. Preoperative simulation and planning of surgical robot setups should accompany advanced robotic surgery if their advantages are to be further pursued. Feedback from the planning system will play an essential role in computer-aided robotic surgery in addition to preoperative detailed geometric information from patient CT/MRI images. Surgical robot setup simulation systems for appropriate trocar site placement have been developed especially for abdominal surgery. The motion of the surgical robot can be simulated and rehearsed with kinematic constraints at the trocar site, and the inverse-kinematics of the robot. Results from simulation using clinical patient data verify the effectiveness of the proposed system. Keywords: Robotic surgery, preoperative setup planning, VR training, inverse-kinematics Paper accepted: 7 November 2004 Published online: 15 January 2005. Available from: www.roboticpublications.com DOI: 10.1581/mrcas.2005.010208 INTRODUCTION The utility of laparoscopic surgical robots has been reported (1–4) , especially in the field of cardiac surgery, in Europe and America since 1998. The use of surgical robots has been reported in over 600 cases worldwide up to May 2000 in digestive organ surgery, obstetrics and gynecology, and cardiac surgery. Good and safe operative results, have been verified through those experiences (5–8) . The merits of introducing surgical robots into operating rooms have included the facts that safe and fine-scale operations can be conducted, trauma is reduced, recovery time is shortened and there is little doctor fatigue from prolonged operations. In the operating room, surgeons must prepare and set up an environ- ment where the robot can have adequate degrees of freedom of motion so that the functions of the robot can be fully performed for each clinical case. As with general laparoscopic surgery, surgeons decide the trocar site placement for approaching the abdominal cavity and insert surgical instruments through a hole of approximately 1 cm diameter in the abdominal wall. In general, two trocar sites for forceps, one for the endoscope and one auxiliary port are chosen and set. The optimal set-up will vary with the type of intervention and the equipment used in the operation. The development of a preoperative planning system for robotic surgery will bring the following benefits: 1. In addition to conventional surgical planning in which doctors determine how to carry out the operation by observing the patient’s CT images, the actual placement of the surgical robot and the procedure for approaching the tumor in a geometrical sense should also be discussed. If the 76 Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85 E 2005 Robotic Publications Ltd. www.roboticpublications.com

ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

Embed Size (px)

Citation preview

Page 1: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

ORIGINAL ARTICLE

Preoperative planning system for surgical robotics setupwith kinematics and haptics

M Hayashibe, N Suzuki, M Hashizume, Y Kakeji, K Konishi, S Suzuki, A Hattori

M Hayashibe, N Suzuki, S Suzuki and A HattoriInstitute for High Dimensional Medical Imaging, The Jikei University School of Medicine, JapanM Hashizume, Y Kakeji and K KonishiCenter for Integration of Advanced Medicine and Innovative Technology, Kyushu University Hospital, JapanCorrespondence to: M Hayashibe, E-mail: [email protected]

AbstractRecently, some useful robotic surgical systems have been developed and applied in many surgical situations.Systems such as the da VinciTM surgical system of Intuitive Surgical Inc., which facilitates minimally invasivesurgery with increased dexterity, are commercially available. Preoperative simulation and planning of surgicalrobot setups should accompany advanced robotic surgery if their advantages are to be further pursued.Feedback from the planning system will play an essential role in computer-aided robotic surgery in additionto preoperative detailed geometric information from patient CT/MRI images. Surgical robot setupsimulation systems for appropriate trocar site placement have been developed especially for abdominalsurgery. The motion of the surgical robot can be simulated and rehearsed with kinematic constraints at thetrocar site, and the inverse-kinematics of the robot. Results from simulation using clinical patient data verifythe effectiveness of the proposed system.

Keywords: Robotic surgery, preoperative setup planning, VR training, inverse-kinematics

Paper accepted: 7 November 2004

Published online: 15 January 2005. Available from: www.roboticpublications.com

DOI: 10.1581/mrcas.2005.010208

INTRODUCTIONThe utility of laparoscopic surgical robots has beenreported (1–4), especially in the field of cardiacsurgery, in Europe and America since 1998. The useof surgical robots has been reported in over 600 casesworldwide up to May 2000 in digestive organsurgery, obstetrics and gynecology, and cardiacsurgery. Good and safe operative results, have beenverified through those experiences (5–8). The meritsof introducing surgical robots into operating roomshave included the facts that safe and fine-scaleoperations can be conducted, trauma is reduced,recovery time is shortened and there is little doctorfatigue from prolonged operations. In the operatingroom, surgeons must prepare and set up an environ-ment where the robot can have adequate degrees offreedom of motion so that the functions of the robotcan be fully performed for each clinical case. As with

general laparoscopic surgery, surgeons decide thetrocar site placement for approaching the abdominalcavity and insert surgical instruments through a holeof approximately 1 cm diameter in the abdominalwall. In general, two trocar sites for forceps, one forthe endoscope and one auxiliary port are chosen andset. The optimal set-up will vary with the type ofintervention and the equipment used in theoperation. The development of a preoperativeplanning system for robotic surgery will bring thefollowing benefits:

1. In addition to conventional surgical planning inwhich doctors determine how to carry out theoperation by observing the patient’s CT images,the actual placement of the surgical robot andthe procedure for approaching the tumor in ageometrical sense should also be discussed. If the

76

Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85 E2005 Robotic Publications Ltd.

www.roboticpublications.com

Page 2: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

geometrical structure of the individual patientand the surgical robot are integrated in virtualspace, surgeons and assistants can confirm themovability of the robot in the operating room.

2. Appropriate placement of the trocar portsmaximizes the movable range of the robot,and significantly influences the success of theoperation. Moreover, the region which therobot can reach and in which it can bemanoeuvred is constrained by the fixed pointsof the trocar sites. Therefore, in each case, thesesites must be carefully chosen for the particularpatient. If a preoperative planning system isavailable, robot positioning which maintains alarge distance from all surrounding obstacles isrealized and the burden to the patient byreopening trocar sites can be avoided.

3. If surgeons, assistants and nurses rehearse byusing a virtual training system, they can developan intuitively common perception about howto set up the robot in the operating room andthe actual setup time in the operating room willbe reduced.

4. The planning system can be effective as aneducational program for doctors who areinexperienced in robotic surgery.

In order to better utilize the patient structureinformation not only for diagnosis but also formedical treatment, some research on surgical simu-lations for operational planning and training hasbeen studied using virtual reality techniques (9–12).As for applications to robotic surgery, some researchgroups have developed a simulation system and animage guided navigation system (13, 14). Optimizedport-placement planning for cardiac surgery wasaddressed in Adhami (13) where a mathematicalalgorithm was implemented. A Data-Fusion systemin which an inner structure virtual model issuperimposed onto the live laparoscopic imageaccording to the direction of scope was developedin Hattori (14).

In this paper, we report on the development of asurgical robot setup simulation system for appro-priate trocar site placement for abdominal surgery.The motion of the surgical robot is simulated andrehearsed with computation of collisions betweenthe robot arms, constraints at the trocar site, and theinverse-kinematics of the robot. Being integratedwith a haptic interface, surgeons can push and drag

the arms of the virtual surgical robot which has thesame kinematics as the real one. In this interactiveplanning system, medical staff can discuss andvalidate the planned ports. Additionally, they canhave a common perception ahead of the interven-tion; and as an educational tool, surgeons can learnand experience how to set up surgical robots forlaparoscopy. The setup planning of the da Vincisystem was targeted in this research as representativeof commercially available surgical robots.

METHODGeometric modeling of da Vinci manipulatorThe slave robot of the da Vinci system consists oftwo forceps manipulators and one laparoscopeholding arm. EndowristTM instruments located atthe tip of the forceps provide a full range of motionand ability allowing the instruments to rotatethrough more than 360 degrees through tinyincisions. Forceps can open and shut in a grippingmode and have two wrist joints in the abdominalcavity. They have a total of three degrees of freedomaccording to the revolution around their axes. AnInSiteTM vision system with a high resolution 3Dendoscope provides 3D images of the operativefield. Two optical lenses are installed in the tip ofone scope to provide a stereoscopic view.

Each link shape and distances between links of theda Vinci were measured in detail, and a geometricmodel was then created and reconstructed in 3DCAD software as shown in Figure 1. Figure 1(a)shows a view of the whole 3D CAD model andFigure 1(b) shows the model of the two forcepsmanipulators and the scope manipulator. Each linkis described with its mechanical information includ-ing the child-parent relationship based on Denavit-Hartenberg notation. Figure 2 shows the linkmechanism of the da Vinci manipulator holdingforceps. The corresponding link parameters aresummarized in Table 1. Links 1 and 10 are prismaticjoints and links 2,6 and 11 are revolute joints. Aparallel linkage mechanism is formed with links7,9. This axis of rotation crosses the revolute axisof link 6 at right angles. Therefore, forceps held bythis arm are able to make a pivoting motion aroundthe fixed point on the abdominal wall. If each jointangle is hierarchically applied from the root link ofthe robot, the position and direction of the endeffector will be determined. As the end effectors ofthe da Vinci, modeling of long-tip forceps, cautery

Preoperative planning system for surgical robotics setup 77

E2005 Robotic Publications Ltd. Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85

www.roboticpublications.com

Page 3: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

with spatula, cadiere forceps and round tip scissorshave been designated as the necessary tools forlaparoscopic cholecystectomy. Some of the forcepsmodels available in this planning system are depictedin Figure 3. Computer graphic models are depictedon the left side and the corresponding pictures of thereal things are shown on the right side.

(a)

(b)

Figure 1 Reconstructed 3D CAD model of the da Vincisystem. (a) whole view (b) forceps manipulator and scopemanipulator.

Figure 2 Link mechanism of the da Vinci manipulator.

Table 1 Link parameters of the da Vinci manipulator

Figure 3 End effectors of the da Vinci forcepsreconstructed in CAD.

78 Hayashibe, Suzuki, Hashizume, Kakeji, Konishi, Suzuki, Hattori

Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85 E2005 Robotic Publications Ltd.

www.roboticpublications.com

Page 4: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

Camera calibration of the da Vinci laparoscopeThe laparoscopic image of a virtual da Vinci modelshould give the same view as the actual laparoscopicimage. The right and left camera parameters ofthe da Vinci laparoscope have been calibrated usingactual image information. A checkerboard wasmeasured with the laparoscope as shown in Figure 4and the synchronized images of the stereovisionwere captured for 16 frames to acquire the scenes ofdifferently oriented checkerboards. Left and rightinternal camera parameters and external parametersbetween the left and right camera coordinates couldbe computed. Details of the calibration methodare given elsewhere (15, 16) and consequently theexplanation here has been minimized. A pin-holecamera model is assumed and the relationshipbetween the 2D position m in camera imagecoordinates and 3D position of this point M inworld coordinates are represented as follows:

where

s denotes the scale factor and (R,t) is the externalparameter between the world and camera coordi-nates, A implies the matrix of the internal parameterincluding the optical centroid and focal distance. Pis called the projection matrix. This projectionmatrix can be calculated by least squares estimationusing the known pairs of m and M as follows:

If there are n images of m reference points in theplane,

where m̂ (Pi, Mj) denotes the re-projection point ofith image’s reference point Mj into the cameracoordinates. If this point nearly equals the actualcorresponding point in the camera image, theprojection matrix can be evaluated as correctlyestimated.

As to lens distortion, the radial distortionmodel (17) was used. Once the projection matrixhas been calculated by the above least-squares

method without lens distortion consideration, theevaluation function is re-minimized introducingthe distortion-corrected image points. To confirmthe precision of the identification, the correspon-dences of the edges of a checkerboard that weredetected from the stereo pictures and 3D edgepoints were reconstructed from the calibratedparameters. The edge points in the virtual spacewere viewed at the same angle and with the samefocal length as the left eye of the da Vinci. Whetheror not identification of the parameter had beenperformed correctly was checked by superimposi-tion with the actual laparoscopic image shownin Figure 5. Figure 5 (a) and (b) are the edgereconstruction results from different positions of thecheckerboard. The average error between thecomputed edge point and the actual edge portionin the image was 1.8 pixels. The cause of the error isconsidered to be a small parallax of the da Vincistereoscope. However, the precision is high enoughto use this parameter for the virtual view for the daVinci setup planning.

Creation of typical patient structure modelThis system is aimed at the examination of theoperational plan for approaching, in advance, theaffected part and as an educational program fordoctors and staff inexperienced in robotic surgery.Therefore, a typical patient structure model forlaparoscopic surgery is required for the virtualrehearsal of the robotic surgery setup. A wholebody structure of a typical Japanese female wasreconstructed from 400 slices of MRI scans at 4 mm

Figure 4 Appearance of laparoscope calibration of daVinci.

Preoperative planning system for surgical robotics setup 79

E2005 Robotic Publications Ltd. Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85

www.roboticpublications.com

Page 5: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

intervals. A geometric model of a raised abdomenwith CO2 insufflation was created from this typical3D patient model by our previously developedgeometry edit program using the PHANTOMTM

haptic interface (SensAble Technologies, USA,www.sensable.com). For gastroenterological surgery,there are patient models of the stomach, liver,intestines, and kidney along with texture informa-tion. This typical model can be used for generalplanning and in educational trials. Indeed, for clinicaluse and fine planning, the preoperative CT/MRIimage data for each patient should be segmented.Suitable acquisitions have therefore to be performedon each patient. In the final section, we show theresult of this simulation using the preoperativemedical image of a clinical case. Figure 6 (a) showsa model of a surface-rendering image of the wholebody of a typical patient and Figure 6 (b) is thecorresponding model for laparoscopic surgery withinsufflation of the abdominal cavity.

System configurationThe purpose of this system is to develop a platformthat could perform the setup simulation for roboticsurgery in virtual space. The configuration of thissystem is shown in Figure 7. We adapted thePHANTOM interface to handle and set the roboticarms against the patient structure model in the VRenvironment. An intuitive surgical robot setupsimulation was enabled with haptic sensation. Inorder to realize the platform, this system consisted ofthree parts: haptics, kinematics and graphic modules.As to the haptics interface, GHOSTTM ofPHANTOM SDK was used to implement theinteraction with the virtual da Vinci model andvirtual patient. The main process is divided into

(a)

(b)

Figure 5 Superimposition of 3D reconstructed checkerboard and video image.

Figure 6 (a) Typical 3D patient model (b) withinsufflation of the abdominal cavity.

Figure 7 System Configuration.

80 Hayashibe, Suzuki, Hashizume, Kakeji, Konishi, Suzuki, Hattori

Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85 E2005 Robotic Publications Ltd.

www.roboticpublications.com

Page 6: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

graphics and haptic pipelines as illustrated inFigure 8. Callback functions allow the synchroniza-tion of object information contained in the hapticand graphic loops. When the application is ready torender a new scene it issues an update graphics callthat queried the states’ nodes with graphics call-backs. One such node is gstPHANTOM, whichchanges state if the user moves the interface handle.Thus, the user can touch the surface of the patientand the da Vinci model.

The positioning of the da Vinci manipulator armis performed in two stages like the actual system.First, each joint angle from link 5 to the rooted link1 illustrated in Figure 2 is positioned, grabbing eachconnected arm. gstDynamic nodes are used togenerate the motion when the link is pushed bythe PHANToM interface. When they change statedue to interaction with one of their children, agraphic callback function is activated. The inertialforce and moment, at a rate proportional to themass of each link, is provided as a reaction forceto help create a physically intuitive operation.Concern about the link from link 6 to a tip, the

corresponding motion of connected links is gener-ated by inverse-kinematics computation when thetip of the forceps is grabbed and dragged in 3D spaceas in Figure 9. With this interactive interface,surgeons can rehearse the intervention for pre-operative planning to enable an optimal device setupin the operating room.

The procedure for setup planning in this system issummarized as follows:

1. position the whole da Vinci model against thepatient model.

2. position the arm holding the laparoscope to geta view of the target organ.

3. guide the arm holding the forceps by viewingthe simulated laparoscopic image.

4. confirm the movable range of the manipulator ifthe trocar site placement is decided.

5. reposition the arm holding the forceps so as notto collide with anything within the moveablerange of the manipulator.

Once the trocar site has been determined, themanipulator makes a pivoting motion around theincision point. The arm’s movable range, whichdepends on the fixed point, is shown in the planningwindow. During the simulation, if the armapproaches the border of the movable range, awarning is provided as an unrecommended opera-tion. Setup planning can be conducted followingthe same procedure as the actual way in which anendoscope is first inserted and then forceps areguided with reference to the laparoscopic image.

Figure 8 Process overview.

Figure 9 Motion of da Vinci manipulator solved byinverse-kinematics. Figure 10 Simulated laparoscopic image.

Preoperative planning system for surgical robotics setup 81

E2005 Robotic Publications Ltd. Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85

www.roboticpublications.com

Page 7: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

Figure 10 shows the simulated laparoscopic image.This system is able to provide stereoscopic visionusing liquid crystal shutter glasses and quad buffer-ing. The specification of the PC is shown in Table 2.

RESULTSPreoperative planning for laparoscopiccholecystectomyLaparoscopic cholecystectomy is an operation forgallbladder stones or polyp, in which the gallbladderis excised and removed under a laparoscope,

without carrying out a big incision in the abdomenas was done previously. It can be called a typicallaproscopic operative procedure. The initial pose ofthe surgical robot and the incision site of the robotarm in laparoscopic cholecystectomy have beendiscussed and planned in this developed system.Figure 12 shows the appearance of a da Vinci setupsimulation, which was operated by Dr. Hashizumeat Kyushu University Hospital. Figure 11 shows theappearance of the actual setup of the da Vinci systemfor laparoscopic cholecystectomy. Here, setup is doneusing a gypsum phantom produced from a liverspecimen preserved in formalin.

The arm holding the forceps and the laparoscopecamera arm of the da Vinci system could be guidedby referring to the entire and camera views of thepatient model as in Figure 13. Surgeons proficient inrobotic surgery have tested and evaluated thissystem. Qualitatively, the general opinion was thatthe robot motion was smooth and very similar tothe actual robot, and that this system would be veryuseful as a discussion and educational tool forrobotic setup. Moreover, there was the indicationthat although to some extent precise structure data isused, the rendering delay is not a cause for concern.The number of polygons of the 3D patient model isapproximately 110,000. The frame rate for thissystem is 38 fps with the PC in Table 2. If the sub-window of the virtual laparoscopic image isrendered, the frame rate reduces to 25 fps. As tothe movable range presentation of arms, there wasthe evaluation that it is convenient to checkwhether or not the mutual manipulator is too closeto interfere.

Table 2 PC specification

PC DELL Precision 650

CPU Dual Xeon Processor 2.8 GHz

Bus Speed 533 MHz

Main Memory 2 GByte RAM

Video Card nVidia Quadro FX1000

Video Memory 128 MByte SDRAM

OS WindowsXP

Graphics Library OpenGL 1.2

Figure 12 Appearance of setup simulation in this system(at Kyushu University Hospital, Japan).

Figure 11 Laparoscopic cholecystectomy setup withactual equipment. (a) whole system (b) forceps andcamera.

82 Hayashibe, Suzuki, Hashizume, Kakeji, Konishi, Suzuki, Hattori

Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85 E2005 Robotic Publications Ltd.

www.roboticpublications.com

Page 8: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

Evaluation of positioning errorIt is important for everyone to be able to correctlyoperate this platform for robotic surgery setupsimulation, to ensure the flexibility and reproduci-bility of the system. In order to check the influenceof error and individual differences of positioning,the simple task was set of guiding a forceps tip to thecenter of the gallbladder and performed about 10times each by two doctors and two engineers. Thepositioning error was calculated. Moreover, thesame task was also performed with stereovision.Figure 14 is the result where subjects A and B aredoctors and subjects C and D are engineers. Theaverage error and standard deviation between theguided forceps tip position and the center of thegallbladder were graphed for ten trials. Theindividual difference was approximately 4 mmmaximum without stereovision and 2.5 mm max-imum with stereovision. Moreover, the operationalerror in the case of stereovision was smaller for eachsubject compared with the case of no-stereovision.The maximum absolute error was 7 mm withoutstereovision and 4.5 mm with it. Although someindividual differences and errors can be seen, it isconsidered to be within the tolerance level as a setupsimulation. However, if the objects are smallinternal organs, operation with stereovision wouldbe better.

Simulation result with clinical exampleFor clinical use and detailed planning, the pre-operative CT/MRI image data of each patientshould be segmented. Suitable acquisitions musttherefore be performed on each patient. In thissection, we show the result of a simulation using aclinical example. The medical images of the patient,who was actually operated on with a da Vincisystem, were used and processed in this system forclinical evaluation. This patient was scanned usingMRCP (Magnetic Resonance Cholangiopan-creatography). In MRCP, gallbladder, bile ductand hepatic duct can be visualized and segmentedwithout contrast enhancement. The region of theliver was also reconstructed into 3D surface data. Inclinical cases, only the region of the abdomen isscanned as patient data. Therefore, we conducted anICP registration (18) to replace the liver model ofa typical whole body model with the particularpatient liver model. Figure 15 depicts the results ofthe laparoscopic cholecystectomy setup simulation.The triangle shows the positional relationship of thetwo forceps ports and the camera port. Here, thedistance from the right forceps port to the leftforceps port was 219.6mm, the distance of the rightforceps port to the camera port was 114.7mm andthe distance of the left forceps port to the cameraport was 115.1mm. We could confirm the feasibility

Figure 13 Surgical robot setup simulation for laparoscopic cholecystectomy. Laparoscopic camera view is depicted inthe sub-window. Each arm’s movable range, which depends on the fixed point, is shown. (a) top view (b) obliqueperspective view.

Preoperative planning system for surgical robotics setup 83

E2005 Robotic Publications Ltd. Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85

www.roboticpublications.com

Page 9: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

of the surgical robot setup simulation which reflectseach patient’s structure information by using pre-operatively scanned images.

CONCLUSIONThe conclusions of this paper are summarized asfollows:

1. A surgical robot setup simulation system forabdominal surgery has been developed. Themotion of the surgical robot could be simulatedand rehearsed preoperatively with the kinematicconstraints at the trocar site, and the inverse-kinematics of the surgical robot. Being inte-grated with a haptic interface, surgeons couldpush and drag the arms of the virtual surgicalrobot in a manner that has kinematics consistentwith the real robot.

2. Simulation experiments using clinical patientdata verified the functionality and performance.A good evaluation by surgeons was obtained forthe program by which the procedure adaptedto an actual operational robot setup can bepracticed.

A statistical evaluation and verification of thissystem should be conducted by increasing theapplied clinical examples. In our group, thisplatform for surgical robots is being extended, alongwith the development of a surgical simulator whichcan provide a tele-training program and practiceprogram for improved operative techniques for theda Vinci (19). Moreover, the setup simulation systemis being developed in another typical robot opera-tion system ZEUS originally from ComputerMotion Inc., USA. ZEUS is a system which consistsof independent robots with three bodies unlike theda Vinci. Since it is a matter of opinion whether ithas more setup flexibility, operational trial and errorare needed to compare it with the da Vinci system.In this way the impact of preoperative planningwould be even greater.

REFERENCES1 Guthart GS, Salisbury JK. The Intuitive Telesurgery System.

Proceedings of the IEEE International Conference onRobotics and Automation. 2000:618–21.

2 Salisbury JK. The heart of microsurgery. MechanicalEngineering Magazine ASME Int’l. 1998;120(12):47–51.

3 Reichenspurner H, Damiano R, Mack M, Boehm M, MeiserDB, Elgass R, et al. Use of the voice-controlled andcomputer-assisted surgical system zeus for endoscopiccoronary artery bypass grafting. J Thorac Cardiovasc Surg.1999;118(1):11–16.

4 Wang YF, Uecker DR, Lee C, Wang Y. A speech-directedmulti-modal man-machine interface for robotically enhancedsurgery. Proceedings of the First International Symposium on

Figure 14 Positioning error of tip of forceps. (above)without stereovision (bottom) with stereovision.

Figure 15 Surgical robot setup simulation using aclinical example. Patient surface model of gallbladder, bileduct, hepatic duct and liver is used for the simulation ofcholecystectomy.

84 Hayashibe, Suzuki, Hashizume, Kakeji, Konishi, Suzuki, Hattori

Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85 E2005 Robotic Publications Ltd.

www.roboticpublications.com

Page 10: ORIGINAL ARTICLE Preoperative planning system for surgical ...hayashibe/076_MRJ010208.pdf · ORIGINAL ARTICLE Preoperative planning system for surgical robotics setup with kinematics

Medical Robotics and Computer AssistedSurgery.1994:176–83.

5 Shennib H, Bastawisy A, Mack MJ, Moll FH. Computer-assisted telemanipulation: An enabling technology forendoscopic coronary artery bypass. Ann thorac Surg.1998;66:1060–3.doi:10.1016/S0003-4975(98)00659-6

6 Kavoussi LR, Moore RG, Adams JB, Partin AW. Comparisonof robotic versus human laparoscopic camera control. J Urol.1995;154:2134–6.doi:10.1097/00005392-199512000-00048

7 Marescaux J, Smith MK, Folscher D, Jamali F, Malassagne B.Telerobotic laparoscopic cholecystectomy: Initial clinicalexperience with 25 patients. Annals of Surgery.2001;234(1):1–7.doi:10.1097/00000658-200107000-00001

8 Margossian H, Garcia-Ruiz A, Falcone T, Goldberg J, AttaranM, Gagner M. Robotically assisted laparoscopicmicrosurgical uterine horn anastomosis. Fertil Steril.1998;80:530–4.doi:10.1016/S0015-0282(98)00196-4

9 Robb RA, Hanson DP Camp JJ. Computer-aided surgeryplanning and rehearsalat Mayo Clinic. Computer.1996;29:39–47.doi:10.1109/2.481435

10 Berkley J, Weghorst S, Gladstone H, Raugi G, Berg D,Ganter M. Fast Finite Element Modeling for SurgicalSimulation. MMVR7. 1999:55–61.

11 Cotin S, Delingette H, Ayache N. Real-time elasticdeformations of soft tissues for surgery simulation. IEEEtransactions on visualization and computer graphics.1999;5(1):62–73.doi:10.1109/2945.764872

12 Suzuki S, Suzuki N, Hattori A, Uchiyama A, Kobayashi A.Sphere-Filled Organ Model for Virtual Surgery System.IEEE transactions on medical imaging. 2004;23(6):714–22.doi:10.1109/TMI.2004.826947

13 Adhami L, Coste-Maniere E. A versatile system for computerintegrated mini-invasive robotic surgery. Lecture Notesin Computer Science 2488: MICCAI2002. 2002:272–281.

14 Hattori A, Suzuki N, Hashizume M, Akahoshi T, Konishi K,Yamaguchi S, et al. A robotic surgery system (da Vinci) withimage guided function System architecture andcholecystectomy application. Medicine Meets Virtual Reality11, IOS Press.2003:110–16.

15 Zhang Z. A flexible new technique for camera calibration.IEEE Transactions on Pattern Analysis and MachineIntelligence. 2000;22(11):1330–4.doi:10.1109/34.888718

16 Tsai RY. A versatile camera calibration technique for highaccuracy 3D machine vision metrology using off-the-shelf TVcameras and lenses. IEEE J. Robotics Automat. 1987;RA-3(4):323–44.

17 Brown DC. Lens Distortion for Close-RangePhotogrammetry. Photometric Engineering.1971;37(8):855–66.

18 Besl P, McKay N. A Method for Registration of 3-D Shapes.IEEE Transactions on Pattern Analysis and MachineIntelligence. 1992;PAMI-14(2):239–56.doi:10.1109/34.121791

19 Suzuki S, Suzuki N, Hashizume M, Kakeji Y, Konishi K, HattoriA, et al. Tele-training simulation for the surgical robot system‘‘da Vinci’’. Proceedings of Computer Assisted Radiologyand Surgery. 2004:86–91.

Preoperative planning system for surgical robotics setup 85

E2005 Robotic Publications Ltd. Int J Medical Robotics and Computer Assisted Surgery 2005;1(2):76–85

www.roboticpublications.com