OaCTrMpGewRcCwct
Opispdwrttfra
FbtUpDC
H1
2
ADULT UROLOGY
7
DESIGN OF FUNCTIONAL SIMULATION OF RENAL CANCERIN VIRTUAL REALITY ENVIRONMENTS
BODO E. KNUDSEN, GORD CAMPBELL, ANDREW KENNEDY, JUSTIN AMANN, DARREN T. BEIKO,JAMES D. WATTERSON, BEN H. CHEW, JOHN D. DENSTEDT, and STEPHEN E. PAUTLER
ABSTRACTbjectives. The preoperative planning of partial nephrectomy can be facilitated by the ability to view the tumornd surrounding tissue in three-dimensional (3D) virtual reality (VR). A technique to convert Digital Imaging andommunications in Medicine computed tomography scan data into a fully 3D VR environment was developed.he model can be transferred to a personal computer, allowing the surgeon to view the 3D model in the operatingoom.ethods. Computed tomography data from a patient with multifocal renal masses was converted into a 3D
olygonal mesh using Amira running on a desktop personal computer with Windows XP Professional. A Siliconraphics Monster Onyx2 running the Linux operating system was used to view the 3D stereo model in the VRnvironments: either the CAVE or a specialized desk called the Immersadesk. An application to view and interactith the model on a desktop personal computer was written in C��.esults. A 3D model of the kidney, the multiple tumors, and the associated systems was created. The modelould be viewed and manipulated in a true VR environment and on a desktop personal computer.onclusions. This project completed two major goals. First, a 3D model of a kidney containing multiple massesas created and viewed in a VR environment. Second, an interface to display the model on a desktop personalomputer in the operating room was created. This is the first step in bringing VR technology to the operating roomo assist the surgeon directly. UROLOGY 66: 732–735, 2005. © 2005 Elsevier Inc.
Fsihddftwo
mirssosuvct
pen partial nephrectomy has been shown to pro-vide effective, long-term tumor control, with
reservation of renal function in patients with local-zed renal tumors.1 More recently, minimally inva-ive laparoscopic techniques to perform partial ne-hrectomy have been developed in an attempt touplicate the open surgical experience.2 Comparedith open surgery, laparoscopic partial nephrectomy
esults in lower analgesic requirements postopera-ively, earlier hospital discharge, and more rapid re-urn to normal activities. Currently used techniquesor laparoscopic partial nephrectomy may, however,esult in a greater incidence of positive marginsnd an increase in intraoperative complications.
rom the Division of Urology, The Ohio State University, Colum-us, Ohio; Integrated Manufacturing Techologies Institute, Na-ional Research Council of Canada, London, Ontario; Division ofrology, University of Western Ontario, London, Ontario; De-artment of Urology, Queen’s University, Kingston, Ontario; andivision of Urology, University of Ottawa, Ottawa, Ontario,anada.Reprint requests: Bodo E. Knudsen, M.D., 4835 University
ospitals Clinic, 456 West 10th Avenue, Columbus, OH, 43210-228. E-mail: [email protected]: October 14, 2004, accepted (with revisions): April
s7, 2005
© 2005 ELSEVIER INC.32 ALL RIGHTS RESERVED
urthermore, the warm ischemic time has beenhown to be longer during laparoscopic than dur-ng open partial nephrectomy.3 In addition, tactileaptic feedback is lost during laparoscopic proce-ures and may make localizing the tumor moreifficult than during an open procedure. There-ore, additional refinements are required so thathe results of laparoscopic partial nephrectomyill meet or exceed the results of the equivalentpen surgical procedure.The ability to view a kidney and an associated renalass in a virtual reality (VR) environment may facil-
tate preoperative planning and successful surgicalemoval of a renal tumor. Our goal was to design aystem that can seamlessly convert a patient’s cross-ectional imaging data (computed tomography [CT]r magnetic resonance imaging) into a three-dimen-ional (3D) VR image that can be viewed and manip-lated by the surgeon preoperatively in a 3D VR en-ironment and in the operating room on a personalomputer. This report focused on the initial stepsaken in achieving this goal.
MATERIAL AND METHODS
A 79-year-old man initially presented for a workup of micro-
copic hematuria. Ultrasonography suggested the presence of bi-0090-4295/05/$30.00doi:10.1016/j.urology.2005.04.060
lrfnot
ITmravFc
TtmMNBvbi
Asvfdltsr
(OiCtT
Ctvet(
sSumtm
a
dasebw
tsw(esbslvtsv(
dctv
FS
Fv
U
ateral renal masses. Subsequent cross-sectional CT and magneticesonance imaging revealed bilateral renal tumors ranging in sizerom approximately 1 to 2.2 cm. The metastatic workup wasegative. The patient was presented with his possible treatmentptions but elected not to proceed with surgical management athat time, instead opting for follow-up imaging.
The patient’s CT data was encoded and stored in Digitalmaging and Communications in Medicine (DICOM) format.his is a standardized format for encoding and transmittingedical data. The format includes a header, which contains
elevant patient data and the imaging modality used, and thectual image data. The segmentation process follows and in-olves extracting the target data from the raw DICOM data.or this project, two methods of segmentation were used andompared.
The first method of segmentation was performed usingKSegmentation1.py (Robarts Research Institute, London, On-
ario, Canada). This program uses an algorithm termed “mathe-atical morphology.” Once segmented, the data are output inedical Image NetCDF (MINC) format, which is based on theetCDF (University Corporation for Atmospheric Research,oulder, Colo) generic data format. The segmented data are con-erted to a 3D polygonal mesh using a Python/VTK script createdy Atamai Incorporated (London, Ontario, Canada). The result-ng 3D polygonal mesh is displayed using OpenGL.
The second method for segmentation was performed usingmira (TGS, San Diego, Calif). Amira is a commercially available
oftware package that performs advanced 3D visualization andolume modeling. Although Amira offers an automated optionor segmentation, a manual method was used for this project. Theata to be segmented were marked on the individual slices, al-
owing for more precise segmentation. A generated surface washen applied to the model using Amira. In addition, Amiramoothes the segmentation data and removes artifacts to bettereflect the original data.
Both segmentation processes create a standard 3D objectOBJ) file. Regardless of the segmentation process used, theBJ files are handled identically at this point. The files are
mported into 3D Studio Max (Discreet, Montreal, Quebec,anada) using a plug-in called OBJ2MAX, version 3.1. Addi-
ional smoothing and optimization of the models is then done.he final model is exported to the viewer.A Silicon Graphics Onyx2 Reality Monster (Mountain View,
alif) running the Linux operating system was used to displayhe final models in a VR environment. The models could beiewed and interacted within one of two environments, either annclosed room called the CAVE (FakeSpace Systems, Marshall-own, Iowa) or using a specialized desk called ImmersadeskFakeSpace Systems).
To view the model on a Windows-based personal computer, apecialized program was written in C�� using Microsoft Visualtudio.NET (Microsoft, Redmond, Wash), and OpenGL wassed to display the model. An interface was created to allowanipulation of the model in the first and third person perspec-
ives. The controls use the keyboard and mouse and allow for fullanipulation of the model in three dimensions.The University of Western Ontario Research Ethics Board
pproved this study.
RESULTS
The two segmentation processes were able to pro-uce models successfully from the DICOM CT datand were imported into 3D Studio Max for finalmoothing and optimization. However, we consid-red the model created using TKSegmentation1.py toe unsatisfactory. The model produced using Amira
as superior owing to the additional precision that mROLOGY 66 (4), 2005
he software provided. 3D Studio Max was used tomooth and optimize the segmented data to create aorking 3D model (Fig. 1). The completed model
Fig. 2) was then exported and viewed in two differ-nt 3D VR environments, the CAVE and the Immer-adesk. In the virtual environments, the model coulde manipulated to allow for viewing from any per-pective, including from inside the kidney (eg, col-ecting system). Specialized eyeglasses are required toiew the model in this environment. Video clips ofhe model were recorded in Amira. The clip demon-trates the model, but, because it is only a recordedideo, it lacks the 3D VR features of the actual modelVideo clip 5).The model was transferred to a desktop Win-
ows-based personal computer. An interface wasreated that allowed the model to be viewed usinghe first and third person perspectives. The cameraiewpoint is controlled using a combination of the
IGURE 1. Mesh 3D kidney model as viewed in 3Dtudio Max.
IGURE 2. Completed model before export to 3D VRiewer.
ouse and keyboard. With this system, the model
733
ci
pihipdnfcttdvpslv
noalmcwtjcmtr
qdbttHfslmsct
naestT
ptfittsaac
etmgtesi
hmtvuddviisOmesc
tesrlsmsp
VmCswtpwbw
7
ould be fully manipulated, allowing viewing fromnfinite angles.
COMMENT
Conventional radical nephrectomy has been therimary treatment for the removal and cure of local-zed renal tumors. The preservation of renal functionas become an increasingly important consideration
n recent years, resulting in growing prominence forartial nephrectomy as a treatment option.1 The in-ications for partial nephrectomy are widening andow include patients with normal contralateral renal
unction. The goal is to achieve the same local cancerontrol as that achieved with open radical nephrec-omy while preserving overall renal function.4 Al-hough no randomized trial has been published toate comparing the oncologic control with radicalersus partial nephrectomy, results have shown thatartial nephrectomy does provide durable long-termuccess.1,2 Furthermore, for small, unilateral tumorsess than 4 cm in diameter, the cancer-specific sur-ival rate at 10 years approaches 100%.1Laparoscopic partial nephrectomy is an emerging
ephron-sparing treatment option for the treatmentf renal tumors, and the results approach thosechieved with open surgery.2 The advantages of theaparoscopic approach include shorter hospital stays,
ore rapid convalescence, and decreased use of nar-otics postoperatively.3 Potential disadvantages existith the laparoscopic approach, including longer in-
raoperative warm ischemic times and increased ma-or intraoperative and postoperative urologic compli-ations. Therefore, improvements in the techniqueust be continually sought to enhance the advan-
ages further and eliminate the disadvantages of lapa-oscopic partial nephrectomy.One challenge during partial nephrectomy is ade-
uately localizing the neoplasm and determining theepth of tumor involvement. Small renal tumors maye completely intraparenchymal and not seen at all athe surface of the kidney. Furthermore, some pa-ients have underlying disease processes, such as vonippel-Lindau disease, that can predispose to multi-
ocal renal tumors.5 Real-time color Doppler ultra-onography may be used intraoperatively to aid inocating a renal tumor and to gain additional infor-
ation with regard to the tumor size, depth of inva-ion, presence of prominent blood vessels in the vi-inity of the tumor, and the existence of other satelliteumors in the kidney.6
Although real-time ultrasonography can be of sig-ificant assistance during partial nephrectomy, it hasnumber of limitations. The operating room is an
lectronically noisy environment because of the sub-tantial quantity of electric medical equipment usedhat can degrade the quality of the ultrasound image.
he ultrasound probe may not be able to visualize all e34
arts of the kidney owing to limitations imparted byhe wound size and shape. Small lesions may be dif-cult to see, especially if excess pressure is applied tohe ultrasound transducer. It may be difficult to dis-inguish small, hypoechoic renal tumors from renalcarring. Finally, a practical limitation is the need forradiologist to be present during the procedure for
n extended period. This may not be practical at someenters.7Three-dimensional VR imaging of the kidney may
ventually obviate or reduce the need for intraopera-ive ultrasonography. By being able to view a 3D VRodel of the kidney during the procedure, the sur-
eon may be better able to perform the resection ofhe renal mass accurately. The entire kidney will beasily viewed, and small lesions that were previouslyeen on cross-sectional imaging should be readilydentifiable.
Coll et al.8 used volume-rendered CT to createigh-quality 3D images in 60 patients with renalasses to facilitate the preoperative and intraopera-
ive evaluations. In addition, a 3 to 5-minute longideotape was prepared and viewed before the patientnderwent open partial nephrectomy. This videoemonstrated the position of the kidney; location andepth of the tumor extension, renal arteries, andeins; and the relationship of the tumor to the collect-ng system. The investigators demonstrated that thismaging technique may avert the need for more inva-ive preoperative imaging such as renal angiography.ne important difference in our study was that theodels we used can be fully manipulated in the VR
nvironment, which provides infinite viewing per-pectives and, therefore, a theoretical benefit overurrent 3D volume-rendered CT images.Certain challenges must be overcome before this
echnology can become a core component in the op-rating room. The process of converting the cross-ectional imaging data into the 3D VR model is cur-ently both labor and time intensive. Identifying thisimitation, we have been able to automate several keyteps of the process. Still, our present method re-ains quite labor intensive and would not be feasible
hould multiple models be required within a shorteriod.Ultimately, to streamline the process of creating 3DR images, it would be ideal to incorporate the seg-entation and rendering software directly into theT workstation. This would eliminate the cumber-
ome steps of exporting the DICOM data to a separateorkstation. Furthermore, if future automation of
he various steps in the segmentation and renderingrocess can be done within the CT workstation, itould provide an efficient system that could readilye incorporated into the standard preoperativeorkup of patients with renal masses.Currently, our models can be viewed in the VR
nvironments (CAVE and Immersadesk, Fakespace
UROLOGY 66 (4), 2005
SpOnwhtpmlhlmirv
aspmecT3htttamoitnsb
sp
smcmTnc
sepdws
sn
nn
om
f2
og1
ci
s
uei
a
av7
tp
a
FG
U
ystems), as well as on a desktop personal com-uter using a keyboard and mouse combination.ur goal is to allow the surgeon to view and ma-ipulate the model during the surgical procedurehile maintaining surgical sterility. An efficientuman/machine interface is required. Viewing op-ions include using a flat-screen LCD monitor sus-ended above the operative field or using a head-ounted LCD worn by the surgeon. For a
aparoscopic procedure, the surgeon could use theead-mounted LCD display to visualize both the
aparoscopic camera image and the VR data. Head-ounted displays are equivalent to standard mon-
tors in terms of surgical efficacy and may help toeduce neck strain by providing alignment of theisual and motor axis.9,10
The current mouse/keyboard interface is not suit-ble for the operating room environment. It is vitalurgeons not be distracted from their focus on therocedure, and the human/machine interactionsust be natural and nonintrusive. In the VR viewing
nvironments, PINCH Gloves (Fakespace Systems)ombined with the Flock of Birds Electromagneticracker (Ascension Technology, Burlington, Vt; Fig.) are used to manipulate the models by a series ofand gestures and eye movements. An advantage ofhis technique is that it allows the user to manipulatehe model in the VR environment in a fashion similaro reality. Miniaturization of the technology wouldllow it to be adopted in the operating room environ-ent. Ideally, the surgeon would have sensors placed
n his hands underneath the surgical gloves in a non-ntrusive fashion. Alternatively, a voice-actuated in-erface could be developed. The model could be ma-ipulated using a series of commands spoken by theurgeon. This interface would be similar to the ro-
IGURE 3. CAVE VR viewing environment with PINCHloves and Flock of Birds Electromagenetic Tracker.
otic system, AESOP, used by many laparoscopic
ROLOGY 66 (4), 2005
urgeons to control the camera system during therocedure.11,12
CONCLUSIONS
A 3D VR model was created using the CT cross-ectional imaging data obtained from a patient withultifocal bilateral renal masses. The model was suc-
essfully viewed and manipulated in a VR environ-ent (CAVE and Immersadesk, FakeSpace Systems).he VR model was also successfully viewed and ma-ipulated on a desktop Windows-based personalomputer after a custom interface was created.This project successfully demonstrated the first
teps necessary to bring VR technology into the op-rating room to facilitate performance of partial ne-hrectomy. This technology could be applied to tra-itional open, as well as laparoscopic, surgery. Futureork is needed to validate this approach to demon-
trate a tangible benefit to operative outcomes.
REFERENCES1. Fergany AF, Hafez KS, and Novick AC: Long-term re-
ults of nephron sparing surgery for localized renal cell carci-oma: 10-year followup. J Urol 163: 442–445, 2000.
2. Gill IS, Desai MM, Kaouk JH, et al: Laparoscopic partialephrectomy for renal tumor: duplicating open surgical tech-iques. J Urol 167(2 Pt 1): 469–470, 2002.
3. Gill IS, Matin SF, Desai MM, et al: Comparative analysisf laparoscopic versus open partial nephrectomy for renal tu-ors in 200 patients. J Urol 170: 64–68, 2003.
4. Van PH: Partial nephrectomy: the standard approachor small renal cell carcinoma? Curr Opin Urol 13: 431–432,003.
5. Choyke PL, Pavlovich CP, Daryanani KD, et al: Intra-perative ultrasound during renal parenchymal sparing sur-ery for hereditary renal cancers: a 10-year experience. J Urol65: 397–400, 2001.
6. Walther MM, Choyke PL, Hayes W, et al: Evaluation ofolor Doppler intraoperative ultrasound in parenchymal spar-ng renal surgery. J Urol 152(6 Pt 1): 1984–1987, 1994.
7. Choyke PL, and Daryanani K: Intraoperative ultra-ound of the kidney. Ultrasound Q 17: 245–253, 2001.
8. Coll DM, Uzzo RG, Herts BR, et al: 3-dimensional vol-me rendered computerized tomography for preoperativevaluation and intraoperative treatment of patients undergo-ng nephron sparing surgery. J Urol 161: 1097–1102, 1999.
9. Hanna G, and Cuschieri A: Image display technologynd image processing. World J Surg 25: 1419–1427, 2001.
10. Herron DM, Lantis JC, Maykel J, et al: The 3-D monitornd head-mounted display: a quantitative evaluation of ad-anced laparoscopic viewing technologies. Surg Endosc 13:51–755, 1999.11. Krapichler C, Haubner M, Engelbrecht R, et al: VR in-
eraction techniques for medical imaging applications. Com-ut Methods Programs Biomed 56: 65–74, 1998.12. Unger SW, Unger HM, and Bass RT: AESOP robotic
rm. Surg Endosc 8: 1131, 1994.
Video Clips cited in this article can be foundon the internet at: http://www.goldjournal.net
735
Recommended
