Design of functional simulation of renal cancer in virtual reality environments

  • Published on

  • View

  • Download

Embed Size (px)


<ul><li><p>D ULY</p><p>BO KED</p><p>ACOb ctoman realCo n dTh owinrooM withpo nalGr stemen d thwit ittenRe s, anco entCo irst,wa n incomputer in the operating roomwas created. This is the first step in bringing VR technology to the operating roomto assist the surgeon directly. UROLOGY 66: 732735, 2005. 2005 Elsevier Inc.</p><p>Oprizesivphduwirestivturforresan</p><p>FrobustionUrparDivCa</p><p>RHo122</p><p>S27,</p><p>ADULT UROLOGY</p><p>73penpartial nephrectomyhasbeen shown topro-vide effective, long-term tumor control, with</p><p>eservation of renal function in patients with local-d renal tumors.1 More recently, minimally inva-e laparoscopic techniques to perform partial ne-rectomy have been developed in an attempt toplicate the open surgical experience.2 Comparedth open surgery, laparoscopic partial nephrectomyults in lower analgesic requirements postopera-ely, earlier hospital discharge, and more rapid re-n to normal activities. Currently used techniqueslaparoscopic partial nephrectomy may, however,ult in a greater incidence of positive marginsd an increase in intraoperative complications.</p><p>Furthermore, the warm ischemic time has beenshown to be longer during laparoscopic than dur-ing open partial nephrectomy.3 In addition, tactilehaptic feedback is lost during laparoscopic proce-dures and may make localizing the tumor moredifficult than during an open procedure. There-fore, additional refinements are required so thatthe results of laparoscopic partial nephrectomywill meet or exceed the results of the equivalentopen surgical procedure.The ability to view a kidney and an associated renal</p><p>mass in a virtual reality (VR) environment may facil-itate preoperative planning and successful surgicalremoval of a renal tumor. Our goal was to design asystem that can seamlessly convert a patients cross-sectional imaging data (computed tomography [CT]or magnetic resonance imaging) into a three-dimen-sional (3D) VR image that can be viewed andmanip-ulated by the surgeon preoperatively in a 3D VR en-vironment and in the operating room on a personalcomputer. This report focused on the initial stepstaken in achieving this goal.</p><p>m the Division of Urology, The Ohio State University, Colum-, Ohio; Integrated Manufacturing Techologies Institute, Na-al Research Council of Canada, London, Ontario; Division ofology, University of Western Ontario, London, Ontario; De-tment of Urology, Queens University, Kingston, Ontario; andision of Urology, University of Ottawa, Ottawa, Ontario,nada.eprint requests: Bodo E. Knudsen, M.D., 4835 UniversityESIGN OF FUNCTIONAL SIMIN VIRTUAL REALIT</p><p>DO E. KNUDSEN, GORD CAMPBELL, ANDREWJAMES D. WATTERSON, BEN H. CHEW, JOHN</p><p>ABSTRjectives. The preoperative planning of partial nephred surrounding tissue in three-dimensional (3D) virtualmmunications in Medicine computed tomography scaemodel can be transferred to a personal computer, allm.ethods. Computed tomography data from a patientlygonal mesh using Amira running on a desktop persoaphics Monster Onyx2 running the Linux operating syvironments: either the CAVE or a specialized desk calleh the model on a desktop personal computer was wrsults. A 3D model of the kidney, the multiple tumoruld be viewed and manipulated in a true VR environmnclusions. This project completed two major goals. Fs created and viewed in a VR environment. Second, aAsco</p><p>spitals Clinic, 456 West 10th Avenue, Columbus, OH, 43210-8. E-mail: knudsen-1@medctr.osu.eduubmitted: October 14, 2004, accepted (with revisions): April2005</p><p> 2005 ELSEVIER INC.2 ALL RIGHTS RESERVEDATION OF RENAL CANCERENVIRONMENTSNNEDY, JUSTIN AMANN, DARREN T. BEIKO,. DENSTEDT, and STEPHEN E. PAUTLER</p><p>Ty can be facilitated by the ability to view the tumor</p><p>ity (VR). A technique to convert Digital Imaging andata into a fully 3D VR environment was developed.g the surgeon to view the 3Dmodel in the operating</p><p>multifocal renal masses was converted into a 3Dcomputer with Windows XP Professional. A Siliconwas used to view the 3D stereo model in the VR</p><p>e Immersadesk. An application to view and interactin C.d the associated systems was created. The modeland on a desktop personal computer.a 3D model of a kidney containing multiple massesterface to display the model on a desktop personalMATERIAL AND METHODS</p><p>79-year-old man initially presented for a workup of micro-pic hematuria. Ultrasonography suggested the presence of bi-</p><p>0090-4295/05/$30.00doi:10.1016/j.urology.2005.04.060</p></li><li><p>lateresfronegopttha</p><p>TImThmerelactvolFocom</p><p>TTKtarmaMeNeBouverbying</p><p>TAmsofvolfordatlowthesmrefl</p><p>B(OOBimCatioTh</p><p>ACathevieenctow(Fa</p><p>TspeStuusemativema</p><p>Tapp</p><p>Tduansmerebewa</p><p>thesmwo(Fensadbesplecviethestrvid(VT</p><p>docrethviemo</p><p>FIGStu</p><p>FIGvie</p><p>URral renalmasses. Subsequent cross-sectionalCTandmagneticonance imaging revealed bilateral renal tumors ranging in sizem approximately 1 to 2.2 cm. The metastatic workup wasative. The patient was presented with his possible treatmentions but elected not to proceed with surgical management att time, instead opting for follow-up imaging.he patients CT data was encoded and stored in Digitalaging and Communications in Medicine (DICOM) is a standardized format for encoding and transmittingdical data. The format includes a header, which containsevant patient data and the imaging modality used, and theual image data. The segmentation process follows and in-ves extracting the target data from the raw DICOM data.r this project, two methods of segmentation were used andpared.he first method of segmentation was performed (Robarts Research Institute, London, On-io, Canada). This program uses an algorithm termed mathe-tical morphology. Once segmented, the data are output indical Image NetCDF (MINC) format, which is based on thetCDF (University Corporation for Atmospheric Research,lder, Colo) generic data format. The segmented data are con-ted to a 3Dpolygonalmeshusing a Python/VTK script createdAtamai Incorporated (London, Ontario, Canada). The result-3D polygonal mesh is displayed using OpenGL.he second method for segmentation was performed usingira (TGS, SanDiego, Calif). Amira is a commercially availabletware package that performs advanced 3D visualization andume modeling. Although Amira offers an automated optionsegmentation, amanualmethodwasused for this project. Thea to be segmented were marked on the individual slices, al-ing for more precise segmentation. A generated surface wasn applied to the model using Amira. In addition, Amiraoothes the segmentation data and removes artifacts to betterect the original data.oth segmentation processes create a standard 3D objectBJ) file. Regardless of the segmentation process used, theJ files are handled identically at this point. The files areported into 3D Studio Max (Discreet, Montreal, Quebec,nada) using a plug-in called OBJ2MAX, version 3.1. Addi-nal smoothing and optimization of the models is then done.e final model is exported to the viewer.Silicon Graphics Onyx2 Reality Monster (Mountain View,</p><p>lif) running the Linux operating system was used to displayfinal models in a VR environment. The models could be</p><p>wed and interacted within one of two environments, either anlosed room called the CAVE (FakeSpace Systems, Marshall-n, Iowa) or using a specialized desk called ImmersadeskkeSpace Systems).o view themodel on aWindows-based personal computer, acialized programwas written in C usingMicrosoft Visualdio.NET (Microsoft, Redmond, Wash), and OpenGL wasd to display the model. An interface was created to allownipulation of the model in the first and third person perspec-s. The controls use the keyboard andmouse and allow for fullnipulation of the model in three dimensions.he University of Western Ontario Research Ethics Boardroved this study.</p><p>RESULTS</p><p>he two segmentation processes were able to pro-ce models successfully from the DICOM CT datad were imported into 3D Studio Max for finaloothing and optimization. However, we consid-d themodel created tounsatisfactory. The model produced using Amiras superior owing to the additional precision thatOLOGY 66 (4), 2005software provided. 3D Studio Max was used toooth and optimize the segmented data to create arking 3D model (Fig. 1). The completed modelig. 2) was then exported and viewed in two differ-t 3D VR environments, the CAVE and the Immer-esk. In the virtual environments, themodel couldmanipulated to allow for viewing from any per-ective, including from inside the kidney (eg, col-ting system). Specialized eyeglasses are required tow the model in this environment. Video clips ofmodel were recorded in Amira. The clip demon-</p><p>ates the model, but, because it is only a recordedeo, it lacks the 3D VR features of the actual modelideo clip 5).he model was transferred to a desktop Win-ws-based personal computer. An interface wasated that allowed the model to be viewed usinge first and third person perspectives. The camerawpoint is controlled using a combination of theuse and keyboard. With this system, the model</p><p>URE 1. Mesh 3D kidney model as viewed in 3Ddio Max.</p><p>URE 2. Completed model before export to 3D VRwer.733</p></li><li><p>coinfi</p><p>CprizehainpadicnofuncotomthodavepasulesvivL</p><p>neofachlapmocowitrajorcatmutagroO</p><p>qudebethetieHifocsolocmasiocintumA</p><p>nifia nelestathaTh</p><p>patheficthetinscaa rancenT</p><p>evtivmogetheeasseeideC</p><p>higmativvidundedeveingimsivOnmoenspcuC</p><p>tecerasecrenlimstemashpeU</p><p>VRmeCTsowotheprwobewoC</p><p>en</p><p>73uld be fully manipulated, allowing viewing fromnite angles.</p><p>COMMENT</p><p>onventional radical nephrectomy has been theimary treatment for the removal and cure of local-d renal tumors. The preservation of renal functions become an increasingly important considerationrecent years, resulting in growing prominence forrtial nephrectomy as a treatment option.1 The in-ations for partial nephrectomy are widening andw include patients with normal contralateral renalction. The goal is to achieve the same local cancerntrol as that achieved with open radical nephrec-y while preserving overall renal function.4 Al-ugh no randomized trial has been published tote comparing the oncologic control with radicalrsus partial nephrectomy, results have shown thatrtial nephrectomy does provide durable long-termccess.1,2 Furthermore, for small, unilateral tumorss than 4 cm in diameter, the cancer-specific sur-al rate at 10 years approaches 100%.1aparoscopic partial nephrectomy is an emergingphron-sparing treatment option for the treatmentrenal tumors, and the results approach thoseieved with open surgery.2 The advantages of thearoscopic approach include shorter hospital stays,re rapid convalescence, and decreased use of nar-tics postoperatively.3 Potential disadvantages existth the laparoscopic approach, including longer in-operative warm ischemic times and increased ma-intraoperative and postoperative urologic compli-ions. Therefore, improvements in the techniquest be continually sought to enhance the advan-es further and eliminate the disadvantages of lapa-scopic partial challenge during partial nephrectomy is ade-</p><p>ately localizing the neoplasm and determining thepth of tumor involvement. Small renal tumorsmaycompletely intraparenchymal and not seen at all atsurface of the kidney. Furthermore, some pa-</p><p>nts have underlying disease processes, such as vonppel-Lindau disease, that can predispose to multi-al renal tumors.5 Real-time color Doppler ultra-nography may be used intraoperatively to aid inating a renal tumor and to gain additional infor-tion with regard to the tumor size, depth of inva-n, presence of prominent blood vessels in the vi-ity of the tumor, and the existenceof other satelliteors in the kidney.6lthough real-time ultrasonography can be of sig-cant assistance during partial nephrectomy, it hasumber of limitations. The operating room is anctronically noisy environment because of the sub-ntial quantity of electric medical equipment usedt can degrade the quality of the ultrasound image.e ultrasound probemay not be able to visualize all4rts of the kidney owing to limitations imparted bywound size and shape. Small lesions may be dif-</p><p>ult to see, especially if excess pressure is applied toultrasound transducer. It may be difficult to dis-guish small, hypoechoic renal tumors from renalrring. Finally, a practical limitation is the need foradiologist to be present during the procedure forextendedperiod.Thismaynotbepractical at someters.7hree-dimensional VR imaging of the kidney may</p><p>entually obviate or reduce the need for intraopera-e ultrasonography. By being able to view a 3D VRdel of the kidney during the procedure, the sur-on may be better able to perform the resection ofrenal mass accurately. The entire kidney will beily viewed, and small lesions that were previouslyn on cross-sectional imaging should be readilyntifiable.oll et al.8 used volume-rendered CT to createh-quality 3D images in 60 patients with renalsses to facilitate the preoperative and intraopera-e evaluations. In addition, a 3 to 5-minute longeotapewasprepared andviewedbefore thepatientderwent open partial nephrectomy. This videomonstrated thepositionof thekidney; locationandpth of the tumor extension, renal arteries, andins; and the relationship of the tumor to the collect-system. The investigators demonstrated that thisaging techniquemay avert the need formore inva-e preoperative imaging such as renal angiography.e important difference in our study was that thedels we used can be fully manipulated in the VRvironment, which provides infinite viewing per-ectives and, therefore, a theoretical benefit overrrent 3D volume-rendered CT images.ertain challenges must be overcome before thishnology can become a core component in the op-ting room. The process of converting the cross-tional imaging data into the 3D VR model is cur-tly both labor and time intensive. Identifying thisitation, we have been able to automate several keyps of the process. Still, our present method re-ins quite labor intensive andwould not be feasibleould multiple models be required within a shortriod.ltimately, to streamline the process of creating 3Dimages, it would be ideal to incorporate the seg-ntation and rendering software directly into theworkstation. This would eliminate the cumber-</p><p>me stepsof exporting theDICOMdata to a separaterkstation. Furthermore, if future automation ofvarious steps in the segmentation and rendering</p><p>ocess can be done within the CT workstation, ituld provide an efficient system that could readilyincorporated into the standard preoperativerkup of patients with renal masses.urrently, our models can be viewed in the VR</p><p>vironments (CAVE and Immersadesk, FakespaceUROLOGY 66 (4), 2005</p></li><li><p>SypuOunipwhhutiopemolaphelapmoitoredvisT</p><p>absuprmuencoTr3)hathithetoallmeonintternipsubo</p><p>surgeons to control the camera system during theprocedure.11,12</p><p>AsecmucesmeThnipcoT</p><p>steeraphditwostr</p><p>sulno</p><p>nepniq</p><p>ofmo</p><p>for200</p><p>opger16</p><p>coling</p><p>sou</p><p>umevaing</p><p>and1</p><p>andvan751</p><p>1terpu</p><p>1arm</p><p>FIGGlo</p><p>URstems), as well as on a desktop personal com-ter using a keyboard and mouse combination.r goal is to allow the surgeon to view and ma-ulate the model during the surgical procedureile maintaini...</p></li></ul>


View more >