Report of the Joint Working Group on Image-Guided ... › programs_resources › ... · 2D, 3D, 4D two-, three-, and four-dimensional ACRIN American College of Radiology Imaging Network

Report of the Joint Working Group onImage-Guided Diagnosis and Treatment

April 12–14, 1999Washington, D.C.

Sponsored byU.S. Public Health Service’s Office on Women’s Health

andNational Cancer Institute

EditorsFaina Shtern, M.D. Daniel Winfield, M.S.U.S. Public Health Service Research Triangle InstituteOffice on Women’s Health

ContributorsJames H. Anderson, Ph.D.The Johns Hopkins University School of Medicine

Stanley Baum, M.D.Hospital of the University of Pennsylvania

Stephen G. Bown, M.D., F.R.C.P.University College London Medical School

George T.Y. Chen, Ph.D.University of Chicago

Steven L. Dawson, M.D.Massachusetts General Hospital

Ferenc A. Jolesz, M.D.Harvard Medical School

Robert B. Lufkin, M.D.UCLA Medical Center

Charles A. Pelizzari, Ph.D.University of Chicago

Julian G. Rosenman, Ph.D., M.D.UNC-Chapel Hill School of Medicine

Russell H. Taylor, Ph.D.The Johns Hopkins University

Yona Tadir, M.D.University of California at Irvine

S. James Zinreich, M.D.The Johns Hopkins Medical Institutions

AcknowledgmentsDaniel Sullivan, M.D. Wanda K. Jones, Dr.P.H. Robert E. Wittes, M.D.National Cancer Institute U.S. Public Health Service National Cancer Institute

Office on Women’s Health

Image-Guided Diagnosis and Treatment ii April 12-14, 1999

Contents

Acronyms.....................................................................................................................................................................iv

Planning Committee ....................................................................................................................................................v

Speakers and Panelists ...............................................................................................................................................vi

Introduction .................................................................................................................................................................1Goals of the Joint PHS OWH/NCI Working Group ..................................................................................1

Session 1: Current and Future Clinical Applications of Image-Guided, Computer-Assisted Interventions .....3Clinical Applications..................................................................................................................................3

Brain .................................................................................................................................................3Head and Neck..................................................................................................................................4Breast ................................................................................................................................................4Abdomen and Pelvis..........................................................................................................................4Gynecology and Reproduction Systems ............................................................................................5Bone ..................................................................................................................................................6

Technical Requirements.............................................................................................................................7Operative Planning and Surgical Simulators ...................................................................................7Intraprocedure Imaging and Endoscopy ..........................................................................................7Registration and Segmentation .........................................................................................................7Anatomical and Physiological Models .............................................................................................7Surgical Instrumentation, Tooling, and Robotics .............................................................................7Systems Architecture, Integration, and User Interfaces....................................................................7

Research Priorities .....................................................................................................................................8Short Term ........................................................................................................................................8Intermediate Term.............................................................................................................................8Long Term.........................................................................................................................................8

References..................................................................................................................................................8

Session 2: Medical Image Computing for Image-Guided Treatment..................................................................10Examples..................................................................................................................................................10

Clinical Analysis, Change Detection, and Time Series Analysis ....................................................10Real-Time 3D Brain Shift Compensation........................................................................................10Volumetric Display and Analysis of 3D Data .................................................................................10Visualization and Virtual Reality (VR) in Image-Guided Surgery..................................................11

Technical Problem Areas.........................................................................................................................13Data Management, Communication, and Visualization .................................................................13Access to Computing Resources......................................................................................................13Segmentation...................................................................................................................................14Multimodality Registration and Fusion ..........................................................................................14Realistic Anatomical Modeling.......................................................................................................15Validation........................................................................................................................................16Atlases.............................................................................................................................................16Plan Optimization ...........................................................................................................................17

Research Priorities ...................................................................................................................................17Short Term ......................................................................................................................................17Intermediate Term...........................................................................................................................17Long Term.......................................................................................................................................18

Image-Guided Diagnosis and Treatment iii April 12-14, 1999

Session 3: Computer-Assisted Interventional Systems .........................................................................................19Systems-Oriented, Multidisciplinary, Team-Based Research ..................................................................19Specific Technology/Research Needs......................................................................................................20

Tissue and Organ Deformation Modeling ......................................................................................20Sensors for Tissue-Tool Interactions ..............................................................................................21Image-Guided Localized Therapy...................................................................................................21

Bibliography ............................................................................................................................................21

Session 4: Treatment Modalities ............................................................................................................................23Interstitial Laser Therapy for Tumors of Solid Organs ............................................................................23Miniature Photon Radiosurgery System for Image-Guided Therapy in the Operating Room..................23References................................................................................................................................................24

Session 5: Machine Design and Clinical Framework ...........................................................................................25System Integration ...................................................................................................................................25Barriers to Overcome...............................................................................................................................25Clinical Benefits of New Technology ......................................................................................................25Breakthrough Technologies .....................................................................................................................25Industry Panel ..........................................................................................................................................26

Interventional MRI..........................................................................................................................26Ideal Characteristics of Equipment ................................................................................................26

Session 6: Summary Roundtable of Professional Societies ..................................................................................28Organizational Issues ...............................................................................................................................28Training Issues.........................................................................................................................................28Industrial Partnerships..............................................................................................................................28Recommendations....................................................................................................................................28

Organization ...................................................................................................................................28Training ..........................................................................................................................................28

Image-Guided Diagnosis and Treatment iv April 12-14, 1999

Acronyms

2D, 3D, 4D two-, three-, and four-dimensional

ACRIN American College of Radiology Imaging Network

CAD computer-aided diagnosis

CAIS computer-assisted interventional systems

CAS computer-assisted surgery

CT computed tomography

DICOM Digital Imaging and Communications in Medicine

FDA U.S. Food and Drug Administration

fMRI functional magnetic resonance imaging

HIFU high intensity focused ultrasound

IGT image-guided treatment

ILP interstitial laser photocoagulation

LITT laser-induced thermotherapy

LSM least squares template matching

MEMS microelectromechanical systems

MR magnetic resonance

MRI magnetic resonance imaging

MRS magnetic resonance spectroscopy

NCI National Cancer Institute

NIH National Institutes of Health

NSF National Science Foundation

PDT photodynamic therapy

PET positron emission tomography

PHS OWH U.S. Public Health Service’s Office on Women’s Health

RF radio frequency

VR virtual reality

Image-Guided Diagnosis and Treatment v April 12-14, 1999

Planning Committee

The U.S. Public Health Service’s Office on Women’s Health and the National Cancer Institute gratefullyacknowledge the members of the planning committee, who devoted time and effort to the development of theconference agenda.

Chair: Faina Shtern, M.D. U.S. Public Health Service’sOffice on Women’s Health

Members: Stanley Baum, M.D. Hospital of the University of PennsylvaniaFerenc A. Jolesz, M.D. Harvard Medical SchoolDaniel C. Sullivan, M.D. National Cancer Institute

Image-Guided Diagnosis and Treatment vi April 12-14, 1999

Speakers and Panelists

Session 1: Current and Future Clinical Applications of Image-Guided, Computer-AssistedInterventions

Session Leaders:Stephen G. Bown, M.D., F.R.C.P. University College London Medical SchoolRobert B. Lufkin, M.D. University of California at Los Angeles Medical

CenterSpeakers: Jonathan Allis, D.Phil. Siemens Medical Systems, Inc.

Stephen G. Bown, M.D., F.R.C.P. University College London Medical SchoolRichard D. Bucholz, M.D., F.A.C.S. St. Louis University School of MedicineKevin Cleary, Ph.D. Georgetown University Medical CenterWladyslaw M.W. Gedroyc,

M.B.B.S., M.R.C.P., F.R.C.P. St. Mary’s Hospital, LondonSteven E. Harms, M.D., F.A.C.R. University of Arkansas for Medical SciencesKullervo Hynynen, Ph.D. Harvard Medical SchoolBranislav Jaramaz, Ph.D. University of Pittsburgh Medical CenterFrank Langlotz, Ph.D. M.E. Mueller Institute for Biomechanics,

University of Bern, SwitzerlandMartin G. Mack, M.D. University Hospital of FrankfurtRainer M.M. Seibel, M.D. Mulheimer Radiology Institute, University of

Witten/HerdeckeYona Tadir, M.D. University of California at IrvineS. James Zinreich, M.D. The Johns Hopkins Medical Institutions

Panelists: M. Peter Heilbrun, M.D. University of Utah School of MedicineFerenc A. Jolesz, M.D. Harvard Medical School

Session 2: Medical Image Computing for Image-Guided TreatmentSession Leaders: George T.Y. Chen, Ph.D. University of Chicago

Charles A. Pelizzari, Ph.D. University of ChicagoS. James Zinreich, M.D. The Johns Hopkins Medical Institutions

Speakers: Michael J. Ackerman, Ph.D. National Library of MedicineGeorge T.Y. Chen, Ph.D. University of ChicagoJames S. Duncan, Ph.D. Yale University Medical CenterHenry Fuchs, Ph.D. University of North Carolina at Chapel HillGuido Gerig, Ph.D. University of North Carolina at Chapel HillFerenc A. Jolesz, M.D. Harvard Medical SchoolSandy Napel, Ph.D. Stanford University Medical CenterCharles A. Pelizzari, Ph.D. University of ChicagoRichard A. Robb, Ph.D. Mayo Foundation and ClinicJulian G. Rosenman, Ph.D., M.D. University of North Carolina at Chapel Hill School of

MedicineDemetri Terzopoulos, Ph.D. University of Toronto

Panelists: Kurt R. Smith, D.Sc. Sofamor Danek GroupArthur W. Toga, Ph.D. University of California at Los Angeles School of

MedicineWilliam M. Wells, Ph.D. Harvard Medical School

Image-Guided Diagnosis and Treatment vii April 12-14, 1999

Session 3: Computer-Assisted Interventional SystemsSession Leaders:Steven L. Dawson, M.D. Massachusetts General Hospital

Russell H. Taylor, Ph.D. The Johns Hopkins UniversitySpeakers: George T.Y. Chen, Ph.D. University of Chicago

Steven L. Dawson, M.D. Massachusetts General HospitalRichard M. Satava, M.D., F.A.C.S. Yale University Medical CenterRussell H. Taylor, Ph.D. The Johns Hopkins University

Panelists: Grigore C. Burdea, Ph.D. Rutgers UniversityFrank Langlotz, Ph.D. M.E. Mueller Institute for Biomechanics,

University of Bern, SwitzerlandGregory Merril HT Medical Systems, Inc.

Session 4: Treatment ModalitiesSession Leaders:Stephen G. Bown, M.D., F.R.C.P. University College London Medical School

Julian G. Rosenman, Ph.D., M.D. University of North Carolina at Chapel Hill School ofMedicine

Speakers: Stephen G. Bown, M.D., F.R.C.P. University College London Medical SchoolKullervo Hynynen, Ph.D. Harvard Medical SchoolFerenc A. Jolesz, M.D. Harvard Medical SchoolJulian G. Rosenman, Ph.D., M.D. University of North Carolina at Chapel Hill School of

MedicinePanelist: Donald O. Smith, Ph.D. Photoelectron Corporation

Session 5: Machine Design and Clinical FrameworkSession Leaders:James H. Anderson, Ph.D. The Johns Hopkins University School of Medicine

Ferenc A. Jolesz, M.D. Harvard Medical SchoolSpeakers: James H. Anderson, Ph.D. The Johns Hopkins University School of Medicine

Leon Kaufman, Ph.D. Toshiba America MRI, Inc.Panelists: Raymond Damadian, M.D. FONAR Corporation

Maurice Ferre, M.D. Visualization Technology, Inc.Rainer Graumann, Ph.D. Siemens Medical Systems, Inc.Alastair J. Martin, Ph.D. Philips Medical Systems, Inc.Kurt R. Smith, D.Sc. Sofamor Danek GroupKirby G. Vosburgh, Ph.D. General Electric Corporate Research and Development

Session 6: Summary Roundtable of Professional SocietiesSession Leader:Stanley Baum, M.D. Hospital of the University of PennsylvaniaSpeakers: Thomas J. Brady, M.D. Massachusetts General Hospital

Steven L. Dawson, M.D. Massachusetts General HospitalMichael J. Pentecost, M.D. Georgetown University Medical Center

Image-Guided Diagnosis and Treatment 1 April 12-14, 1999

Introduction

In March 1996, the U.S. Public Health Service’sOffice on Women’s Health (PHS OWH) established aFederal Multi-Agency Consortium for Imaging andOther Technologies to Improve Women’s Health.This consortium facilitates technology transfer fromlaboratories to patients. The membership of theconsortium includes, but is not limited to, theNational Cancer Institute (NCI), Food and DrugAdministration (FDA), Health Care FinancingAdministration, Central Intelligence Agency,Department of Defense, Department of Energy, andNational Aeronautics and Space Administration. Theactivities of this consortium have been critical forsharing expertise, resources, and technologies bymultiple government agencies, industry, andacademia for the advancement of novel breastimaging for early diagnosis of cancer, such as digitalmammography, magnetic resonance imaging (MRI)and spectroscopy (MRS), ultrasound, nuclearmedicine, and positron emission tomography (PET),as well as related image display, analysis,transmission, and storage and minimally invasivebiopsy and treatment.

The consortium sponsored a public conferenceentitled “Technology Transfer Workshop on BreastCancer Detection, Diagnosis, and Treatment”convened on May 1-2, 1997.1 During this meeting,consortium members developed recommendations forthe scientific and technologic projects critical foradvancement of novel breast imaging.

Subsequently, PHS OWH and NCI jointly sponsoredthe establishment of several working groups to definefurther the research agenda in the areas of breastimaging examined by the May 1997 conference.These groups focused on specific recommendationsfor research priorities and technology developmentand transfer opportunities across multiple areas ofbreast imaging:

• Functional imaging (e.g., PET, MRI and MRS,and optical imaging and spectroscopy) for theachievement of comprehensive in vivo cellularand ultimately molecular biologic tissuecharacterization2

• Image processing, computer-aided diagnosis(CAD), and three-dimensional digital display forenhanced lesion visualization and radiologicimage interpretation3

• Telemammography, teleradiology, and relatedinformation management4

• Digital X-ray mammography, with an emphasison digital display technologies and workstationdesign for image interpretation5

• Image-guided diagnosis and treatment forpotential replacement of open surgery withminimally invasive and/or noninvasiveinterventions

• Methodological issues for diagnostic andscreening trials for imaging technologies, withspecific focus on the development of computermodels for analysis of patient outcomes and cost-effectiveness.6

This article summarizes the results of the Conferenceof the Joint PHS OWH/NCI Working Group onImage-Guided Diagnosis and Treatment. Almost 70international scientific leaders, representing clinicalpractice, academic research, government agencies andlaboratories, and medical imaging systemmanufacturers, attended the meeting held April 12-14, 1999, in Washington, D.C. This article describesthe group’s findings and recommendations.

Goals of the Joint PHS OWH/NCIWorking Group

1) To review the state of the art in image-guideddiagnosis and treatment, including current andfuture clinical applications and technicalchallenges and the potential impact on criticaldiseases, with an emphasis on women’s health.

2) To outline short- and long-term researchpriorities to advance imaging tools for guidance,planning, control, and monitoring ofinterventional procedures.

3) To consider “ideal” characteristics of equipmentand the framework for image-guided diagnosisand treatment.

4) To identify technical limitations and developproblem statement(s) seeking new or emergingtechnologies.

To achieve these goals, the meeting consisted of thefollowing sessions.


Session 1: Current and Future ClinicalApplications of Image-Guided, Computer AssistedInterventions addressed clinical applications acrossa range of medical specialties and organ systems,including brain; breast; gastrointestinal; gynecology;head and neck; spine; liver; orthopedics; plasticsurgery; and ear, nose and throat.

Session 2: Medical Image Computing for Image-Guided Treatment reviewed the critical role of state-of-the-art 2D and 3D image processing andvisualization technologies in image-guidedinterventions.

Session 3: Computer-Assisted InterventionalSystems included a series of presentations illustratingsuch technologies as telepresence/telesurgery,surgical simulators, therapy planning systems, andmedical robotics.

Session 4: Treatment Modalities reviewed currentpractice and emerging opportunities with newtreatment modalities, including radiation therapy,thermal ablation, focused ultrasound, and genetherapy.

Session 5: Machine Design and ClinicalFramework provided an opportunity for bothacademic and industrial participants to provide theirperspectives on the key issues surrounding broaderimplementation of image-guided interventions and thecharacteristics of “ideal” equipment for image-guideddiagnosis and treatment.

Session 6: Summary Roundtable of ProfessionalSocieties developed a statement on the organizationaland training issues surrounding image-guideddiagnosis and treatment.

Working Session: Working group members met toformulate consensus reports describing (1) the stateof the art and current fundamental clinical/technicalroadblocks, (2) technical parameters required to meetcurrent and future clinical needs, and (3) futurepriorities in technology development and relatedbasic and clinical research.

Summary Session: The consensus reports werepresented during the summary session.

Subsequent to the working group meeting, its leadersdeveloped written summary reports with input fromsession participants. These summary reports havebeen integrated into this article with editorial inputfrom the working group chairs and sponsors.

References1. Ascher S, Shtern F, Winfield D, et al. Final report

of the technology transfer workshop on breastcancer detection, diagnosis, and treatment.Washington, D.C., USA. May 1-2, 1997. AcadRadiol 1998; 5:S465-501.

2. Shtern F, Winfield D, eds. Report of the jointworking group on quantitative in vivo functionalimaging in oncology. Acad Radiol (in press).

3. Shtern F, Winfield D, eds. Report of the workinggroup on digital mammography: Computer-aideddiagnosis and 3D image analysis and display.Acad Radiol 1999; 6(suppl. 5): S221-S256.

4. Shtern F, Winfield D, eds. Report of the jointworking group on telemammography/teleradiology and information management. AcadRadiol (in press).

5. Shtern F, Winfield D, eds. Report of the workinggroup on digital mammography: Digital displaysand workstation design. Acad Radiol 1999;6(suppl. 4):S193-S220.

6. Methodological issues in diagnostic clinical trials:Health services and outcomes research inradiology. Symposium proceedings. WashingtonD.C., USA, March 15-16, 1998. Acad Radiol1999; 6(suppl. 1):S1-136.


Session 1: Current and Future Clinical Applications of Image-Guided,Computer-Assisted Interventions

Several applications of image-guided and/orcomputer-assisted interventions already have had alarge impact in many clinical areas. This articlefocuses on the technical challenges and validationrequirements of new investigational applications thathave yet to enter the clinical mainstream. Researchpriorities are outlined for imaging tools forguidelines, treatment planning, control, andmonitoring of interventional procedures. In particular,the session participants identified current and futureclinical challenges that can be addressed by theadvancement of image-guided diagnosis andtreatment.

Clinical ApplicationsBrainDiagnosis and treatment of brain disorders have along history of image-guided interventions andassociated devices. For example, image-guidedstereotatic biopsy is accepted in clinical practice.Most recently, MRI has been investigated as aguidance tool in disease management in the followingareas: stereotactic brain biopsy and electrodeplacement, image-guided tumor resection, and image-guided tumor ablation. Using MR as the roadmap,tools such as the MR-compatible neuroendoscopemay have a dramatic effect on minimally invasiveneurosurgery.

MRI is now recognized as one of the most promisingimaging techniques for the detection of intracranialpathologies. With the development of the MR-compatible stereotactic frame, stereotactic brainbiopsy based on MR imaging data is now routinelyperformed in many institutions. Stereotacticcoordinates and the optimal angle of the probeinsertion are calculated with high accuracy based onMR imaging. New interventional MR magnets withan open design and in-room monitors now enableframeless stereotactic brain biopsy and minimallyinvasive treatment.

With appropriately equipped, open-design MRsystems, several groups are using MR during surgicalprocedures such as craniotomy. The ultimate valueand indications of the technique remain to be defined.

Image-guided thermal ablation of brain tumors thusfar appears to be a safe, minimally invasive treatment

that deserves further study. Unlike radiation, it is notassociated with cumulative toxicity and may serve asa palliative alternative for end-stage patients notwishing to undergo open craniotomy.1

For example, Figure 1-1 shows postprocessed, phase-sensitive, two-dimensional, fast, low-angle shot MRimages acquired and displayed during and after laser-induced thermotherapy (LITT) of an intracranialmetastasis. The calculated temperatures are color-coded, gradually increasing the heat-affected zonewith increasing maximum temperatures. Images Athrough D were acquired during LITT 1, 4, 8, and 10minutes after starting the laser therapy. Images E and

Figure 1-1: MR images during and after laser-induced thermotherapy of an intracranialmetastasis.Source: In vivo MRI thermometry using a phase-sensitivesequence: Preliminary experience during MRI-guided laser-induced interstitial thermotherapy of brain tumors. ThomasKahn, M.D.; Thorsten Harth, Ph.D.; Jürgen C.W. Kiwit, M.D.;Hans-Joachim Schwarzmaier, M.D.; Christoph Wald, M.D.;and Ulrich Mödder, M.D. JMRI. Copyright © 1998, ISMRM.Reprinted by permission of Wiley-Liss, Inc., a subsidiary ofJohn Wiley & Sons, Inc.


F show the temperature distribution 2 and 4 minutesafter switching off the laser. The temperature returnedto baseline values; however, some scattered pixelsremain unchanged during cooling.

Head and NeckImage-guided biopsy and endoscopic sinus surgeryare in use and widely accepted. MRI is rapidlyreplacing computed tomography (CT) in image-guided aspiration of head and neck lesions becauseMRI permits precise needle placement intocomparatively inaccessible areas, such as the skullbase and submandibular region, where beam-hardening artifacts limit the effectiveness of CT.Another advantage is the superior tip localization ofMR by obtaining an oblique or orthogonal imagealong the course of the needle, similar to that ofultrasound.

Palliative management options for recurrent head andneck cancers are limited by the proximity of vitalvascular and neural structures and the aggressivenature of these tumors. Wide local resection of theselesions may result in functional and cosmeticdeformities. Most head and neck tumors are usuallytreated by surgery and/or radiation therapy. MR-guided minimally invasive thermal ablation couldsomeday be another alternative if thermal energydelivery would be controlled and monitoredaccurately with imaging.2 Nodal metastases to thehead and neck (prior to radical neck dissection)provide a valuable model for image-guided thermalablation of such metastases throughout the body.Although the efficacy of interstitial laser therapyneeds to be demonstrated further in larger series,preliminary clinical trials have shown promisingresults.

BreastBreast carcinoma is a leading cause of death forwomen in the United States and Europe. Because ofhigh soft tissue contrast, MR has been able to detectbreast carcinomas not visible with the usualmammographic techniques.

The most common clinical scenario that leads to MR-guided breast biopsy is the detection of additionalenhancing lesions on breast MRI performed for localstaging of breast cancer. In this setting, MRIlocalization wires may be placed so that these lesionscan be excised during excision of primary lesions.This technique also may be used to clearly define themargins of abnormal enhancement so that a lesionwith its surrounding intraductal component may beexcised in one setting. With the recent development

of breast biopsy surface coils, the breast now can becompressed and stabilized while the patient is in aprone position. Although experience is limited todate, successful MR-guided biopsy has beenperformed on lesions that are not detectable or easilylocalized by other modalities.

In recent years, management of breast cancer hasmoved toward breast conservation, with the goal ofmaximized cosmesis without compromising overallsurvival. Several ablation techniques have beeninvestigated for treatment of breast tumors. A recentpreliminary clinical trial of interstitial laserphotocoagulation (ILP) for breast cancer, followed bysurgery, showed excellent correlation of MRappearance and pathology.3 The study suggests thatMR-guided interstitial laser photocoagulation forbreast cancer is a potentially useful tool. Anotherstudy of MR-guided high intensity focused ultrasound(HIFU) therapy for breast fibroadenomas is underway.

Several issues should be addressed before ablationtechniques can replace lumpectomy. One reservationconcerns the loss of material for histopathologicexamination when the whole tumor is ablated, iftreatment options may depend on the histologicinformation. Secondly, the enhancing lesions seen onMRI 48 to 72 hours after ablation may be caused byinflammatory response or residual tumor, thusnecessitating multiple biopsies of the treated margin.

Abdomen and PelvisPercutaneous interventional procedures in theabdomen and pelvis are mostly performed forbiopsies, but they also are done for sympathectomiesand drainages. CT-, fluoro-, and ultrasound-guidedpunctures have been reported in the literature fornearly 30 years. In 1967, Nordenstrom reported thefirst series of percutaneous fluoro-guided lymph nodebiopsies.4 Approaches such as transperitoneal,translumbar, and transvascular procedures usingvarious imaging modalities have been described.

Fine needle biopsies are most often performed forcytologic and histologic diagnosis in the abdomen,retroperitoneum, pelvis, lymph nodes, bones, andjoints. In general, the results of percutaneousaspiration techniques are very good. Success rates ofmore than 80% have been reported for cytologicdiagnosis. Sonographic and CT-guided biopsies areestablished diagnostic techniques. However,ultrasound-guided methods are being replaced moreand more by CT guidance. Although new MRItechniques for clinical treatments in the abdomen


have the potential to replace a large number ofconventional techniques, they must be used carefullyin order to minimize possible injury to vitalstructures. Biopsies of large tumor masses orabscesses are possible as well as percutaneous tumortherapy with ethanol ablation, radiofrequency (RF),lasers, or cryotherapy.

In the near future, more treatments in the abdomenand pelvis will be performed. Cryogenic, RF, or lasertechniques have great potential for treating liver andother tumors, while MR-guided drainage of largecysts and abscesses in the liver, pancreas,retroperitoneum, kidneys, abdominal wall, and pelvisis an area of active investigation.5 In the future,combined MRI and endoscope therapy, especially forobstetrics, gallbladder, urinary tract, colon, andsympthectomy at all levels of the spines, may be used

routinely if endocoils and/or MRI-compatibleendoscopic systems are developed.

Gynecology and Reproductive SystemGenital organs are relatively accessible for clinicalevaluation. Diagnostic tools such as X rays, CT scans,and MR imaging are rarely needed; however, somediagnostic tools, such as ultrasonography andlaparoscopy, are used in gynecology more than in anyother medical or surgical discipline. In order tounderstand the background of these trends andpossible future developments, one must consider theimaging technologies currently available to theclinician, those that are under development, and thenature of genital organs/disease needs. Most of thesetechnologies are presented in Tables 1-1 and 1-2.

Table 1-1: Deep Imaging Techniques in Gynecology and Reproduction

X-Ray CT MRI US 3D US

ColorDoppler

USUS plus

UltraguideFocused

US OtherUterusEndometrium

Cancer Op A RD RDHyperplasia A F? F?DUB A F?

MyometriumFibroids A Op A RD F? F?Adenomyosis Op A RD F?Malformation A Op A RD

OvariesCancer A A RD A RD F?Cysts A RD RD

Fallopian TubesTubal pathol. A A F?

Pelvic CavityMetastatic spread A A F? F? RDEndometriosis Op A F?

Retro-Periton.Node metast. A A F? RD

PregnancyWell-being A A RD AFetal surgery A RD RD F?

AssistedReproductionEgg collection A RDLaser-assistedhatching

Comp.Image

AbbreviationsUS = UltrasoundA = Technique available Op = Optional, but not used routinelyF? = Might be available in the future RD = In research and development


The deep penetrating imaging diagnostic tools listedin Table 1-1 provide a hard copy documentation. Theendoscopic diagnostic procedures listed in Table 1-2are usually not connected to any objectivedocumentation tool, rendering future evaluationimpossible. Although it is easy to store suchprocedures on videotape, which is standard practicein some clinics, it is not practical to review such tapesin a consultation or to compare current and pastimages. Moreover, diagnosis based only on theclinician’s experience may be inaccurate if notexposed to a second opinion. Recent progress incomputer image processing and cost reductions indigital storage call for improved documentation

following any kind of minimally invasive procedurewith endoscopes.

BoneThe incidence of dislocation following primary totalhip replacement surgery is between 2% and 6% andeven higher following revisions.6 It is, therefore, themost commonly occurring early complicationfollowing hip replacement surgery. Dislocation of atotal hip replacement causes significant distress to thepatient and additional costs to relocate the hip.Impingement between the neck of the femoral implantand the rim of the acetabular component can lead todislocations and also advanced wear of the acetabular

Table 1-2: Surface Imaging Techniques in Gynecology and Reproduction

Laparoscopy Hysteroscopy ColposcopyConventional PDDT Conventional PDDT Conventional PDDT

VulvaWarts A RDVIN A RD

VaginaWarts A RDVAIN A RD

CervixCIN A RD A RD

UterusEndometrium

Cancer A RDHyperplasia A RDDUB A RD

MyometriumFibroids AAdenomyosis RDMalformation A

OvariesCancer A RDCysts A

Ovaries-IVF OpFallopian TubesTubal pathol. A

Pelvic CavityMetastatic spread A RDEndometriosis A RD

Retro-Periton.Node metast. A F?

PregnancyWell-being A- Fetosc.Fetal surgery Fetosc.

AbbreviationsPDDT = Photodynamic diagnosis and treatmentA = Technique available Op = Optional, but not used routinelyF? = Might be available in the future RD = In research and development


rim, resulting in polyethylene wear debris shown toaccelerate loosening of implant bone interfaces. Thecauses of impingement and dislocation aremultifactorial; however, the most common cause ofboth impingement and dislocation is malposition ofthe acetabular component.

A system has been developed to permit accurateplacement of the acetabular component during totalhip replacement surgery. The system includes twocomponents: (1) a preoperative planner and range ofmotion simulator and (2) an intraoperative image-guided surgery system. The preoperative plannerallows the surgeon to specify the position of theimplant components within the pelvis and the femur,based upon preoperative CT images. A kinematicrange of motion simulator determines range of motionbased upon the specific bone and implant geometryand alignment, and predicts the leg positions in whichthe prosthetic or bone impingement occurs. Thefeedback provided by the simulator permits thesurgeon to determine the optimal, patient-specificacetabular implant alignment for any implant systemand determines an “envelope” of safe range ofmotion.

Technical RequirementsDramatic advances have been made in image-guidedprocedures in the brain during the past few years butrelatively little attention has been given to otherregions of the body. The technical requirements forimage-guided procedures beyond the rigid structureof the head are addressed in the following areas:

• Operative planning and surgical simulators

• Intraprocedure imaging and endoscopy

• Registration and segmentation

• Anatomical and physiological modeling

• Surgical instrumentation, tooling, and robotics

• Systems architecture, integration, and userinterfaces

Operative Planning and Surgical SimulatorsCurrent models for operative planning and surgicalsimulators are not sophisticated enough for realisticsystems. There are still many research issues in tissuemodeling, including deformable modeling. For thesesystems to be clinically useful, patient-specificmodels must be incorporated. Finally, for applicationswhere the sense of touch or force is important, betterhaptic interfaces are needed.

Intraprocedure Imaging and EndoscopyHardware developments are required to reduce sizeand cost while improving imaging resolution.Interventional MRI and the associated instruments aregenerally seen as too expensive, while othermodalities, such as CT and fluoroscopy, involveionizing radiation.7 Endoscopy problems includelimited visibility, difficulty with knowing where oneis in relation to the anatomy, and difficulty in dealingwith complications.

Registration and SegmentationThe major technical problems with image registrationinclude the need for manual intervention, limitedrobustness, the lack of methods for accurate real-timeregistration of a nonrigid object, and the limitedaccuracy of fiducial-free registration methods.Segmentation techniques are generally seen as slowand manually intensive. The problem of anatomicalmotion between imaging and surgery needs to beaddressed. Finally, there is a lack of standards fordetermining performance requirements, assessingaccuracy, and validation of algorithms.

Anatomical and Physiological ModelsCurrent models are not realistic enough, and softtissue modeling is a fundamental problem.Developing an accurate model that incorporatesphenomena such as hemodynamics is a complex task.Other issues include the development of patient-specific models, computational efficiency, andvalidation.

Surgical Instrumentation, Tooling, andRoboticsThe major technical challenge is developingtechnology that is safe, reliable, and easy to use in theoperating room. The equipment also should becompatible with imaging modalities such as MRI andCT. Other problems include accuracy, suitable man-machine interfaces, and real-time navigation. Cost,liability, and FDA considerations limit the use of thistechnology.

Systems Architecture, Integration, and UserInterfacesThe major technical challenge is to create a devicethat is powerful yet easy to use. Many users believethat current image-guided systems remain too difficultto use in the operating room and that a skilledtechnician usually is required. The user interface is akey issue. Effective user interface design requirescollaboration of experts from various fields.Conveying the information that surgeons need in a


format they can use is still a problem. Other factorslimiting the use of these technologies include the lackof complete component technologies, economicjustification, and liability issues. The use of differentfile formats by different medical imaging devicemanufacturers also is a problem, but this might beresolved by the DICOM medical imaging standard.

Research PrioritiesShort Term

• Improve resources for multicenter trials (e.g.,ACRIN).

• Develop better user interfaces for wide clinicalacceptance.

• Improve validation and create uniform standardsof thermal monitoring technology.

• Develop better ablation technology to targetlarger lesions with smaller access.

• Optimize image-guided systems for nonrigidanatomy, focusing on ease of use andregistration.

• Improve soft tissue stereotaxis.

• Develop better contrast agents for tumordefinition.

• Develop better interactive control of thermalablation.

• Conduct multicenter cost and/or efficacy studiesof the following:- Intraoperative MRI of brain- Image-guided percutaneous disk ablation

versus microdiscectomy- Palliation for recurrent head and neck tumors

(also study pain control and morbidity)- LITT versus liver resection for colorectal

metastases- LITT versus no treatment following failed

treatment for colorectal metastases- Image-guided percutaneous ablation versus

surgery for hepatoma- Percutaneous image-guided thermal ablation of

fibroids versus myomectomy- Image-guided joint replacement/reconstruction

systems.

• Conduct feasibility studies of the following:- Image-guided thermal ablation of metastatic

lymph nodes in the head and neck- Thermal breast cancer ablation followed by

surgery- Image-guided percutaneous bone ablation and

structural repair related to metastases.

Intermediate Term

• Conduct feasibility studies of the following:- New treatment effector technologies (e.g.,

ablation, robots, gene therapy).- Pedicle screws, other than thoracic level and

percutaneous placement- Image-guided percutaneous ablation of

gynecologic malignancy prior to definitivesurgery.

• Improve target delineation to better determineeloquent brain areas and tumor margins.

• Research safety and quality-of-life issues relatedto thermal breast cancer ablation without surgery

• Conduct multicenter trials of thermal ablation ofbreast cancer and fibroadenoma.

• Research the role of growth factor stimulation insurgery versus LITT or other thermal ablation(animal model).

• Automate segmentation of bone-tissue margins.

• Standardize commercial image-guidedorthopedic technology components.

Long Term

• Incorporate tissue elastic properties into modelsof brain tissue deformation; use open MR andother new imaging technologies data to validatethese models.

References1. Kahn T, Harth T, Kiwit JC, Schwarzmaier HJ,

Wald C, Modder U. In vivo MRI thermometryusing a phase-sensitive sequence: preliminaryexperience during MRI-guided laser-inducedinterstitial thermotherapy of brain tumors. JMagn Reson Imaging 1998; 8:160-164.

2. Castro D, Saxton RE, Lufkin R. Interstitialphotoablative laser therapy guided by magneticresonance imaging for the treatment of deeptumors. Semin Surg Oncol 1992; 8:233-241.

3. Mumtaz H, Hall-Craggs MA, Wotherspoon A, etal. Laser therapy for breast cancer: MR imagingand histopathologic correlation. Radiology 1996;200:651-658.

4. Nordenstrom B. Transthoracic needle biopsy. N Engl J Med 1967; 276:1081-1082.5. Vogl TJ, Mack MG, Straub R, Roggan A, Felix R.

MR-guided laser-induced thermotherapy (LITT)of liver metastases: results of survival rate.ISMRM 1997; 773.


6. McCollum DE, Gray WJ. Dislocation after totalhip arthroplasty: causes and prevention. ClinOrthop 1990; 261:159-170.

7. Seibel RM, Gronemeyer DH, Sorensen RA.Percutaneous nucleotomy with CT andfluoroscopic guidance. J Vasc Interv Radiol1992; 3:571-576.


Session 2: Medical Image Computing for Image-Guided Treatment

The goal of medical image computing in the contextof image-guided treatment (IGT) is to create andmanipulate three (or higher) dimensionalrepresentations of relevant patient data to enhance theability to detect disease and to plan and delivertherapy. Dimensions beyond three may be temporal(e.g., the synthesis of multiple scans from a singlemodality at several times or a time sequence ofimages from a cine modality, such as digitalsubtraction angiography, ultrasound, or fast MRI).Additionally, the patient representation may includemultiple anatomical signals or functional values ateach point in space. For example, metabolicinformation from nuclear medicine, several MRIintensities from multiple image sequences, blood flowinformation from functional MRI (fMRI), andintensity from CT all might be available and useful.Merging all the relevant portions of this data into asingle representation can define a vector-valued fieldin the 3D or 4D (location plus time) patient space.For the purpose of the present discussion, thecomponents of this vector field beyond the first areequivalent, as far as their computationalrequirements, to additional dimensions of the patientmodel.

ExamplesClinical Analysis, Change Detection, andTime Series AnalysisGuido Gerig, Ph.D., University of North Carolinaat Chapel HillA least squares template matching (LSM) has beenused for the precise measurement of patientpositioning in a series of digital images. This newdevelopment is driven by applying image analysis inorder to check patient position during radiotherapytreatment. Accurate information about patientposition is gained by employing electronic portalimages acquired during radiation treatment sessions.The problem with such megavoltage X-ray imagery isits extremely low contrast, rendering reliable featureextraction a difficult task and thus favoring the LSMapproach.

LSM is an iterative and area-based fitting methodespecially suitable for attaining very high precision orfor processing low-contrast, noisy, and blurredimages. The automatic quality control—a componentoften missing in commonly used image matching

methods—is achieved by self-diagnostic measuressupervising the iterative procedure.

The present system includes both field edgealignment and 2D anatomy displacementmeasurement. A very promising success rate of over90% was reported with 500 portal images. Digitallyreconstructed radiographs were used as simulatedportal images with known ground truth, allowing themeasurement errors to be analyzed in more detail.Furthermore, results for the multimodal matchbetween a digitally reconstructed radiographreference image and therapeutic portal imagespromise an approach that might significantly increasethe accuracy of a treatment.

Real-Time 3D Brain Shift CompensationJames S. Duncan, Ph.D., Yale UniversityThe use of surgical navigation systems has become astandard method to assist the neurosurgeon innavigating within an intraoperative environment an inplanning and guiding the surgery. However, thesesystems are subject to inaccuracy caused byintraoperative brain movement (i.e., brain shift), sincecommercial systems typically assume that theintracranial structures are rigid. Experiments showbrain shift of up to several millimeters, making it thecause of the dominant error in the system.

Addressing this problem requires an image-basedbrain shift compensation system based on anintraoperatively guided deformable model, such asthat under development at Yale University. A set ofbrain surface points has been recorded during thesurgery and used to guide and/or validate modelpredictions. Initial results show that such a systemlimits the error between its brain surface predictionand real brain surface to within 0.5 mm. This is asignificant improvement over the systems that arebased on the rigid brain assumption, which in thiscase would have an error of 3 mm or greater. Futurework is aimed at richer intraoperative data acquisitionand nonhomogeneous brain tissue modeling.

Volumetric Display and Analysis of 3D DataSandy Napel, Ph.D., Stanford UniversityThe past several years have seen an explosion in thedetail and amount of medical imaging data that can beroutinely produced during the course of a cross-sectional imaging examination. Ten years ago, a


typical CT examination generated between 30 and 50images; today’s helical CT scanners generatehundreds of overlapping slices for interpretation.Spatial resolution and sampling, particularly in thethrough-plane direction, also have improved. Similartrends are evident in MR and ultrasound. Althoughthese and other modalities have become moresophisticated, the dominant method for radiologicalinterpretation—that is, visual assessment of each ofthe cross-sectional images generated by themodality—has not changed. Furthermore, as thenumber of images increases, so does the time requiredfor interpretation.

While radiologists may be keeping up in 1999, it isdoubtful that the current paradigm will be possible inthe near future. Consider the introduction of multiple-detector ring helical CT, which can image a contrastbolus as it travels from above the renal arteries to thetoes in 1 minute and can generate over a thousand2.5-mm-thick slices spaced every 1.25 mm. Not onlydoes the time and, therefore, the cost of interpretationsignificantly increase, but fatigue and other factorsmight compromise diagnostic accuracy.

The new paradigm of radiological interpretation willbe based upon treating the acquired image data as avolume to be explored and from which to extractimages and quantitative data that document thecondition of the patient. Although several volumevisualization techniques have been available forseveral years (e.g., maximum intensity projection,surface rendering, volume rendering, flat and curvedreformatted planes, thin slab renderings), they havebeen used largely to supplement the diagnosis madeby assessment of the primary source images and forconveying findings to referring physicians. In the newparadigm of radiological interpretation, these andother techniques, including segmentation andcomputer-aided diagnosis, exist as choices that can bemade as part of the exploration process. However, theconcept of diagnosis based on these methods, perhapswithout ever viewing the primary source images, isnew and must be validated for every possiblediagnosis. Nevertheless, in the new world of 1,000+images per examination, diagnosis based on review ofsource images is not yet validated and may not bepossible.

Visualization and Virtual Reality (VR) inImage-Guided SurgeryRichard A. Robb, Ph.D., Mayo Foundation andClinicInteractive visualization, manipulation, andmeasurement of multimodality 3D medical images onstandard computer workstations has been developed,used, and evaluated in a variety of biomedicalapplications for more than a decade. Thesecapabilities have provided scientists, physicians, andsurgeons with powerful and flexible computationalsupport for basic biological studies and for medicaldiagnosis and treatment. Comprehensive softwaresystems, ANALYZE and VRASP, developed at theMayo Clinic have been applied to a variety ofbiological, medical, and surgical problems and usedon significant numbers of patients at manyinstitutions. This scope of clinical experience hasfostered continual refinement of approaches andtechniques, especially 3D volume imagesegmentation, classification, registration, andrendering and has provided useful information andinsights related to the practical clinical usefulness ofcomputer-aided procedures and their impact onmedical treatment outcome and cost.

This experience has led to using virtual realitytechnology in computer-assisted surgery (CAS). VRoffers the promise of highly interactive, naturalcontrol of the visualization process, providingrealistic simulations of surgery for training, planning,and rehearsal. The Mayo Clinic has developedefficient methods for the production of accuratemodels of anatomic structures computed from patient-specific volumetric image data (e.g., CT or MRI).The models can be enhanced with textures mappedfrom photographic samples of the actual anatomy.When used on a VR system, such models providerealistic and interactive capabilities for patient-specific surgical training, surgery planning andprocedure rehearsal. VR technology also can bedeployed in the operating room to provide thesurgeon with online, intraoperative access to allpreoperative planning data and experience, translatedfaithfully to the patient on the operating table.Additionally, these preoperative data and models canbe fused with real-time data in the operating room toprovide enhanced reality visualizations during theactual surgical procedures.


Virtual endoscopy is a new method of diagnosis usingcomputer processing of 3D image data (e.g., CT orMRI) to provide simulated visualizations of patient-specific organs similar or equivalent to thoseproduced by standard endoscopic procedures.Conventional endoscopy is invasive and oftenuncomfortable for patients. It can have serious sideeffects, such as perforation, infection, andhemorrhage. Virtual endoscopy visualization avoidsthese risks and can minimize difficulties and decreasemorbidity when used before actual endoscopicprocedures. In addition, there are many body regionsnot compatible with real endoscopy that can beexplored with virtual endoscopy. Eventually, virtualendoscopy may replace many forms of realendoscopy.

Other applications of VR technology in medicinebeing developed include anesthesiology training,virtual histology, and virtual biology. Thesetechniques provide faithful virtual simulations fortraining, planning, rehearsing, and/or analyzingmedical and/or biological image data.

There remains a critical need to refine and validatethree-dimensional (e.g., CAS or VR) visualizationsand simulated procedures before they are acceptablefor routine clinical use. The Mayo Clinic has used theVisible Human Dataset from the National Library ofMedicine to develop and test these procedures and toevaluate their use in a variety of clinical applications.

Specific clinical protocols are developed to evaluatevirtual surgery against surgical outcomes and tocompare virtual endoscopy with real endoscopy.Informative and dynamic on-screen navigation guideswill help the surgeon or physician determine bodyorientation and precise anatomical localization whileperforming the virtual procedures. Additionally, theadjunctive value of full 3D imaging (e.g., looking“outside” of the normal field of view) during thevirtual surgical procedure or endoscopic exam isbeing evaluated. Quantitative analyses of localgeometric and densitometric properties obtained fromthe virtual procedures (“virtual biopsy”) are beingdeveloped and compared with other direct measures.

Register

Merged representation

Planning

Segment

Atlas

Model

Diagnosis

Treatment

X rayMRICTMRAfMRIUltrasoundNuclear medicine

Figure 2-1: Interaction of computing tasks in planning and delivery of IGT.

Table 2-1: Technical Problem Areas

• Data management, communication, andvisualization

• Access to computing resources

• Segmentation

• Multimodality registration and fusion

• Realistic anatomical modeling

• Validation

• Atlases

• Plan optimization


Preliminary results suggest that these virtualprocedures can provide accurate, reproducible, andclinically useful visualizations and measurements.These studies will help drive improvements in andlend credibility to virtual procedures and simulationsas routine clinical tools. CAS and VR-assisteddiagnostic and treatment systems hold significantpromise for optimizing many medical procedures,minimizing patient risk and morbidity, and reducinghealth care costs.

Technical Problem AreasThe synthesis of many of the individual computingtasks into the planning and delivery of IGT isschematically illustrated in Figure 2-1. The box at theleft represents data management and communication,where all relevant patient data are brought into theappropriate systems for subsequent steps.Registration and segmentation are performed toextract the essential components of the multimodalitypatient model used in treatment planning anddelivery. Atlases and patient models informregistration, segmentation, and planning and are inturn informed by the multimodality patientrepresentation. In the case of IGT, the plan optimizesthe proposed treatment based on all the availableinformation. In the case of image-guided diagnosis,the merged patient representation may be used in aCAD step. On completion of the plan, treatmentmight be delivered with image guidance and withiterative modification of the plan as an image-guidedprocedure unfolds.

As demonstrated throughout this section, computingis pervasive in the planning and delivery of IGT.There are, however, technical problem areas that needadditional research support (Table 2-1).

Data Management, Communication, andVisualizationThe volume of image data available per study, as wellas the number of studies performed, has increasedrapidly in recent years, and it is quite reasonable toexpect that fully developed IGT regimes will generateand use terabytes of data per year (1 terabyte = 1million megabytes). Continued research on optimizedmethods to store and communicate this information isrequired. The contents of image data archives will notconsist solely of the primary image data themselves.Full use of multimodality IGT planning will requiresaving multiple copies of some datasets that areregistered with other datasets and/or developing new,efficient methods to produce registered image

datasets on demand. There may be other patient-specific and general information (e.g., functional,pathological, clinical) in addition to the primaryimage data. Partially processed images, segmentationresults, and deformed and synthesized datasets alsomust be available for timely and efficient use. Thesheer volume of these data, and the interactive naturein which they may be used, means that rapiddevelopment in data management, communication,and visualization methods are key to the practicalrealization of IGT. Moreover, communication ofinformation among collaborators, both withininstitutions and throughout the world, is of extremeimportance to continued research progress in areasrelated to IGT. Continued research and developmentin compression, communication, and dataorganization methods is critical to the success of theoverall IGT research enterprise.

An important development is the enhancement ofcapabilities for data manipulation by the IGT plannerand therapist. Visualization of IGT datasets, bothsingle-modality and multimodality, requiresconsiderable further research. Although someprogress has been made in rendering 2D, 3D, or 4Dviews of portions of image datasets, more powerfuland flexible methods for exploration of all availableimage data are needed. The techniques of scientificvisualization as used in many other application areasshould be adapted and developed to suit the particularneeds of medical imaging and IGT. Seemingly simpletasks such as navigating a CT dataset will becomeprohibitive as the number of slices generated bymultiring scanners grows to 1,000 and beyond. Oneaspect of visualization that has not been well enoughdeveloped is how to convey to an end user (e.g.,surgeon, radiation therapy machine, robot) a usefulestimate of uncertainty in the plan, which can be usedto optimize the delivery of therapy.

Continued improvement in display methods also isimportant. The accuracy with which IGT can bedelivered depends on (1) the resolution andorientation of acquired image data, (2) the format inwhich the data are presented to the user, and (3) thequality of the display of information. The demands ondisplay quality, as with many aspects of IGT, areapplication dependent.

Access to Computing ResourcesIncorporating adequate levels of realism in thebiomechanics that underlie simulations of IGT willconsume computer processing resources. For research


in prototype development, and production use, of IGTsystems, it will be necessary to ensure access toleading computing technology. Requests forsupercomputer time and state-of-the-art computinghardware should be viewed favorably. The same istrue for displays, which as mentioned earlier mightlimit the utility or accuracy of IGT systems. State-of-the-art visualization hardware might be required toachieve adequate realism in displays at usefulinteractive rates.

SegmentationDespite decades of intensive research, segmentationremains an outstanding problem in the continueddevelopment of IGT. The magnitude of this“segmentation bottleneck” will become greater as thenumber and size of image datasets grow. In addition,the problems of segmentation and registration(discussed below) are intertwined, with eachdepending on and facilitating the other. For example,multispectral segmentation assumes that the varioussignals have a known geometric relationship to eachother and to the patient, which is the essence ofregistration. Also, many registration methods dependon identifying corresponding points, lines, curves,surfaces, or regions in multiple datasets, which is afunction of segmentation.

Two fundamental tasks are required of segmentationmethods in the context of IGT. First, image datasetsmust be labeled—that is, a functional, morphological,anatomical, or other identity must be assigned tovoxels or regions in the datasets. Second, datasetsmust be measured—that is, the geometric shapes andrelationships of objects and regions within thedatasets must be quantified. Labels and measurementsshould have credible estimates of uncertainty thatshould inform the planning and delivery of IGT.Although the generation of these estimates during thesegmentation process is assumed to be done, thisproblem has not been solved, and further research isrequired. As mentioned earlier, methods fortransmitting these estimates in a useful way to endusers is of considerable importance.

Continued development of the user interface isrequired to optimize the user’s ability to segmentwhat is seen in images. Humans usually can recognizepatterns and perceive objects in images much moreeffectively than automated segmentation algorithmscan. Research therefore is warranted in thedevelopment of tools to assist users further and toallow quick and easy labeling and measurement of

perceived objects. Also, modeling human processesfor perceiving objects in images should lead to furtherdevelopments in automated segmentation algorithms.

High-level knowledge should be incorporated into theautomated segmentation process. Automatedalgorithms should start out “knowing” what they arelooking for, what it should look like, and where itshould be found. The classic embodiment of suchhigh-level knowledge is an atlas, which is discussedfurther later in this section. Given a computationallywell defined atlas, the atlas objects can be mapped to,or associated with, regions in the image. Suchmappings must consider intersubject variations in thesize, shape, and position of organs; distortions due tosubject position, pathology, or physiology; andanatomical abnormality. Probabilistic representationsof population variability in anatomy and functionshould be included in such an atlas to aidsegmentation and to evaluate the uncertainties inlabeling and geometry of extracted features.Continued development of methods for constructingand using atlases should be given a high priority.Another promising means for incorporating high-levelknowledge into segmentation is to model the image-formation process, taking into account what is knownabout the anatomy or biological function beingimaged and the physics of the imaging system.

Another promising approach to incorporatingsegmentation into IGT is to develop probabilityimages that might indicate, for example, thelikelihood of finding tumor cells at a certain densityat a particular point within a multimodality dataset.Treatment could be tailored to the distribution of riskand response probability resulting from such a“fuzzy” model. This approach will require anunderstanding of contrast enhancement mechanisms,development of new contrast agents, incorporation ofnew biological and cellular imaging modalities intoexisting schemes of multimodality registration andfusion, and development of theoretical modelsrelevant to this conceptual form of segmentation.

Multimodality Registration and FusionRegistration is the process of determining coordinatetransformations that map points corresponding to thesame anatomical location in the patient in multipleimage datasets. Once the relevant coordinatetransformations are known, information from multiplesources can be merged; this synthesis is called imageor data fusion. A second important use of registrationis to transfer information between an image-based


model of the patient and the actual patient; this is theessence of image guidance.

As mentioned earlier, registration often depends onsegmentation. Furthermore, registered datasets maybe used to improve segmentation techniques.Research progress in the registration andsegmentation fields thus can benefit each other.

One problem is registration in the presence ofanatomical motion or distortion. Many methods havebeen developed for registration of medical imagedatasets, but none is capable of dealing realisticallywith distortions between multiple image datasets insuch anatomical regions as the breast or the abdomen.Simple models that allow global distortions, such asanisotropic linear scaling, shearing, or warping, havebeen somewhat successful in dealing with such effectsas brain swelling or shrinkage. However, solvinggeneral registration problems in every area of thebody requires coupling registration with realisticbiomechanical models of motion and distortionand/or with atlas-based descriptions of anatomicalstructure and variability.

A second problem is registration of one 3D datasetwith a 2D dataset. For example, a radiograph may beused to infer the position or orientation of a structureor a location defined in the context of a 3D dataset,such as CT or MRI. Since information is lost in theprojection onto the plane of the radiograph, the 3Dposition of objects from a single radiograph cannot bedetermined exactly except under certain limitedcircumstances (e.g., when the 3D positions of asufficient set of landmark points is known and theirprojections can be uniquely identified). In general,solution for the full 3D orientation of the skull, chest,or pelvis from a single radiograph is an unsolvedproblem, even when rigid anatomies can be assumed.

Registration of 3D breast images, for example MRI,with radiographs such as mammograms demonstratesboth problems. The issues of 2D/3D ambiguity, lackof unique landmarks, and severe anatomical distortionall come together. Further research on these problemsshould be given a high priority.

As mentioned earlier, incorporating new biological-and/or molecular-based imaging modalities into themore traditional set of modalities may lead to newtypes of diagnostic and therapeutic approaches.Means for registering new and current modalitiesneed to be developed. This research will be ongoingfor many years as newer modalities become available.

An essential stage in the development of anyregistration method must be a characterization of itsaccuracy under clinically relevant conditions. Beyondthis, ideally it should be possible to give anuncertainty estimate for the accuracy of registrationon a case-by-case basis. This information should thenbe incorporated into the planning process, and atreatment design that is minimally sensitive to theknown uncertainties and expected errors in everystage of the IGT process should be used. Manyregistration methods provide an estimate ofuncertainty for each case; however, many others donot. Research into appropriate methods of generatinguncertainty estimates and communicating confidencelimits to users is important for the success of IGT.Any automated procedure should be able to recognizeand report when it has failed to achieve a reliableresult.

Realistic Anatomical ModelingThe areas of segmentation, registration, simulation,atlas construction, and planning and delivery of IGTall depend on accurate modeling of the motion anddistortion of anatomical structures. The presenttechnique for modeling deformable structures,however, provides the necessary interactivity for onlythe most limited situations. Considerable progress hasbeen made in modeling cardiac motion, but acomplete biomechanical model for characterizingmotion and distortion of any part of the body seemsmany years away. Research in this direction should begiven high priority, as should intermediateapproaches based on contemporary computer scienceresearch that can model the behavior of definedanatomical areas under practically relevantconditions.

As with segmentation and registration, anatomicalmodeling is intertwined with several of the otherresearch areas described in this section. Segmentationand registration both need realistic modeling to makeprogress toward more general, highly automatedsolutions. Modeling treatments during the planningprocess requires realistic treatment of the motion,distortion, and interactions of anatomical structures.Incorporating uncertainty into the planning processdepends on the ability to predict where and howobjects will move during procedures and on theprobable differences between the static patient modeldefined during planning and the actual position of theanatomy during therapy. Presentation of informationduring IGT should use realistic representations of


anatomy, including deformation of anatomicalstructures by the intervention.

Anatomical modeling is an area where access to high-performance computing resources will be ofparticular importance.

ValidationAll of the steps in planning and delivering IGT mustbe characterized as to their accuracy, reproducibility,reliability, and robustness in the presence of expecteddeficiencies in input data. A database of standard“ground truth” data against which to test newlydeveloped segmentation and registration methodswould be an important community research resource.Such a database could consist of well-characterizedimage data and simulated data incorporated withknown information. For example, a standard lesiondatabase for tumor detection could have simulatedlesions of known size, contrast, and other features.Similar databases of normal anatomy with agreed-upon labeling could be used to test segmentationmethods. Such a database would need to incorporaterealistic intersubject variability in size, shape,position, and image appearance of anatomicalstructures. This corresponds closely to thedevelopment of population-based atlases, which isdiscussed below.

Validation databases for registration should includewell-characterized examples of intra- andintermodality combinations of images. Determiningground truth usually is difficult except, for example,in brain images where stereotactic frames or fiducialscan be used to establish a trusted registration. Theavailability of high-quality standard datasets fortesting multimodality registration methods, togetherwith best estimates of the “correct” registration,would be very useful.

An area of medical image computing for whichvalidation methods have not been well developed isvisualization. There is no accepted standard by whichthe accuracy of a visualization, or of proceduresperformed based on visualization, may becharacterized. Research should be pursued on how tocharacterize accuracy of visualizations and how totranslate the uncertainties, including those introducedby the viewer’s interpretation, into inaccuracies intreatments.

AtlasesThe atlas is the embodiment of prior knowledgeconcerning structure and function as manifested inimage data. Even if not perfect, atlases can be useful.They must be computationally tractable—that is, theinformation they contain must be expressed in a formthat can be used effectively by analysis programs.The utility of atlases can be increased by increasingthe amount of information associated with eachanatomical point (e.g., nomenclature, normalhistology, interactions with other organs or systems).Their utility also will be improved by includinginformation concerning the range of normalvariability of structure and function. This allowsconfidence limits to be placed on quantities derivedfrom atlas-based analyses and on the decisions basedon them.

Frequently, atlases are mapped into the coordinatesystem defined by a particular subject’s image studiesin order to carry the atlas labels into the image data.Assuming the atlas is valid and the mapping iscorrect, this step alone can accomplish a good deal ofthe segmentation task, because many voxels will belabeled. This mapping process is a registration taskbecause the coordinate transformation must bedefined to map homologous anatomical points.Depending on the complexity of the anatomicalmotion and deformation that must be modeled toadequately map the general anatomy of the atlas ontothe specific anatomy of the patient, further researchmay be needed to perform this step adequately.

Most of the attention in atlas construction has focusedon brain atlases, where the motions and distortionsfrom the atlas to the individual subject are moderatelycomplex and where the benefits of atlas use incomplex segmentation tasks were first appreciated.What will be needed in the future are atlases coveringthe head and neck, thorax, abdomen, pelvis, andextremities. Development of extracranial atlases, andthe tools needed to construct and use them whiletaking account of shape variability and distortion,should be given high priority in the near tointermediate future. Incorporation of as manyfunctional and anatomic modalities as possible willincrease the usefulness of atlases and also should besupported. Synthesis of macroscopic and microscopicinformation in an atlas also will be an important areaof research.


Plan OptimizationThe link between the virtual space of medical imagesand the physical space of an IGT procedure is thetreatment plan. The planning stage of IGT is highlystructured in radiotherapy, for example, where thecomplete geometry and time course of the irradiationto be delivered is simulated.

Planning or simulation integrates results fromsegmentation, registration, atlases, modeling, andvisualization in order to provide the planner with theinformation needed to understand the treatmentsituation and to design a plan. Weaknesses in theseareas limit the ability to produce complex and optimalplans in an acceptable time frame. Imperfections insegmentation; a lack of rich, timely visualization frommultimodality datasets; and the unavailability ofsufficiently realistic patient models are particularlylimiting.

Radiotherapy also provides an illustration of somelimitations of manual planning with respect tooptimization. The parameter space within which theplan must be optimized (e.g., size, shape, number and3D orientation of radiation beams; radiation type andenergy; spatial variation of intensity within beamapertures) is too large to allow true optimization.Standard or “forward” planning techniques involve(1) choosing a set of plan parameters that are likely tobe good, based on past experience with similar casesituations, (2) computing the resulting radiation dosedistributions, and (3) evaluating some figures of meritthat express the success of the plan at fully treatingthe targets while minimizing collateral damage tonormal organs. In consultation with medical andtechnical colleagues, the planner identifies how theresult might be improved, and the process is iterateduntil the result is acceptable.

An alternative approach is so-called “inverse”planning, where a desired result (e.g., dosedistribution) is specified and an optimizationalgorithm finds a set of treatment parameters to givethat result within specified tolerances. Some work hasbeen done in this area, but the parameter space mustbe sharply limited in order to produce a plan in atimely manner. Continued research on optimizationstrategies for automated planning, and onincorporating biological endpoints and clinicalacceptability into plan evaluation and comparison,should be supported.

An important aspect of a plan is deliverability—thatis, the practicality of the plan. Does it require

impossible approaches to the patient? Does itminimize the time for the procedure relative to otheracceptable plans? Is it sensitive to small deviationsfrom ideal positioning of the patient or equipment? Isit sensitive to uncertainty in location of an internalanatomical structure whose position cannot beprecisely measured at the time of therapy? Asmentioned in earlier sections, all available estimatesof uncertainty during imaging, segmentation,registration, planning, and delivery should beincorporated into the treatment design. The goal is todeliver a treatment whose probability of success willbe minimally impacted by the known uncertainties.


• Develop validation databases for registration.Real and synthetic datasets, known truth, andstandard validation methodology should bedeveloped and a mechanism for theirdissemination should be provided.

• Develop validation databases for segmentationand for lesion detection.

• Improve capabilities for image registrationincorporating deformation. Both 3D/3D and themore difficult 3D/2D problems must beaddressed.

• Expand access to state-of-the-art computing andvisualization.

• Improve capability for navigation of extremelylarge image datasets.

• Investigate methods for estimating uncertainty ateach stage and for propagating through the entireIGT process. Investigate means for ensuringdelivered treatments are minimally sensitive tolikely uncertainties.

• Improve segmentation capabilities based on userinterface to allow users to classify what they cansee with minimal time and interaction.

• Continue developing deformable modeling ofanatomical structures using current technologies.

Intermediate Term

• Improve capabilities for realistic modeling ofanatomical motion and deformation, including asmuch biomechanical information as is feasible.

• Develop probabilistic atlases for relevant bodyregions, including characterization of population


variability in shape and position. Incorporate asmuch additional information linked to the basicanatomical description as possible.

• Incorporate microscopic, biologically basedimaging into standard medical imaging.

• Develop widely applicable segmentationmethods based on high-level knowledge.

• Investigate probabilistic descriptions of diseasefrom multiple image modalities and theimplications for IGT.

Long Term

• Develop realistic biomechanical models of allrelevant regions and organ systems.

• Develop complete segmentation and registrationcapabilities adequate for all relevant anatomicalregions and organs.


Session 3: Computer-Assisted Interventional Systems

Integrating imaging with therapy will enable the nextera in medicine’s history. As surgery becomes lessinvasive, new sources of vision are required toexamine the area outside the surgeon’s portal of view.This vision is provided by imaging. Fundamental tobringing imaging into treatment is the completeintegration of computers with interventions.Computer integration enables therapeutic innovation.

The tasks facing the physicians, engineers, andscientists who will bring this next medical era throughits evolution must be faced by teams of researchersworking together. Medicine is conservative andtradition-bound, with a language which appearsobscure to those outside. Similarly, engineeringseems foreign and obscure to medical practitioners.As clinical hurdles to innovation are identified, eachdiscipline commonly attempts to leap to a solutionfrom its own perspective. Yet for the sake ofefficiency—in time, cost, effort, and time-to-acceptance—the coming problems must be solved byteams of specialists working together. It will serve nopurpose to develop solutions devoid of input from thecollaborating specialty. The problems to be solvedmust represent unmet clinical needs. Technologydevelopment and related research must be clinicallydriven.

Computer-integrated therapy systems includesimulators for training, pretreatment planning, andrapid prototyping. Systems also include localizedtherapies—both noninvasive therapy modalities, suchas HIFU and radiation beam therapy, and minimallyinvasive therapies, such as radiation seed implants,localized chemotherapy, RF ablation, andcryotherapy. Despite many differences of detail, thereare significant synergies that can be achieved bypursuing solutions that are adaptable to multipletreatment modalities, imaging technologies, andorgan systems. Significant research barriers must beovercome to extend current capabilities to mobile anddeforming organ systems. Similarly, novel andversatile delivery systems can enable the developmentof novel therapies requiring accurate and consistentdelivery.

The development of computer-integrated therapysystems rests on common research and technologyneeds, including the following:

• Computer science: image processing, modeling,planning, image registration, real-time computingand communication, and reliability.

• Interfacial technologies: sensors, imagingdevices, robotics devices, and user interface.

• Systems infrastructure: test beds and modularcomponents to support research,experimentation, and deployment.

Systems-Oriented, Multidisciplinary,Team-Based ResearchA program of systems-oriented research on computer-assisted interventional systems (CAIS) should beestablished. Such research should address bothfundamental engineering and scientific problemsassociated with CAIS, with specific activitiesmotivated or driven by realistic clinical problems.This research is inherently multidisciplinary in nature,and it is best pursued through collaborative effortsincorporating both clinical and engineeringresearchers. Clinical application cannot be anafterthought, and clinicians should be involved in allphases of the work. Similarly, it is important thatengineering researchers not be treated as mereimplementers for the clinicians. There arefundamental algorithmic and technology barriers thatmust be addressed within the context of CAIS andtest beds.

One fundamental challenge is finding a means to fundand support such team-oriented research activities.Although there are some potentially pertinentprograms within government agencies (e.g., NationalScience Foundation [NSF], National Institutes ofHealth [NIH]), it traditionally has been very difficultto secure the necessary level of financial andinstitutional support. Means must be found toovercome the institutional and other barriers that haveinhibited the establishment of effective clinician-engineer teams.

Many of the technologies associated with CAIS areinterdependent, forming a web. They are often bestpursued within the context of an overall program ortest bed application. For example, developing meansto deliver patterns of localized injections accuratelyinto a deforming, mobile organ may require advancesin real-time imaging, segmentation, deformable


registration, mechatronics, and biomechanics. Betterbiomechanical models of such organs will make itpossible to plan therapy patterns more accurately andcan be combined with real-time force and imagesensing to help deliver the planned patterns.Similarly, the availability of accurate force and real-time 3D imaging capabilities can be crucial indeveloping and validating biomechanical models thatcan predict organ deformations. Finally, theavailability of accurate and repeatable means fordelivering patterns of therapy to targeted anatomy cansignificantly speed up the development and validationof new treatment options by reducing experimentalvariability.

Some mechanisms for promoting this synergy include

• Creation of jointly funded and jointly reviewedNSF/NIH and other multiagency programs

• Funding shared technology infrastructureresearch

• Promoting common software toolkits andreplicable experimental hardware components

• Encouraging consortia to develop shared testbeds and fund the development of such test beds

• Funding standards activities and multi-institutional working groups to define robustsystem architectures with nonproprietaryinterfaces and activities for the development andmaintenance of these standards

• Actively involving industry and academia in thedevelopment of standards, “best practice”guidelines, test beds, and took kits.

CAIS currently is a small market compared to otheruses of some of the backbone technologies (e.g., 3Dgraphics and computing). This can be an advantage,in the sense that other commercial applications canoften drive technology and cost reductions. However,this cannot substitute for crucial research anddevelopment support for adapting these advances tothe CAIS environment. Means need to be found toencourage incorporating such advances into a broadlysupported infrastructure for biomedical research. Onepossibility might be to create a funding mechanismthat would allow research proposals to include anexplicit task (and appropriate support) fordevelopment, testing, and maintenance of thetechnology backbone.

Specific Technology/Research NeedsA number of specific technology areas are crucial forimage-guided CAIS:

• Tissue and organ property modeling• Integration of planning and control

• Robotics systems for treatment delivery

• User interfaces (e.g., haptic, visualization)

• Sensors for feedback

• Representation standards

• Interface standards

• Validation of systems.

Several of these areas are discussed in greater detailbelow.

Tissue and Organ Deformation ModelingAdvances in biomechanical modeling of tissue anddeformable organs are needed to support therapyplanning, real-time control of therapy delivery, andsimulation. (The role of such modeling in therapyplanning and delivery has been discussed brieflyabove.)

Tissue modeling will enable the development ofrealistic simulators for training, rehearsal, and deviceprototyping. Procedural simulation will become thestandard of learning for both novice and experiencedphysicians, as well as provide engineers a means toperform medical procedures for which they aredesigning new devices. Such simulators also have animportant role in therapy planning.

The development of computer-based simulators formedical learning has not progressed to a level ofdemonstrable transfer of knowledge gained fromcurrently available simulators. One major hurdle toeffective learning is the ability to accurately representsimulated organ characteristics such as weight,elasticity, deformation, and texture. Because there isso little established science in this field, developingthis knowledge will require multiyear collaborationsamong physicians, haptics designers, programmers,and engineers. However, the resulting body ofknowledge will be used to assemble proceduralsimulators that will permit medical learning to takeplace without putting real patients at risk. Realisticmedical simulation also will permit rehearsal ofdifficult procedures and will open the opportunity forbiomedical engineers to perform a procedure forwhich they are designing new implantable devices,


instruments or drug delivery systems. Theinfrastructure technologies that such an effort will useinclude sensors to gather data on organcharacteristics; robotics to accurately manipulateorgans during study; and modeling science, includinghaptics interface design and graphics programming.

Infrastructure and Enabling TechnologyAdvancing tissue and organ deformation modeling isdirectly synergistic with a number of other areas:

• Sensor technology (see below)

• Real-time imaging

• Robotics

• More general modeling, representation methods,segmentation, and registration.


• Conduct experimental studies on in-vivo organsystems

Intermediate Term

• Characterize normal and abnormal tissues inbreast, prostate, and liver.

• Integrate normal and abnormal tissue models intographics and haptic simulation.

Long Term

• Develop real-time control to drive therapydevices.

• Incorporate real-time image feedback.

Spanning the Time Line

• Develop better representation and computationmethods that permit incorporation ofuncertainties and approximations withpredictable error.

• Establish high-fidelity anatomic representationswith embedded physiology, permitting silicondevice prototyping.

Sensors for Tissue-Tool InteractionsA variety of sensors, in addition to real-time imagesensors, will serve as new diagnostic tools, therapymonitors and force reflectance monitors.Microelectromechanical systems (MEMS)technologies from integrated circuit manufacturingwill be used to design biosensors that permit tissue

analysis for immediate in situ diagnosis, drugmonitoring from steady state delivery systems, andmonitoring tissue-tool interactions during minimallyinvasive therapy. These sensors must be integratedinto an overall information infrastructure for theireffective use in CAIS.

Infrastructure and Enabling TechnologyAdvances in sensing are directly synergistic with anumber of other areas:

• Tissue property modeling

• Stereotactic navigation

• Real time imaging

• Robotics

• More general modeling, representation methods,segmentation, and registration.


• Develop force and touch sensing (e.g., ininjection needles, palpation devices).

• Devise integrated imaging sensors (e.g., catheter-based ultrasound, MR sensors built into needlesand catheters).

Intermediate/Long Term

• Build biological sensors into therapy and biopsydevices.


• Enable tissue identification.

• Integrate navigation and robotic positioningdevices with sensors.

• Allow image/sensor “fusion.”

• Allow monitoring and control of therapydelivery.

Image-Guided Localized TherapyThere is a need to develop broadly usable andflexible systems for planning and delivering patternsof noninvasive or minimally invasive local therapies.As discussed earlier, such systems are needed (1) totake advantage of the many potential synergiesspanning multiple treatment modalities, imagemodalities, and organ systems and (2) to promote themore rapid development of novel therapy options.Broadly, moving beyond rigid organs like bones andrigidly contained organs like the brain is needed toaccommodate mobile and deforming structures.


Infrastructure and Enabling TechnologyThe development of such systems for localizedtherapy is directly synergistic with the entire range oftechnologies discussed in this session and throughoutthis article. A systems-oriented approach is vitallyneeded to advance these technologies and theireffective application.


• Enable bone and spine biopsies.

• Facilitate vertebroplasty.

• Focus on other stereotactic interventions whereorgan deformation may be ignored.

Medium Term

• Transition to nonrigid organs such as the breast,liver, and prostate.

• Integrate advances in tissue sensing andmonitoring into therapy delivery devices.

Long Term

• Enable dynamic local therapy in mobile organs,such as the heart.


• Develop multi-institutional preclinical activities.

• Develop common engineering toolkits andshared test beds.

BibliographyChen E, Marcus B. Force feedback for surgical

simulation. Proc IEEE 1998; 86:524-530.Delingette H. Toward realistic soft-tissue modeling in

medical simulation. Proc IEEE 1998; 86:512-523.

Delp SL, Zajac FR. Force and moment generatingcapacity of lower limb muscles before and aftertendon lengthening. Clin Ortho Rel Res 1992;284:247-259.

Hettinger LJ, Tannen RS, Geiselman E., et al.Surgical strike: interface design across taskdomains. Proceedings of the Fourth AnnualSymposium on Human Interaction with ComplexSystems 1998; 131-136.

Johnston R, Weiss P. Analysis of virtual realitytechnology applied in education. MITAT 1997;6:126-127.

Meglan DA, Raju R, Merril GL, et al. Teleos virtualenvironment for simulation-based surgical

education. In Satava, Morgan, Sieburg, et al.,eds. Interactive technology and the new paradigmfor healthcare. Washington , DC: IOS Press,1995; 346-351.

Orlansky J, Dahlman CJ, Hammon CP, et al. Thevalue of simulation for training (IDA paper).Alexandria, VA: Institute for Defense Analyses,1994; 2982.

Peifer JW, Curtis WD, Sinclair MJ. Applied virtualreality for simulation of endoscopic retrogradecholangiopancreatography. In Sieburg,Weghorst, Morgan, eds. Health care in theinformation age. Washington, DC: IOS Press,1996; 36-42.

Playter R, Raibert M. A virtual reality simulator usingadvanced haptic feedback. MITAT 1997; 6:117-121.

Satava RM. Virtual reality surgical simulator: the firststeps. Surg Endosc 1993; 7:203-205.

Taylor A, Colchester A, eds. Medical imagecomputing and computer-assisted interventions(MICCAI-99). Cambridge, MA: SpringerLecture Notes in Computer Science, 1999.

Taylor RH, Bekey GB, Funda J. Proc.of NSFworkshop on computer-assisted surgery.Washington, DC: National Science Foundation,1993.

Taylor RH, et al., eds. Computer-integrated surgery.Cambridge, MA: MIT Press, 1996.

Wells W, Kikinis R, eds. Proc. medical imagecomputing and computer-assisted interventions(MICCAI-98). Cambridge, MA: SpringerLecture Notes in Computer Science, 1998.

Wilson MS, Middlebrook A, Sutton C. MIST VR: avirtual reality trainer for laparoscopic surgeryassesses performance. Ann R Col Surg Engl1997; 79:403-404.


Session 4: Treatment Modalities

While the working group reviewed a wide varietyof thermal and radiation treatments, this paperfocuses on two examples: (1) laser surgery and(2) radiosurgery.

Interstitial Laser Therapy for Tumorsof Solid OrgansLaser beams delivered interstitially via flexible fibersprovide a convenient means of treating lesions in thecenter of solid organs with minimal effects on theoverlying normal tissues. In most cases, the fibers canbe inserted through needles positionedpercutaneously. However, the techniques arecritically dependent on imaging to position fiberscorrectly and to monitor and assess therapy. The keyto success is matching the extent of the lesion withthe extent of the laser-induced necrosis, and themethod is only valid if the true margins of the lesionto be treated can be defined sufficiently accurately.Two forms of tissue destruction can be used: thermaland photochemical. Both have the advantage of nocumulative toxicity, so treatments can be repeated atthe same site if the initial treatment proves to beincomplete.

The thermal technique of ILP involves gentle heatingof the tissue to coagulate a volume of about 1.5 cm indiameter around each fiber tip, larger volumes beingtreatable by using multiple fibers or diffuser fibers.All tissue within the treated area (cells and connectivetissue) is necrosed, and healing is by resorption of thenecrosed tissue with scarring or regeneration. ILP issuitable for lesions in relatively large organs in whichthe surrounding normal tissue can tolerate minimalthermal damage, such as isolated hepatic metastases.It is being studied as a possible alternative to widelocal excision in the initial management of small,localized breast cancers as well as for treatingfibroadenomas of the breast, fibroids of the uterus,and benign prostatic hypertrophy.

The photochemical technique is photodynamictherapy (PDT) in which low-power red light is usedto activate a previously administered photosensitizingdrug. There is no increase in tissue temperature andlittle effect on connective tissue. PDT therefore issafer than ILP near the edge of small organs becausethere is less risk to their mechanical integrity.Although most research so far on PDT has been done

on tumors of the skin and hollow organs, interest hasextended recently to tumors of solid organs, such asthe prostate and pancreas, and the peripheral part ofthe lungs. Animal studies have shown that thesetissues and surrounding normal structures can toleratePDT. Clinical trials have started on localized cancersof the prostate (recurrence after radiotherapy) andpancreas. Tumor necrosis can be achieved with noserious complications, but it is too early to judge whatrole this may have in the overall management of thesepatients.

Contrast-enhanced CT or MR scans best identify ILP-and PDT-treated areas as new zones ofdevascularization, but these are only clearlydemarcated a day or more after treatment. DynamicMR imaging can pick up the temperature changesinduced by ILP in real time, and it may provepossible to correlate these changes with the finalextent of heat-induced necrosis. Real-time monitoringof PDT is more difficult because there are notemperature changes and new techniques are required,perhaps PET or MRS.

The results available so far suggest that ILP and PDTare simple, safe, and effective and warrant moredetailed and extensive studies of their efficacy in arange of solid organs.

Miniature Photon RadiosurgerySystem for Image-Guided Therapy inthe Operating RoomA miniature photon radiosurgery system of 50 kvp Xrays suitable for use in the operating room has beendescribed by Dinsmore and colleagues.1 Briefly, thissource consists of an electronics package (16x11x7cm powered by 12 volts dc) to which is attached a 10-cm long, 3.2-mm diameter probe. When the device isturned on, X rays are emitted from the tip of theprobe. When the probe is inserted into tissue, thedose rate varies approximately as the inverse thirdpower of the distance from the source, and at 1 cmthe typical dose rate is 3 Gy/minute. The rapid falloffof dose results in sharp boundaries between normaland necrotic tissue.

More than 150 metastatic brain tumors have beentreated with photon radiosurgery. In these cases, theprobe tip was placed interstitially into the tumor.


Interstitial treatment of breast tumors also is inprogress. Animal studies of interstitial treatment ofthe kidney and liver have been performed inpreparation for laparascopic clinical use. Percutaneous treatment of these and other organs alsois planned. Skin tumors are being treated by using aconical applicator with a flat end, which serves toposition the probe tip above the tumor. Sphericalapplicators are being used to treat the surgical bed oftumors resected from the breast, brain, and colorectalregion. In principle, any surgical bed, whereverlocated and of various shapes and sizes, can betreated using appropriately designed applicators. Ananimal study is in progress to evaluate the use of thephoton radiosurgery system for the treatment ofmacular degeneration. In this case, the probe isequipped with a microcollimator with a diametersmall enough to be inserted into the eye.

References1. Dinsmore M, Harte KJ, Sliski AP, et al. A new

miniature x-ray source for interstitialradiosurgery: device description. Med Phys1996; 23:45-52.


Session 5: Machine Design and Clinical Framework

System IntegrationFollowing the experimental, scientific evaluation of apotential image-guided procedure, there is a need forsystem integration, engineering, and validation of thetools and instruments used for a clinical operation.This system integration should precede clinicalfeasibility testing. Within the framework of aprocedure, there are individual components thatrequire product development and regulatory approval.Only after these particular elements are adapted to theentire procedure can full integration be accomplished.

Barriers to OvercomeThe lack of standards has been a significant problemin this emerging medical field. The full integration ofmultiple diverse technologies requires theestablishment of standards. The presence of standardcomponents will facilitate and promote systemintegration and make the process substantially easier.

Rapid integration of the components of image-guidedtherapy usually results in early clinical evaluation.Early clinical trials are frequently performed withoutwell-defined outcome measures and are essentiallyrestricted to the validation of individual components.The fully integrated system, however, should bevalidated on short-term outcome measures, such asimmediate complications or lack of postoperativesymptoms. Yet the reliance on short-term outcomemeasures to test the effectiveness and feasibility ofnovel procedures is a major problem. Longer termoutcome measures are essential to help estimate thepotential versus existing market and to conduct anypreliminary research on the possibility ofreimbursement.

Clinical Benefits of New TechnologyWhile reduced invasiveness and cost are obviousjustifications of a minimally invasive procedure, theremust be sufficient clinical evidence that the procedureis safe and results in fewer complications than theconventional procedure. In addition, there is a need toshow improved efficacy and clinical outcome notonly for peer recognition but for widespread patientacceptance.

Breakthrough TechnologiesIn the current research and development environmentof image-guided therapy, there are demands for bettertarget definition using new imaging techniques,optical imaging methods, functional imaging, tumor-seeking contrast agents, genetic or molecular markers.At the same time, there is a need for improvedtherapy delivery systems, such as transcutaneusenergy deposition, gene therapy, targeted drugdelivery, and user interfaces, including datapresentation, effectors, sensors, and display systems.

The academic-clinical environment for image-guidedtherapy is a multidisciplinary, multidepartmental, and,in most cases, multi-institutional effort. There is alsoa great step from the basic to the translationalresearch endeavor. This type of research benefitsgreatly from on-site technology development. Itsometimes is difficult to focus on clinicalimplementation of specific applications in the sameenvironment where the bioengineering, computing,and other technology-dependent components areundertaken. Because of the limitation of nonacademicsites to combine technology development withclinical implementation, this type of research ismainly restricted to academic institutions whereclinical sciences coexist with biomedical engineeringfacilities. There is the possibility to use off-siteacademic-clinical codevelopers.

Industrial partnerships are very important for theintegration of multiple technologies, especially whennonmedical technologies are used. Nevertheless,industrial support is needed for technologyassessment. This support is relatively easy to obtainwhen there are multiple test bed applications for asingle system by a single manufacturer. It is quitedifficult, however, when the integration of diagnosticand therapy devices and computers includes multiplevendors.

The industrial design and architecture of image-guided system requires a high degree of modularity.The industrial architecture and design features shouldinclude safety considerations and significantdocumentation for the reinforcement of regulations.Architecture definition influences standards andinterfaces as well.


Industry PanelInterventional MRILeon Kaufman, Ph.D., Toshiba America MRI,Inc.Interventional MRI has been in use for one and a halfdecades. In the mid-1980s, Lufkin (UCLA) startedusing closed MRI systems for MR-guided needlebiopsy and aspiration, surgical planning, stereotaxy,and monitoring laser and RF ablation therapy.Mueller and colleagues (Massachusetts GeneralHospital) performed MR-guided aspiration biopsy inhumans in 1986. In 1988, open MRI becamecommercially available (Toshiba AKA Diasonics),and Seibel and Groenemeyer used open MRI onpatients for biopsies and tumor treatment and, in1989, published on guidance of a needle-biopsytreatment of tumors with chemicals, catheterinsertion, visualization of the distribution of themedication, oblique slices to align with certainstructures, monitoring of the treatment, and use oflasers. In 1991, Zamorano (Wayne State), Talton(Dynamic Digital Disp.), Hanwehr (Georgetown),Weghorst (University of Washington) and others atIEEE’s Strategic Defense Initiative TechnologyApplications Symposium discussed intraoperativeMRI, automated operation, preplanning, displayneeds, guidance from the MRI system, radiotherapymonitoring, and system and subsystem configurationsand designs. Also in 1991, MR fluoroscopy becamecommercially available, including in-gantry roomdisplay and ghost imaging (Toshiba). GE announceda dedicated interventional MRI system in 1994. Asthis 1999 meeting proves, interventional MRI is stillthe subject of academic meetings rather than apractice in the community. Why is this the case?

For Toshiba, the team that designed andmanufactured the first open MRI, openness was anaccident and interventional MRI an afterthought. Theintent was to develop a low-cost, reliable system thatwas easy to install and maintain. Openness fell out ofthe design of what was to be a low-cost imagingsystem, and it quickly found a claustrophobic patientmarket that made it successful. Nevertheless, anoperator can charge more for the use of MRI in aninterventional procedure as compared to diagnosis. Ifthat is the case, why are all other widely availableopen systems designed for imaging instead ofintervention? Why is GE the only company with adedicated interventional MRI system? Evidently, theothers do not see a market that merits the investmentin a dedicated design. That should be a subject fordiscussion.

Ideal Characteristics of EquipmentAlastair J. Martin, Ph.D., Philips MedicalSystems, Inc.During the past decade, there has been a significantinterest in the development of MR systems forinterventional applications. This follows in thefootsteps of computed tomography, which was firstused in the surgical suite in the early 1980s. CT,however, has failed to establish itself as a routine partof surgical practice outside of a very limited numberof centers. With this in mind, one must consider whatis necessary for MR to succeed in this forum. Thepresence of an MR system in a therapeuticenvironment imposes certain restrictions on bothconventional instrumentation and personnel. Thebenefits offered by the MR system must definitivelyoutweigh these limitations in order for interventionalMR to gain widespread acceptance. Thus, it isdesirable for the MR system to have the full gambit ofcapabilities that are exhibited on state-of-the-artdiagnostic scanners. This includes MR angiography,diffusion and perfusion imaging, fMRI, real-timeimaging, MR thermometry, and MR spectroscopy.Moreover, these capabilities must be offered to theinterventionist in a way that is compatible with andintegrated into their conventional therapeuticenvironment.

An ideal system for image-guided diagnosis andtreatment must exhibit these properties but also mustbe cost-effective to both the industrial developer andthe clinical user. This implies that the resourcesrequired to develop an interventional MR systemmust be in line with the potential market and rates orreimbursement for these products. Clinicians,however, must demonstrate both the benefits of usingthe technology as well as its cost-effectiveness. Largecapital expenditures make the cost-effectivenessjustification much more difficult; thus, there is astrong incentive to contain costs. Theseconsiderations argue against the development of ahighly specialized interventional MR system. A moreprudent approach that is being used by severalvendors is to customize existing products to thespecial needs of the interventional environment. Thisapproach requires the use of a conventional MRsystem as the starting point.

There are currently two major genre of MR scanners:cylindrical bore and biplanar. The biplanarconfiguration offers lateral access to the patient whileat isocenter, but these systems have limited fieldstrength, which, in turn, negatively affects imagequality and imposes functional limitations. Short-bore


cylindrical systems, particularly those exhibiting aflared opening, offer some degree of patient accessduring scanning and provide all of the benefits thatMR has to offer. A platform such as this has theadditional advantages of being acceptable forconventional diagnostic scans when not being usedfor therapy and will be upgradeable with futuredevelopments to diagnostic scanners.

System performance, patient access, and cost aremitigating factors that tend to oppose one another andtherefore it is difficult to define an idealinterventional MR system. The continuing evolutionof applications that may require or benefit from MRimaging guidance also complicates matters. Thedesirable system features that are realisticallyachievable are as follows:

• Approximate the capabilities of a currentdiagnostic scanner.

• Offer acceptable patient access and/or providefast and easy patient transport.

• Build the interventional MR system upon anaccepted diagnostic standard.

• Integrate MR into the existing therapeuticenvironment (as opposed to vice versa).

• Develop an interactive and intuitive interface.

• Minimize the extent and impact of the magnetic,gradient, and RF fields.

• Minimize acoustic noise production.

Ultimately, the ideal system will be defined by thespecific application(s) it will perform. In the interim,when the niche of image guidance in therapy isestablished, it will be necessary for both industrialand clinical partners to make judicious choicesbetween system properties that are often in conflict.


Session 6: Summary Roundtable of Professional Societies

A paradigm shift in medicine has occured. Imaginghas become an important, if not critical, factor in thecare of patients beyond simply diagnosis. This hasresulted from several decades of advances in theimaging sciences. Developments in hardware andsoftware in CT, MR, ultrasound, and nuclearmedicine, in concert with clinician-directedapplications, has made this shift possible.

Organizational IssuesA similar paradigm shift in image-guided therapy isbeginning. This revolution requires synergisticinteractions between scientists, bioengineers, andclinicians from many disciplines. The underlyingscience that enables this field of study must berigorously investigated to maximize the delivery ofhealth care. As such, centers of excellence in image-guided therapy are essential to break down thetraditional barriers, foster collaborative interactions,and elevate this discipline to include hypothesis-driven science.

Within academic health centers, image-guided andcomputer-assisted intervention does not have aclearly defined structure or home. Unlike othermultidisciplinary and multidepartmental programs(e.g., MRI, neuroscience), image-guided, computer-assisted intervention does not have a single advocateor champion.

Training IssuesThis lack of structure in image-guided and computer-assisted intervention makes it difficult to establish anapplicant pool and criteria for research and clinicaltraining. At present, people train elsewhere and bringexpertise to a clinical program rather than learningthe expertise within the program.

Industrial PartnershipsMost academic partnerships with industry tend to bemade with departments and schools so thatlongstanding relationships are established. In image-guided and computer-assisted interventions, the“partnerships” tend to be with individuals andfrequently lack long-term commitment.

RecommendationsOrganizationIt is suggested that the National Institutes of Healthestablish centers for image-guided diagnosis andtreatment similar to what is being done with “In VivoCellular and Molecular Imaging Centers.” These newcenters for image-guided diagnosis and treatmentshould have the following criteria:

• The center should be established within adepartment or as a formal collaboration ofseveral departments, thereby having an advocatewithin the medical center

• The center should be multidisciplinary and havemultidepartmental support.

• The center should be able to support on-sitetechnology development with strong basicscience and system engineering.

• It is desirable that the sponsoring departmenthave a successful track record in industrialpartnerships.

TrainingThe faculty in the funded center should beencouraged to apply for training grants for graduatestudents, as well as research and clinical fellows.Support from training grants should be sought fromall appropriate federal agencies, as well as industry.

Documents

Report of the Joint Working Group on Image-Guided ... › programs_resources › ... · 2D, 3D, 4D two-, three-, and four-dimensional ACRIN American College of Radiology Imaging Network