053 handbook of medical imaging processing and analysis isaac bankman

1. H A N D B O O K O F MEDICAL IMAGING

2. Editorial Advisory Board Dr. William Brody Dr. Elias Zerhouni President Chairman, Department of Radiology Johns Hopkins University and Radiological Science Johns Hopkins Medical Institutions Section Editors Dr. Rangaraj M. Rangayyan Dr. Richard A. Robb Department of Electrical and Computer Engineering Director, Biomedical Imaging Resource University of Calgary Mayo Foundation Dr. Roger P. Woods Dr. H. K. Huang Division of Brain Mapping Department of Radiology UCLA School of Medicine Childrens Hospital of Los Angeles/ University of Southern California Academic Press Series in Biomedical Engineering Joseph Bronzino, Series Editor The focus of this series will be twofold. First, the series will produce a set of core text/ references for biomedical engineering undergraduate and graduate courses. With biomedical engineers coming from a variety of engineering and biomedical backgrounds, it will be necessary to create new cross-disciplinary teaching and self-study books. Secondly, the series will also develop handbooks for each of the major subject areas of biomedical engineering. Joseph Bronzino, the series editor, is one of the most renowned biomedical engineers in the world. He is the Vernon Roosa Professor of Applied Science at Trinity College in Hartford, Connecticut. 3. H A N D B O O K O F MEDICAL IMAGING P R O C E S S I N G A N D A N A LY S I S Editor-in-Chief Isaac N. Bankman, PhD Applied Physics Laboratory Johns Hopkins University Laurel, Maryland San Diego / San Francisco / New York / Boston / London / Sydney / Tokyo 4. This book is printed on acid-free paper. ?s Copyright # 2000 by Academic Press All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Requests for permission to make copies of any part of the work should be mailed to: Permissions Department, Harcourt, Inc., 6277 Sea Harbor Drive, Orlando, Florida, 32887-6777. ACADEMIC PRESS A Harcourt Science and Technology Company 525 B Street, Suite 1900, San Diego, CA 92101-4495, USA http://www.academicpress.com Academic Press Harcourt Place, 32 Jamestown Road, London NW1 7BY, UK Library of Congress Catalog Card Number: 00-101315 International Standard Book Number: 0-12-077790-8 Printed in the United States of America 00 01 02 03 04 COB 9 8 7 6 5 4 3 2 1 5. To Lisa, Judy, and Danny 6. Contents Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii I Enhancement 1 Fundamental Enhancement Techniques Raman B. Paranjape. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Adaptive Image Filtering Carl-Fredrik Westin, Hans Knutsson, and Ron Kikinis. . . . . . . . . . . . . . . . . . . . . . . 19 3 Enhancement by Multiscale Nonlinear Operators Andrew Laine and Walter Huda . . . . . . . . . . . . . . . . . . . . 33 4 Medical Image Enhancement with Hybrid Filters Wei Qian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 II Segmentation 5 Overview and Fundamentals of Medical Image Segmentation Jadwiga Rogowska . . . . . . . . . . . . . . . . . . . . . 69 6 Image Segmentation by Fuzzy Clustering: Methods and Issues Melanie A. Sutton, James C. Bezdek, Tobias C. Cahoon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7 Segmentation with Neural Networks Axel Wismuller, Frank Vietze, and Dominik R. Dersch . . . . . . . . . . . . . . 107 8 Deformable Models Tim McInerney and Demetri Terzopoulos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 9 Shape Constraints in Deformable Models Lawrence H. Staib, Xiaolan Zeng, James S. Duncan, Robert T. Schultz, and Amit Chakraborty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 10 Gradient Vector Flow Deformable Models Chenyang Xu and Jerry L. Prince . . . . . . . . . . . . . . . . . . . . . . . . 159 11 Fully Automated Hybrid Segmentation of the Brain M. Stella Atkins and Blair T. Mackiewich. . . . . . . . . . . . . 171 12 Volumetric Segmentation Alberto F. Goldszal and Dzung L. Pham . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 13 Partial Volume Segmentation with Voxel Histograms David H. Laidlaw, Kurt W. Fleischer, and Alan H. Barr . . 195 III Quantication 14 Two-Dimensional Shape and Texture Quantication Isaac N. Bankman, Thomas S. Spisz, and Sotiris Pavlopoulos 215 15 Texture Analysis in Three Dimensions as a Cue to Medical Diagnosis Vassili A. Kovalev and Maria Petrou. . . . 231 16 Computational Neuroanatomy Using Shape Transformations Christos Davatzikos . . . . . . . . . . . . . . . . . . . . 249 17 Arterial Tree Morphometry Roger Johnson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 18 Image-Based Computational Biomechanics of the Musculoskeletal System Edmund Y. Chao, N. Inoue, J.J. Elias, and F.J. Frassica . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 19 Three-Dimensional Bone Angle Quantication Jens A. Richolt, Nobuhiko Hata, Ron Kikinis, Jens Kordelle, and Michael B. Millis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 20 Database Selection and Feature Extraction for Neural Networks Bin Zheng . . . . . . . . . . . . . . . . . . . . . . . . . 311 21 Quantitative Image Analysis for Estimation of Breast Cancer Risk Martin J. Yaffe, Jeffrey W. Byng, and Norman F. Boyd. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 22 Classication of Breast Lesions in Mammograms Yulei Jiang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 23 Quantitative Analysis of Cardiac Function Osman Ratib. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 24 Image Processing and Analysis in Tagged Cardiac MRI William S. Kerwin, Nael F. Osman, and Jerry L. Prince . 375 25 Image Interpolation and Resampling Philippe Thevenaz, Thierry Blu, and Michael Unser . . . . . . . . . . . . . . . . 393 IV Registration 26 Physical Basis of Spatial Distortions in Magnetic Resonance Images Peter Jezzard. . . . . . . . . . . . . . . . . . . . . 425 27 Physical and Biological Bases of Spatial Distortions in Positron Emission Tomography Images Magnus Dahlbom and Sung-Cheng (Henry) Huang. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 28 Biological Underpinnings of Anatomic Consistency and Variability in the Human Brain N. Tzourio-Mazoyer, F. Crivello, M. Joliot, and B. Mazoyer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449 29 Spatial Transformation Models Roger P. Woods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 vii 7. 30 Validation of Registration Accuracy Roger P. Woods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 31 Landmark-Based Registration Using Features Identied Through Differential Geometry Xavier Pennec, Nicholas Ayache, and Jean-Philippe Thirion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499 32 Image Registration Using Chamfer Matching Marcel Van Herk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515 33 Within-Modality Registration Using Intensity-Based Cost Functions Roger P. Woods. . . . . . . . . . . . . . . . . . . . 529 34 Across-Modality Registration Using Intensity-Based Cost Functions Derek L.G. Hill and David J. Hawkes . . . . . 537 35 Talairach Space as a Tool for Intersubject Standardization in the Brain Jack L. Lancaster and Peter T. Fox . . . . . 555 36 Warping Strategies for Intersubject Registration Paul M. Thompson and Arthur W. Toga . . . . . . . . . . . . . . . . . 569 37 Optimizing the Resampling of Registered Images William F. Eddy and Terence K. Young . . . . . . . . . . . . . . . . . 603 38 Clinical Applications of Image Registration Robert Knowlton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613 39 Registration for Image-Guided Surgery Eric Grimson and Ron Kikinis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623 40 Image Registration and the Construction of Multidimensional Brain Atlases Arthur W. Toga and Paul M. Thompson. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635 V Visualization 41 Visualization Pathways in Biomedicine Meiyappan Solaiyappan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659 42 Three-Dimensional Visualization in Medicine and Biology Richard A. Robb . . . . . . . . . . . . . . . . . . . . . . . . . 685 43 Volume Visualization in Medicine Arie E. Kaufman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 713 44 Fast Isosurface Extraction Methods for Large Image Data Sets Yarden Livnat, Steven G. Parker, and Christopher R. Johnson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 731 45 Morphometric Methods for Virtual Endoscopy Ronald M. Summers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 747 VI Compression Storage and Communication 46 Fundamentals and Standards of Compression and Communication Stephen P. Yanek, Quentin E. Dolecek, Robert L. Holland, and Joan E. Fetter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 759 47 Medical Image Archive and Retrieval Albert Wong and Shyh-Liang Lou . . . . . . . . . . . . . . . . . . . . . . . . . . . . 771 48 Image Standardization in PACS Ewa Pietka . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 783 49 Quality Evaluation for Compressed Medical Images: Fundamentals Pamela Cosman, Robert Gray, and Richard Olshen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 803 50 Quality Evaluation for Compressed Medical Images: Diagnostic Accuracy Pamela Cosman, Robert Gray, and Richard Olshen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 821 51 Quality Evaluation for Compressed Medical Images: Statistical Issues Pamela Cosman, Robert Gray, and Richard Olshen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 841 52 Three-Dimensional Image Compression with Wavelet Transforms Jun Wang and H.K. Huang . . . . . . . . . . . . . 851 53 Medical Image Processing and Analysis Software Thomas S. Spisz and Isaac N. Bankman . . . . . . . . . . . . . . . . 863 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895 viii 8. Foreword The development of medical imaging over the past three decades has been truly revolutionary. For example, in cardi- ology specialized three-dimensional motion estimation algorithms allow myocardial motion and strain measurements using tagged cardiac magnetic resonance imaging. In mammography, shape and texture analysis techniques are used to facilitate the diagnosis of breast cancer and assess its risk. Three-dimensional volumetric visualization of CT and MRI data of the spine, internal organs and the brain has become the standard for routine patient diagnostic care. What is perhaps most remarkable about these advances in medical imaging is the fact that the challenges have required signicant innovation in computational techniques for nearly all aspects of image processing in various elds. The use of multiple imaging modalities on a single patient, for example MRI and PET, requires sophisticated algorithms for image registration and pattern matching. Automated recognition and diagnosis require image segmentation, quantication and enhancement tools. Algorithms for image segmentation and visualization are employed broadly through many applications using all of the digital imaging modalities. And nally, the widespread availability of medical images in digital format has spurred the search for efcient and effective image compression and communication methods. Advancing the frontiers of medical imaging requires the knowledge and application of the latest image manipulation methods. In Handbook of Medical Imaging, Dr. Bankman has assembled a comprehensive summary of the state-of-the-art in image processing and analysis tools for diagnostic and therapeutic applications of medical imaging. Chapters cover a broad spectrum of topics presented by authors who are highly expert in their respective elds. For all those who are working in this exciting eld, the Handbook should become a standard reference text in medical imaging. William R. Brody President, John Hopkins University ix 9. Preface The discoveries of seminal physical phenomena such as X-rays, ultrasound, radioactivity, and magnetic resonance, and the development of imaging instruments that harness them have provided some of the most effective diagnostic tools in medicine. The medical imaging community is now able to probe into the structure, function, and pathology of the human body with a diversity of imaging systems. These systems are also used for planning treatment and surgery, as well as for imaging in biology. Data sets in two, three, or more dimensions convey increasingly vast and detailed information for clinical or research applications. This information has to be interpreted in a timely and accurate manner to benet health care. The examination is qualitative in some cases, quantitative in others; some images need to be registered with each other or with templates, many must be compressed and archived. To assist visual interpretation of medical images, the international imaging community has developed numerous automated techniques which have their merits, limitations, and realm of application. This Handbook presents concepts and digital techniques for processing and analyzing medical images after they have been generated or digitized. It is organized into six sections that correspond to the fundamental classes of algorithms: enhancement, segmentation, quantication, registration, visualization, and a section that covers compression, storage, and communication. The last chapter describes some software packages for medical image processing and analysis. I Enhancement Enhancement algorithms are used to reduce image noise and increase the contrast of structures of interest. In images where the distinction between normal and abnormal tissue is subtle, accurate interpretation may become difcult if noise levels are relatively high. In many cases, enhancement improves the quality of the image and facilitates diagnosis. Enhancement techniques are generally used to provide a clearer image for a human observer, but they can also form a preprocessing step for subsequent automated analysis. The chapters in this section present diverse techniques for image enhancement including linear, nonlinear, xed, adaptive, pixel-based, or multi-scale methods. II Segmentation Segmentation is the stage where a signicant commitment is made during automated analysis by delineating structures of interest and discriminating them from background tissue. This separation, which is generally effortless and swift for the human visual system, can become a considerable challenge in algorithm development. In many cases the segmentation approach dictates the outcome of the entire analysis, since measurements and other processing steps are based on segmented regions. Segmentation algorithms operate on the intensity or texture variations of the image using techniques that include thresholding, region growing, deformable templates, and pattern recognition techniques such as neural networks and fuzzy clustering. Hybrid segmentation and volumetric segmentation are also addressed in this section. III Quantication Quantication algorithms are applied to segmented structures to extract the essential diagnostic information such as shape, size, texture, angle, and motion. Because the types of measurement and tissue vary considerably, numerous techniques that address specic applications have been developed. Chapters in this section cover shape and texture quantication in two- and three-dimensional data, the use of shape transformations to characterize structures, arterial tree morphometry, image-based techniques for musculoskeletal biomechanics, image analysis in mammography, and quantication of cardiac function. In applications where different kinds of tissue must be classied, the effectiveness of quantication depends signicantly on the selection of database and image features, as discussed in this section. A comprehensive chapter covers the choices and pitfalls of image interpolation, a technique included in many automated systems and used particularly in registration. IV Registration Registration of two images of the same part of the body is essential for many applications where the correspondence between the two images conveys the desired information. These two images can be produced by different modalities, for example CT and MRI, can be taken from the same patient with the same instrument at different times, or can belong to two different subjects. Comparison of acquired images with digital anatomic atlas templates also requires registration algorithms. These algorithms must account for the distortions between the two images, which may be caused by differences between the imaging methods, their artifacts, soft tissue elasticity, and variability among subjects. This section explains the physical and biological factors that introduce distortions, presents various linear and nonlinear registration algorithms, describes the Talairach space for brain registration, and addresses interpolation issues inherent in registration. Chapters that describe clinical applications and brain atlases illustrate the current and potential contributions of registration techniques in medicine. xi 10. V Visualization Visualization is a relatively new area that is contributing signicantly to medicine and biology. While automated systems are good at making precise quantitative measurements, the complete examination of medical images is accomplished by the visual system and experience of the human observer. The eld of visualization includes graphics hardware and software specically designed to facilitate visual inspection of medical and biological data. In some cases such as volumetric data, visualization techniques are essential to enable effective visual inspection. This section starts with the evolution of visualization techniques and presents the fundamental concepts and algorithms used for rendering, display, manipulation, and modeling of multidimensional data, as well as related quantitative evaluation tools. Fast surface extraction techniques, volume visualization, and virtual endoscopy are discussed in detail, and applications are illustrated in two and three dimensions. VI Compression, Storage, and Communication Compression, storage, and communication of medical images are related functions for which demand has recently increased signicantly. Medical images need to be stored in an efcient and convenient manner for subsequent retrieval. In many cases images have to be shared among multiple sites, and communication of images requires compression, specialized formats, and standards. Lossless image compression techniques ensure that all the original information will remain in the image after compression but they do not reduce the amount of data considerably. Lossy compression techniques can produce signicant savings in storage but eliminate some information from the image. This section covers fundamental concepts in medical image compression, storage and communication, and introduces related standards such as JPEG, DICOM, and HL-7. Picture archiving and communication systems (PACS) are described and techniques for preprocessing images before storage are discussed. Three chapters address lossy compression issues and one introduces an efcient three-dimensional image compression technique based on the wavelet transform. Acknowledgments This Handbook is the product of a relatively large international team which reects the diversity of the medical imaging community. It has been a great privilege and pleasure for me to interact with the authors. I would like to express my most sincere thanks to the section editors, Bernie Huang, Rangaraj Rangayyan, Richard Robb, and Roger Woods, for their initiative, insight, coordination, and perseverance. The journey of the Handbook was set on its course with the guidance of two distinguished leaders who served on the advisory board of the Handbook: William Brody, president of Johns Hopkins University, and Elias Zerhouni, director of the Radiology and Radiological Science Department at Hopkins. I appreciate the vision and encouragement of Joel Claypool who initiated this Handbook at Academic Press and allowed the journey to progress smoothly in all its phases and for all involved. I also thank Julie Bolduc from Academic Press and Marty Tenney from Textbook Writers Associates for coordinating the com- pilation of the Handbook so effectively. My deepest gratitude goes to my wife, Lisa, and my children, Judy and Danny, for enduring and encouraging the journey graciously. Isaac N. Bankman Johns Hopkins University xii 11. Contributors M. Stella Atkins School of Computing Science Simon Fraser University Burnaby, BC V5A 1S6, Canada Chapter 11 Nicholas Ayache INRIA Sophia-Projet Epidaure 06902 Sophia Antipolis Cedex, France Chapter 31 Isaac N. Bankman Applied Physics Laboratory Johns Hopkins University Laurel, MD 20723 Chapters 14, 53 Alan H. Barr Computer Graphics Laboratory Department of Computer Science Division of Engineering and Applied Science California Institute of Technology Pasadena, CA 91125 Chapter 13 James C. Bezdek Computer Science Department University of West Florida Pensacola, FL 32514 Chapter 6 Thierry Blu Swiss Federal Institute of Technology- Lausanne EPFL/DMT/IOA/Biomedical Imaging Group CH-1015 Lausanne, Switzerland Chapter 25 Norman F. Boyd Division of Clinical Epidemiology and Biostatistics Ontario Cancer Institute Toronto, ON, Canada Chapter 21 Jeffrey W. Byng Health Imaging Eastman Kodak Company Toronto, ON, Canada Chapter 21 Tobias C. Cahoon Computer Science Department University of West Florida Pensacola, FL 32514 Chapter 6 Amit Chakraborty Siemens Corporate Research Princeton, NJ 08540 Chapter 9 Edmund Y. S. Chao Orthopaedic Biomechanics Laboratory Johns Hopkins University School of Medicine Baltimore, MD 21205 Chapter 18 Pamela Cosman Department of Electrical and Computer Engineering University of California at San Diego La Jolla, CA 92093-0407 Chapters 49, 50, 51 Fabrice Crivello Groupe d'Imagerie Neurofonctionelle (GIN) Universite de Caen GIP Cyceron 14074 Caen Cedex, France Chapter 28 Magnus Dahlbom Division of Nuclear Medicine Department of Molecular and Medical Pharmacology UCLA School of Medicine Los Angeles, CA 90095-6942 Chapter 27 Christos Davatzikos Department of Radiology Johns Hopkins University School of Medicine Baltimore, MD 21287 Chapter 16 Dominik R. Dersch Crux Cybernetics Sydney, Australia Chapter 7 Quentin E. Dolecek Applied Physics Laboratory Johns Hopkins University Laurel, MD 20723 Chapter 46 James S. Duncan Image Processing and Analysis Group Departments of Diagnostic Radiology and Electrical Engineering Yale University New Haven, CT 06520-8042 Chapter 9 William F. Eddy Department of Statistics Carnegie Mellon University Pittsburgh, PA 15213 Chapter 37 John J. Elias Orthopaedic Biomechanics Laboratory Johns Hopkins University Baltimore, MD 21205-2196 Chapter 18 Joan E. Fetter Applied Physics Laboratory Johns Hopkins University Laurel, MD 20723 Chapter 46 Kurt W. Fleischer Pixar Animation Studios Richmond, CA 94804 Chapter 13 Peter T. Fox Research Imaging Center University of Texas Health Science Center at San Antonio San Antonio, TX 78232 Chapter 35 Frank J. Frassica Orthopaedic Biomechanics Laboratory Johns Hopkins University Baltimore, MD 21205-2196 Chapter 18 Alberto F. Goldszal Imaging Sciences Program, Clinical Center National Institutes of Health Bethesda, MD 20892 Chapter 12 Robert Gray Stanford University Palo Alto, CA Chapters 49, 50, 51 Eric Grimson Artical Intelligence Laboratory Massachusetts Institute of Technology Cambridge, MA 02139 Chapter 39 xiii 12. Nobuhiko Hata Surgical Planning Lab Department of Radiology Brigham & Women's Hospital Harvard Medical School Boston, MA 02115 Chapters 6, 19 David J. Hawkes Radiological Sciences King's College London Guy's Hospital London SE1 9RT, United Kingdom Chapter 34 Derek L.G. Hill Radiological Sciences King's College London Guy's Hospital London SE1 9RT, United Kingdom Chapter 34 Robert L. Holland Applied Physics Laboratory Johns Hopkins University Laurel, MD 20723 Chapter 46 Sung-Cheng (Henry) Huang Division of Nuclear Medicine Department of Molecular and Medical Pharmacology UCLA School of Medicine Los Angeles, CA 90095-6942 Chapter 27 H.K. Huang Department of Radiology Childrens Hospital of Los Angeles/University of Southern California Los Angeles, CA 90027 Chapter 52 Walter Huda Director, Radiological Physics Department of Radiology SUNY Upstate Medical University Syracuse, NY 13210 Chapter 3 Nozomu Inoue Orthopaedic Biomechanics Laboratory Johns Hopkins University Baltimore, MD 21205-2196 Chapter 18 Peter Jezzard FMRIB Centre John Radcliffe Hospital Headington, Oxford OX3 9DU, United Kingdom Chapter 26 Yulei Jiang Department of Radiology The University of Chicago Chicago, IL 60637 Chapter 22 Roger Johnson Biomedical Engineering Department Marquette University Milwaukee, WI 53233-1881 Chapter 17 Christopher R. Johnson Center for Scientic Computing and Imaging Department of Computer Science University of Utah Salt Lake City, UT 84112 Chapter 44 Marc Joliot Groupe d'Imagerie Neurofonctionelle (GIN) Universite de Caen GIP Cyceron 14074 Caen Cedex, France Chapter 28 Arie E. Kaufman Department of Computer Science State University of New York at Stony Brook Stony Brook, NY 11794-4400 Chapter 43 William S. Kerwin Center for Imaging Science Department of Electrical and Computer Engineering Johns Hopkins University Baltimore, MD 21218 Chapter 24 Ron Kikinis Surgical Planning Laboratory Brigham & Women's Hospital Harvard Medical School Boston, MA 02115 Chapters 2, 19, 39 Robert Knowlton Epilepsy Center University of Alabama School of Medicine Birmingham, AL 35205 Chapter 38 Hans Knutsson Computer Vision Lab Department of Electrical Engineering Linkoping University Linkoping, Sweden Chapter 2 Jens Kordelle Surgical Planning Lab Department of Radiology Brigham & Women's Hospital Harvard Medical School Boston, MA 02115 Chapter 19 Vassili A. Kovalev Institute of Engineering Cybernetics Belarus Academy of Sciences 220012 Minsk, Belarus Chapter 15 David H. Laidlaw Computer Science Department Brown University Providence, RI 02912 Chapter 13 Andrew Laine Department of Biomedical Engineering Columbia University New York, NY 10027 Chapter 3 Jack L. Lancaster Research Imaging Center University of Texas Health Science Center at San Antonio San Antonio, TX 78232 Chapter 35 Yarden Livnat Center for Scientic Computing and Imaging Department of Computer Science University of Utah Salt Lake City, UT 84112 Chapter 44 Shyh-Liang Lou Laboratory for Radiological Informatics Department of Radiology University of California at San Francisco San Francisco, CA 94903 Chapter 47 Blair T. Mackiewich School of Computing Science Simon Fraser University Burnaby, BC V5A 1S6, Canada Chapter 11 Bernard Mazoyer Groupe d'Imagerie Neurofonctionelle (GIN) Universite de Caen GIP Cyceron 14074 Caen Cedex, France Chapter 28 xiv 13. Tim McInerney Department of Computer Science University of Toronto Toronto, ON M5S 3H5, Canada Chapter 8 Michael B. Millis Department of Orthopaedic Surgery Children's Hospital Boston, MA 02115 Chapter 19 Richard Olshen Stanford University Palo Alto, CA Chapters 49, 50, 51 Nael F. Osman Center for Imaging Science Department of Electrical and Computer Engineering Johns Hopkins University Baltimore, MD 21218 Chapter 24 Raman B. Paranjape Electronic Systems Engineering University of Regina Regina, SASK S4S 0A2, Canada Chapter 1 Steven G. Parker Center for Scientic Computing and Imaging Department of Computer Science University of Utah Salt Lake City, UT 84112 Chapter 44 Sotiris Pavlopoulos Institute of Communication and Computer Systems National Technical University of Athens Athens 157-73, Greece Chapter 14 Xavier Pennec INRIA Sophia-Projet Epidaure 06902 Sophia Antipolis Cedex, France Chapter 31 Maria Petrou School of Electronic Engineering Information Technologies and Maths University of Surrey Guildford GU2 7XH, United Kingdom Chapter 15 Dzung L. Pham Laboratory of Personality and Cognition National Institute on Aging National Institutes of Health Baltimore, MD Chapter 12 Ewa Pietka Silesian University of Technology Division of Biomedical Electronics PL. 44-101 Gliwice, Poland Chapter 48 Jerry L. Prince Center for Imaging Science Department of Electrical and Computer Engineering Johns Hopkins University Baltimore, MD 21218 Chapters 10, 24 Wei Qian Department of Radiology College of Medicine and the H. Lee Moftt Cancer and Research Institute University of South Florida Tampa, FL 33612 Chapter 4 Osman Ratib Department of Radiological Sciences UCLA School of Medicine Los Angeles, CA 90095-1721 Chapter 23 Jens A. Richolt Orthopaedic University Clinic "Friedrichsheim" D-60528 Frankfurt, Germany Chapter 19 Richard A. Robb Director, Biomedical Imaging Resource Mayo Foundation Rochester, MN 55905 Chapter 42 Jadwiga Rogowska McLean Brain Imaging Center Harvard Medical School Belmont, MA 02478 Chapter 5 Robert T. Schultz Yale University Child Study Center New Haven, CT 06520 Chapter 9 Meiyappan Solaiyappan Department of Radiology Johns Hopkins University School of Medicine Baltimore, MD 21210 Chapter 41 Thomas S. Spisz Applied Physics Laboratory Johns Hopkins University Laurel, MD 20723 Chapter 53 Lawrence H. Staib Image Processing and Analysis Group Departments of Diagnostic Radiology and Electrical Engineering Yale University New Haven, CT 06520-8042 Chapter 9 Ronald M. Summers Diagnostic Radiology Department Warren Grant Magnuson Clinical Center National Institutes of Health Bethesda, MD 20892-1182 Chapter 45 Melanie A. Sutton Computer Science Department University of West Florida Pensacola, FL 32514 Chapter 6 Demetri Terzopoulos Department of Computer Science University of Toronto Toronto, ON M5S 3H5, Canada Chapter 8 Philippe Thevenaz Swiss Federal Institute of Technology- Lausanne EPFL/DMT/IOA/Biomedical Imaging Group CH-1015 Lausanne, Switzerland Chapter 25 Jean-Philippe Thirion INRIA Sophia-Projet Epidaure 06902 Sophia Antipolis Cedex, France Chapter 31 Paul M. Thompson Department of Neurology Lab of Neuro-Imaging and Brain Mapping Division UCLA School of Medicine Los Angeles, CA 90095-1769 Chapters 36, 40 Arthur W. Toga Department of Neurology Lab of Neuro-Imaging and Brain Mapping Division UCLA School of Medicine Los Angeles, CA 90095-1769 Chapters 36, 40 Nathalie Tzourio-Mazoyer Groupe d'Imagerie Neurofonctionelle (GIN) Universite de Caen GIP Cyceron 14074 Caen Cedex, France Chapter 28 xv 14. Michael Unser Swiss Federal Institute of Technology- Lausanne EPFL/DMT/IOA/Biomedical Imaging Group CH-1015 Lausanne, Switzerland Chapter 25 Marcel Van Herk Radiotherapy Department The Netherlands Cancer Institute 1066 CX Amsterdam, The Netherlands Chapter 32 Frank Vietze Institut fur Radiologische Diagnostik Ludwig-Maximilians-Universitat Munchen Klinikum Innenstadt 80336 Munchen, Germany Chapter 7 Jun Wang Collabria Menlo Park, CA Chapter 52 Carl-Fredrik Westin Surgical Planning Lab Brigham & Women's Hospital Harvard Medical School Boston, MA 02115 Chapter 2 Axel Wismuller Institut fur Radiologische Diagnostik Ludwig-Maximilians-Universitat Munchen Klinikum Innenstadt 80336 Munchen, Germany Chapter 7 Albert Wong Laboratory for Radiological Informatics Department of Radiology University of California at San Francisco San Francisco, CA 94903 Chapter 47 Roger P. Woods Division of Brain Mapping Department of Neurology Neuropsychiatric Institute UCLA School of Medicine Los Angeles, CA 90095-7085 Chapters 29, 30, 33 Chenyang Xu Center for Imaging Science Department of Electrical and Computer Engineering Johns Hopkins University Baltimore, MD 21218 Chapter 10 Martin J. Yaffe Department of Medical Imaging University of Toronto Toronto, ON M4N 3M5, Canada Chapter 21 Stephen P. Yanek Applied Physics Laboratory Johns Hopkins University Laurel, MD 20723 Chapter 46 Terence K. Young Mobil Exploration & Producing Technical Center Dallas, TX 75265-0232 Chapter 37 Xiaolan Zeng Image Processing and Analysis Group Departments of Diagnostic Radiology and Electrical Engineering Yale University New Haven, CT 06520-8042 Chapter 9 Bin Zheng Radiological Imaging Division University of Pittsburgh Pittsburgh, PA 15261 Chapter 20 xvi 15. I Enhancement 1 Fundamental Enhancement Techniques Raman B. Paranjape . . . . . . . . . . . . . . . . . . 3 2 Adaptive Image Filtering Carl-Fredrik Westin, Hans Knutsson, and Ron Kikinis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3 Enhancement by Multiscale Nonlinear Operators Andrew Laine and Walter Huda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4 Medical Image Enhancement with Hybrid Filters Wei Qian . . . . . . . . . . . . . . . . . . . 57 Rangaraj M. Rangayyan University of Calgary M edical images are often deteriorated by noise due to various sources of interference and other phenomena that affect the measurement processes in imaging and data acquisition systems. The nature of the physiological system under investigation and the procedures used in imaging also diminish the contrast and the visibility of details. For example, planar projection nuclear medicine images obtained using a gamma camera as well as single-photon emission computed tomography (SPECT) are severely degraded by Poisson noise that is inherent in the photon emission and counting processes. Although mammograms (X-ray images of the breast) are not much affected by noise, they have limited contrast because of the nature and superimposition of the soft tissues of the breast, which is compressed during the imaging procedure. The small differences that may exist between normal and abnormal tissues are confounded by noise and artifacts, often making direct analysis of the acquired images difcult. In all of the cases just mentioned, some improvement in the appearance and visual quality of the images, even if only subjective, may assist in their interpretation by a medical specialist. Image enhancement techniques are mathematical techniques that are aimed at realizing improvement in the quality of a given image. The result is another image that demonstrates certain features in a manner that is better in some sense as compared to their appearance in the original image. One may also derive or compute multiple processed versions of the original image, each presenting a selected feature in an enhanced appearance. Simple image enhancement techniques are developed and applied in an ad hoc manner. Advanced techniques that are optimized with reference to certain specic requirements and objective criteria are also available. Although most enhancement techniques are applied with the aim of generating improved images for use by a human observer, some techniques are used to derive images that are meant for use by a subsequent algorithm for computer processing. Examples of the former category are techniques to remove noise, enhance contrast, and sharpen the details in a given image. The latter category includes many techniques in the former, but has an expanded range of possibilities, including edge detection and object segmentation. If used inappropriately, enhancement techniques themselves may increase noise while improving contrast, they may eliminate small details and edge sharpness while removing noise, and they may produce artifacts in general. Users need to be cautious to avoid these pitfalls in the pursuit of the best possible enhanced image. 1 16. The rst chapter, by Paranjape, provides an introduction to basic techniques, including histogram manipulation, mean and median ltering, edge enhancement, and image averaging and subtraction, as well as the Butterworth lter. Applications illustrate contrast enhancement, noise suppression, edge enhancement, and mappings for image display systems. Dental radiographic images and CT images of the brain are used to present the effects of the various operations. Most of the methods described in this chapter belong to the ad hoc category and provide good results when the enhancement need is not very demanding. The histogram equalization technique is theoretically well founded with the criterion of maximal entropy, aiming for a uniform histogram or gray-level probability density function. However, this technique may have limited success on many medical images because they typically have details of a wide range of size and small gray-level differences between different tissue types. The equalization procedure based on the global probability with a quantized output gray scale may obliterate small details and differences. One solution is the locally adaptive histogram equalization technique described in this chapter. The limitations of the fundamental techniques motivated the development of adaptive and spatially variable processing techniques. The second chapter by Westin et al. presents the design of the adaptive Wiener lter. The Wiener lter is an optimal lter derived with respect to a certain objective criterion. Westin et al. describe how the Wiener lter may be designed to adapt to local and spatially variable details in images. The lter is cast as a combination of low-pass and high-pass lters, with factors that control their relative weights. Application of the techniques to CT and MR images is illustrated. The third chapter by Laine et al. focuses on nonlinear contrast enhancement techniques for radiographic images, in particular mammographic images. A common problem in contrast or edge enhancement is the accompanying but undesired noise amplication. A wavelet-based framework is described by Laine et al. to perform combined contrast enhancement and denoising, that is, suppression of the noise present in the input image and/or control of noise amplication in the enhancement process. The basic unsharp masking and subtracting Laplacian techniques are included as special cases of a more general system for contrast enhancement. The fourth and nal chapter of the section, by Qian, describes a hybrid lter incorporating an adaptive multistage nonlinear lter and a multiresolution/multiorientation wavelet transform. The methods address image enhancement with noise suppression, as well as decomposition and selective reconstruc- tion of wavelet-based subimages. Application of the methods to enhance microcalcication clusters and masses in mammograms is illustrated. Together, the chapters in this section present an array of techniques for image enhancement: from linear to nonlinear, from xed to adaptive, and from pixel-based to multiscale methods. Each method serves a specic need and has its own realm of applications. Given the diverse nature of medical images and their associated problems, it would be difcult to prescribe a single method that can serve a range of problems. An investigator is well advised to study the images and their enhancement needs, and to explore a range of techniques, each of which may individually satisfy a subset of the requirements. A collection of processed images may be called for in order to meet all the requirements. 2 I Enhancement 17. 1 Fundamental Enhancement Techniques Raman B. Paranjape University of Regina 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Preliminaries and Denitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 Pixel Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1 Compensation for Nonlinear Characteristics of Display or Print Media3.2 Intensity Scaling3.3 Histogram Equalization 4 Local Operators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.1 Noise Suppression by Mean Filtering4.2 Noise Suppression by Median Filtering4.3 Edge Enhancement4.4 Local-Area Histogram Equalization 5 Operations with Multiple Images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.1 Noise Suppression by Image Averaging5.2 Change Enhancement by Image Subtraction 6 Frequency Domain Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 7 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1 Introduction Image enhancement techniques are used to rene a given image, so that desired image features become easier to perceive for the human visual system or more likely to be detected by automated image analysis systems [1, 13]. Image enhancement allows the observer to see details in images that may not be immediately observable in the original image. This may be the case, for example, when the dynamic range of the data and that of the display are not commensurate, when the image has a high level of noise or when contrast is insufcient [4, 5, 8, 9]. Fundamentally, image enhancement is the transformation or mapping of one image to another [10, 14]. This transformation is not necessarily one-to-one, so that two different input images may transform into the same or similar output images after enhancement. More commonly, one may want to generate multiple enhanced versions of a given image. This aspect also means that enhancement techniques may be irreversible. Often the enhancement of certain features in images is accompanied by undesirable effects. Valuable image information may be lost or the enhanced image may be a poor representation of the original. Furthermore, enhancement algorithms cannot be expected to provide information that is not present in the original image. If the image does not contain the feature to be enhanced, noise or other unwanted image components may be inadvertently enhanced without any benet to the user. In this chapter we present established image enhancement algorithms commonly used for medical images. Initial concepts and denitions are presented in Section 2. Pixel-based enhancement techniques described in Section 3 are transformations applied to each pixel without utilizing specically the information in the neighborhood of the pixel. Section 4 presents enhancement with local operators that modify the value of each pixel using the pixels in a local neighborhood. Enhancement that can be achieved with multiple images of the same scene is outlined in Section 5. Spectral domain lters that can be used for enhancement are presented in Section 6. The techniques described in this chapter are applicable to dental and medical images as illustrated in the gures. 2 Preliminaries and Denitions We dene a digital image as a two-dimensional array of numbers that represents the real, continuous intensity distribution of a spatial signal. The continuous spatial signal is Copyright # 2000 by Academic Press. All rights of reproduction in any form reserved. 3 18. sampled at regular intervals and the intensity is quantized to a nite number of levels. Each element of the array is referred to as a picture element or pixel. The digital image is dened as a spatially distributed intensity signal f m; n, where f is the intensity of the pixel, and m and n dene the position of the pixel along a pair of orthogonal axes usually dened as horizontal and vertical. We shall assume that the image has M rows and N columns and that the digital image has P quantized levels of intensity (gray levels) with values ranging from 0 to P 1. The histogram of an image, commonly used in image enhancement and image characterization, is dened as a vector that contains the count of the number of pixels in the image at each gray level. The histogram, hi, can be dened as hi M1 m0 N1 n0 df m; n i; i 0; 1; . . . ; P 1; where dw 1 w 0; 0 otherwise:A useful image enhancement operation is convolution using local operators, also known as kernels. Considering a kernel wk; l to be an array of 2K 162L 1 coefcients where the point k; l 0; 0 is the center of the kernel, convolution of the image with the kernel is dened by: gm; n wk; l f m; n K kK L lL wk; l ? f m k; n l; where gm; n is the outcome of the convolution or output image. To convolve an image with a kernel, the kernel is centered on an image pixel m; n, the point-by-point products of the kernel coefcients and corresponding image pixels are obtained, and the subsequent summation of these products is used as the pixel value of the output image at m; n. The complete output image gm; n is obtained by repeating the same operation on all pixels of the original image [4, 5, 13]. A convolution kernel can be applied to an image in order to effect a specic enhancement operation or change in the image characteristics. This typically results in desirable attributes being amplied and undesirable attributes being suppressed. The specic values of the kernel coefcients depend on the different types of enhancement that may be desired. Attention is needed at the boundaries of the image where parts of the kernel extend beyond the input image. One approach is to simply use the portion of the kernel that overlaps the input image. This approach can, however, lead to artifacts at the boundaries of the output image. In this chapter we have chosen to simply not apply the lter in parts of the input image where the kernel extends beyond the image. As a result, the output images are typically smaller than the input image by the size of the kernel. The Fourier transform Fu; v of an image f m; n is dened as Fu; v 1 MN M1 m0 N1 n0 f m; ne2pjum M vn N ; u 0; 1; 2; . . . ; M 1 v 0; 1; 2; . . . ; N 1; where u and v are the spatial frequency parameters. The Fourier transform provides the spectral representation of an image, which can be modied to enhance desired properties. A spatial- domain image can be obtained from a spectral-domain image with the inverse Fourier transform given by f m; n M1 u0 N1 v0 Fu; ve2pj um M vn N ; m 0; 1; 2; . . . ; M 1; n 0; 1; 2; . . . ; N 1: The forward or inverse Fourier transform of an N6N image, computed directly with the preceding denitions, requires a number of complex multiplications and additions propor- tional to N2 . By decomposing the expressions and eliminating redundancies, the fast Fourier transform (FFT) algorithm reduces the number of operations to the order of N log2N [5]. The computational advantage of the FFT is signicant and increases with increasing N. When N 64 the number of operations are reduced by an order of magnitude and when N 1024, by two orders of magnitude. 3 Pixel Operations In this section we present methods of image enhancement that depend only upon the pixel gray level and do not take into account the pixel neighborhood or whole-image characteristics. 3.1 Compensation for Nonlinear Characteristics of Display or Print Media Digital images are generally displayed on cathode ray tube (CRT) type display systems or printed using some type of photographic emulsion. Most display mechanisms have nonlinear intensity characteristics that result in a nonlinear intensity prole of the image when it is observed on the display. This effect can be described succinctly by the equation em; n Cf m; n; where f m; n is the acquired intensity image, em; n represents the actual intensity output by the display system, and C() is a nonlinear display system operator. In order to correct for the nonlinear characteristics of the display, a transform that is the inverse of the display's nonlinearity must be applied [14, 16]. 4 I Enhancement 19. gm; n Tem; n%C1 Cf m; n gm; n%f m; n; where T is a nonlinear operator which is approximately equal to C1 , the inverse of the display system operator, and gm; n is the output image. Determination of the characteristics of the nonlinearity could be difcult in practice. In general, if a linear intensity wedge is imaged, one can obtain a test image that captures the complete intensity scale of the image acquisition system. However, an intensity measurement device that is linear is then required to assess the output of the display system, in order to determine its actual nonlinear characteristics. A slightly exaggerated example of this type of a transform is presented in Fig.1. Figure 1a presents a simulated CRT display with a logarithmic characteristic. This characteristic tends to suppress the dynamic range of the image decreasing the contrast. Figure 1b presents the same image after an inverse transformation to correct for the display nonlinearity. Although these operations do in principle correct for the display, the primary mechanism for review and analysis of image information is the human visual system, which is fundamentally a nonlinear reception system and adapts locally to the intensities presented. 3.2 Intensity Scaling Intensity scaling is a method of image enhancement that can be used when the dynamic range of the acquired image data signicantly exceeds the characteristics of the display system, or vice versa. It may also be the case that image information is present in specic narrow intensity bands that may be of special interest to the observer. Intensity scaling allows the observer to focus on specic intensity bands in the image by modifying the image such that the intensity band of interest spans the dynamic range of the display [14, 16]. For example, if f1 and f2 are known to dene the intensity band of interest, a scaling transformation may be dened as e f f1 f f2 0 otherwise @ g e f1 f2 f1' ? fmax; where e is an intermediate image, g is the output image, and fmax is the maximum intensity of the display. These operations may be seen through the images in Fig. 2. Figure 2a presents an image with detail in the intensity band from 90 to 170 that may be of interest to, for example a gum specialist. The image, however, is displayed such that all gray levels in the range 0 to 255 are seen. Figure 2b shows the histogram of the input image and Fig. 2c presents the same image with the 90-to-170 intensity band stretched across the output band of the display. Figure 2d shows the histogram of the output image with the intensities that were initially between 90 and 170, but are now stretched over the range 0 to 255. The detail in the narrow band is now easily perceived; however, details outside the band are completely suppressed. (a) (b) FIGURE 1 (a) Original image as seen on a poor-quality CRT-type display. This image has poor contrast, and details are difcult to perceive especially in the brighter parts of the image such as in areas with high tooth density or near lling material. (b) The nonlinearity of the display is reversed by the transformation, and structural details become more visible. Details within the image such as the location of amalgam, the cavity preparation liner, tooth structures, and bony structures are better visualized. 1 Fundamental Enhancement Techniques 5 20. 3.3 Histogram Equalization Although intensity scaling can be very effective in enhancing image information present in specic intensity bands, often information is not available a priori to identify the useful intensity bands. In such cases, it may be more useful to maximize the information conveyed from the image to the user by distributing the intensity information in the image as uniformly as possible over the available intensity band [3, 6, 7]. This approach is based on an approximate realization of an information-theoretic approach in which the normalized histogram of the image is interpreted as the probability density function of the intensity of the image. In histogram equalization, the histogram of the input image is mapped to a new maximally-at histogram. As indicated in Section 2, the histogram is dened as hi, with 0 to P 1 gray levels in the image. The total number of pixels in the image, M*N, is also the sum of all the values in hi. Thus, in order to distribute most uniformly the intensity (b)(a) (c) (d) FIGURE 2 (a) Input image where details of interest are in the 90-to-170 gray level band. This intensity band identies the bony structures in this image and provides an example of a feature that may be of dental interest. (b) Histogram of the input image in (a). (c) This output image selectively shows the intensity band of interest stretched over the entire dynamic range of the display. This specic enhancement may be potentially useful in highlighting features or characteristics of bony tissue in dental X-ray imagery. This technique may be also effective in focusing attention on other image features such as bony lamina dura or recurrent caries. (d) Histogram of the output image in (c). This histogram shows the gray levels in the original image in the 90-to- 170 intensity band stretched over 0 to 255. 6 I Enhancement 21. prole of the image, each bin of the histogram should have a pixel count of M N=P. It is, in general, possible to move the pixels with a given intensity to another intensity, resulting in an increase in the pixel count in the new intensity bin. On the other hand, there is no acceptable way to reduce or divide the pixel count at a specic intensity in order to reduce the pixel count to the desired M N=P. In order to achieve approximate unifor- mity, the average value of the pixel count over a number of pixel values can be made close to the uniform level. A simple and readily available procedure for redistribution of the pixels in the image is based on the normalized cumulative histogram, dened as Hj 1 M ? N j i0 hi; j 0; 1; . . . P 1: The normalized cumulative histogram can be used as a mapping between the original gray levels in the image and the new gray levels required for enhancement. The enhanced image gm; n will have a maximally uniform histogram if it is dened as gm; n P 1 ? Hf m; n: Figure 3a presents an original dental image where the gray levels are not uniformly distributed, while the associated histogram and cumulative histogram are shown in Figs 3b and 3c, respectively. The cumulative histogram is then used to map the gray levels of the input images to the output image shown in Fig. 3d. Figure 3e presents the histogram of Fig. 3d, and Fig. 3f shows the corresponding cumulative histogram. Figure 3f should ideally be a straight line from 0; 0 to P 1; P 1, but in fact only approximates this line to the extent possible given the initial distribution of gray levels. Figure 3g through 1 show the enhancement of a brain MRI image with the same steps as above. 4 Local Operators Local operators enhance the image by providing a new value for each pixel in a manner that depends only on that pixel and others in a neighborhood around it. Many local operators are linear spatial lters implemented with a kernel convolution, some are nonlinear operators, and others impart histogram equalization within a neighborhood. In this section we present a set of established standard lters commonly used for enhancement. These can be easily extended to obtain slightly modied results by increasing the size of the neighborhood while maintaining the structure and function of the operator. 4.1 Noise Suppression by Mean Filtering Mean ltering can be achieved by convolving the image with a 2K 162L 1 kernel where each coefcient has a value equal to the reciprocal of the number of coefcients in the kernel. For example, when L K 1, we obtain wk; l 1=9 1=9 1=9 1=9 1=9 1=9 1=9 1=9 1=9 V b` bX W ba bY ; referred to as the 363 averaging kernel or mask. Typically, this type of smoothing reduces noise in the image, but at the expense of the sharpness of edges [4, 5, 12, 13]. Examples of the application of this kernel are seen in Fig. 4(ad). Note that the size of the kernel is a critical factor in the successful application of this type of enhancement. Image details that are small relative to the size of the kernel are signicantly suppressed, while image details signicantly larger than the kernel size are affected moderately. The degree of noise suppression is related to the size of the kernel, with greater suppression achieved by larger kernels. 4.2 Noise Suppression by Median Filtering Median ltering is a common nonlinear method for noise suppression that has unique characteristics. It does not use convolution to process the image with a kernel of coefcients. Rather, in each position of the kernel frame, a pixel of the input image contained in the frame is selected to become the output pixel located at the coordinates of the kernel center. The kernel frame is centered on each pixel m; n of the original image, and the median value of pixels within the kernel frame is computed. The pixel at the coordinates m; n of the output image is set to this median value. In general, median lters do not have the same smoothing characteristics as the mean lter [4, 5, 8, 9, 15]. Features that are smaller than half the size of the median lter kernel are completely removed by the lter. Large discontinuities such as edges and large changes in image intensity are not affected in terms of gray level intensity by the median lter, although their positions may be shifted by a few pixels. This nonlinear operation of the median lter allows signicant reduction of specic types of noise. For example, ``shot noise'' may be removed completely from an image without attenuation of signicant edges or image characteristics. Figure 5 presents typical results of median ltering. 4.3 Edge Enhancement Edge enhancement in images is of unique importance because the human visual system uses edges as a key factor in the comprehension of the contents of an image [2, 4, 5, 10,13, 14]. Edges in different orientations can be selectively identied and 1 Fundamental Enhancement Techniques 7 22. (c) (b) (d) (a) FIGURE 3 (a) Original image where gray levels are not uniformly distributed. Many image details are not well visualized in this image because of the low contrast. (b) Histogram of the original image in (a). Note the nonuniformity of the histogram. (c) Cumulative histogram of the original image in (a). (d) Histogram-equalized image. Contrast is enhanced so that subtle changes in intensity are more readily observable. This may allow earlier detection of pathological structures. (e) Histogram of the enhanced image in (d). Note that the distribution of intensity counts that are greater than the mean value have been distributed over a larger gray level range. (f ) Cumulative histogram of the enhanced image in (d). (g) Original brain MRI image (courtesy of Dr. Christos Dzavatzikos, Johns Hopkins Radiology Department). (h) through (l) same steps as above for brain image. 8 I Enhancement 23. (g) (f ) (h) (e) FIGURE 3 (Continued). 1 Fundamental Enhancement Techniques 9 24. enhanced. The edge-enhanced images may be combined with the original image in order to preserve the context. Horizontal edges and lines are enhanced with wH1k; l 1 1 1 0 0 0 1 1 1 V b` bX W ba bY or wH2k; l 1 1 1 0 0 0 1 1 1 V b` bX W ba bY ; and vertical edges and lines are enhanced with wV1k; l 1 0 1 1 0 1 1 0 1 V b` bX W ba bY or wV2k; l 1 0 1 1 0 1 1 0 1 V b` bX W ba bY : The omnidirectional kernel (unsharp mask) enhances edges in all directions: KHPk; l 1=8 1=8 1=8 1=8 1 1=8 1=8 1=8 1=8 V b` bX W ba bY : Note that the application of these kernels to a positive-valued (l)(k) ( j)(i) FIGURE 3 (Continued). 10 I Enhancement 25. image can result in an output image with both positive and negative values. An enhanced image with only positive pixels can be obtained either by adding an offset or by taking the absolute value of each pixel in the output image. If we are interested in displaying edge-only information, this may be a good approach. On the other hand, if we are interested in enhancing edges that are consistent with the kernel and suppressing those that are not, the output image may be added to the original input image. This addition will most likely result in a nonnegative image. Figure 6 illustrates enhancement after the application of the kernels wH1, wV1, and wHP to the image in Fig. 3a and 3g. Figures 6a, b, and c show the absolute value of the output images obtained with wH1, wV1, and wHP, respectively applied to the dental image while 6d, e, and f show the same for the brain image. In Fig. 7, the outputs obtained with these three kernals are added to the original images of Fig. 3a and g. In this manner the edge information is enhanced while retaining the context information of the original image. This is accomplished in one step by (c) (d) (b)(a) FIGURE 4 (a) Original bitewing X-ray image. (b) Original image in (a) corrupted by added Gaussian white noise with maximum amplitude of + 25 gray levels. (c) Image in (b) convolved with the 363 mean lter. The mean lter clearly removes some of the additive noise; however, signicant blurring also occurs. This image would not have signicant clinical value. (d) Image in (b) convolved with the 969 mean lter. This lter has removed almost all of the effects of the additive noise. However, the usefulness of this lter is limited because the lter size is similar to that of signicant structures within the image, causing severe blurring. 1 Fundamental Enhancement Techniques 11 26. convolving the original image with the kernel after adding 1 to its central coefcient. Edge enhancement appears to provide greater contrast than the original imagery when diagnosing pathologies. Edges can be enhanced with several edge operators other than those just mentioned and illustrated. Some of these are described in the chapter entitled `Òverview and Fundamentals of Medical Image Segmentation,'' since they also form the basis for edge-based segmentation. 4.4 Local-Area Histogram Equalization A remarkably effective method of image enhancement is local- area histogram equalization, obtained with a modication of the pixel operation dened in Section 3.3. Local-area histogram equalization applies the concepts of whole-image histogram equalization to small, overlapping local areas of the image [7, 11]. It is a nonlinear operation and can signicantly increase the observability of subtle features in (b)(a) (c) FIGURE 5 (a) Image in Fig. 4b enhanced with a 363 median lter. The median lter is not as effective in noise removal as the mean lter of the same size; however, edges are not as severely degraded by the median lter. (b) Image in Fig. 4a with added shot noise. (c) Image in gure 5(b) enhanced by a 363 median lter. The median lter is able to signicantly enhance this image, allowing almost all shot noise to be eliminated. This image has good diagnostic value. 12 I Enhancement 27. (d) (a) (b) (c) FIGURE 6 (a) Absolute value of output image after convolution of wH1 with the image in Fig. 3a. (b) Absolute value of output image after convolution of wV1 with the image in Fig. 3a. (c) Absolute value of output image after convolution of wHP. (d through f ) same as a, b, and c using image in Fig. 3g. (e) (f ) 1 Fundamental Enhancement Techniques 13 28. (a) (b) (c) (d) (f )(e) FIGURE 7 (a) Sum of original image in Fig. 3a and its convolution with wH1, (b) with wV1, and (c) with wHP. (d through f ) same as a, b, and c using image in Fig. 3g. 14 I Enhancement 29. the image. The method formulated as shown next is applied at each pixel m; n of the input image. hLAm; ni K kK L lL df m l; n k i; i 0; 1; . . . P 1 HLAm; nj 1 2K 1 ? 2L 1 j i0 hLAm; ni; j 0; 1; . . . P 1 gm; n P 1 ? HLAm; nf m; n where hLAm; ni is the local-area histogram, HLAm; nj is the local-area cumulative histogram, and gm; n is the output image. Figure 8 shows the output image obtained by enhancing the image in Fig.2a with local-area histogram equalization using K L 15 or a 31631 kernel size. Local-area histogram equalization is a computationally intensive enhancement technique. The computational com- plexity of the algorithm goes up as the square of the size of the kernel. It should be noted that since the transformation that is applied to the image depends on the local neighborhood only, each pixel is transformed in a unique way. This results in higher visibility for hidden details spanning very few pixels in relation to the size of the full image. A signicant limitation of this method is that the mapping between the input and output images is nonlinear and highly nonmonotonic. This means that it is inappropriate to make quantitative measurements of pixel intensity on the output image, as the same gray level may be transformed one way in one part of the image and a completely different way in another part. 5 Operations with Multiple Images This section outlines two enhancement methods that require more than one image of the same scene. In both methods, the images have to be registered and their dynamic ranges have to be comparable to provide a viable outcome. 5.1 Noise Suppression by Image Averaging Noise suppression using image averaging relies on three basic assumptions: (1) that a relatively large number of input images are available, (2) that each input image has been corrupted by the same type of additive noise, and (3) that the additive noise is random with zero mean value and independent of the image. When these assumptions hold, it may be advantageous to acquire multiple images with the specic purpose of using image averaging [1] since with this approach even severely corrupted images can be signicantly enhanced. Each of the noisy images aim; n can be represented by aim; n f m; n dim; n; where f m; n is the underlying noise-free image, and dim; n is the additive noise in that image. If a total of Q images are available, the averaged image is gm; n 1 Q Q i1 aim; n such that E gm; nf g f m; n and sg sd Q p ; where Ef ? g is the expected value operator, sg is the standard deviation of gm; n, and sd is that of the noise. Noise suppression is more effective for larger values of Q. 5.2 Change Enhancement by Image Subtraction Image subtraction is generally performed between two images that have signicant similarities between them. The purpose of image subtraction is to enhance the differences between two images (1). Images that are not captured under the same or very similar conditions may need to be registered [17]. This may be the case if the images have been acquired at different times or under different settings. The output image may have a very small dynamic range and may need to be rescaled to the available display range. Given two images f1m; n and f2m; n, the rescaled output image gm; n is obtained with FIGURE 8 Output image obtained when local-area histogram equalization was applied to the image in Fig. 2a. Note that the local-area histogram equalization produces very high-contrast images, emphasizing detail that may otherwise be imperceptible. This type of enhancement is computationally very intensive and it may be useful only for discovery purposes to determine if any evidence of a feature exists. 1 Fundamental Enhancement Techniques 15 30. bm; n f1m; n f2m; n gm; n fmax ? bm; n minfbm; ng maxfbm; ng minfbm; ng where fmax is the maximum gray level value available, bm; n is the unstretched difference image, and minfbm; ng and maxfbm; ng are the minimal and maximal values in bm; n, respectively. 6 Frequency Domain Techniques Linear lters used for enhancement can also be implemented in the frequency domain by modifying the Fourier transform of the original image and taking the inverse Fourier transform. When an image gm; n is obtained by convolving an original image f m; n with a kernel wm; n, gm; n wm; n f m; n; the convolution theorem states that Gu; v, the Fourier transform of gm; n, is given by Gu; v Wu; vFu; v; where Wu; v and Fu; v are the Fourier transforms of the kernel and the image, respectively. Therefore, enhancement can be achieved directly in the frequency domain by multiplying Fu; v, pixel-by-pixel, by an appropriate Wu; v and forming the enhanced image with the inverse Fourier transform of the product. Noise suppression or image smoothing can be obtained by eliminating the high-frequency components of Fu; v, while edge enhancement can be achieved by eliminating its low-frequency components. Since the spectral ltering process depends on a selection of frequency parameters as high or low, each pair u; v is quantied with a measure of distance from the origin of the frequency plane, Du; v u2 v2 p ; which can be compared to a threshold DT to determine if u; v is high or low. The simplest approach to image smoothing is the ideal low-pass lter WLu; v, dened to be 1 when Du; v DT and 0 otherwise. Similarly, the ideal high-pass lter WH u; v can be dened to be 1 when Du; v ! DT and 0 otherwise. However, these lters are not typically used in practice, because images that they produce generally have spurious structures that appear as intensity ripples, known as ringing [5]. The inverse Fourier transform of the rectangular window WLu; v or WH u; v has oscillations, and its convolution with the spatial-domain image produces the ringing. Because ringing is associated with the abrupt 1 to 0 discontinuity of the ideal lters, a lter that imparts a smooth transition between the desired frequencies and the attenuated ones is used to avoid ringing. The commonly used Butterworth low-pass and high-pass lters are dened respectively as BLu; v 1 1 c Du; v=DT 2n and BH u; v 1 1 c DT =Du; v 2n ; where c is a coefcient that adjusts the position of the transition and n determines its steepness. If c 1, these two functions take the value 0.5 when Du; v DT . Another common choice for c is 2 p 1, which yields 0.707 ( 3 dB) at the cutoff DT . The most common choice of n is 1; higher values yield steeper transitions. The threshold DT is generally set by considering the power of the image that will be contained in the preserved frequencies. The set S of frequency parameters u; v that belong to the preserved region, i.e., Du; v DT for low-pass and Du; v ! DT for high-pass, determines the amount of retained image power. The percentage of total power that the retained power constitutes is given by b u;v [ S Fu; vj j 2 Vu;v Fu; vj j2 6100 and is used generally to guide the selection of the cutoff threshold. In Fig. 9a, circles with radii rb that correspond to ve different b values are shown on the Fourier transform of an original MRI image in Fig. 9e. The u v 0 point of the transform is in the center of the image in Fig. 9a. The Butterworth low-pass lter obtained by setting DT equal to rbfor b 90% , with c 1 and n 1, is shown in Fig. 9b where bright points indicate high values of the function. The corresponding ltered image in Fig. 9f shows the effects of smoothing. A high-pass Butterworth lter with DT set at the 95% level is shown in Fig. 9d, and its output in Fig. 9h highlights the highest frequency components that form 5% of the image power. Figure 9c shows a band-pass lter formed by the conjunction of a low-pass lter at 95% and a high-pass lter at 75%, while the output image of this band-pass lter is in Fig. 9g. 7 Concluding Remarks This chapter has focused on fundamental enhancement techniques used on medical and dental images. These techniques have been effective in many applications and are commonly used in practice. Typically, the techniques presented in this chapter form a rst line of algorithms in attempts to 16 I Enhancement 31. enhance image information. After these algorithms have been applied and adjusted for best outcome, additional image enhancement may be required to improve image quality further. Computationally more intensive algorithms may then be considered to take advantage of context-based and object- based information in the image. Examples and discussions of such techniques are presented in subsequent chapters. Acknowledgments The author thanks Dr. Bill Moyer for providing access to data and for discussions. Mr. Mark Sluser and Mr. Rick Yue supported this work. NSERC and the Government of Saskatchewan, Post Secondary Education, provided nancial support for this project. TRLabs Regina provided laboratory facilities. References 1. de Graaf, C. N., and Viergever, M. A. (ed.), Information Processing in Medical Imaging. Plenum Press, New York, 1988. 2. Fong, Y. S., Pomala-Roez, C. A., and Wong, X. H., Comparison study of non-linear lters in image processing applications. Opti. Eng. 28(7), 749760 (1989). 3. Frei, W., Image enhancement by image hyberbolization. Comp. Graph. Image Process. 6, 286294 (1977). FIGURE 9 Filtering with the Butterworth lter. (a) Fourier transform of MRI image in (e); the ve circles correspond to the b values 75, 90, 95, 99, and 99.5%. (b) Fourier transform of low-pass lter with b 90% which provides the output image in (f ). (c) Band-pass lter with band b 75% to b 90% whose output is in (g). (d) High-pass lter with b 95%, which yields the image in (h). (Courtesy of Dr. Patricia Murphy, Johns Hopkins University Applied Physics Laboratory.) 1 Fundamental Enhancement Techniques 17 32. 4. Gonzalez, R. C., and Wintz, P., Digital Image Processing. Addison-Wesley, Reading, MA, 1987. 5. Gonzalez, R. C., and Woods, R. E., Digital Image Processing Addison-Wesley, Reading, MA, 1992. 6. Hall, E. H. Almost uniform distributions from image enhancement, IEEE Trans. Comp. C-23(2), 207208 (1974). 7. Ketchum, D. J., Real-time image enhancement techniques. Proc. SPIE/OSA 74, 120125 (1976). 8. Lewis, R., Practical Digital Image Processing. Ellis Horwood, West Sussex, UK, 1990. 9. Low, A., Introduction to Computer Vision and Image Processing. McGraw-Hill, U.K., 1991. 10. Niblack, W., An Introduction to Digital Image Processing. Prentice Hall, Englewood Cliffs, NJ, 1986. 11. Pizer, S. M., Amburn, P., Austin, R., Cromartie, R., Geselowitz, A., Geer, T., tar Haar Remeny, J., Zimmerman, J. B., and Zuiderveld, K., Adaptive histogram equalization and its variations. Comp. Vision Graph. Image Process. 39, 355368 (1987). 12. Restrepo, A., and Bovik, A., An adaptive trimmed mean lter for image restoration. IEEE Trans. Acoustics, Speech, Signal Process. ASSP-36(8), 88138818 (1988). 13. Rosenfeld, A., and Kak, A., Digital Picture Processing. Academic Press, New York, 1982. 14. Russ, J. The Image Processing Handbook, 2nd Ed. CRC Press, Boca Raton, FL 1994. 15. Sinha, P. K., and Hong, Q. H., An improved median lter. IEE Trans. Med. Imaging MI-9(3), 345346 (1990). 16. Wahl, F., Digital Image Signal Processing. Artech House, Norwood, MA, 1987. 17. Watkins, C., Sadun, A., and Marenka, A., Modern Image Processing: Warping, Morphing and Classical Techniques. Academic Press, London, 1993. 18 I Enhancement 33. 2 Adaptive Image Filtering Carl-Fredrik Westin Ron Kikinis Harvard Medical School Hans Knutsson Linkoping University 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2 Multidimensional Spatial Frequencies and Filtering . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.1 Spatial Frequency2.2 Filtering2.3 Unsharp Masking 3 Random Fields and Wiener Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1 Autocorrelation and Power Spectrum3.2 The Wiener Filter 4 Adaptive Wiener Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 4.1 Local Adaptation4.2 Nonlinear Extension by a Visibility Function 5 Anisotropic Adaptive Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.1 Anisotropic Adaptive Filtering in Two Dimensions5.2 Multidimensional Anisotropic Adaptive Filtering5.3 Adaptation Process5.4 Estimation of Multidimensional Local Anisotropy Bias5.5 Tensor Mapping5.6 Examples of Anisotropic Filtering in 2D and 3D References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 1 Introduction Adaptive lters are commonly used in image processing to enhance or restore data by removing noise without signicantly blurring the structures in the image. The adaptive ltering literature is vast and cannot adequately be summarized in a short chapter. However, a large part of the literature concerns one-dimensional (1D) signals [1]. Such methods are not directly applicable to image processing and there are no straightforward ways to extend 1D techniques to higher dimensions primarily because there is no unique ordering of data points in dimensions higher than one. Since higher- dimensional medical image data are not uncommon (2D images, 3D volumes, 4D time-volumes), we have chosen to focus this chapter on adaptive ltering techniques that can be generalized to multidimensional signals. This chapter assumes that the reader is familiar with the fundamentals of 1D signal processing [2]. Section 2 addresses spatial frequency and ltering. 2D spatial signals and their Fourier transforms are shown to illuminate the similarities to signals in one dimension. Unsharp masking is described as an example of simple image enhancement by spatial ltering. Section 3 covers random elds and is intended as a primer for the Wiener lter, which is introduced in Section 3.2. The Wiener formulation gives a lowpass lter with a frequency characteristic adapted to the noise level in the image. The higher the noise level, the more smoothing of the data. In Section 4 adaptive Wiener formulations are presented. By introducing local adaptation of the lters, a solution more suitable for nonstationary signals such as images can be obtained. For example, by using a ``visibility function,'' which is based on local edge strength, the lters can be made locally adaptive to structures in the image so that areas with edges are less blurred. Section 5 is about adaptive anisotropic ltering. By allowing lters to change from circularly/spherically symmetric (isotropic) to shapes that are closely related to the image structure, more noise can be suppressed without severe blurring of lines and edges. A computationally efcient way of implementing shift-variant anisotropic lters based on a non-linear combination of shift-invariant lter responses is described. 2 Multidimensional Spatial Frequencies and Filtering At a conceptual level, there is a great deal of similarity between 1D signal processing and signal processing in higher dimensions. For example, the intuition and knowledge gained from working with the 1D Fourier transform extends fairly straight- forwardly to higher dimensions. For overviews of signal processing techniques in 2D see Lim [3], or Granlund and Knutsson for higher dimensional signal processing [4]. 2.1 Spatial Frequency The only difference between the classical notions of time frequencies and spatial frequencies is the function variable used: Instead of time, the variable is spatial position in the Copyright # 2000 by Academic Press. All rights of reproduction in any form reserved. 19 34. latter case. A multidimensional sinusoidal function can be written as f cosxT ê; 1 where x is the spatial position vector and ê is a normalized vector dening the orientation of the wave x;ê [ rn . The signal is constant on all hyperplanes normal to ê. For any 1D function g, the multidimensional function f gxT ê 2 will have a Fourier transform in which all energy is concentrated on a line through the origin with direction e. For example, for a plane in three dimensions given by f dxT ê; 3 where d denotes the Dirac function, the Fourier transform will be concentrated on a line with orientation normal to the plane, and the function along this line will be constant. Examples of 2D sinusoidal functions and their Fourier transforms are shown in Fig. 1. The top gures show a transform pair of a sinusoid of fairly low frequency. The Fourier transform, which contains two Dirac impulse functions, is shown to the left and may be expressed as F1 du x1 du x1; 4 where u denotes the 2D frequency variable, x1 the spatial frequency of the signal. The bottom gures show the transform pair of a signal consisting of the same signal in F1 plus another sinusoidal signal with frequency o2. F2 F1 du x2 du x2: 5 F1 F1 F2 f1 f2 f1 FIGURE 1 (Top) Sinusoidal signal with low spatial frequency. (Bottom) Sum of the top signal and a sinusoidal with a higher spatial frequency. 20 I Enhancement 35. 2.2 Filtering Linear ltering of a signal can be seen as a controlled scaling of the signal components in the frequency domain. Reducing the components in the center of the frequency domain (low frequencies), gives the high-frequency components an increased relative importance, and thus highpass ltering is performed. Filters can be made very selective. Any of the Fourier coefcients can be changed independently of the others. For example, let H be a constant function minus a pair of Dirac functions symmetrically centered in the Fourier domain with a distance jx1j from the center, H 1 du x1 du x1: 6 This lter, known as a notch lter, will leave all frequency components untouched, except the component that corre- sponds to the sinusoid in Fig. 1, which will be completely removed. A weighting of the Dirac functions will control how much of the component is removed. For example, the lter H 1 0:9du x1 du x1; 7 will reduce the signal component to 10% of its original value. The result of the application of this lter to the signal F1 F2. (Fig. 1, bottom) is shown in Fig. 2. The lower-frequency component is almost invisible. Filters for practical applications have to be more general than ``remove sinusoidal component cosxT x.'' In image enhancement, lters are designed to remove noise that is spread out all over the frequency domain. It is a difcult task to design lters that remove as much noise as possible without removing important parts of the signal. 2.3 Unsharp Masking Unsharp masking, an old technique known to photographers, is used to change the relative highpass content in an image by subtracting a blurred (lowpass ltered) version of the image [5]. This can be done optically by rst developing an unsharp picture on a negative lm and then using this lm as a mask in a second development step. Mathematically, unsharp masking can be expressed as ^f af bflp 8 where a and b are positive constants, a ! b. When processing digital image data, it is desirable to keep the local mean of the image unchanged. If the coefcients in the lowpass lter flp are normalized, i.e., their sum equals 1, the following formulation of unsharp masking ensures unchanged local mean in the image: ^f 1 a b af bflp: 9 By expanding the expression in the parentheses af bflp aflp af flp bflp, we can write Eq. (9) as ^f flp a a b f flp; 10 which provides a more intuitive way of writing unsharp masking. Further rewriting yields ^f flp gf flp 11 flp gfhp; 12 where g can be seen as a gain factor of the high frequencies. For HF1 F2 h f1 f2 FIGURE 2 Notch ltering of the signal f1 f2, a sum of the sinusoids. The application of the lter h in Eq. (7) reduces the low-frequency component to one-tenth of its original value. 2 Adaptive Image Filtering 21 36. g 1, the lter is the identity map, flp fhp flp f flp f , and the output image equals the input image. For g41 the relative highpass content in the image is increased, resulting in higher contrast; for g51 the relative highpass content is decreased and the image gets blurred. This process can be visualized by looking at the corresponding lters involved. In the Fourier domain, the lowpass image flp can be written as the product of a lowpass lter Hlp and the Fourier transform of the original image, Flp HlpF: 13 Figure 3 (top left) shows Hlp and the highpass lter that can be constructed thereof, 1 Hlp (top right). Figure 3 (bottom left) shows the identity lter from adding the two top lters, and (bottom right) a lter where the highpass component has been amplied by a factor of 2. Figure 4 shows a slice of a CT data set through a skull (left), and the result of unsharp masking with g 2, i.e., a doubling of the highpass part of the image (right). The lowpass lter used (flp in Eq. (11)) was a Gaussian lter with a standard deviation of 4, implemented on a 21621 grid. A natural question arises. How should the parameters for the lowpass lter be chosen? It would be advantageous if the lter could adapt to the image automatically either through an a priori model of the data or by some estimation process. There Lowpass lter, Hlp Highpass lter, 1 Hlp Hlp Hhp Hlp 2Hhp FIGURE 3 Visualization of the lters involved in unsharp masking. See also Plate 1. 22 I Enhancement 37. are two main categories of such adaptive lters, lters for image enhancement and lters for image restoration. The two categories mainly differ in the view of the data that is to be ltered. The method of unsharp masking belongs to the rst category, image enhancement. The image is made crisper by increasing the image contrast. The input image was not considered to be degraded in any way, and the purpose of the algorithm was just to improve the appearance of the image. In image restoration, as the name implies, the image data are modeled as being degraded by a (normally unknown) process, and the task at hand is to `ùndo'' this degradation and restore the image. The models of image degradation commonly involve random noise processes. Before we introduce the well- known Wiener lter, a short description of stochastic processes in multiple dimensions is given. In multiple dimensions stochastic processes are customary referred to as random elds. 3 Random Fields and Wiener Filtering A central problem in the application of random elds is the estimation of various statistical parameters from real data. The Wiener lter that will be discussed later requires knowledge of the spectral content of the image signal and the background noise. In practice these are, in general, not known and have to be estimated from the image. 3.1 Autocorrelation and Power Spectrum A collection of an innite number of random variables dened on an n-dimensional space x [ Rn is called a random eld. The autocorrelation function of a random eld f x is dened as the expected value of the product of two samples of the random eld, Rff x; xH Eff xf xH g; 14 where E denotes the statistical expectation operator. A random eld is said to be stationary if the expectation value and the autocorrelation function are shift-invariant, i.e., the expectation value is independent of the spatial position vector x, and the autocorrelation is a function only of s x xH . Rff s Eff x sf xg: 15 The power spectrum of a stationary random process is dened by the Fourier transform of the autocorrelation function. Sff fRff : 16 Since the autocorrelation function is always symmetric, the power spectrum is always a real function. FIGURE 4 Filtering CT data using unsharp masking. The highpass information of the original image (left) is twice as high in the result image (right). Note how details have been amplied. This technique works well due to the lack of noise in the image. 2 Adaptive Image Filtering 23 38. A random process nx is called a white noise process if Rnns s2 nds: 17 From Eq. (16), the power spectrum of a white process is a constant: Snn s2 n: 18 The Wiener-Khinchin theorem [6] states that Eff 2 xg R0 1 2pn Sudu; 19 where n is the dimension of the signal (the theorem follows directly from Eq. 16, by taking the inverse Fourier transform of Su for s 0). This means that the integral of the power spectrum of any process is positive. It can also be shown that the power spectrum is always nonnegative [6]. The cross-correlation function of two random processes is dened as Rfg x; xH Eff xgxH g 20 3.2 The Wiener Filter The restoration problem is essentially one of optimal ltering with respect to some error criterion. The mean-squared error (MSE) criterion has for