Supervised Dimensionality Reduction - Universiteit .Erik Satie, Gnossienne No. 3. Contents 0 Introduction

  • View
    214

  • Download
    0

Embed Size (px)

Text of Supervised Dimensionality Reduction - Universiteit .Erik Satie, Gnossienne No. 3. Contents 0...

  • Supervised Dimensionality Reductionand Contextual Pattern Recognition

    in Medical Image Processing

    Marco Loog

  • this book was typeset by M. Loog using LATEX2 cover design by M. Loog

    ISBN 9039338043

    This work is subject to copyright. All rights are reserved, whether the whole or part of the material isconcerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting,reproduction on microfilm or in any other form or by any other means, and storage in data banks, synapticweights, or hidden variables in electronic, mechanical, virtual or any other way. Permission for duplica-tion of this publication or parts thereof must always be obtained in writing from the author. Violationsare liable for prosecution.

    Copyright c 2004 Marco Loog

    Printed by Ponsen & Looijen, Wageningen, The Netherlands

  • Supervised Dimensionality Reductionand Contextual Pattern Recognition

    in Medical Image Processing

    Gesuperviseerde dimensionaliteitsreductieen contextuele patroonherkenningin de medische beeldverwerking

    (met een samenvatting in het Nederlands)

    proefschrift

    ter verkrijging van de graad van doctor aan de Universiteit Utrecht op gezagvan de Rector Magnificus, prof.dr. W. H. Gispen, ingevolge het besluit van hetCollege voor Promoties in het openbaar te verdedigen op dinsdag 14 september2004 des middags te 1245 uur

    door

    Marco Loog

    geboren op 2 juni 1973 te Willemstad, Curacao

  • promotor Prof.dr.ir. M. A. ViergeverImage Sciences InstituteUniversity Medical Center Utrecht, The Netherlands

    copromotoren Dr. B. van GinnekenImage Sciences InstituteUniversity Medical Center Utrecht, The Netherlands

    Dr.ir. R. P. W. DuinFaculty of Electrical Engineering, Mathematics and Computer ScienceDelft University of Technology, The Netherlands

    The research described in this thesis was carried out at the Image Sciences Institute, University MedicalCenter Utrecht, the Netherlands, under the auspices of ImagO, the Utrecht Graduate School for Biomed-ical Image Sciences. The project was financially supported by the Dutch Ministry of Economic Affairswithin the framework of the innovation-driven research program (IOP image processing, project numberIBV98002).

    Financial support for publication of this thesis was kindly provided by Philips Medical Systems Neder-land B.V. (Medical IT - Advanced Development), the Rontgen Stichting Utrecht, and Utrecht University.

  • beoordelingscommissie Prof.dr. J. J. DuistermaatDepartment of MathematicsUtrecht University, The Netherlands

    Prof.dr. R. D. GillDepartment of MathematicsUtrecht University, The Netherlands

    Prof.dr.ir. B. M. ter Haar RomenyDepartment of Biomedical EngineeringEindhoven University of Technology, The Netherlands

    Prof.dr. J. KittlerDepartment of Electronic and Electrical EngineeringUniversity of Surrey, United Kingdom

    Prof.dr. M. ProkopDepartment of RadiologyUniversity Medical Center Utrecht, The Netherlands

  • de maniere a obtenir un creuxErik Satie, Gnossienne No. 3

  • Contents

    0 Introduction + Summary 10.1 On Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.2 On Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40.3 On Image Processing for CAD in Chest Radiography . . . . . . . . . . . . . . . . . . . . . . 50.4 On Self-Containedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    1 A Heteroscedastic Extension of LDA:The Chernoff Criterion 71.1 The Chernoff Criterion: Two-Class Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.2 The Multi-Class Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.4 Discussion + Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

    2 The Canonical Contextual Correlation Projection 212.1 Supervised Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.2 LDA + a Direct Approach to Incorporating Context . . . . . . . . . . . . . . . . . . . . . . . 232.3 Canonical Contextual Correlation Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.4 An Illustrative Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.5 Discussion + Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

    3 Nonparametric Local LinearDimensionality Reduction for Regression 353.1 Local Linear Dimensionality Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.2 Relative Influence of Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.3 Concluding Remarks + Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

    4 Iterated Contextual Pixel Classification 394.1 Iterated Contextual Pixel Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414.2 Experimental Setup + Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

  • 5 Segmentation of the Posterior Ribsin Chest Radiographsusing Iterated Contextual Pixel Classification 595.1 Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605.2 Iterated Contextual Pixel Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615.3 Experiments, Results, + Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.4 Discussion + Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

    6 Suppression of Bony Structuresfrom Projection Chest Radiographsby Dual Energy Faking 776.1 Materials + Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796.2 Pilot + Leave-One-Out Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 816.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836.4 Discussion + Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

    7 Notes 95

    Bibliography 99

    Een samenvatting in het Nederlands (A Summary in Dutch) 109

    Acknowledgements 111

    Published + Submitted Articles 113

    Curriculum Vitae 117

  • 0

    Introduction + Summary

    We write the year 2004 CE. The last few years have witnessed a significant increase in the number ofsupervised methods employed in diverse image processing tasks. Especially in medical image analysisthe use of, for example, supervised shape and appearance modelling [16, 18] has increased considerablyand has proven to be successful.

    This thesis focuses on applying supervised pattern recognition methods [22, 28, 37, 47, 55, 90] in medi-cal image processing. We consider a local, pixel-based approach in which image segmentation, regression,and filtering tasks are solved using descriptors of the local image content (features) based on which de-cisions are made that provide a class label (in case of image segmentation) or a gray value (in case offiltering or regression) for every pixel. The basic probabilistic decision problem, underlyingimplicitlyor explicitlyall the methods presented in this thesis, can be stated in terms of a conditional probabilityoptimization problem

    = argmaxyY

    P(y|x) (1)

    in which x Rd is a d-dimensional vector of measurements, i.e., a feature vector, describing the localimage content and y is an quantity that takes values from a set Y. Typically, in a classification task, Y is adiscrete set of labels and in case of regression, Y equals R. Based on the maximization in Equation (1), toevery vector x (which is associated to a pixel in an image), a particular from Y is associated.

    This approach isbecause of its local naturequite different from the shape and appearance meth-ods mentioned in the beginning of this chapter which try to solve image processing tasks in a more globalway. A recent comparative study [42] shows that in image segmentation, pixel-based approaches cancompete with shape and appearance models, providing an interesting alternative to the latter.

    The methodological part of the thesis consists of three dimensionality reduction methods (Chapters 1,2, and 3) that can aid the extraction of relevant features to be used for performing image segmentation orregression. Furthermore, in Chapter 4, an iterative segmentation scheme is developed which draws fromclassical pattern recognition and machine learning methods. Chapters 5 and 6 present the application ofthese techniques in two problems related to computer-aided diagnosis (CAD) in chest radiography. Chap-ter 5 considers the task of segmenting the posterior ribs while Chapter 6 presents a regression frameworkto suppress the bony structures in chest radiographs.

    In the remainder of this introductory chapter, we provide an outline and summary of Chapters 1 to 6.

  • 0.1 On Features

    Picking good features is the essence of pattern recognition, as Ballard and Brown put it terse and in-sightful in their book on computer vision [3]. Indeed, it seems that there is not much more to it. Onceone or more good features have been selected1, solving the actual pattern recognition task is easy, if nottrivial. Clearly, the principal problem is to determine these good feature