19 Matrix Preconditioning Techniques and .19 Matrix Preconditioning Techniques ... The Cambridge

  • View
    213

  • Download
    0

Embed Size (px)

Text of 19 Matrix Preconditioning Techniques and .19 Matrix Preconditioning Techniques ... The Cambridge

  • CAMBRIDGE MONOGRAPHS ONAPPLIED AND COMPUTATIONALMATHEMATICS

    Series EditorsM. J. ABLOWITZ, S. H. DAVIS, E. J. HINCH, A. ISERLES,J. OCKENDON, P. J. OLVER

    19 Matrix Preconditioning Techniquesand Applications

    Cambridge University Press www.cambridge.org

    Cambridge University Press0521838282 - Matrix Preconditioning Techniques and ApplicationsKe ChenFrontmatterMore information

    http://www.cambridge.org/0521838282http://www.cambridge.orghttp://www.cambridge.org

  • The Cambridge Monographs on Applied and Computational Mathematicsreflects the crucial role of mathematical and computational techniques in con-temporary science. The series publishes expositions on all aspects of applicableand numerical mathematics, with an emphasis on new developments in this fast-moving area of research.

    State-of-the-art methods and algorithms as well as modern mathematicaldescriptions of physical and mechanical ideas are presented in a manner suitedto graduate research students and professionals alike. Sound pedagogical pre-sentation is a prerequisite. It is intended that books in the series will serve toinform a new generation of researchers.

    Also in this series:

    1. A Practical Guide to Pseudospectral Methods, Bengt Fornberg2. Dynamical Systems and Numerical Analysis, A. M. Stuart and A. R.

    Humphries3. Level Set Methods and Fast Marching Methods, J. A. Sethian4. The Numerical Solution of Integral Equations of the Second Kind, Kendall

    E. Atkinson5. Orthogonal Rational Functions, Adhemar Bultheel, Pablo Gonzalez-Vera,

    Erik Hendiksen, and Olav Njastad6. The Theory of Composites, Graeme W. Milton7. Geometry and Topology for Mesh Generation Herbert Edelsbrunner8. SchwarzChristoffel Mapping Tofin A. Driscoll and Lloyd N. Trefethen9. High-Order Methods for Incompressible Fluid Flow, M. O. Deville, P. F.

    Fischer and E. H. Mund10. Practical Extrapolation Methods, Avram Sidi11. Generalized Riemann Problems in Computational Fluid Dynamics,

    Matania Ben-Artzi and Joseph Falcovitz12. Radial Basis Functions: Theory and Implementations, Martin Buhmann13. Iterative Krylov Methods for Large Linear Systems, Henk A. van der Vorst14. Simulating Hamiltonian Dynamics, Benedict Leimkuhler and Sebastian

    Reich15. Collocation Methods for Volterra Integral and Related Functional

    Equations, Hermann Brunner16. Topology for Computing, Afra J. Zomorodian17. Scattered Data Approximation, Holger Wendland

    Cambridge University Press www.cambridge.org

    Cambridge University Press0521838282 - Matrix Preconditioning Techniques and ApplicationsKe ChenFrontmatterMore information

    http://www.cambridge.org/0521838282http://www.cambridge.orghttp://www.cambridge.org

  • Matrix Preconditioning Techniquesand Applications

    KE CHENReader in Mathematics

    Department of Mathematical SciencesThe University of Liverpool

    Cambridge University Press www.cambridge.org

    Cambridge University Press0521838282 - Matrix Preconditioning Techniques and ApplicationsKe ChenFrontmatterMore information

    http://www.cambridge.org/0521838282http://www.cambridge.orghttp://www.cambridge.org

  • cambridge university pressCambridge, New York, Melbourne, Madrid, Cape Town, Singapore, Sao Paulo

    Cambridge University PressThe Edinburgh Building, Cambridge CB2 2RU, UK

    www.cambridge.orgInformation on this title: www.cambridge.org/9780521838283

    C Cambridge University Press 2005

    This book is in copyright. Subject to statutory exceptionand to the provisions of relevant collective licensing agreements,

    no reproduction of any part may take place withoutthe written permission of Cambridge University Press.

    First published 2005

    Printed in the United Kingdom at the University Press, Cambridge

    A catalogue record for this book is available from the British Library

    ISBN-13 978-0-521-83828-3 hardbackISBN-10 0-521-83828-2 hardback

    MATLAB r is a trademark of The MathWorks, Inc. and is used with permission. TheMathWorks does not warrant the accuracy of the text or exercises in this book. This books

    use or discussion of MATLAB r software or related products does not constituteendorsement or sponsorship by The MathWorks of a particular pedagogical approach or

    particular use of the MATLAB r software.

    Cambridge University Press has no responsibility for the persistence or accuracy of URLsfor external or third-party internet websites referred to in this book, and does not guarantee

    that any content on such websites is, or will remain, accurate or appropriate.

    Cambridge University Press www.cambridge.org

    Cambridge University Press0521838282 - Matrix Preconditioning Techniques and ApplicationsKe ChenFrontmatterMore information

    http://www.cambridge.org/0521838282http://www.cambridge.orghttp://www.cambridge.org

  • Dedicated toZhuang and Leo Ling Yi

    and the loving memories of my late parents Wan-Qing and Wen-Fang

    Cambridge University Press www.cambridge.org

    Cambridge University Press0521838282 - Matrix Preconditioning Techniques and ApplicationsKe ChenFrontmatterMore information

    http://www.cambridge.org/0521838282http://www.cambridge.orghttp://www.cambridge.org

  • In deciding what to investigate, how to formulate ideas and what problemsto focus on, the individual mathematician has to be guided ultimately bytheir own sense of values. There are no clear rules, or rather if you onlyfollow old rules you do not create anything worthwhile.

    Sir Michael Atiyah (FRS, Fields Medallist 1966). Whats it allabout? UK EPSRC Newsline Journal Mathematics (2001)

    Cambridge University Press www.cambridge.org

    Cambridge University Press0521838282 - Matrix Preconditioning Techniques and ApplicationsKe ChenFrontmatterMore information

    http://www.cambridge.org/0521838282http://www.cambridge.orghttp://www.cambridge.org

  • Contents

    Preface page xiiiNomenclature xxi

    1 Introduction 11.1 Direct and iterative solvers, types of preconditioning 21.2 Norms and condition number 41.3 Perturbation theories for linear systems and eigenvalues 91.4 The Arnoldi iterations and decomposition 111.5 Clustering characterization, field of values and

    -pseudospectrum 161.6 Fast Fourier transforms and fast wavelet transforms 191.7 Numerical solution techniques for practical equations 411.8 Common theories on preconditioned systems 611.9 Guide to software development and the supplied Mfiles 62

    2 Direct methods 662.1 The LU decomposition and variants 682.2 The NewtonSchulzHotelling method 752.3 The GaussJordan decomposition and variants 762.4 The QR decomposition 822.5 Special matrices and their direct inversion 852.6 Ordering algorithms for better sparsity 1002.7 Discussion of software and the supplied Mfiles 106

    3 Iterative methods 1103.1 Solution complexity and expectations 1113.2 Introduction to residual correction 1123.3 Classical iterative methods 1133.4 The conjugate gradient method: the SPD case 119

    vii

    Cambridge University Press www.cambridge.org

    Cambridge University Press0521838282 - Matrix Preconditioning Techniques and ApplicationsKe ChenFrontmatterMore information

    http://www.cambridge.org/0521838282http://www.cambridge.orghttp://www.cambridge.org

  • viii Contents

    3.5 The conjugate gradient normal method: the unsymmetric case 1303.6 The generalized minimal residual method: GMRES 1333.7 The GMRES algorithm in complex arithmetic 1413.8 Matrix free iterative solvers: the fast multipole methods 1443.9 Discussion of software and the supplied Mfiles 162

    4 Matrix splitting preconditioners [T1]: direct approximationof Ann 165

    4.1 Banded preconditioner 1664.2 Banded arrow preconditioner 1674.3 Block arrow preconditioner from DDM ordering 1684.4 Triangular preconditioners 1714.5 ILU preconditioners 1724.6 Fast circulant preconditioners 1764.7 Singular operator splitting preconditioners 1824.8 Preconditioning the fast multipole method 1854.9 Numerical experiments 1864.10 Discussion of software and the supplied Mfiles 187

    5 Approximate inverse preconditioners [T2]: directapproximation of A1nn 191

    5.1 How to characterize A1 in terms of A 1925.2 Banded preconditioner 1955.3 Polynomial preconditioner pk(A) 1955.4 General and adaptive sparse approximate inverses 1995.5 AINV type preconditioner 2115.6 Multi-stage preconditioners 2135.7 The dual tolerance self-preconditioning method 2245.8 Near neighbour splitting for singular integral equations 2275.9 Numerical experiments 2375.10 Discussion of software and the supplied Mfiles 238

    6 Multilevel methods and preconditioners [T3]: coarse gridapproximation 240

    6.1 Multigrid method for linear PDEs 2416.2 Multigrid method for nonlinear PDEs 2596.3 Multigrid method for linear integral equations 2636.4 Algebraic multigrid methods 2706.5 Multilevel domain decomposition preconditioners for

    GMRES 2796.6 Discussion of software and the supplied Mfiles 286

    Cambridge University Press www.cambridge.org

    Cambridge University Press0521838282 - Matrix Preconditioning Techniques and ApplicationsKe ChenFrontmatterMore information

    http://www.cambridge.org/0521838282http://www.cambridge.orghttp://www.cambridge.org

  • Contents ix

    7 Multilevel recursive Schur complementspreconditioners [T4] 289

    7.1 Multilevel functional partition: AMLI approximated Schur 2907.2 Multilevel geometrical partition: exact Schur 2957.3 Multilevel algebraic partition: permutation-based Schur 3007.4 Appendix: the FEM hierarchical basis 3057.5 Discussion of software and the supplied Mfiles 309

    8 Sparse wavelet preconditioners [T5]: approximationof Ann and A1nn 310

    8.1 Introduction to multiresolution and orthogonal wavelets 3118.2 Operator compression by wavelets and sparsi