TEXTS AND READINGS IN MATHEMATICS 19

Linear Algebra Second Edition

Texts and Readings in Mathematics

AdWory Editor C. S. Seshadri, Chermai Mathematical Institute, Chennai.

ManagingEditor Rajendra Bhatia, Indian Statistieallnstitute, New Delhi.

Editors V. S. Borkar, Tata InstituteofFundamental Research, Mumbai. R. L. Karandikar, Indian Statisticallnstitute, New Delhi. C. Musili, University ofHyderabad, Hyderabad. K. H. Paranjape, Institute ofMathematical Sciences, Chennai. T. R. Ramadas, Tata Institute ofFundamental Research, Mumbai. V. S. Sunder, InstituteofMathematieal Sciences, Chennai.

Already Publisbed Volumes R. B. Bapat: Linear Algebra and Linear Models (Second Edition) R. Bhatia: Fourier Series C. Musili: Representations ofFinite Groups H. Helson: Linear Algebra ( Second Edition) D. Sarason: Notes on Complex Function Theory M. G. Nadkarni: Basic Ergodic Theory(Second Edition) H. Helson: Harmonie Analysis (Second Edition) K. Chandrasekharan: A Course on Integration Theory K. Chandrasekharan: A Course on Topological Groups R. Bhatia(ed.): Analysis, Geometryand Probability K. R. Davidson: C· - Aigebras by Example M. Bhattacharjee et a/.: Notes on Infinite Permutation Groups V. S. Sunder: Functional Analysis - Spectral Theory V. S. Varadarajan: Algebra in Ancient and Modem Tim es M. G. Nadkarni: Spectral Theoryof Dynamical Systems A. Borei: Semisimple Groupsand Riemannian Symmetrie Spaees M. Marcolli: Seiberg-Witten Gauge Theory A.Botteher and S. M. Grudsky: Toeplitz Matriees, Asymptotie Linear

Algebra and Functional Analysis

Linear Algebra Second Edition

A. Ramachandra Rao Indian Statistical Institute

Calcutta

and

P. Bhimasankaram Indian Statistical Institute

Hydembad

l1:lQl@. HINDUST AN U UU U BOOK AGENCY

Published by

Hindustan Book Agency (India) P 19 Green Park Extension New Delhi 110 016 India

email: hba@vsnl.com http://www.hindbook.com

Copyright © 2000 by Hindustan Book Agency (India)

No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, e1ectronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without written permission from the copyright owner, who has also the sole right to grant licences for translation into other languages and publication thereof.

All export rights for this edition vest exelusively with Hindustan Book Agency (lndia). Unauthorized export is a violation of Copyright Law and is subject to legal action.

Produced from camera ready copy supplied by the Author.

ISBN 978-81-85931-26-5 ISBN 978-93-86279-01-9 (eBook) DOI 10.1007/978-93-86279-01-9

To

my wife

Vani and children

Rajeev Padmaja

- A.R.R.

To

my wife

Vijaya and children

Chandu Chandana

- P.B.

Preface

There have been broadly three approaches to the treatment of Lin­ear Algebra, viz., those emphasizing vector spaces, determinants and elementary operations. The books following the first approach (for ex­ample, those by Halmos and Hoffman and Kunze) mainly use the setting of linear transformations with a view to possible extensions of the results to Banach and Hilbert spaces. The books following the second approach (for example that by Mirsky) study Matrix Algebra making heavy use of determinants. The last approach is generally used in service courses where the emphasis is on the basic operations and computations involv­ing matrices with not much importance given for proofs and concepts.

For several applications in Science and Engineering, matrix setting is of importance. However, the vector space approach is more suit­able for geometrie intuition leading to transparent proofs besides being amenable for generalization to infinite-dimensional spaces. The Indian Schoölled by Professors C. R. Raa and S. K. Mitra has successfully used this approach to great advantage. We follow them in this book and sys· tematically develop the elemenatry parts of matrix theory exploiting the properties of row and column spaces of matrices. The book is meant to be a text book at the honours level for students of Mathematics and/or Statistics though it can be used with advantage by students of other subjects like Physics, Computer Science, Engineering, Operational Re­search, etc. It can also serve as a reference book for scientists in various other disciplines.

The developments in Linear Algebra during the past few decades have brought into focus se ver al techniques like rank-factorization, gen­eralized inverses and singular value decomposition, which are hitherto not included in elementary text books. These techniques are actually simple enough to be taught at undergraduate (honours) level and, when properly used, provide a better understanding of the topic besides giving simpler proofs of results, thus enablillg the student to learn the subject without tears. One of our aims in writing this book is to provide a treatment incorporating these.

The book is organized in two parts. The results in Chapters 1 through

viii Preface

6 are valid ,over any field while in Chapters 7, 8 and 9 the field is assumed to be the real field 'IR or the complex field C. We think this is very useful because, without this separation, one may have to spend a huge amount of time to decide whether a (frequently used) result is true over a general field. Fields other than IR and C are very much used in subjects like Statistics, Computer Science, Co ding Theory, Combinatorics, etc. However, a student not interested in general fields would not lose much by taking the field to be IR or C throughout. We assure the reader that the proofs in Chapters 1-6 are no more complicated than they would be if the field were assumed to be IR or C.

In an earlier version of the book, there were two chapters, one on Linear Programming and the other on Statistical Applications, which we have dropped for the sake of uniformity and brevity. We have, moreover, simplified some of the proofs and the presentation of the material in several places and included a discussion on the locus of a general equation of the second degree in the plane.

We mention a few general features of tne book. Many of the proce­dures are given in the form of algorithms which can easily be converted to computer programmes in any high levellanguage. However we have not tried to give the best computationally stable algorithms as they usu­ally need a lot more theory and are available in computer packages. We have included a large number of examples illustrating almost every defi­nition, result, procedure, etc. However, the real highlight of the book is the large collection of exercises given at the ends of the sections. Hints are provided along with the exercises for a few tricky ones; more hints and solutions are given at the end of the book for the rest. We urge each student to attempt as many exercises as possible without looking at the solutions. However, one should not feel discouraged if he or she needs frequent help of the solutions as there are many exercises which are either tough or lengthy. We have marked with an asterisk exercises and topics which, we think, are somewhat difficult alld may be omitted in a first reading.

We had to omit some important topics like non-negative matricas and matrix analysis mainly because of the constraint on the length of the book. Though we have corrected the errors in the earlier version of the book which were brought to our notice, some errors (typographical or even conceptual) are bound to remain in spite of our best efforts. We shall be grateful if the readers inform us of any errors found or

Preface ix

suggestions for improvement. The list of references given at the end of the book is by no means

exhaustive and includes mainly those which have influenced us consider­ably in one form or another. More references can be obtained from some of these.

Regarding the numbering of results: we have numbered theorems, lemmas, definitions, examples, algorithms etc. (but not corollaries) in the same sequence for ease in tracing them. Section 4.5 refers to the fifth section in Chapter 4. Exercise 4.5.2 is the second exercise in Section 4.5. (4.5.2) refers to the second numbered equation (or displayed item) in Section 4.5 and is different from Theorem 4.5.2. An index of symbols and notations and a fairly detailed subject index are given at the end of the book.

Though this book contains material which can be covered in just about one year, it can also be used to teach a one-semester course as outlined on the following page.

It is a pleasure to acknowledge the help received from various per­sons. P. Bhimasankaram is particularly grateful to Professor C. R. Rao and Professor S. K. Mitra from whom he learnt Linear Algebra. Their thoughts on the subject greatly influenced his own understanding of the subject and are reflected to some extent in the present work. However, the present authors are solely responsible for any shortcomings in this book.

We are thankful to the authorities of the Indian Statistical Institute for providing us the necessary facilities while writing this book. We also thank the colleagues with whom we had discussions and the students of several batches of B. Stat. (Hons.) and M. Stat. at the Institute for their intelligent querries and for bearing with our experimentation using preliminary vers ions of this book.

Finally we thank Hindustan Book Agency for their efforts in bringing out this book in a neat form.

A. Ramachandra Rao P. Bhimasankaram

Suggestions for a One-Semester Course

A one-semester course of about 50 lectures can be taught from the book by covering the following portions: For shorter courses, one has to carefully choose the topics. Here, i.j.k refers to the item i.j.k, which may be a theorem, example or definition, etc., and i.j refers to the whole of Section i.j excluding difficult examples or results indicated by an asterisk, if any.

1.1-1.5,1.6.1-1.6.4,1.6.6,1.7.1-1.7.4,1.8.1, 1.8.2. 2.1-2.2,2.3.1,2.3.4,2.4-2.6, Idea ofpartitioned matrices using (2.7.2). 3.1-3.2, 3.3.1-3.3.7, 3.4.1, 3.5.1-3.5.12, 3.6.1-3.6.3, (3.7.1), 3.8.1,

3.8.2, (3.8.8), 3.9.2, 3.9.3, 3.9.7. 4.1-4.2,4.3.1,4.4.1-4.4.3,4.4.7-4.4.10,4.5.1-4.5.3. 5.1-5.3,5.5.1,5.5.2. 6.3 (assurne properties of permutations), 6.4.1-6.4.4 (omit proofs),

6.5.1,6.6.1-6.6.4,6.7.1. 7.1, 7.2.1-7.2.3, 7.2.7, 7.3.1, 7.3.8-7.3.10, 7.4.1-7.4.4, 7.4.6-7.4.9,

7.5.1-7.5.6,7.6.1-7.6.6 (excluding 7.6.3). 8.1,8.2.1-8.2.7,8.3.1,8.3.2,8.3.7,8.3.10,8.4.1, 8.4.2, 8.7.1, 8.7.2. 9.1,9.2 , 9.3.1-9.3.5,9.4.1,9.4.2,9.4.5,9.4.8,9.4.9,9.6.1. The algorithms in the above topics may be omitted except 4.3.1,

4.4.10 and 7.4.8.

Contents

o Preliminaries 0.1 Relations 0.2 Flmctions 0.3 Groups .. 0.4 Rings and fields . 0.5 Polynomials

1 Vector spaces 1.1 Introduction ... .. ..... . . . 1.2 Vector space: axiomatic definition 1.3 Subspaces . . . . . . 1.4 Linear independence 1.5 Basis and dimension 1.6 Calculus of subspaces 1. 7 Direct sum and complement 1.8 Isomorphism .. 1.9 Quotient space* ...... .

2 Algebra of matrices 2.1 Introduction ......... . . . . . 2.2 Linear transformations and matrices 2.3 Operations on matrices .... . 2.4 Properties of matrix operations .. . 2.5 Matrices with special structures . . . 2.6 Rows and columns of a matrix product . 2.7 Partitioned matrices .. .. .. . . .. .

3 Rank and inverse 3.1 Introduction . . . ...... . 3.2 Row space and column space 3.3 Inverse of a matrix . . . . . . 3.4 Properties of inverse . . . . . 3.5 Rank of a product of matrices .

1 1 2 6 9

10

14 14 16 22 30 36 46 51 58 63

67 67 67 78 83 91 96

101

108 108 108 112 116 120

xii

3.6 Rank-factorization of a matrix 3.7 Rank of a sum and projectors . 3.8 Inverse of a partitioned matrix 3.9 Rank of real and complex matrices 3.10 Change of basis . . . . . . . . . . . 3.11 Rank and nullity of linear transformations·

4 Elementary operations and reduced forms 4.1 Introduction......... ... 4.2 Elementary operations .... . . 4.3 Sweeping out a row or a column 4.4 Echelon form . . . . . . 4.5 Normal form ..... . 4.6 Hermite canonical form

5 Linear equations 5.1 Introduction . 5.2 Homogeneous systems 5.3 General linear systems 5.4 Generalized inverse of a matrix 5.5 Sweep-out method for solving Ax = b 5.6 Compact form of Gauss-Doolittle method

6 Determinants 6.1 Introduction. 6.2 6.3 6.4 6.5 6.6 6.7

Permutations Determinant and its elementary properties . Cofactors and expansion theorems Determinant of a product . . . . . . Classical adjoint and inverse. . . . . Determinant of a partitioned matrix

7 Inner product and orthogonality 7.1 Introduction. 7.2 Inner product .. . ...... . . 7.3 Norm ..... . ... . .... . 7.4 Orthogonality and orthonormal basis. 7.5 Orthogonal complement and orthogonal projector . 7.6 Orthogonal and unitary matrices . ........ .

128 132 138 143 146 152

157 157 157 163 165 174 179

185 185 187 188 194 200 210

218 218 218 223 229 237 240 246

248 248 250 253 259 266 273

xiii

8 Eigenvalues 280 8.1 Introduction.......... 280 8.2 Characteristic roots ..... 280 8.3 Eigenvectors and eigenspaces 284 8.4 Cayley-Hamilton theorem and minimal polynomial 292 8.5 Spectral representation of a semi-simple matrix* 296 8.6 Jordan canonical form* 304 8.7 Spectral theorem . . . . . . . . 309 8.8 Singular value decomposition* . 318 8.9 Computational methods 322

9 Quadratic forms 328 9.1 Introduction............ 328 9.2 Classification of quadratic forms 329 9.3 Rank and signature . . . . . 331 9.4 p.d. and n.n.d. matrices . . . 339 9.5 Extrema of quadratic forms . 346 9.6 Simultaneous diagonalization 351 9.7 Square-root method 358 9.8 Hermitian forms* . . . . . . . 365

References 367

More hints and solutions 369

List of symbols 407

Index 410