Springer Series in Statistics Advisors: D. Brillinger, S. Fienberg, J. Gani, J. Hartigan, K. Krickeberg

Springer Series in Statistics

Andrews/Henberg: Data: A Collection of Problems from Many Fields for the Student and Research Worker.

Anscombe: Computing in Statistical Science through APL. Berger: Statistical Decision Theory and Bayesian Analysis, 2nd edition. Bremaud: Point Processes and Queues: Martingale Dynamics. BrockwelljDavis: Time Series: Theory and Methods. DaleyjVere-lones: An Introduction to the Theory of Point Processes. Dzhaparidze: Parameter Estimation and Hypothesis Testing in Spectral Analysis

of Stationary Time Series. Farrell: Multivariate Calculation. GoodmanjKJUskal: Measures of Association for Cross Classifications. Hartigan: Bayes Theory. Heyer: Theory of Statistical Experiments. lolliffe: Principal Component Analysis. Kres: Statistical Tables for Multivariate Analysis. Leadbetter/LindgrenjRootzen: Extremes and Related Properties of Random

Sequences and Processes. Le Cam: Asymptotic Methods in Statistical Decision Theory. Manoukian: Modern Concepts and Theorems of Mathematical Statistics. Miller, II'.: Simulaneous Statistical Inference, 2nd edition. Moste/lerjWa/lace: Applied Bayesian and Classical Inference: The Case of The

Fedem/ist Papers. Pollard: Convergence of Stochastic Processes. Pmu/Gibbolls: Concepts of Nonparametric Theory. Read/Cressie: Goodness-of-Fit Statistics for Discrete Multivariate Data. Reiss: Approximate Distributions of Order Statistics: With Applications to

Nonparametric Statistics. Sachs: Applied Statistics: A Handbook of Techniques, 2nd edition. Seneta: Non-Negative Matrices and Markov Chains. Siegmund: Sequential Analysis: Tests and Confidence Intervals. Tong: The Multivariate Normal Distribution Vapnik: Estimation of Dependences Based on Empirical Data. West/Hamson: Bayesian Forecasting and Dynamic Models Wolter: Introduction to Variance Estimation. Yag/om: Correlation Theory of Stationary and Related Random Functions I:

Basic Results. Yaglom: Correlation Theory of Stationary and Related Random Functions II:

Supplementary Notes and References.

Y.L. Tong

The Multivariate Normal Distribution

Springer-Verlag New York Berlin Heidelberg London Paris Tokyo Hong Kong

Y.L. Tong School of Mathematics Georgia Institute of Technology Atlanta, GA 30332-0160 U.S.A.

AMS Mathematics Subject Classifications (1980): 60E05, 62H99

Library of Congress Cataloging in Publication Data Tong, Y.L. (Yung Liang), 1935-

The multivariate normal distribution I Y.L. Tong. P. cm. - (Springer series in statistics)

Includes bibliographical references. ISBN-13:978-1-4613-9657-4 (alk. paper) 1. Distribution (Probability theory) 2. Multivariate analysis.

1. Title. II. Series. QA273.6.T67 1990 519.2'4-dc20 89-21929

CIP

Printed on acid-free paper

© 1990 by Springer-Verlag New York Inc. Softcover reprint of the hardcover 1st edition 1990

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag, 175 Fifth Avenue, New York, NY 10010, U.S.A.), except for brief excerpts in connection with reviews or scholarly analysis. Use in connec­tion with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

Typeset by Asco Trade Typesetting Ltd., Hong Kong.

9 ~ 7 6 5 4 3 2 1

ISBN-13:978-1-4613-9657-4 e-ISBN-13:978-1-4613-9655-0 DOl: to.1 007/978-1-4613-9655-0

To My Family

Contents

Preface Basic Notation and Numbering System

CHAPTER 1

Introduction

1.1. Some Fundamental Properties 1.2. Historical Remarks 1.3. Characterization 1.4. Scope and Organization

CHAPTER 2

The Bivariate Normal Distribution

2.1. Some Distribution Properties 2.2. The Distribution Function and Sampling Distributions 2.3. Dependence and the Correlation Coefficient

Problems

CHAPTER 3

Fundamental Properties and Sampling Distributions of the Multivariate Normal Distribution

3.1. Preliminaries 3.2. Definitions of the Multivariate Normal Distribution 3.3. Basic Distribution Properties 3.4. Regression and Correlation 3.5. Sampling Distributions

Problems

Xl

Xlll

1

1 2 3 3

6

7 14 19 21

23

23 26 30 35 47 59

viii

CHAPTER 4

Other Related Properties

4.1. The Elliptically Contoured Family of Distributions and the Multivariate Normal

4.2. Log-Concavity and Unimodality Properties 4.3. MTP2 and MRR2 Properties 4.4. Schur-Concavity Property 4.5. Arrangement-Increasing Property

Problems

CHAPTER 5

Positively Dependent and Exchangeable Normal Variables

5.1. Positively Dependent Normal Variables 5.2. Permutation-Symmetric Normal Variables 5.3. Exchangeable Normal Variables

Problems

CHAPTER 6

Order Statistics of Normal Variables

6~1. Order Statistics of Exchangeable Normal Variables 6.2. Positive Dependence of Order Statistics of Normal Variables 6.3. Distributions of Certain Partial Sums and Linear

Combinations of Order Statistics 6.4. Miscellaneous Results

Problems

CHAPTER 7

Related Inequalities

7.1. Introduction 7.2. Dependence-Related Inequalities 7.3. Dimension-Related Inequalities 7.4. Probability Inequalities for Asymmetric Geometric Regions 7.5. Other Related Inequalities

Problems

CHAPTER 8

Statistical Computing Related to the Multivariate Normal Distribution

8.1. Generation of Multivariate Normal Variates 8.2. Evaluation and Approximations of Multivariate Normal

Probability Integrals 8.3. Computation of One-Sided and Two-Sided Multivariate

Normal Probability Integrals 8.4. The Tables

Problems

Conterits

62

62 68 73 79 84 89

91

92 104 108 120

123

123 130

136 140 147

150

150 152 154 161 169 177

181

181

186

193 194 199

Contents

CHAPTER 9

The Multivariate t Distribution

9.1. Distribution Properties 9.2. Probability Inequalities 9.3. Convergence to the Multivariate Normal Distribution 9.4. Tables for Exchangeable t Variables

Problems

References

Appendix-Tables

Author Index

Subject Index

ix

202 204 207 211 213 216

219

229

261

265

Preface

The multivariate normal distribution has played a predominant role in the historical development of statistical theory, and has made its appearance in various areas of applications. Although many of the results concerning the multivariate normal distribution are classical, there are important new results which have been reported recently in the literature but cannot be found in most books on multivariate analysis. These results are often obtained by showing that the multivariate normal density function belongs to certain large families of density functions. Thus, useful properties of such families immedi­ately hold for the multivariate normal distribution.

This book attempts to provide a comprehensive and coherent treatment of the classical and new results related to the multivariate normal distribution. The material is organized in a unified modern approach, and the main themes are dependence, probability inequalities, and their roles in theory and applica­tions. Some general properties of a multivariate normal density function are discussed, and results that follow from these properties are reviewed exten­sively. The coverage is, to some extent, a matter of taste and is not intended to be exhaustive, thus more attention is focused on a systematic presentation of results rather than on a complete listing of them.

Most of the classical results on distribution theory, sampling distributions, and correlation analysis are presented in Chapters 2 and 3. Chapter 4 deals with the log-concavity, unimodality, total positivity, Schur-concavity, and arrangement increasing properties of a multivariate normal density function and related results. Notions of dependence and their application to the multi­variate normal distribution are discussed in Chapter 5; not surprisingly, the results involve the covariance matrix of the distribution. Chapter 6 includes distribution theory and dependence results for the order statistics of normal variables. Chapter 7 contains inequalities and bounds for the multivariate

xii Preface

normal distribution, including dependence-related inequalities, dimension­related inequalities, and inequalities for the probability contents of geometric regions in a certain class. Problems on statistical computing, mainly the generation of multivariate normal variates and the evaluation of multivariate normal probability integrals, are treated in Chapter 8; tables of equicoordinate one-sided and two-sided percentage points and probability integrals for ex­changeable normal variables are given in the Appendix. A short chapter (Chapter 9) on the multivariate t distribution presents results concerning related distribution theory and convergence to the multivariate normal distri­bution. Chapters 2-9 contain sets of complementary problems. Finally, a combined list of references can be found at the end of the volume.

This book assumes a basic knowledge of matrix algebra and mathematical statistics at the undergraduate level, and is accessible to graduate students and advanced uhndergraduate students in statistics, mathematics, and related applied areas. Although it is not intended as a textbook, it can be used as a main reference in a course on multivariate analysis. And, of course, it can be used as a reference book on the multivariate normal distribution by researchers.

This work was partially supported by National Science Foundation grants DMS-8502346 and DMS-8801327 at Georgia Institute of Technology. Need­less to say, I am indebted to the extensive literature in related areas. Professors Theodore W. Anderson, Herbert A. David, Kai-Tai Fang, Kumar Joag-Dev, Mark E. Johnson, Samuel Kotz, and Moshe Shaked read all or parts of the manuscript, and their comments and suggestions resulted in numerous signifi­cant improvements. However, I am solely responsible for errors and omissions. I am grateful to Professors Ingram Olkin and Frank Proschan for their inspiration, continuing encouragement, and constructively critical comments, and to Professor Milton Sobel for his strong influence on my work concerning the multivariate t distribution. I wish to thank Ms. Annette Rohrs for her skillful typing and wonderful cooperation, and also, the staff at Springer­Verlag for the neat appearance of the volume. Finally, I thank my wife Ai-Chuan and our children Frank, Betty, and Lily for their understanding and support. Frank read Chapter 1 and made some helpful comments, and Betty spent many long hours with me at the office.

Atlanta, Georgia November 1988

Y.L. TONG

Basic Notation and Numbering System

All vectors and matrices are in boldface type and, unless specified otherwise, all vectors are column vectors.

The following notation is used throughout this book:

(1) 91 = (-00, (0). (2) 9ln = {x: x = (Xl' ... , X n)', -00 < Xi < 00 for i = 1, ... , n}.

(3) ~(z) = ~e-Z2/2, -00 < Z < 00. y' 2n

fz 1 (4) q,(z) = ;;:ce-u2/2 du, -00 < Z < 00.

-00 y' 2n (5) The symbol "0" denotes the end of a proof or the end of an example. (6) For an n x n symmetric matrix 1:, 1: > 0 denotes that 1: is positive definite. (7) %(Jl, (12) denotes a univariate normal distribution with mean Jl and vari-

ance (12.

(8) .¥" (Jl, 1:) denotes a multivariate normal distribution with mean vector Jl and covariance matrix 1:.

Definitions, propositions, theorems, lemmas, facts, examples, remarks, and equations are numbered sequentially within each section. Results which are of general interest are stated as propositions, and results which concern only the multivariate normal distribution are given as theorems.