•

•

•

COLLINEARITY IN MIXED MODELS

by

Sandra S. Stinnett

Department of BiostatisticsUniversity of North Carolina

Institute of StatisticsMimeo Series No. 2125T

December 1993

•

•

•

COLLINEARITY IN MIXED MODELS

by

Sandra S. Stinnett

A dissertation submitted to the faculty of The University of North Carolina atChapel Hill in partial fulfillment of the requirements for the degree of Doctor ofPublic Health in the Department of Biostatistics.

Chapel Hill

1993

Approved by:

I/} ;J j, I / (;;/~~/~ Advisor

Reader

Reader

Reader

Reader

•

. ..

SANDRA S. STINNETT Collinearity in Mixed Models (Under the direction of

RONALD W. HELMS)

ABSTRACT

Methods for detection of collinearity in the GLUM are well established,

however no previously published research has been directed toward extending

these methods for use in the mixed model. In the mixed model, collinearity in

the fixed effects arises from ill-conditioning of (X't"'X), leading to inflated

elements of vl/J) =(X't"'xr'. Mixed model diagnostics, analogous to those

used in ordinary least squares regression, were defined using (X'I-'X) in place

of X'X. A procedure for their use was specified and illustrated by applying the

diagnostics to a data set. Preliminary analyses revealed that variation in two

factors, the number of variables in the random effects and constraints on the

covariance of the random effects, produced different collinearity diagnostics for

models with the same variables in the fixed effects.

In order to generalize the behavior of the diagnostics, they were applied

to experimental data with two types of known predetermined collinearities with

increasingly tighter dependencies. The focus of the research was the behavior

of the diagnostics when different random effects were in models containing the

same fixed effects, for a given type and level of dependency and a specified

covariance matrix 11. A procedure, developed for computing (X'I-'X) directly

rather than through fitting of actual models, permitted a pure assessment of the

impact of varying the random effects since the "noise" of the estimation

ii

process was bypassed. The diagnostics were able to pinpoint the

dependencies in the experimental data. The results demonstrated that adding

variables, especially collinear variables, to the random effects diminished the

impact of collinearity in the fixed effects. Repetition of the experiment using

a difference covariance structure produced similar results.

These experimental results were explored in actual data by creating

similar dependencies and varying the variables in the random effects. The

pattern of diminished collinearity was seen when collinear variables were added

to the random effects, though some departure from the experimental results

was seen and attributed to the differences in dependencies, covariances and

the estimation process.

iii

•

•

•

ACKNOWLEDGEMENTS

My sincerest thanks go to my advisor, Ron Helms, for the extended

discussions and helpful advice and support that not only made this dissertation

possible, but also made the task of creating it enjoyable. I have learned a great

deal from him both during my tenure as a student and during this endeavor.

I thank the other members of my committee for their keen interest in the topic

and their responsive feedback. Thank you, Keith Muller, Gerardo Heiss, Gary

Koch and Lloyd Edwards.

Since a dissertation is a culmination of many years of education, I thank

those who have contributed to mine and enabled me to get to this point.

Special thanks go to Dennis Gillings, under whose influence I learned many

analysis strategies and practical aspects of statistics. My work, under his

direction, in the Biometric Consulting Laboratory was a focal point in shaping

my career. For those years, I am grateful. I also especially thank Gary Koch

for ten years of advice, friendship and shared endeavors. He has been a true

mentor, permitting me to work beside him as I was able. His tutelage in

statistics and his professional advice have been invaluable. I also give a special

thanks to Barry Margolin for supporting my pursuit of a grant for developing the

department's statistical consulting course and for allowing me to teach it for

two years after the grant was received. This experience has impacted the

iv

direction of my career enormously. I am very appreciative for the opportunity

he gave me.

I also thank my Chapel Hill friends for sustaining friendships during this

work. The Chambless and Schlitt families have been extensions of my own

and have nurtured my progress. I thank my good friend Ellen Sim Snyder for

much encouragement. Our shared experience of creating dissertations has

made the path to the finish easier. I also thank my friends elsewhere and my

family for their enduring support for many years.

I am very grateful to my mother, Sudie Stinnett, who has taught me,

especially by example, the importance of education. As I become more like her

in pursuit of knowledge and filled with curiosity, I become more grateful for her

example. She has been my greatest supporter throughout the pursuit of this

degree in every way possible. Thank you, Mother, for everything.

v

•

To my mother, Sudie

and

To the memory of my father, Lee

vi

..TABLE OF CONTENTS

Chapter Page

I. REVIEW OF LITERATURE 1

1.1 Introduction.................................. 1

1.2 Collinearity in the General Linear Univariate Model (GLUM) .. 2

1.2.1 General Linear Univariate Model (GLUM) Notation ... 3

1.2.2 Detection and Diagnosis of Collinearity . . . . . . . . . .. 4

1.2.2.11.2.2.21.2.2.31.2.2.41.2.2.5

Foundation for Collinearity Assessment . . .. 5Measures of Collinearity . . . . . . . . . . . . .. 5Steps in the BKW Diagnostic Procedure 11Computation of Collinearity Diagnostics 14Impact of Collinearity . . . . . . . . . . . . . . .. 15

1.2.3 Related Issues 16

1.2.3.11.2.3.21.2.3.31.2.3.4

Parameterization 16Scaling of Columns 16Mean Centering . . . . . . . . . . . . . . . . . . .. 17Distinctions and Additional Aspects 22

Collinearity and Correlation (22); Collinearity,Conditioning, Weak Data and Short Data(23); Assessment of Damaging Collinearity(24); Collinearity-Influential Observationsl27); Maverick Interlopers and RagingControversies (28)

1.2.4 Procedures for Resolution of Collinearity . . . . . . . . .. 29

1.2.4.11.2.4.2

1.2.4.31.2.4.4

Column Scaling .. . . . . . . . . . . . . . . . . . . 30Deletion of Variable(s) Involved in

Dependencies . . . . . . . . . . . . . . . . . . 30Introduction of New Data 31Bayesian-type Techniques . . . . . . . . . . . .. 31

vii

Pure Bayesian (31); Mixed-Estimation (32)

1.2.4.5 Biased Regression Techniques 33

Ridge Regression (34); Principal ComponentsRegression (36)

1.3 The Mixed Effects Model (MIXMOD) 39

1.3.1 Introduction 391.3.2 Definition, Notation and Assumptions 411.3.3 Features of the Covariance Structure 431.3.4 Estimation of Parameters 45

1.3.4.1 Estimation Principles 451.3.4.2 Computing Algorithms 461.3.4.3 Extant Procedures for Estimation of

p, A, and 02 49

Estimation Steps (50)

A

1.3.5 Variance of IJ 511.3.6 Prediction 521.3.7 Objectives of Mixed Model Analysis 53

1.4 Collinearity Diagnostics for Mixed Models . . . . . . . . . . . . .. 53

II. COLLINEARITY DIAGNOSTICS FOR THE MIXED MODEL:AN OVERVIEW 55

2.1 Introduction 552.2 Type of Data 552.3 Specifics of the Mixed Model 562.4 The Diagnostic Measures 562.5 Methods of Computation 602.6 Employing the Diagnostic Procedure 60

2.6.1 Examine Condition Indexes 612.6.2 Look for Gaps in "10/30 Progression" of Cis 632.6.3 Examine the Variance Decomposition Proportions 632.6.4 Determine Involved Variables 642.6.5 Perform Auxiliary Regressions 652.6.6 Determine Uninvolved Variables 66

2.7 Illustration of ,the Collinearity Diagnostics 67

viii

..

•

2.7.1 Description of the Data 672.7.2 GLUM Example and Diagnostics 692.7.3 Mixed Model Examples and Diagnostics 76

2.7.3.1 A Simple Mixed Model 762.7.3.2 GLUM for Longitudinal Data 812.7.3.3 Mixed Model with Multiple

Independent Variables . . . . . . . . . . .. 84

2.8 Factors Impacting Collinearity in the Mixed Model 92

2.8.1 Number and Nature of Variables in Z 93

2.8.1.1 Two variables, Pair-wise collinear . . . . . . .. 932.8.1.2 One Variable 932.8.1.3 Summary 94

2.8.2 Structure of A 102

2.8.2.1 Constrained, One Off Diagonal ElementEqual to Zero 102

2.8.2.2 Constrained, All Off Diagonal ElementsEqual to Zero 102

2.8.2.3 Summary 102

2.8.3 Different Response, Same Fixed andRandom Effects 105

2.9 Summary and Implications of Results . . . . . . . . . . . . . . .. 108

2.9.1 Initial Conclusions . . . . . . . . . . . . . . . . . . . . . . .. 1082.9.2 Implications for Subsequent Research 110

III. APPLICATION OF DIAGNOSTICS TO EXPERIMENTAL DATA .... 111

3.1 Introduction................................. 111

3.2 The GLUM Experiment 112

3.2.1 The Basic Data 1133.2.2 The Dependency Sets 1133.2.3 The Data Series 1143.2.4 The Issues Addressed 1143.2.5 Results 114

3.2.5.1 Simple Dependency: Two Variables. . . .. 1153.2.5.2 Simple Dependency: Three Variables . . .. 115

ix

3.2.5.3 Coexisting Dependency: Two Variables,Three Variables (Nonoverlapping) ... 116

3.3 The Mixed Model Experiment 122

3.3.1 The Basic Data . . . . . . . . . . . . . . . . . . . . . . . . .. 1223.3.2 The Dependency Sets 1233.3.3 The Data Series .........•.••............ 1233.3.4 The Issues Addressed 1243.3.5 Mixed Model Matrices and Parameters . . . . . . . . .. 1333.3.6 Values for 4, 0'-, and V" . . . . . . . . . . . . . . . . . . .. 1353.3.7 Results 136

3.3.7.1 Simple Dependency: Two Variables . . . .. 137

Baseline (137); Adding Variables to RandomEffects (137)

3.3.7.2 Simple Dependency: Three Variables .... 139

Baseline (139); Adding Variables to RandomEffects (140)

3.3.7.3 Coexisting Dependency: Two Variables,Three Variables (Nonoverlapping) ... 142

Baseline (142); Adding Variables to RandomEffects (145)

•

3.4 Effect of Adding Variables to Random Effects 153

3.5 Summary of Results 157

IV. IMPACT OF RANDOM EFFECTS COVARIANCEON COLLINEARITY 158

4.1 Introduction................................. 1584.2 The Mixed Model Experiment 1594.3 Results 160

4.3.1 Simple Dependency: Two Variables 160

4.3.1.1 Adding Variables to Random Effects. . . .. 1604.3.1.2 Comparison to Experiment 1 162

4.3.2 Simple Dependency: Three Variables 162

x

..

4.3.2.1 Adding Variables to Random Effects . . . .. 1624.3.2.2 Comparison to Experiment 1 164

4.3.3 Coexisting Dependency: Two Variables,Three Variables (Nonoverlapping) 164

4.3.3.1 Adding Variables to Random Effects . . . .. 1644.3.3.2 Comparison to Experiment 1 171

4.4 Summary of Results 172

V. DATA EXAMPLES OF THE IMPACT OF RANDOMEFFECTS ON COlliNEARITY 174

5.1 Introduction 1745.2 The Data 1765.3 Example 1: One Simple Dependency 1765.4 Example 2: Overlapping Dependencies 1815.5 Summary 185

VI. EXAMPLE MIXED MODEL DATA ANALYSISUSING COlliNEARITY DIAGNOSTICS 187

6.1 Introduction................................. 1876.2 Model Fitting Strategy 1886.3 Model Fitting Example 1896.4 Summary................................... 212

VII. SUMMARY AND RECOMMENDATIONS FORFUTURE RESEARCH 213

7.1 Summary of Research 2137.2 Directions for Future Research 217

APPENDIX 1: GENERAL LINEAR UNIVARIATE MODEL (GLUM)COlliNEARITY DIAGNOSTICS . . . . . . . . . . . . .. 220

APPENDIX 2: MIXED MODEL BASELINECOlliNEARITY DIAGNOSTICS . . . . . . . . . . . . .. 227

APPENDIX 3: MIXED MODEL EXPERIMENT 1COlliNEARITY DIAGNOSTICS . . . . . . . . . . . . .. 234

APPENDIX 4: MIXED MODEL EXPERIMENT 2COlliNEARITY DIAGNOSTICS . . . . . . . . . . . . .. 269

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

xi

LIST OF TABLES

Table 1.1: GLUM Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 4

Table 1.2: The BKW Diagnostic Procedure ..... . . . . . . . . . . . . .. 11

Table 1.3: Details of the BKW Diagnostic Procedure 13

Table 1.4: Selected Collinearity Measures 20

Table 1.5: Structures and Models for the Components ofV(Yk) =I.t=~4~· + u2vk •••••••••••••••••••• 45

Table 1.6: Aspects of Mixed Model Estimation 45

Table 2.1: Row-oriented Format for Presentation of CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 61

Table 2.2: Descriptive Statistics for 34 Black Female Childrenwith 527 Total Pulmonary Function Studies 69

Table 2.3: Correlation Coefficients of GLUM Regressors 70

Table 2.4: FVC: GLUM Results and Collinearity Diagnostics 71

Table 2.5: GLUM Auxiliary Regressions for Pulmonary Data 73

Table 2.6: Number of Pulmonary Studies for Each Subject 76

Table 2.7: A Simple Mixed Model: FVC as a Function of Age 79

Table 2.8: FVC: GLUM Results and Collinearity Diagnosticsfor 527 Observations on 34 Subjects . . 83

Table 2.9: FVC: MIXMOD Results and Collinearity Diagnostics. . . .. 86

Table 2.10: FVC (Z = Int,Ht,Wt): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

xii

Table 2.11: FVC (Z = Int,Age,Wt): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Table 2.12: FVC (Z = Int,Age,Ht): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Table 2.13: FVC (Z =Int,Age): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Table 2.14: FVC (Z = Int,Ht): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Table 2.15: FVC (Z =Int,Wt): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Table 2.16: Summary of Results of Altering Z Matrix by DeletingVariables from FVC Mixed Model With All ThreeVariables in Z 101

Table 2.17: FVC(One element lJ. =0): MIXMOD Results andCollinearity Diagnostics . . . . . . . . . . . . . . . . . . . .. 103

Table 2.18: FVC(AII off diag lJ. =0): MIXMOD Results andCollinearity Diagnostics . . . . . . . . . . . . . . . . . . . .. 104

Table 2.19: Summary of Results of Altering Z Matrix byConstraining lJ. Compared to FVC Mixed ModelWith Unconstrained lJ. . . . . . . . . • • . . . . . . • • • . .• 105

Table 2.20: VMAX50'l6: MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 107

Table 3.1: The GLUM Experiment, Series X1{i}, ;=0, ..., 4(One Contrived Near Dependency): ConditionIndexes and Variance-Decomposition Proportions . .. 118

Table 3.2: The GLUM Experiment, Series X2{j}, j=O, n., 4(One Contrived Near Dependency): ConditionIndexes and Variance-Decomposition Proportions 119

Table 3.3: The GLUM Experiment, Series X3{i,j}, ;=2;j=0, ..., 4(Two Contrived Near Dependencies): ConditionIndexes and Variance-Decomposition Proportions . .. 120

Table 3.4: The GLUM Experiment, Series X3{i,j}, ;=0, ..., 4; j=2(Two Contrived Near Dependencies): ConditionIndexes and Variance-Decomposition Proportions 121

xiii

Table 3.5: Variables in Fixed and Random Effects of MixedModel Experiment 126

Table 3.6: Impact of Collinearity in Fixed Effects for DifferentCombinations of Variables in Random Effects forX1{4} 154

Table 5.1: Impact of Collinearity in Fixed Effects for Different.Combinations of Variables in Random Effects forExample 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 178

Table 5.2: Impact of Constraining Covariance of Random Effects Ii.for Model with Intercept, Height, and Height2 inFixed Effects . . . . . . . . . . . . . . . . . . . . . . . . . . .. 180

Table 5.3: Impact of Collinearity in Fixed Effects for DifferentCombinations of Variables in Random Effects forExample 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 182

Table 5.4: Impact of Constraining Covariance of Random Effects Ii.for Model with Intercept, Age, Height, and Heighein Fixed Effects 184

Table 6.1:

Table 6.2:

Table 6.3:

Table 6.4:

Table 6.5:

Table 6.6:

Table 6.7:

Table 6.8:

Results for Model 1 in Step 1 of Model Fitting

Results for Model 2 in Step 1 of Model Fitting

Results for Model 3 in Step 1 of Model Fitting

Results for Model 1 in Step 2 of Model Fitting

Results for Model 2 in Step 2 of Model Fitting

Results for Model 1 in Step 3 of Model Fitting

Results for Model 1 in Step 4 of Model Fitting

Results for Model 2 in Step 4 of Model Fitting

191

192

193

195

196

198

200

201

Table 6.9: Results for Alternative Model 2 in Step 1 203

Table 6.10: Summary of Model Fitting . . . . . . . . . . . . . . . . . . . . .. 204

xiv

..

LIST OF FIGURES

Figure 2.1: Employing the Diagnostic Procedure 62

Figure 2.2: Values of FVC Plotted Against Age 74

Figure 2.3: Values of FVC Plotted against Height . . . . . . . . . . . . . . . 75

Figure 2.4: Values of FVC Plotted Against Weight 75

Figure 2.5: Values of FVC Predicted from Mixed Model withAge in X and in Z . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Figure 2.6: Values of FVC Predicted from Mixed Model withAge, Height and Weight in X and in Z,Plotted Against Age . . . . . . . . . . . . . . . . . . . . . . . . 88

Figure 2.7: Values of FVC Predicted from Mixed Model withAge, Height and Weight in X and in Z,Plotted against Height 89

Figure 2.8: Values of FVC Predicted from Mixed Model withAge, Height and Weight in X and in Z,Plotted Against Weight . . . . . . . . . . . . . . . . . . . . . . 90

Figure 3.1: Condition Index for BX3 and Wi byNumber of Random Effects Variables 138

Figure 3.2: Condition Index for BX1, BX2, and Zj byNumber of Random Effects Variables 140

Figure 3.3: Condition Index for BX3 and Wi byLevels of Wi and Zj' No Random Effects Variables . .. 143

Figure 3.4: Condition Index for BX1, BX2 and Zj byLevels of Wi and Zj' No Random Effects Variables .. 144

Figure 3.5: Condition Index for BX3 and Wi byNumber of Random Effects Variables, at Zo . . . . . .. 146

xv

Figure 3.6: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z, · ...... 146

Figure 3.7: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z2 · ...... 147

Figure 3.8: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z3 · ...... 147

Figure 3.9: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z4 · ...... 148

Figure 3.10: Condition Index for aX1, aX2, and Zj byNumber of Random Effects Variables, at Wo · ..... 148

Figure 3.11 : Condition Index for aX1, aX2, and Zj byNumber of Random Effects Variables, at W, · ..... 149

Figure 3.12: Condition Index for aX1, aX2, and Zj byNumber of Random Effects Variables, at W2 · ..... 149

Figure 3.13: Condition Index for ax1, aX2, and Zj byNumber of Random Effects Variables, at W3 · ..... 150

Figure 3.14: Condition Index for aX1, aX2, and Zj byNumber of Random Effects Variables, at W4 · ..... 150

Figure 4.1: Condition Index for aX3 and Wi byNumber of Random Effects Variables ........... 161

Figure 4.2: Condition Index for ax1, aX2, and ~ byNumber of Random Effects Variables ........... 163

Figure 4.3: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Zo · ...... 166

Figure 4.4: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z, · ...... 166

Figure 4.5: Condition Index for aX3 and Wi by ..Number of Random Effects Variables, at Z2 · ...... 167

Figure 4.6: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z3 · ...... 167

Figure 4.7: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z4 · ...... 168

xvi

•

Figure 4.8: Condition Index for BX1, BX2, and ~ byNumber of Random Effects Variables, at Wo 168

Figure 4.9: Condition Index for BX1, BX2, and ~ byNumber of Random Effects Variables, at W 1 •••••• 169

Figure 4.10: Condition Index for BX1, BX2, and Zj byNumber of Random Effects Variables, at W2 •••••• 169

Figure 4.11: Condition Index for BX1, BX2, and Zj byNumber of Random Effects Variables, at W 3 •••••• 170

Figure 4.12: Condition Index for BX1, BX2, and Zj byNumber of Random Effects Variables, at W 4 •••••• 170

Figure 6. 1: Values of FVC Predicted from Final Mixed Modelwith Intercept, Age and Weight in X and in Zat Mean Weight, Plotted Against Age . . . . . . . . . .. 210

Figure 6.2: Values of FVC Predicted from Final Mixed Modelwith Intercept, Age and Weight in X and in Zat Mean Weight, Plotted Against Weight . . . . . . . .. 210

Figure 6.3: Values of FVC Predicted from Final Mixed Modelwith Intercept, Age and Weight in X and in Zat Mean Age, Plotted Against Height 211

xvii

•

..

1.1 Introduction

CHAPTER I

REVIEW OF LITERATURE

•

•

The evolution of computers has enabled investigators to statistically

analyze large data sets with great facility. However, the analysis of such data

brings with it the concern of how to diagnose data-related problems. It is

difficult to examine interrelationships among variables in large data sets and

thus to become aware of ill-conditioned data. III-conditioning or coJlinearity

among the variables may produce model results that are incorrect. These

considerations have led, in recent years, to the development of diagnostic tools

for data analysis, primarily in the context of ordinary least squares regression.

However, other types of statistical analyses also require tools for assessing

conditioning, especially when large data sets are analyzed in which the

interrelationships among variables are not easily seen. Recently, the

diagnostics used in least square regression have been extended to other types

of modeling. (See, for example, Schindler (1986) for extensions to logistic

regression, the Buckley-James model and Cox's proportional hazards model.)

However, little if any research has been conducted for extensions of

diagnostics to the mixed model. Initially, this chapter presents an overview of

collinearity, its assessments and remedies, in the context of the general linear

univariate model. Then the mixed model, its notation and recent development,

is presented. Finally, the extension of collinearity assessment to the mixed

model is discussed.

1.2 Collinearity in the General Unear Univariate Model (GLUM)

According to Belsley, Kuh and Welsch (BKW) (1980), "no precise

definition of collinearity exists in the literature." They state (BKW 1980, P 85)

that two variates are exactly collinear if the data vectors representing them lie

on the same line (Le., in a subspace of dimension one). Further, k variates are

exactly collinear if the vectors that represent them lie in a subspace of

dimension less than k, Le., if one of the vectors is an exact linear combination

of the others, or equivalently, if the vectors are linearly dependent. In practice

and in this dissertation, we are concerned with the situation in which there are

~ collinearities among variables rather than exact collinearities. Statisticians

use the terms collinearity and multicollinearity to designate cases in which the

data vectors are approximately, but not exactly, collinear.

Collinearity may occur for several reasons. Rawlings (1988, p 274)

states that geometrically, collinearity results when at least one dimension of the

X-space is very poorly defined in the sense that there is almost no dispersion

among the data points in that dimension. A single variable that has little

dispersion will be collinear with the intercept term in the model. Other reasons

are listed by Rawlings (1988, pp 327-328). One cause of collinearity is due to

2

•

•

..

..

"

•

•

mathematical constraints on variables; another is due to the transformations of

original variables to create new ones. Other causes are related to study design

and sampling. Inadequate study designs may produce levels of experimental

factors that are not orthogonal, i.e., are nearly collinear. Inadequate sampling

may create dependencies which are an artifact of the data collection process

and may not be present in other samples. Collinearity also may be due to the

system or process being studied and thus will be present in all data from the

system.

BKW (1980) emphasize that collinearity is not a "statistical" problem (in

the sense of probability distributions, etc.), but rather a data problem.

Nevertheless, collinearity in the data impacts the statistical model in several

ways. First, due to redundancies, the contributions of collinear variables are

often inseparable. Second, estimates of model parameters are often imprecise,

Le., the estimated coefficients of all variables involved in the collinearity have

high variances. This is in comparison to variances of estimates in a model with

nearly orthogonal variables. Finally, predicted values can be inconsistent with

underlying science.

1.2.1 General Linear Univariate Model (GLUM) Notation

After discussing the concepts of collinearity in general terms, we now

introduce notation and define terms mathematically. The general linear

univariate model of full rank (GLUM-FR) is represented by the model equation

Y = X/J + e, (1.2.1)

where Y is an Nx 1 response vector, X is a full column rank Nxp matrix of

3

explanatory variables, Il is a

p x 1 vector of parameter

estimates and e is an N x 1

vector of unobservable errors.

The random errors have

E[e] =0 and V[e] =021.

Table 1.1 GLUM NOTATIONModel equation, data from the kth subject:Yk = XJl + " k= 1, 2, ..., NwhereYk :0: measurement of a dependent variable

from the kth subject, k =1, 2, ... , Nx.: =~ of the design matrix for the kth subjectII = vector of primary parametersek - an unobservable error term

Ele] =0, Vle] - crtAssumptions lead to:

ElY] =X/l, VlYl -crt

Notation for the kth subject is presented in Table 1.1 above. The least squares

estimate of Il, the estimate that minimizes the sum of the squared error terms,

is

(1.2.2)

A.

The variance of Il is

(1.2.3)I

In the GLUM, collinearity arises from ill-conditioning of X and X'X, leading toA.

inflated elements in the variance of /1...

1.2.2 Detection and Diagnosis of Collinearity

Historically, many procedures have been used to detect the presence of

collinearity. Several are reviewed by BKW (1980, pp 92-98) and Belsley

(1991, pp 26-37) with respect to their deficiencies; these are the forerunners

of currently used diagnostic tools. Numerical analysts have used techniques

to assess collinearity with a view toward obtaining a matrix A which is

conditioned well enough for the solution to the equation Az = c to be obtained

with numerical stability. This is relevant to the statistical process of obtainingA.

a solution to (X 'X)/1 = X'V. BKW (1980) apply the techniques of numerical

4

..

..

analysts to obtain indices of conditioning that 1) indicate the presence of

collinearity, 2) detect the variables that are involved in particular dependencies,

and 3) assess the degree to which the estimates of the coefficients are

degraded by the presence of the collinearities.

1.2.2.1 Foundation for Collinearity Assessment

Informative collinearity diagnostic tools can be derived from the singular

value decomposition of X or the spectral decomposition of X'X. In the singular­

value decomposition of X,

(1.2.4)

where U'U =V'V =Ip, 1\ = Diag(A, ~A2 ~ ... ~Ap) contains the singular values of

X on its diagonal, X has full column rank and p =rank(X). Note that Ap> 0 is the

smallest singular value.

The spectral decomposition of X'X (eigenanalysis) is related to the

singular-value decomposition of X. In the spectral decomposition of X'X,

(1.2.5)

•

where V is an orthogonal matrix that diagonalizes X'X. The diagonal elements

of 1\2 = Diag(A,2 ~A22~ ... ~Ap2) are the eigenvalues of X'X and the columns of

V are the eigenvectors of X'X. The matrix of right singular vectors of X is also

the matrix of eigenvectors of X'X. The positive square root of the jth

eigenvalue A/ is the jth singular value Aj •

1.2.2.2 Measures of Collinearity

Several measures useful in diagnosing collinearity are described in many

textbooks on linear regression [Kleinbaum, Kupper and Muller (1988); Myers

5

(1990); Neter, Wasserman, and Kutner (1985); Rawlings (1988); and

Chatterjee and Price (1991)] and are available in most software packages in

current use. Related matrix concepts are presented in Strang (1980). The

primary measures used for diagnosing collinearity are summarized in this

section. The first involves the eigenanalysis of X'X from which is obtained the

condition index (CI) and condition number (CN) of the matrix and another

related measure, the multicollinearity index (MCI). Other measures described

in this section are the variance decomposition proportions (VDP) and the

variance inflation factor (VIF). These diagnostics are usually applied after the

X matrix has been scaled to have equal column lengths. This is accomplished

by dividing the elements of each column vector by the square root of the sum

of squares of the elements of that column. There are varying views on the

issue of centering the data, in addition to scaling it. Centering and scaling are

discussed in section 1.2.3. [Also, Chatterjee and Price (1991) state that

principal components analysis and ridge regression can be used to detect and

pinpoint collinearity. The use of these procedures as alternative estimation

techniques is discussed in section 1.2.4.5.]

Eigenanalysis of X'X' Definitions As described above, the matrix X'X

can be decomposed as a· product of orthogonal and diagonal matrices that

contain its eigenvectors and eigenvalues. Using the singular values of X,

several measures of conditioning are defined. The condition index (CI) is

defined as the ratio of the largest singular value to the jth singular value,

CI = A,IAj • (1.2.6)

The condition number (CN) is defined as the ratio of the largest singular value

6

..

to the smallest singular value,

CN = A,IAp ' (1.2.7)

The multicollinearity Index (MCI) is defined as the sum of the ratios of the

square of the smallest eigenvalue to the square of each eigenvalue:

p A4

MC/=r..!- .j-' A~'J

(1.2.8)

..

Eigenanalysis ofX'X' Utility According to Rawlings (1988, p 276), the

eigenvalues, A/, measure dispersion in dimensions corresponding to principal

component axes of the X-space. The last principal component axis identifies

the dimension with least dispersion. Most authors state that "small"

eigenvalues indicate singularities in the X matrix. However, BKW (1980, p 96

and 104) state that few can agree on how small "small" is and that zero is the

wrong standard of comparison. According to BKW (1980, p 101), the

condition number (the largest condition index) of a matrix provides more useful

summary information on potential problems in calculations involving the matrix

than do the eigenvalues. The larger the condition number, the more iII-

conditioned the matrix. Specifically, it is a measure of the sensitivity of the

matrix to small changes in the components of the equations for which solutions

are sought. Belsley (1991, p 71) claims that the condition number gives a

multiplication factor by which imprecision in the data can be inflated to produce

even greater imprecision in the solution to a linear system of equations. He

states that it is a factor by which a 1% relative change in the data X could

effect a relative shift in the least squares estimates (Belsley 1991, 184). Also,

Belsley (1991, p 51) states that it measures the distance of a matrix from

7

singularity (or collinearity). Further, the condition number gauges the ability of

a matrix to be inverted and gives an upper bound on the "elasticity" of the

diagonal elements of the matrix (X'Xr' with respect to any element of the X

matrix. (The diagonal elements are proportional to the variances of the least

squares estimates.) (Belsley 1991, P 51). Weak dependencies are indicated by

condition indexes of 5 to 10, moderate to strong dependencies by values of 30

to 100, and serious dependencies by values greater than 100 (BKW 1980, P

101). In addition, the number of large condition indexes indicates the number

of contributing dependencies. According to Rawlings (1988), values of the

multicollinearity index (Mel) near 1 indicate high collinearity; values near 2

indicate no collinearity.

Variance Decomposition Proportions (VDP): Definition The estimate of

/1 in the GLUM was given in (1.2.2) and its variance in (1.2.3). Using the

singular value decomposition of X, (1.2.3) can be rewritten asA.

V(/l(p xPi) =u2(X '(p XN,X(N xpi)"' = u2v(px PII\-2(p xpIV'(PX pi

where

(1.2.9)

1/A~ 0

o 1/A~

o 0A.

Then for the kth component of /1,

0 v,

0 V2 (1.2.10).

1/A;.

vp

~

(1.2.11)

The k,jth variance decomposition and the sum of the p components of the jth

8

decomposition are

k=1, ... ,p. (1.2.12)

Since t/Jk is the variance of the kth regression coefficient, the variance

proportions are

k,j=1, ... ,p. (1.2.13)

•

"it< is the proportion of the variance of 1Jk attributable to the collinearity

indicated by AtVariance Decomposition Proportions (VDP): Utility The components

involved in the dependencies will have small eigenvalues in the denominators

of (1.2.12) and thus, large variances. A large "it< indicates that the kth

independent variable is a major contributor to jth principal component. A

subset of regressors with large variance proportions associated with the same

small eigenvalue indicates dependencies in that subset. Usually, the variance

proportions are displayed in a table that identifies the associated eigenvalue

and/or condition index.

Two qualifications are described by Belsley (1991, P 60-61). First, if

some collinear variables, say Set 1, are orthogonal to all other variables in a

model, say Set 2, then the collinearity may affect only the variables in Set 1.

Thus, all variances may not be affected by the collinearity. Second, collinearity

is present only if two or more variables are involved in the dependencies.

Variance Inflation Factor (VIF): Definition If the data are centered and

scaled, then the diagonal elements of the inverse of the correlation matrix of

9

A

X (which corresponds to the correlation matrix of Il, except for the intercept) are

called the variance inflation factors. The relationship between the variance of

the regression coefficients and the variance inflation factors is given by

(1.2.14)

where Xk is the kth column of X, centered and scaled to unit length (Rawlings

1988, p 277).

Variance Inflation Factor (VIF): Utility The diagnostic value of the

variance inflation factor is evident when it is written as follows:

_ 1VIFk - -- ,

1-Rf(1.2.15)

where Rk2 is the multiple correlation coefficient of Xk regressed on the other

variables. A "high" VIF indicates an R/ near one and thus, a collinearity. If Xk

is orthogonal to the other variables, VIFk will be 1.0. Thus, the VIF is a

A

measure of how many times larger the V(fJk) will be for collinear data than for

orthogonal data (Mansfield and Helms, 1982). This measure is not as useful

as the other diagnostic tools, however, because it cannot identify multiple

dependencies and because there is no clear-cut definition of what values of the

VIF are "high" and which are "low." It is of some use in detecting overall

problems not involving the intercept. Further, if collinearity is present, the VIFs

will be high, but the converse is not necessarily true. Because lack of a high

VIF often occurs when there are variables collinear with the intercept, Belsley

(1991, P 29) suggests using a VIF computed from uncentered data to enhance

10

..

detection of collinearities involving the intercept. The uncentered form of the

VIF is less general since it only applies when there is an intercept term in the

model.

1.2.2.3 Steps in the BKW Diagnostic Procedure

•

A process for diagnosing collinearity has developed empirically, based on

the work of many researchers. Belsley, Kuh and Welch summarized much of

this research in their book in 1980. Based on a series of experiments with

varying levels of contrived collinearity, they proposed a process for detecting

and assessing the extent and impact of collinearity. Belsley (1991) provided

an extension of the previous work based on another decade of research, his

own and that of others. Additional experimentation provided more information

regarding the diagnostics, especially when more than one dependency is

present. The steps in the procedure advocated by these authors are abstracted

here.

Table 1.2 The BKW Diagnostic Procedure·

1. Determine X.2. Scale the columns of X to equal length.3. Obtain condition indexes and variance

decomposition proportions.4. Determine the number of near dependencies5. Determine which variables are involved.6. Determine auxiliary regressions.7. Determine unaffected variables.

·Source: Belsley 1991, pp 134-135.

According to BKW (1980, p 112), there are two conditions which

should be satisfied in order to identify a "degrading" collinearity. First, a

11

singular value must have a high condition index. Second, condition one must

be associated with high variance decomposition proportions for two or more

estimated regression coefficient variances. After identifying the variables

involved in dependencies, the nature of the dependencies can be explored by

regressing the variates implicated on the others. Specific steps to be employed

for a given model are listed by BKW (1980, pp 157-158) as follows:

12

.,

,

OJ

Table 1.3 Details of the BKW Diagnostic Procedure"

1. Scale the data matrix X to have unit column length.2. Obtain the singular-value decomposition of X, and from this calculate:

a. the condition indexes fill.b. the n matrix of variance-decomposition proportions ...

3. Determine the number and relative strengths of the near dependencies bythe condition indexes exceeding some chosen threshold fl·, such as fI"=10, or 15, or 30.

4. Examine the condition indexes for the presence of competing dependencies(roughly equal condition indexes) and dominating dependencies (highcondition indexes---exceeding the threshold determined for step 3--­coexisting with even larger indexes.)

5. Determine the involvement (and the resulting degradation to the regressionestimates) of the variates in the near dependencies. For this step, somethreshold variance-decomposition proportion, n°, must be chosen (n" =0.5has worked well in practice). Three cases are to be considered.Case 1. Only one near dependency present. A variate is involved in, andits estimated coefficient degraded by, the single near dependency if it isone of two or more variates with variance-decomposition proportions inexcess of some threshold value n°, such as 0.50. Presumably, if only onehigh variance-decomposition proportion is associated with this singlehighest condition index, no degradation is exhibited.Case 2. Competing dependencies. Here involvement is determined byaggregating the variance-decomposition proportions over the competingcondition indexes .... Those variates whose aggregate proportions exceedthe threshold n" are involved in at least one of the competingdependencies, and therefore have degraded coefficient estimates. In thiscase, it is not possible exactly to determine in which of the competing neardependencies the variates are involved.Case 3. Dominating dependencies. In this case (1) we cannot rule out theinvolvement of a given variate in a dominated dependency if its variance isbeing greatly determined by a dominating dependency, and (2) we cannotassume the noninvolvement of a variate even if it is the only one with ahigh proportion of its variance associated with the dominated conditionindex--other variates can well have their joint involvement obscured by thedominating near dependency. In this case additional analysis, such asauxiliary regression, is warranted, directly to investigate the descriptiverelations among all of the variates potentially involved....

6. Form the auxiliary regressions. Once the number of near dependencies hasbeen determined, auxiliary regression among the indicated variates can berun to display the relations....

7. Determine those variates that remain unaffected by the presence of thecollinear relations. ...

"Source: Belsley, Kuh and Welch 1980, pp 157-158.

13

According to BKW (1980, pp 158-159), the quality of the regression can

be analyzed after the diagnostic steps have been conducted. They suggested

that the following can be learned: 1) how many near dependencies plague a

given set of data and what they are; 2) which variates have coefficient

estimates adversely affected by the presence of those dependencies; 3)

whether estimates of interest are included among those with inflated

confidence intervals, and therefore whether corrective action is warranted; 4)

whether prediction intervals based on the estimated model are greatly inflated

by the presence of ill-conditioned data; 5) whether specific coefficient

estimates of interest are relatively isolated from the ill effects of collinearity and

therefore trustworthy in spite of ill-conditioned data.

1.2.2.4 Computation of Collinearity Diagnostics

The review of diagnostic procedures described thus far assumes that the

analyst will compute them from scratch. Computation can be done easily using

the SAS IMl procedure; this was the path taken for this dissertation.

Alternatively, Velleman and Welsch (1981) present formulas useful in

computing diagnostics when the regression itself is computed by a previously

written or packaged program and illustrate computations using results from a

computer package. In addition, the article alerts the reader to nuances in

computations and in interpretation of diagnostics. Several points are

noteworthy in regard to this dissertation: 1) computation and interpretation of

the VIF is different for regression through the origin; 2) the spectral

decomposition of X'X is not as computationally stable as the singular value

14

..

•

decomposition of X in the computation of eigenvalues and eigenvectors; and

3) rescaling the columns of X can change conditioning. They advocate an

alternative to BKW's unit scaling, proposing that X be scaled such that the

decimal point separates the digits trusted by the analyst from those likely to be

in error.

1.2.2.5 Impact of Collinearity

Computational According to BKW (1980, P 114), the condition number

reflects the meaningfulness of the digits in the solution of the normal equations.

The rule of thumb given is: • .. if data are known to d significant digits, and the

condition number of the matrix A of a linear system Az = c is of the order of

magnitude 10r, then a small change in the data in its last place can (but need

not) affect the solution z= A-'c in the (d-r)th place.· (BKW 1980, P 114). Thus

as the condition number increases, the trustworthiness of the digits, i.e,

computational precision, in the least squares estimates decreases. In addition,

Belsley (1991, p 178) states that in ill conditioned data, small relative changes

in X and Y can produce large relative changes in the least squares estimates.

Statistical As mentioned previously, collinearity causes variances of the

least squares estimates to be unduly large. When this is true, the value of the

estimates for estimation, hypothesis testing and prediction is attenuated. Some

tests may not attain significance due to large variances of estimates and the

separate impact of important variables may be missed if they are collinear.

[See Willan and Watts (1978) for a discussion the impact of collinearity on

parameter confidence regions, tests of hypotheses, effective sample size and

15

predictability.J

1.2.3 Related Issues

1.2.3.1 Parameterization

A linear statistical model Y = X/J + ~ may be rewritten in a parallel form

as Y =(XG-')G!l + ~ • Z6 + ~ where G is a p x p nonsingular matrix.

According to BKW (1980, P 178), the collinearity diagnostics of Z do not

change substantially from those of X, if at all, however their composition may

be altered. Belsley (1991, p 165) states that neither the condition indexes nor

the variance-decomposition proportion of Z are the same as those of X. The

transformation does not reverse the ill-conditioning of X unless the

transformation is ill-conditioned in a manner reflective of the ill-conditioning of

X and designed to offset it. Usually this would not occur since

parameterizations are chosen based on modeling considerations and not on the

ill-conditioning of X. However, even if one did choose a transformation based

on problems in X, its own ill-conditioning would render it unstable

computationally. Thus, the BKW concluded that reparameterization does not

typically remedy the collinearity in the data. However, BKW did state that

"some linear combinations of regression parameters can be estimated even if

ill conditioning prevents precise knowledge of the specific parameters

estimated." (BKW 1980, p 178). In any case, the diagnostics should be

applied to the model actually used, reparameterized or not.

1.2.3.2 Scaling of Columns

As mentioned in section 1.2.2, diagnostics are applied to the X matrix

16

..

after its columns have been scaled. The obvious reason for scaling is to

remove the effect of unequal units in the columns of the X matrix on the

assessment of collinearity. The scaling is essentially a transformation that does

not result in an intrinsically new parameterization. It merely changes the units

of the columns of X and the elements of fl. However, Belsley (1991, P 171)

states that different column scalings of the same X matrix will give different

singular values and thus cause the collinearity diagnostics to vary according to

scaling. BKW give a procedure for optimal scaling (BKW 1980, P 184), which

removes this ambiguity and gives a matrix with minimum condition number (see

also Belsley 1991, p 171-172), but state that scaling for unit length is

sufficient to approximate the optimal scaling. Scaling to unit length is a form

of the more general scaling to equal length; it is accomplished by dividing the

elements of each column vector by the square root of the sum of squares of

the elements of each column XI (the norm of XI' II XIII) so that the resulting

XII II XIII has unit Euclidean length (Belsley 1991, P 135).

1.2.3.3 Mean Centering

There are divergent points of view with respect to centering each column

of the X matrix prior to fitting models and applying diagnostics. According to

BKW, X should not be centered if the data are relevant to a model with a

constant term (BKW 1980, p 98). They state that centering can mask the role

of the constant in any underlying dependencies and produce misleading

diagnostic results. Rawlings (1988) states that centering makes all

independent variables orthogonal to the intercept column and hence, removes

17

"nonessential collinearity," a term initiated by Marquardt (1980).

The topic of centering was addressed in detail by Belsley (1984) and

discussed by Cook (1984), Gunst (1984), Snee and Marquardt (1984), and

Wood (1984). Belsley (1984) says that the least-squares solution becomes ill

conditioned if small relative changes in X and Y can result in large relative

changes in the estimates; this is a numerical issue. This problem is evidenced

by the condition number of the X matrix and manifested by large variances of

the estimates, a statistical issue. Belsley argues that centering does not affect

conditioning of the X matrix because it does not change the inflated variances

and does not change the sensitivity of the data to perturbations. Mean

centered data do have lower condition numbers and thus appear to be well

conditioned, however Belsley states that centering can remove from the data

the information needed to assess conditioning. He provided empirical evidence

of these assertions. Further, Belsley said that data must be "structurally

interpretable," a term describing "data whose form allows a given numerical

relative change also to be meaningfully assessed as unimportant or

inconsequential relative to the real-life situation being modeled." The effect of

relative change, i.e., perturbations, is the basis for Belsley's assessment of

conditioning. He states that perturbations must be carried out on data whose

form makes sense. That is, the data must have an appropriate origin. Since

centering changes the origin, it renders the assessment meaningless. In

contrast to data-dependent centering, Belsley states that model-dependent

centering is acceptable if it gives structural interpretability. That is, it would be

carried out for all such models and is not dependent on the data at hand. It

18

•

..

seems that this is the only case in which Belsley would preform diagnostics on

centered data.

In response to Belsley's article, Cook (1984) states that Belsley focuses

on numerical stability rather than statistical stability, which is related to inflated

variances measured relative to a specific standard design. He says that

choosing a centering is equivalent to choosing the center of a structurally

relevant design space. Gunst (1984) takes another view of centering. He

reacts to Belsley's "dogmatic insistence that there is one correct technique

within which discussions of collinearity must be stratightjacketed." He believes

that centering does not demean diagnostics, but rather that one must

understand the nature of the centering and know where to look for the

appropriate diagnostic. He believes that collinearity is difficult to define and

that no one measure can completely characterize the nature and effects of

collinear variables. Gunst actually redefines collinearity, based on linear

dependencies, and advocates different measures for assessing different aspects

of collinearity. In fact, he re-presents a table which captures many of the uses

of diagnostics presented by BKW (1980). The table, shown below, summarizes

many of the concepts presented thus far in this chapter.

19

Table 1.4 Selected Collinearity Measures

eetection Measures Estimator Effects Precision

Predictor-variable Condition indices Variance inflationcorrelations factors

...Variance inflation factors Estimator correlations s.e. fA), s.e·(9)

Eigenvalues, eigenvectors Curve decolletages Variance decompositionof X'X proportions

Condition indices Volumes of confidenceellipsoids

Source: Gunst (1984)

Gunst advocates a global approach to assessment. He contends that structural

interpretability is a concept to be considered without regard to collinearity. He

states that the "application of structural interpretability to regression implies

that a constant term is included in the model if appropriate and not of

necessity." Gunst states that to properly diagnose collinearity, the X matrix

should be centered or standardized as deemed necessary for correct analysis.

In general, Gunst is supportive of the arguments of Belsley, but he prefers

different collinearity diagnostics.

Snee and Marquardt (1984) are critical of Belsley's article and produce

oblique arguments to counter his points. They state that the "domain of

prediction" is the key to proper centering and that the collinearity diagnostics

must also relate to this domain. That is, if centering captures the appropriate

domain of prediction, diagnostics should be performed on the centered data.

To counter Belsley's argument that mean-centering masks the role of the

intercept, Snee and Marquardt argue that the intercept is of little interest in

most cases; prediction at the center of the data is of more interest. Much of

20

•

Belsley's arguments for not centering revolve around the empirical evidence he

presents based on "small" perturbations of data which reveal their sensitivity,

centered or not. Snee and Marquardt criticize Belsley's approach by stating

that since the "small" perturbations he produced were actually large, his

conclusions are untenable. They claim that diagnostics should be carried out

on the model being fitted.

Wood (1984) states that the decision to center data "depends solely on

the substantive meaning of the data." Further, the goal should be to estimate

a meaningful intercept; this may involve centering some variables and not

others. Reverting to a limited diagnostic, Wood states that the ~2 values can

be used to assess pair-wise collinearity. In an example model, he shows that

the variance inflation factors (1/( 1-R2i)) reveal that independent variables show

high collinearity without centering and low collinearity with centering. Like

Snee and Marquardt (1984), he counters Belsley's conclusion that small

perturbations causing large shifts in coefficients is evidence of collinearity since

the perturbations Belsley used were not small. He uses a "real" example, in

contrast to Belsley's contrived example, to demonstrate that "centering

ameliorates collinearity but does not remove it."

Belsley (1991, pp 175-183) revisits the issue of mean centering,

providing examples to illustrate his claims. He demonstrates, by way of a

counterexample, that even though centered data have better condition numbers

than uncentered data, mean centering does not reduce the impact of

collinearity. That is, from a computational standpoint, small relative changes

in X and Y still produce large relative changes in parameter estimates; from a

21

statistical standpoint, the variances remain inflated. Further, he states that

diagnosing the conditioning of the mean-centered data (which are perfectly

conditioned) overlooks these two issues whereas diagnosing the basic data

does not (Belsley 1991, pp 178-179). Even worse, conditioning diagnostics

applied to mean-centered data provide misleading information about the true

conditioning of a set of data relevant to a regression model having a constant

term and remove information needed to assess problems with variables that are

collinear with the constant (Belsley 1991, p 183). Actually, since the

condition number for mean-centered data provides information about the

sensitivity of the parameter estimates to small changes in the mean-centered

data, i.e., data without "structural interpretability," the magnitude of the

condition number is also uninterpretable (Belsley 1991, P 189).

1.2.3.4 Distinctions and Additional Aspects

Collinearity and Correlation

Often the correlation matrix of regressor variables is examined for "high"

correlations between pairs of variables, as a means of detecting collinearity.

However the two issues are distinct. When two uncorrelated collinear variables

are plotted against each other, it can be seen that a small angle between the

two vectors is not equivalent to a high correlation between them (though two

vectors with a high correlation will have a small angle). Belsley (1991, P 20)

demonstrates that the angle between two variates is small, while the angle

between the centered variables is a right angle, indicating zero correlation.

Also, Belsley (1991, P 26-27) states that 1) a high correlation indicates

22

..

collinearity, but lack of a high correlation does not imply that no collinearity is

present; 2) three or more variables may be collinear while pairs of the variables

are not collinear; 3) the correlation matrix cannot expose the existence or the

number of coexisting collinear relationships; and 4) there is no consensus on

what values of the correlation coefficient should be considered as "large."

Mansfield and Helms (1982) present an example in which regressor

variables for a multiple regression are highly collinear even though no pairwise

correlations are large. They conclude that 1) the existence of large pairwise

correlations is sufficient for determining collinearity, but not necessary; and 2)

that examination of the eigenvalues and eigenvectors of the correlation matrix

is a "necessary and sufficient means" of detecting collinearity.

Collinearity, Conditioning, Weak Data and Short Data

The terms collinearity and conditioning have been used interchangeably

to denote linear dependencies in data. However, the concept of conditioning

is more global than that of collinearity and encompasses it. Collinearity refers

to a near linear dependency among a set of variables; conditioning refers to the

sensitivity of a relationship to perturbations in the data (Belsley 1991, P 7).

Collinearity in the X's can result in sensitivity in the least squares estimates to

small changes in the data.

Belsley defines "weak data" as data that has been robbed "of the

information needed for statistical analysis to proceed in some dimensions with

adequate precision" (Belsley 1991, P 7). Collinearity is a data weakness.

Another is "short data" (defined below).

23

Assessment of Damaging Collinearity

As indicated earlier, not all collinearity is deleterious. Though the

diagnostics reveal the presence of collinearity, they cannot alone determine the

degree to which it is affecting the parameter estimates. However, Belsley

(1991) has developed a statistical test for the presence of weak data, of which

collinearity is one type. If collinearity is present and the test indicates that the

data are weak, then the collinearity is harmful. Basically, the test assesses

whether the variances of the parameter estimates are "too large" by some

standard. Belsley (1991, P 207) uses the actual component of P as the

standard and tests it relative to its estimated variance. This "signal-to-noise"

test is

(1.2.16)

given that Pi is not equal to zero. He states that a test that T is high (low) isA.

also a test that V{fJj) is relatively low (high), which in turn signals the absence

(presence) of weak data. The parameter T is related to the noncentrality

parameter of the t distribution.

In order to operationalize this test, Belsley generalizes (1.2.16) to a

subset of regressors and proposes a measure of signal-to-noise and a test for

its significance. For details, see Belsley (1991), pages 209-213. The

significant points are captured here.

Initially, the P vector is partitioned so that the subset of interest is

A.

isolated from the other components. Similarly, the least squares estimators PA. A.

are partitioned to give rJJ'"p'2r. Then the signal-to-noise of the least squaresA.

estimator P2 of P2 relative to IJ"2 (any arbitrary point) is

24

•

A A

Because Pz - Np2(IJz'V(lJz))' whereA

V(lJz) = u2(X'zM,Xzr' and

it follows that

(1.2.17)

(1.2.18)

(1.2.19)

A A A

(lJz - 'z),V·'(lJz)(/1z - 'z) - r (A). (1.2.20)P2

Here, r (A) denotes the noncentral chi-squared distribution with P2 degrees ofP2

freedom and noncentrality parameter

A

A • (/1z - 'Z)'V·'(/12)(/1z - '2)'

Then the test statistic is derived as

(1.2.21 )

(1.2.22)

where S2 is the residual sum of squares divided by its degrees of freedom (n-p)

and where (n-p)s2/u2 - r n-p' Then the test statistic (/)2 is distributed as a

noncentral F with P2 and n-p degrees of freedom and noncentrality parameter

A as in (1.2.21), which is the same as the signal-to-noise parameter -r. For the

test, .A = T. 2 and

(1.2.23)

This tests that Ao: -r =T. 2 against-r> T. 2• For test size a, calculate

(1.2.24)

•

the (1-a) critical value for the noncentral F with P2 and n-p degrees of freedom

and noncentrality parameter T. 2• If (/)2 s Fcr' accept Ao; if (/)2 > Fcr' reject Ao '

A

Belsley notes that this test requires one to know Pz and V(/1z) in order to

25

provide 7. 2• He then proposes a ·practical and intuitively appealing definition

for an adequate level of signal-to-noise· that does not require this knowledge.

Details are provided in Belsley (1991), pages 216-217. Essentially, he definesA

a ·y-isodensity ellipsoidW for the least-squares estimator Ilz of Ilz' This is

sometimes call the •ellipse ofconcentration.· The y-isodensity ellipsoid definesA

regions of likely and unlikely outcomes for the least-squares estimator Ilz (given

A

X and V(JJz) (Belsley 1991, P 215). This is the ellipsoid of smallest volume thatA A

contains any particular least-squares outcome from I z - Np2(JJz'V(JJz)) with

probability y (Belsley 1991, P 214). Belsley then defines the probabilisticA

distance between ,.z and Ilz relative to I z as the y that determines the

isodensity ellipsoid that is centered on Ilz and includes poz on its boundary

(Belsley 1991, P 215). The is the y such that

•

A A A

(Jrz-Ilz)'Y" (JJz) (Jr2-/2) = I" r p2' (1.2.25)

A level of signal-to-noise y2 is large if it corresponds to a large probabilistic

separation of /1'z from Pz, Le., if it equals a value of )<;2 in the range 0.90 to

1.0. A weak signal-to-noise indicates little separation and has a small value

«0.75) (Belsley 1991, p 215). The magnitude is called

(1.2.26)

or the threshold of adequacy at level y. Then the signal-to-noise r2 of the least-A

squares estimator Ilz of 12 relative to"z is called adequate at level y if

(1.2.27)

..Further details and critical values for testing are provided in Belsley (1991, pp

216-244).

26

•

The pertinence of these results is that a low value of r indicates that

there are inflated variances and data problems either in the form of collinearity

or short data. Belsley clarifies the distinction between collinearity and short

data. -Harmful collinearity is defined as inadequate signal-to-noise occurring

simultaneously with collinearity, while short data is defined as inadequate

signal-to-noise without concurrent collinearity. - (Belsley (1991, p 209) It is

useful to know in which sense the data are weak in order to take corrective

action. Belsley sees this test as complementary to the collinearity diagnostics.

The test can determine the presence of weak data, but not its cause; whereas

the diagnostics can determine whether an existing data weakness is due to

collinearity or to short data.

Collinearity-Influential Observations

Influence diagnostics are aimed at the detecting the disproportionate

effect of an observation on the regression coefficients and other model

parameters. In addition, an influential observation may also induce (or mask)

collinearity. These observations are called collinearity influential (Belsley 1991 I

P 246). Influential observations can be those with outlying values of the

dependent variable Y or those with leveraging values of a row of X, or both.

Leverage indicates that a value is so divergent from that of the other

observations that it overly impacts the estimation process. Several diagnostics

for detecting these collinearity influential observations are given by Belsley

(1991, Chapter 8). One general method attributed to Chatterjee and Hadi

(1988) measures the relative change in the condition number of X that results

from the deletion of a row of X. Belsley gives several weaknesses of the

27

diagnostic, one of which is that it is computationally intensive for a large

number of observations. He also notes that diagnosing the presence of several

coexisting collinearity-influential observations becomes more difficult. Belsley

concludes that the diagnostic process can identify "good" or "bad" data points

in terms of inducing or worsening collinearity. And the decision to correct,

remove or keep them would depend on how they are classified. However,

according to Belsley (1991, P 270), this process does not add anything to

currently used influential-data diagnostics.

A contrasting view was presented by Mason and Gunst (1985). They

suggest that when outlier-induced collinearities occur in X, estimators can

exhibit effects that are associated with both collinearity and outliers. In

particular, collinearities produce large coefficient estimates while leverage

points tend to drive estimates toward zero. They recommend a careful

examination of the nature of collinearity in a data set, providing general

procedures for combining influence and collinearity diagnostics.

Maverick Interlopers and Raging Controversies

After this review, the process of detecting collinearity might seem

straight-forward. However, many researchers still disagree on several issues.

Often, an article by one author is followed by reviews and comments by others.

To read the entire set is both illuminating and entertaining. One can almost be

equally persuaded by each point of view. Stewart (1987) reviewed current

diagnostic measures for collinearity and proposed new ones called collinearity

indices, defined as the square root of the variance inflation factors. He gives

four reasons why a change to these indices is desirable: 1) the indices are

28

...

scale invariant; 2) the nomenclature reflects the purpose of the indices while

that of other measures does not; 3) the indices vary linearly with the relative

distance to exact collinearity, whereas variance inflation factors vary as the

square (This renders them more interpretable and "removes unsightly square

roots from formulas. "); and 4) the indices cast results in terms of relative

errors. Stewart addresses most other issues raised regarding diagnostics,

including the effects of centering and errors in regression variables. He

provides a diagnostic procedure and gives the properties of the indices. This

extensive and impressive article concludes with a suggested documentation for

a regression package that uses collinearity indices.

Comments following the article by Marquardt (1987), Belsley (1987),

Thisted (1987), and Hadi and Velleman (1987) are filled with praise for

Stewart's "lucid and practical article," "his substantive contribution," and his

"time and energy." The praise is followed by each reviewer's criticism, harsh

at times, and re-presentation of his own points of view, often integrated with

those of Stewart, however. Belsley (1987) concludes his review of Stewart's

article by stating that if information is needed regarding multiple dependencies,

then something more than VIFs and Stewart's indices will be needed. In this

case, he says "try mine, you'll like 'em." This is the route taken for this

dissertation.

1.2.4 Procedures for Resolution of Collinearity

.. After the presence and nature of collinearity have been determined, steps

may be undertaken to correct or remedy the ill conditioning in the data. Several

29

measures for resolution of the problems are described in this section. They

range from some simple data-related approaches to the use of Bayesian and

biased regression methods.

1.2.4.1 Column Scaling

A simple approach to improving conditioning is to scale the columns to

unit or equal length as described in section 1.2.3.2. This procedure does not

add or delete data, but transforms existing data in a manner that renders it

more computationally optimal. According to BKW (1980, P 194), column

scaling can often reduce the condition number by a factor of 103 or more.

Marquardt (1980) emphatically presents his views regarding scaling. He

states that the goals of an analyst are "facilitated when the predictor variables

are expressed in a scaling that makes the terms as nearly orthogonal and of

equal size as practicable." He claims that standardization is the only way to do

this. He provides explicit steps for scaling regression data.

1.2.4.2 Deletion of Variable(s) Involved in Dependencies

One seemingly obvious solution for improving ill-conditioning caused by

variables involved in dependencies is to delete redundant variables. When this

is done, the collinearity is removed. However, the model may no longer be

meaningful. Belsley states that this procedure should be avoided. "If an

investigator has reason for including a variate in the regression model in the

first place, there is just that much reason for not excluding it capriciously."

(Belsley 1991, P 301). If collinearity is affecting the parameter estimates, his

conclusion would be that "the data lack the information needed to accomplish

30

the statistical task at hand with precision, not that the model must be molded

into a form that looks good relative to the data." (Belsley 1991, P 304).

Hamilton (1987) shows by way of example that "omitting all but one of

the redundant variables," as is of often advocated, can amount to "throwing

out the baby with the bathwater." He warns of the dangers of discarding

variables as a cure for collinearity because correlated variables are not always

redundant. (This is the other side of the coin for the statement the

uncorrelated variables are not necessarily not collinear.) He restates the view

of several others that backward elimination variable selection procedures be

used in contrast to forward selection in the presence of correlated explanatory

variables.

1.2.4.3 Introduction of New Data

..

Another approach to improving conditioning is to collect and use new

data points in order to provide more variation than that contained in the original

data. However, this is often an impractical approach because of time, budget

or study design constraints. Even if new data are used in the model, there is

no guarantee that the problem will be completely alleviated (BKW, pp 193­

194).

1.2.4.4 Bayesian-type Techniques

Pure Bayesian

BKW (1980, P 194) mention, but do not describe, a purely Bayesian

approach developed in the 1970's. In this technique, prior subjective

information about the parameters in the model is used to improve conditioning.

31

Several disadvantages of the approach are listed. First, it uses subjective

information that is either not obtainable or not trusted by many investigators.

Second, the statement of the exact distribution may be too precise to be

realistic. Third, the theory underlying the decision is neither well understood

nor accepted. Fourth, computer software needed to apply Bayesian methods

is not available widely (BKW 1980, P 194).

Mixed-Estimation

The approach called mixed-estimation, which was also developed in the

1970's, is discussed more fully by BKW (1980 pp 195-196); their description

is paraphrased below. In this technique, supplementary information is added

to the data matrix. For the linear model

Y = X/J + e, (1.2.28)

with E[e] =0 and V[e] =I" restrictions on the elements of IJ are constructed in

the form

c=R/1 + f, (1.2.29)

with E[f] =0 and V[f] =1 2 • R is a matrix of rank r of known constants, c is an

r-vector of specifiable values, and fis a random vector, independent of e, with

mean zero and variance-covariance matrix 12 which is specified by the

investigator. Y and X are augmented to give

[j = [~] P + [f]' (1.2.30)

where

According to BKW (1980, P 195), if I, and 12 are known, the solution to the

equation is obtained using generalized least squares; the unbiased mixed-

32

..

v [fl • [~ ;J.I.(1.2.31 )

estimation estimator is

bME = (X'2:;'X + R'2:2-'Rr'(X'2:;'Y + R'2:2-'c). (1.2.32)

5" an estimate of 2:" is used and 2:2 is given by the investigator as prior

information. The solution may be better conditioned than the problem without

the supplemental prior information.

1.2.4.5 Biased Regression Techniques

It is often desirable to use least squares estimators of the regression

coefficients since they are the best linear unbiased estimators. However, when

the data are collinear, the variance of the estimators may be too large. Opting

for a biased estimator which has a smaller variance is the basis of biased

regression methods. In this case, bias is exchanged for precision.

Several types of biased methods have been proposed, including Stein

shrinkage, ridge regression and principal component regression (Rawlings 1988,

p 337); there are all similar. Rawlings states that biased regression techniques

have not been universally accepted and should be used with caution. He

advises that while biased solutions may be better for estimation purposes, they

may not be better for other purposes, presumably prediction purposes

especially outside the existing X-space. Also, biased solutions may not be

useful in assessing the relative importance of independent variables involved in

a collinearity (Rawlings 1988, p 338). Mason and Gunst (1985) demonstrate

that biased estimators may not be effective alternatives when collinearities are

33

outlier-induced. Swamy, Mehta, Thurman and Iyengar (1985) state that the

determination of whether biased estimation methods can cope with

multicollinearity in least squares regression depends on knowing the estimates

of collinearity implied by both types of estimators. They developed a new

measure of multicollinearity suited to biased estimation. After developing and

applying a formula for measuring collinearity in both cases, they found that "if

one forces a biased regression estimator to satisfy the minimax conditions, then

the other goal of reducing multicollinearity will not be realized." With these

qualifications for perspective, the biased methods are now presented.

Ridge Regression

In ridge regression the ill-conditioning of the X'X matrix is reduced by

adding a small positive constant term to the diagonal elements. The parameter

estimates produced using the augmented X'X matrix are biased. According to

Rawlings (1988, p 338), ridge regression is carried out on the centered and

scaled independent variables Z. The estimate of Pis

b, = (Z'Z + kl)°'Z'Y. (1.2.33)

The variance of Pis

V[b,] = (Z'Z + kl)°'(Z'ZHZ'Z + kl)O'02.

And the bias of b, is

E(b,) - P = [(Z'Z+kl)°'Z'Z - 1]p.

(1.2.34)

(1.2.35)

The choice of a value for k involves several considerations. If k = 0, ridge

regression is equivalent to ordinary least squares regression. According to

Rawlings (1988, P 339), as k increases from 0, several quantities decrease:

1) maximum variance inflation factor, 2) the sum of the variances of the

34

•

estimated regression coefficients, 3) the length of the vector of parameter

estimates, and 4) R2• As k increases, the variance of the parameters decreases

but the bias increases. Thus, the objective of the procedure is to choose a k

that is ·optimal· in reducing variance while minimizing bias. Several choices

of k may be used. Then the estimates of individual regression coefficients can

be plotted against k to give ·ridge traces· (Rawlings 1980, p 339). According

to Rawlings, the value of k chosen is the smallest value where major changes

in the parameter estimates and their variances occur and where R2 has not

decreased too mUCh. The choice is somewhat subjective, but these are useful

guidelines. Rawlings (1988, P 339) also gives a specific equation for

computing k:

(1.2.36)

where p is the number of parameters excluding the intercept, S2 is the residual

mean square estimated from ordinary least squares regression and P(O) is

ordinary least squares regression coefficients, excluding the intercept and

computed with centered and scaled variables (Rawlings 1988, p 339).

Ridge regression has a Bayesian interpretation if the prior probability

distribution of P is assumed to have zero mean and variance-covariance matrix

1(02/k). Then the choice of k expresses the prior belief regarding the variances

of the pistributions of the true regression coefficients. The larger the value of

k, the greater the shrinkage toward zero of the ridge regression estimates from

the ordinary least squares estimates (Rawlings 1988, p 340). [See OmanA

(1982) for a discussion of the choice of the origin towards which P is shrunk.]

In summary, the ridge estimates are weighted averages of the least squares

35

estimates with greatest weight given to the regression coefficients of the

variables involved in the near-singularities (Rawlings 1988, p 341). [See Smith

and Campbell (1980) for a review and critique of ridge regression methods and

the article by Marquardt (1980) for related comments. In particular, Marquardt

discusses the relationship between predictor variable scaling and the a priori

assumptions of ridge regression.]

Two other estimators that are related to the ridge estimator are

mentioned by BKW (1980, P 196). These are the generalized ridge estimator

that has the form

bgr = (X'X + 4r'X'Y, (1.2.37)

where 4 is a positive-definite matrix, and the wedge estimator that has the

form

bw • (Z'Zr'Z'Y where Z = X +kX(X'Xr' . (1.2.38)

Principal Components Regression

In principal components regression, the dimensions of the X-space that

are causing the collinearity are eliminated by restating the model in terms of a

set of orthogonal explanatory variables. This is accomplished by forming a

linear combination of the collinear variables rather than retaining only one (or

a subset) of the collinear variables. Then a dimension is represented by the

combination. These are known as the principal components. They lack simple

interpretation since each is a mixture of the original variables. According to

Chatterjee and Price (1991 ), these new variables enable one not only to obtain

information about collinearity, but also serve as the basis of an alternative

estimation technique. Rawlings' (1988, pp 344-349) description of the process

36

of obtaining the principal components is paraphrased here. First, the singular

value decomposition of the centered and scaled variables Z is obtained. Then

the principal components are the linear combinations of the ~ (column vectors)

that are specified by the eigenvectors of Z. The matrix of sums of squares and

cross products of the principal components is the diagonal matrix of the

eigenvalues. The first principal component has the largest eigenvalue and the

principal components corresponding to the smallest eigenvalues are the

dimension of the Z-space with the least dispersion, typically those dimensions

involved in the collinearity.

Using (1.2.4), the SVD of Z is

Then the linear model

can be written as

Y=2/l+E

Y = ZW'P + E,

(1.2.39)

(1.2.40)

(1.2.41)

since VV' = I. The model can be written in terms of the principal components

as

Y = Wy + E, (1.2.42)

where W = ZV and y= V'p. Then y is the vector of regression coefficients for

the principal components and P is the vector of regression coefficients for the

Z's. Next, Y is regressed on W, the principal components, using ordinary least

squares. The regression coefficients for the principal components are

A

Y=(W'W)-1w,y = 1\2W'Y, (1.2.43)

since W'W = 1\2 =Diag(A/~A/~ ... ~Ap2) are the eigenvalues of Z'Z. The

37

A

variance of y isA

V[y] = ,,-2u2. (1.2.44)

Since the matrices involved in these computations are orthogonal, the

estimates of the regression coefficients and their variances can be computed

individually. Then, the coefficients are tested for significance and eliminated

if nonsignificant. In addition, they are eliminated if they cause a collinearity

problem. Otherwise, they are retained; retained coefficients are denoted by the

subscript g. Finally, the regression coefficients for the principal components

are converted to the regression coefficients for the original variables Z by

.

The estimated variance is

A

/l+ (gl = V(gIY(gl'

2rD+ ] _ V A-2V' 2S IP (gl - (gin S.

(1.2.45)

(1.2.46)

Rawlings suggests being conservative in eliminating principal components

since each elimination is a constraint on the estimates and another increment

of bias. He advocates not eliminating a principal component for which Yj is very

different from zero (Rawlings 1988, p 348).

Mandel (1982) gives the geometric representation and interpretation of

the singular value decomposition and its relation to principal components

regression. For a collinear data set, he illustrates the use of the SVD in

principal components, noting that both the detection and the treatment of

collinearity is greatly facilitated its use. In particular, he echoes the observation

of others that even though regression parameters cannot be estimated precisely

in collinear data, certain linear combination of the coefficients can be estimated

38

with confidence. In addition, he notes that valid predictions can be made under

collinear conditions provided that they take place in the same subspace as the

points on which the regression was computed.

1.3

1.3.1

The Mixed Effects Model (MIXMOD)

Introduction

In medical and public health research, studies involving the observation

of a longitudinal response profile for each subject in two or more groups are

common. The longitudinal dimension, or metameter, might correspond to time

or to varying conditions, such as a sequence of increasing doses of a stimulus

or treatment. Typically, interest lies in determining and modeling one or more

responses over time or under the different conditions. In these studies, the

times or conditions for observation may vary somewhat from subject to subject

and thereby complicate the analysis. Some longitudinal terminology is useful

is referring to such studies. A longitudinal study is (Helms, 1992; Grady and

Helms, 1992):

regularly timed

irregularly timed

consistently timed

if

if

if

the time interval between measurement

occasions is the same throughout the study,

e.g., each month;

the time interval between measurement

occasions differs, e.g., weeks 1, 2, 4, 8, 12

and 24;

all subjects are evaluated on the same

39

schedule, whether or not regularly timed;

inconsistently timed if subjects are evaluated on different schedules,

due to missed appointments or because the

study design called for data collection after a

specified event or episode in the subject's life;

balanced if there are no missing data in a consistently

timed study;

unbalanced if there are missing data and the study is

consistently timed, or if the data are from an

inconsistently time study;

complete if there are no missing data; and

incomplete if there are missing data values.

If longitudinal data are unbalanced or incomplete, then analysis using

standard general linear multivariate model (GLMM) methodology is difficult.

One of the assumptions of GLMM methodology is that the elements in a

column of Yare measurements of the same entity, i.e., measurements at the

same time point or under the same conditions. Thus unbalanced longitudinal

data might violate this assumption, even if the data are complete. Because

data collected longitudinally are rarely complete and because GLMM methods

do not handle missing data, several other analysis approaches have been used

to deal with incomplete and/or unbalanced data. One approach, case-wise

deletion, is to delete all of the data from any subject with any missing data and

then employ methods for analyzing complete data. Another approach is to

impute values, such as the mean of existing observations, to missing data

40

points and then to proceed with the analysis. These approaches involve

manipulation of the data that may result in loss or contamination of

information. A more desirable approach is one in. which the available

incomplete data can be utilized without the constraints imposed by the other

methods.

Recently, statistical methods, in particular the mixed model, have been

developed that permit the analysis of longitudinal data when they are affected

by unbalanced designs, missing data, attrition, time-varying covariates and

other characteristics that make standard multivariate procedures inapplicable.

[See Searle (1988) for a history of the mixed mode!.] Ware (1985) states

several advantages offered by the mixed model. Individuals need not be

observed at the same times or on the same number of occasions. In addition,

time-varying covariates can be included in the model if their contribution to the

expected response can be written linearly. Also, covariates can modify either

the expected value of the dependent variable or its rate of change. Finally,

Ware notes that the mixed model offers other generalities, such as inclusion of

trigonometric functions, that do not complicate the analysis.

1.3.2 Definition, Notation and Assumptions

For the situation in which the kth subject is observed on nk occasions,

the mixed model with fixed population effects and random individual effects is

(Helms, 1992)

y = X IJ + Z d + 8.

The model equation for the kth subject is

41

(1.3.1)

(1.3.2)

where

Y (Nx1)

X (Nxp)

Z (NxKq)

is a vector of the nk observations (dependent variable or

responses) for the kth subject, k= 1,2, ..., K;

= [Y, /I Y2 /I ... /I YK] is the vector of responses from all K

Ksubjects (N= r nk ) (-/I- denotes the vertical concatenation

k-'operator);

is a known fixed effects design matrix for the kth subject (The

columns of Xk represent the values of independent variables.);

= [X, II X2 II ... II XK] is the fixed effects design matrix for the

model;

is a known random effects design matrix for the kth subject (The

columns of Zk represent the values of independent variables.);

= Diag(Z" Z2' "', ZK) is the random effects design matrix for the

model;

..

The following assumptions are made:

/l (p x 1)

dk (q x 1)

d (Kq x 1)

is a vector of unknown constant population parameters, a vector

of fixed effect primary parameters, essentially the same as /l in

univariate models;

is a random vector of unobservable random subject effects for the

kth subject;

= [d, II d2 /I ... II dK] is a vector of random subject effects for the

model;

is an vector of unobservable within-subject random error terms;

42

dk (qx 1)

4 (qxq)

..

...

and

8 (N x 1) = [8, II 82 II ... /I 8K] is a vector of random error terms for the

model.

The following additional assumptions are made:

- NIDq (0, 4);

= V(dk) is the covariance matrix of the random effects (each dk

has the same covariance matrix);

- NIDN (0, UZVk ) independent of the dk; and for k'¢k,

Cov(dk"dk) = 0; Cov(dk,,8k) = 0; and COV(8k,,8k) = 0;

UZ is an unknown scalar within-subject error variance parameter;

Vk (nkxnk) = V(8k ) is the covariance matrix of the random deviations about

the kth subject's random regression line.

These assumptions lead to the following:

Yk - NID (Xk P, ~k);

Y - NN (X P, I);

~ (NxN)

1.3.3

is the V(Yk) =~ = ~ 4 Zk' + UZVk, a positive definite symmetric

covariance matrix for the kth subject (often, it is assumed that

Vk = I..);

= Diag(~" ~2' "., ~) is the covariance matrix of the entire

response vector, Y.

Features of the Covariance Structure

There are several aspects of the mixed model covariance that are

noteworthy. First, the measurements from each subject are correlated, Le.,

43

V(YIl) =Ek is not diagonal. Second, the diagonal elements of Ek are not

necessarily equal. Third, Ek is modeled in terms of a smaller number of

parameters, the elements of 4 and 02. An additional feature of the mixed

model is that the covariance parameters (4 and 02) of ~ = ~ 4~' + o2vll

must be estimated, since they are rarely known. Thus, ~ is estimated. Nested

within the modeling of E is an option of also modeling 4. Finally, prior to the

estimation of the covariance parameters, the structure of Vil must be specified.

The table below provides some of the possibilities for modeling 4 and

structuring Vil • The modeling of t:,. is described in section 1.3.4.3. The

structure of Vil is determined from prior information about the data. Jennrich

and Schluchter (1986) provide a menu (in Table 1) of possible covariance

structures for the incomplete data model. Grady and Helms (1992) explored

a variety of covariance structure models and illustrated methods for comparing

the models with respect to goodness of fit. Louis (1988) states that choosing

among covariance models depends on data structures, subject-area theories,

and available computer packages. Further, he states that the choice affects

estimates and standard errors of fixed effects, diagnostics, interpretations and

extrapolations.

44

..

..

Table 1.5 Structures and Models for the Components ofV(Yk) =~= ~4~' + a2vk

Variance of Random Effects Variance of ek

~4~' 02-vk

Models for 4: Structures for Vk:

(Vk is assumed to be known.)

Linear Vk is diagonalAll parameters used (uncorrelated deviations)

(unconstrained) Diagonal elements equalFewer parameters used and equal one

(constrained) Diagonal elements not equal

Nonlinear Vk is not diagona1(correlated deviations)

Diagonal elements equalDiagonal elements not equal

1.3.4 Estimation of Parameters

There are several aspects to the estimation procedure for the primary

parameters and covariance parameters (fJ, 4, and 02-) of the random effects

model. These are summarized in the table below.

Table 1.6 Aspects of Mixed Model Estimation

Parameters Estimation Principles ComputingEstimated Algorithms

p Maximum Likelihood (ML) EM4 Restricted Maximum Likelihood (REML) Newton-Raphson02- Scoring

1.3.4.1 Estimation Principles

Two principles for the estimation of the mean and covariance parameters

are maximum likelihood (ML) and restricted maximum likelihood (REMLl. [See

45

Searle (1988) for a discussion of several estimation principles and their relative

merits.] Both principles produce sets of simultaneous nonlinear estimation

equations whose solution requires iterative computations. In the ML principle,

reviewed by Harville (1977), the log likelihood function

L(,r,4,p; Y) = constant

(1.3.3)

is maximized for 02,4, and p. One criticism of the ML approach is that the ML

estimators of 02 and ~ do not account for the 10$s in degrees of freedom that

results from the estimation of p.

This "deficiency" of the ML approach can be eliminated using the

restricted maximum likelihood (REML) approach. REML estimators for 02 and

4 maximize L" where (Harville, 1977)

L, (,r,4; Y) = constant

.

(1.3.4)

Here X· is an n x p' matrix whose columns are any p' linearly independent

columns of X and p is any solution of the normal equations

1.3.4.2 Computing Algorithms

(1.3.5)

There are many iterative algorithms that can be used for computing the

ML or REML estimates. Harville (1977), in his review of several algorithms,

46

states that the choice of the "best" algorithm depends on the application and

on computational requirements.

Use of the E-M algorithm to obtain either maximum likelihood (ML) or

restricted maximum likelihood (REML) estimates of the parameters has been

described by Dempster, Laird and Rubin (1977), Laird and Ware (1982),

Fairclough and Helms (1984), Laird, Lange and Stram (1987) and Laird (1988).

The EM algorithm is used for computing estimates of parameters in incomplete

data problems. In each iteration, it consists of two steps: an expectation (E)

step and a maximization (M) step (Laird, 1988). The maximization step is

based on maximizing the likelihood of the "complete data." The rationale for

using this technique in the mixed model situation, which treats data as if it

were complete even though it may be incomplete, lies buried in the history of

the mixed model. Dempster, Laird and Rubin (1976) and Laird and Ware

(1982) describe the connection. It is worth noting at this point that for the

mixed model, the dk can be considered as "missing." It is for this reason that

the EM algorithm can be used in this situation. In effect, the dk are appended

to Y vector as "missing" data. In this manner, they are "estimated." Searle

(1988) prefers to say that they are "predicted" since "the estimation of random

variables is counterintuitive statistically." Laird and Ware (1982) state that

they regard no data as missing; they use the EM algorithm to "estimate"

unobservable (random) parameters, not missing observations. So, in the E­

step, the conditional expectation of the 'complete-data sufficient statistic' is

computed based on the observed data and current estimates of the unknown

parameters. In the M-step, the maximum likelihood estimates of the

47

components of ~ are computed, with the sufficient statistics replaced by the

conditional expectations of the statistics produced in the E-step. [See

Fairclough and Helms (1984) for a more complete summary of the process.]

Laird and Ware (1982) also derived an E-M algorithm for computing ML and

REML estimates in the general linear mixed model for V. = I". and unstructured

~. According to Laird (1988), the EM algorithm may not be the most efficient

algorithm and may be slow to converge. However, it offers a general approach

that can be applied in a wide variety of settings; it is easy to implement; and

it will not converge to parameters values outside the boundary of the parameter

space.

Harville (1977) and Callanan and Harville (1991) reviewed various

algorithms for computing ML and REML estimators, including the Newton-

Raphson procedure and the method of Scoring. The use of Newton-Raphson

and related procedures to estimate parameters of random effects models also

was discussed by Jennrich and Schluchter (1986). The Newton-Raphson

procedure is a gradient procedure that utilizes second-order partial derivatives

of the log-likelihood function. This algorithm can converge in few iterations

provided that its initial values are near the maximum value. However, it may

converge to a stationary point which is not a maximum or it may not converge

at all, if the initial values are poor (Harville, 1977). Apparently, the difficulty

can be overcome by using the "extended" Newton-Raphson procedure. (See

Harville (1977) and Callanan and Harville (1991) for a more complete summary

of the procedures.) The method of Scoring, another gradient procedure, uses

the expected values of the second-order partial derivatives instead of the actual

48

derivatives (Callanan and Harville, 1991).

Forms of the maximum likelihood equations that are useful for

computations may be found in Fairclough and Helms (1986), Jennrich and

Schluchter (1986), Laird, Lange, and Stram (1987), Lindstrom and Bates

(1988) and Callanan and Harville (1991). Jennrich and Schluchter (1986) also

discuss implementation of the algorithms, considering computational efficiency,

modifications to improve convergence, methods for dealing with nonpositive

definite estimates for J:, and constraints on the covariance parameters.

1.3.4.3 Extant Procedures for Estimation of P, A, and a2

The procedures of interest for this dissertation involve maximum

likelihood and restricted maximum likelihood estimation with the E-M algorithm.

The maximum likelihood estimate of P is a solution of the Aitken estimation

equation:

(1.3.6)

A. At A A. At. At. A

where J:k = Zk 4 Zk' + o2vk and 4, 02, and Vk are maximum likelihoodA A

estimates of 4, 02, and Vk ' respectively. In order to compute p, ~ must beA A

estimated, however estimation of ~ requires knowledge of p. Thus, an

A A

iterative algorithm must be used to solve for P and for~. There are several

versions of the maximum likelihood estimation equations for the variance-

covariance parameters, depending upon the structures of 4 and Vk and the

algorithm being used to solve the equations. Fairclough and Helms (1986)

obtained the following maximum likelihood estimating equations for use with

49

A A

the E-M algorithm. These procedures usually require initial estimates of /1, 4,

A

and cr in order to set the iterative process in motion.

Estimation Steps

A A A

1. Estimate /1 using (1.3.6) (For first /1, let E.t =1; this is the OLS

estimate of /1.)A

2. Solve the following equation for each dk, k =1, 2, ..., K: (For the

A

first iteration, let 4 =0 and cr =1.)

[b24-' + z:V;'z.]a.=z:v;'(Y.-X.h).

A

3. Compute cr as

U' =~ [t. (Y.-X~ - Z,.d.) I V;l(Y.-X~-Z,.d.)J.

A

4. Compute 4.

(1.3.7)

(1.3.8)

If 4 does not have a linear covariance structure, Le., is not

modeled, the estimator of 4 is

K.. 1 r .... I4 = K4JdP•.

k-1

(1.3.9)

If 4 ~ have a linear covariance structure, then its model

equation is:

where

H

4 = V(d.) =E ThGh ,h-1

(1.3.10)

Gh denotes a known, constant, symmetric matrix, h =1, 2, ...

50

H;

Th denotes an unknown variance-covariance parameter, h = 1,

2, ... H; and

T = (T1, T2, •••, TH)' denotes the H x 1 vector of variance-

covariance parameters.

Then, the estimator of 4 is

A-

T is the solution of

(1.3.11 )

where the notation <> gil denotes a matrix whose (g,h)-element is

the scalar inside the angled brackets ( <>)and the notation <>g

..

~ [ (Trac~A -1 G,A -1 G" ) ),h] t =

(t Trac~A-'G,A-'iljJ~ ) ), ,k-'

(1.2.12)

denotes a vector whose gth element is the scalar inside the angled

brackets.

5. For each k = 1, 2, ... , K, compute

(1.3.13)

1.3.5

A A

(If 4 has a linear covariance structure, use ~ in place of 4.)

6. Repeat the entire process until the estimates have converged.A A A

Then, the final estimates of the parameters P, 4, and if are used.A

Variance of /l

Exact small-sample expressions for variances and covariances of

MIXMOD maximum likelihood or REML estimators are not available (Helms,

51

1992). Instead, asymptotic expressions for variances, covariances andA.

standard errors are used. The asymptotic variance of /l is given by

(1.3.14) ..

or

(1.3.15)

1.3.6 Prediction

Harville (1977, P 322) states that predicting a future data point from

data to which the mixed model applies can be formulated as problem of

estimating a linear combination of the components of the fixed and randomA. A.

effects, Le., of IJ and d. Helms (1992) gives a "convenient" estimator of a

future data point as

,

K

W = LrJ + E L.dt '*-1

(1.3.16)

where Lk , k =1, 2, ..., K are known, constant matrices, each with a rows. (to is

a xp; Lk is a x q, k ~ 1.) Helms (1992) gives the variance and standard error

of the predicted value as

and

v(w) = L(JL I(1.3.17)

(1.3.18)

A.

with vEH =N-Rank(X II Z) d.f., where Q is a generalized inverse of the coefficient

matrix in the mixed model equations,

52

(1.3.19)

1.3.7 Objectives of Mixed Model Analysis

In general, the objectives of a mixed model analysis are (Hold itch-Davis,

•

Helms and Edwards, 1992) 1) to estimate the fixed effects component and testA

hypotheses about the population regression coefficients, p; 2) to simplify the

fixed effects component by eliminating variables with nonsignificantA

coefficients (/1); 3) to estimate and test hypotheses about Ii, the covariance

matrix of the random effects; 4) to simplify the random effects component by

eliminating variables with small coefficients (6jj); and 5) to simplify the model

for Ii by including large covariances (6•.) and deleting small ones. Methods for

4) and 5) have not been reviewed for this dissertation because they are not

particularly relevant.

1.4 Collinearity Diagnostics for Mixed Models

In the mixed model, collinearity in the fixed effects arises from ill-

conditioning of (t·' /2X) and (X't·'X), leading to inflated elements of

V(/J) =(X'r'xr'. In the GLUM, collinearity stems from X; in the mixed model,

collinearity might stem from X, from Z, or from both. The objective of this

dissertation is to explore the aspects of collinearity described in this chapter for

the GLUM in the context of the mixed model. Some research has been carried

out on influence diagnostics, residual plots and model adequacy for the mixed

model, though often in the context of ANOVA models. [See Louis (1988);

53

Beckman, Nachtsheim, and Cook (1987); and Christensen, Pearson and

Johnson (1992).] However, at this point, no previous work on the topic of

collinearity diagnostics for mixed models has been unearthed.

54

,

..

CHAPTER II

COLLINEARITY DIAGNOSTICS FOR THE MIXED MODEL:AN OVERVIEW

2. 1 Introduction

In this chapter, the parameters of this research are specified. These

include descriptions of: 1) the type of data to which the diagnostics are applied,

2) the specific nature of the mixed model being used, 3) the formal procedure

for collinearity assessment in the mixed model, and 4) methods for fitting the

mixed model and computing the collinearity indices. The complications arising

from the involvement of the matrix I also are addressed. Initial steps in the

process are illustrated by computing the GLUM and MIXMOD collinearity

diagnostics for a data set with some collinear variables. Finally, several factors

unique to the mixed model that may impact collinearity are discussed. This is

a prelude to Chapter 3 in which the behavior of the diagnostics, determined

empirically under a wider variety of conditions, will be reported.

2.2 Type of Data

Exploration of collinearity in the mixed model will be carried out for a

specific type of data: 1) The response vector, V, for each observation is a

sequence of measurements on a continuous scale. 2) The variables comprising

both the fixed effects design matrix, X, and the random effects design matrix,

Z, are also measurements on a continuous scale. 3) The response vector, V,

and the design matrices, X and Z, reflect assessments made on nk occasions

for the kth subject.

2.3 Specifics of the Mixed Model

For the analyses in this dissertation, specific aspects of the mixed model

will be featured; the conclusions reached apply to these circumstances.

Specifically, A is model~d linearly. The structure for Vk is Vk =I. Maximum

likelihood equations are used for parameter estimation.

2.4 The Diagnostic Measures

The mixed model was introduced in section 1.3 and the collinearity

diagnostics for the GLUM were discussed in section 1.2. Now diagnostic

measures, analogous to those used for the GLUM, are formulated for the fixed

effects of the mixed model.

Recall that the model equation for the mixed model is

•

V = X IJ + Z d + e;the estimate of Pis

A

and the estimate of the asymptotic variance of IJ is

(2.2.1)

(2.2.2)

(2.2.3)

A

where I is the estimated covariance matrix of the response vector, Y. When

56

..

the matrix (X'!:·'X) is ill conditioned, the estimates of IJ are likely to be unstable

and their variances inflated. This matrix is analogous to the matrix X'X in the

GLUM and is evaluated similarly for ill conditioning. Collinearity diagnostics for

the mixed model can be obtained through the spectral decomposition of

A

(X'i:"'X). Since E is involved, the measures must be computed after the final

estimation of E. Thus, for the mixed model, the degree of collinearity present

in the data is assessed after the estimation of model parameters. In contrast,

the assessment in the GLUM is done before estimation.

The process of computing the diagnostic measures for the mixed model

is similar to that described in section 1.2.2.2 for the GLUM. However, there

are differences due to the different nature of the models. These are described

in this section.

Eigenanalysis of(X~'X): In order to parallel the GLUM development, an

eigenanalysis is performed on

(2.2.4)

A

which is a scaled version of (X':!:"'X) that has 1's on the diagonal. Letting

(2.2.5)

then W'W is the matrix defined in 2.2.4. The spectral decomposition of

W'W = V1\2V' can be used to obtain its eigenvalues and eigenvectors, as was

done for previously for GLUM using X'X. Using the singular values of W, the

condition indexes can be obtained as described for the GLUM in section

1.2.2.2. The condition index (CI) is defined as the ratio of the largest singular

value to the jth singular value,

57

(2.2.6)

the condition number (CN) is defined as the ratio of the largest singular value

to the smallest singular value,

CN = A,IAp ' (2.2.7)

Variance Decomposition Proportions (VDPs): Similarly, with W as

defined (2.2.5) above, and using the spectral decomposition of W'W, the

A

estimated variance of IJ =(X't"xr' can be decomposed as

•

A A

V(JJ(p>cp,) =(W'(P>cN,W(N>cp,r' = V Cp >cp,I\-2Cp >cp,V'(p>cp" (2.2.8)

A

Then the variance decomposition of IJ can be obtained as described for the

GLUM in section 1.2.2.2. Specifically Vcp>cp,J\-2Cp>CPIV'(P>CPI can be rewritten as

lIA~ 0 0 V,

VJ\-2V' = [v, VII']0 11A~ 0 V2 (2.2.9) •V2 ...

0 0 VA; vII'A.

Then for the kth component of the scaled version of P,2

A (/J P vlcj (2.2.10)V .J = I:-.j.' A~1

The k,jth variance decomposition and the sum of the p components of the jth

decomposition are

2VIcj

(/)k" = ­I A~

1

k=1, ... ,p. (2.2.11)

Since (/)k is the variance of the kth regression coefficient, the variance

proportions are

k,j=1, ... ,p. (2.2.12)

"ik is the proportion of the variance of lik attributable to the collinearity

58

..

indicated by AtVariance Inflation Factors (VIFs): The definitions of MixMod regression

diagnostics are based upon the following idea: (1) Extend the definition of a

diagnostic from GLUM( Y ; X I, crlN) to GLUM( Y ; X I, crY), where V is

known. (2) Use the definition in MixMod(Y; X/J, J: = UZ' +crl), acting as if

J: = crV. This paradigm is not successful for the variance inflation factor. The

VIF definition can be extended from GLUM( Y ; X I, crlN) to the weighted least

squares case, i.e., GLUM( Y ; X I, crY). (That VIF will be called the w.l.s.

VIF.) The interpretation in either model is: VIFj is the ratio of the variance of

A

Pi in the model with p regressors to the variance of the corresponding estimator

in a model with only one regressor, Xi' The attempt to use the VIF from

GLUM( Y ; X P, crY) in the MixMod fails for the following reason. V is known

in GLUM( Y ; X I, crY) and when one moves from a model with p regressors

to a model with only one regressor, the elements of V do not change.

However, in a typical mixed model, the relationships between the columns of

X and Z usually imply that when one removes columns from X (as in going from

a model with p regressors to a model with one regressor), one usually removes

corresponding columns from Z as well. Removing columns from Z changes the

structure of J: as well as its estimate. Even if one were to use the w.l.s. VIF

in the mixed model, the different structure and estimate of J: would prevent the

VIF statistic from having the same interpretation, Le., VIFj would not be theA

ratio of the variance of Pi in the model with p regressors to the variance of the

corresponding estimator in a model with only one regressor, Xi' One could, of

course, compute the mixed model VIFj directly by fitting two models, one with

59

p regressors and one with only the i-th regressor, and compute the VIFi as theA

ratio of the variances of "Pi" from the two models. Although this might be

interesting, it requires substantially more computation than in the GLUM and

one must decide if this diagnostic statistic is worth the additional work.

Belsley (1991) attaches much less importance to the VIF than to the

other diagnostic statistics. Because of this statistic's lesser importance and the

lack of an efficient algorithm for its computation, the characteristics of this

statistic will not be examined in the mixed model setting.

2.5 Methods of Computation

Computations for both the analysis of the mixed model and for the

collinearity diagnostics were carried out using the SAS IML procedure. Macros

for the mixed model analysis using the EM algorithm were developed previously

by Fairclough and Helms (1984) and are provided in Appendix 1. Macros for

computing the diagnostics were developed as part of this research and are

provided in Appendix 2. The diagnostics were computed after the final iteration

of the EM algorithm.

2.6 Employing the Diagnostic Procedure

The basic procedure, as described in Table 1.2 in Chapter 1 for the

GLUM, is paralleled for the mixed model diagnostics. Conceptually, the steps

A

are as follows. First the matrix r 1/2x is determined and its columns scaled to

equal length, i.e., W = (I-1I2X) Diag(X't.-1X)-1/2. Then the condition indices

and variance decomposition proportions are obtained as described in section

60

2.4. These diagnostics are examined in order to determine 1) the number of

near dependencies, 2) which variables are involved in the dependencies, 3)

which auxiliary regressions should be performed, and 4) which variables are

unaffected by the collinearity.

The usual manner of displaying the collinearity diagnostics is in a "row-

oriented format" (Belsley 1991, p 137). The scaled condition indexes and the

variance decomposition proportions are combined into a matrix. The ordered

Cis constitute the first column; each row corresponds to a possible near

cfependency. The VDPs constitute the remaining columns; each is associated

with a column of X or the variance of its parameter estimate. The row-oriented

format is shown in Table 2.1. The objective of the analysis is to examine the

structure and patterning in the rows of the matrix.

Table 2.1 Row-oriented Format for Presentation of Collinearity Diagnostics

Scaled Proportions ofCondition

Index X, X2 X4

V(b, ) V(b 2) ... V(b4 )

CI 1 "11 "12...

"1p

CI 2 "21 "22...

"2p

Clp "p' "p2...

"pp

Adapted from Belsley 1991, p 138

The steps in the diagnostic procedure as outlined in Figure 2.1 are

described in this section.

2.6.1 Examine Condition Indexes

First, the number of near dependencies and their relative strengths are

61

Figure 2. 1 Employing the Diagnostic Procedure

1. Examine Scaled Condition Indices

IAbsolutely

Small?(5-10)

II

No problemsSTOP

I II I

Moderate? Large?(30-100) > 100

I 11 1

II

2. Look for Gap in 10130 Progression of CIsI1 --:-

1 IGap of first kind Gap of second kindbetween small and between large Cislarge Cis separated by magnitudes(1, 3, 5, 50) along 10/30 progression

I 1I 1I Look for competing1 dependencies1 11 .,......- 1

I1

3. Examine the Variance Decomposition ProportionsII

Look for strongest near dependency firstII

Look for next strongest dependency next1I

Look for dominance and competitionII

4. Determine Involved Variables1I

5. Perform Auxiliary Regressions, if NecessaryII

6. Determine Uninvolved Variables

Source: abstracted from Belsley (1991), PP 134-142.

62

..

determined by the scaled condition indexes exceeding a chosen threshold such

as 30. The highest CI indicates the worst dependency in the data. If it is

"absolutely" small « 10), then one can stop the procedure. If, however, it is

moderate (30-100) or large (> 100), then further examination is necessary. If

it is "immense," (> 1000), then smaller indexes that would be considered

moderate or large when considered by themselves are no longer of interest.

2.6.2 Look for Gaps in "10/30 Progression" of Cis

Next, the condition indexes are examined for gaps in their progression.

These aid in determining the number of near dependencies. The relative

strengths of the indexes are determined by their position along the "progression

of 10/30" (Belsley 1991, P 136). This means a progression of 1, 3, 10, 30,

100,300,1000 and so on. Belsley (1991, p 140) defines a "gap of the first

kind" as a separation of a small CI from one that is large; he defines a "gap of

the second kind" as separations between several large Cis by several orders of

magnitude along the "10/30" progression. Gaps of the first kind indicate that

one dependency is present; gaps of the second kind indicate that several

dependencies are present. Belsley (1991, P 141) notes that picking the number

of near dependencies is an art form.

2.6.3 Examine the Variance Decomposition Proportions

Next, the VDPs that correspond to the dependencies indicated by the Cis

are examined. A variable is involved if its VDP associated with the high CI

exceeds a threshold, such as 0.50. If there is only one dependency present,

there will be only one high condition index. Then it is possible to determine the

63

variables involved directly from the variance decomposition proportions. If there

are several high condition indexes, determination of the variables involved in

each is made by aggregating the VDPs of each variable over the set of these

high Cis. Those variables whose aggregate proportions exceed the threshold

(0.50) are involved in at least one of the near dependencies and their

corresponding estimated coefficients are degraded. A variable is considered

involved in a dependency, and its corresponding regression coefficient degraded

by, at least one near dependency if the total proportion of its variance

associated with the set of high scaled condition indexes exceeds a chosen

threshold such as 0.50 (Belsley 1991, P 136).

2.6.4 Determine Involved Variables

A systematic examination begins with the strongest dependency (largest

Cl), usually in the last row of a display. One typically looks for values as large

as 0.8 or 0.9. The columns in which these values are found indicate the

variables that are definitely involved in the strongest near dependency. Then

the VDPs associated with the next largest scaled CI are examined (next to last

row of a display). Here the simultaneous involvement of variables between this

dependency and the strongest dependency is indicated. If this is true, the

VDPs are distributed across the two so that their~ is large even if no single

part is. One continues in this manner until the total collinear structure is

determined.

If there are several competing near dependencies (those with equal Cis),

the determination of the involvement of a variable in each is not always clear.

64

-,

Perform Auxiliary Regressions

The VDPs of the variables involved in at least one of the competing

dependencies can be capriciously distributed among them, confounding the

assessment. However, the VDPs can indicate which variables are involved in

at least one dependency, and thereby have degraded coefficients, even though

they cannot determine which variables are involved specifically in which

dependency (Belsley 1991, p 136).

A dominating near dependency occurs when a CI is larger along the

"10/30" progression than others that exist with it (Belsley 1991, p 132). This

near dependency can be the chief determinant of the variance of the coefficient

of a variable, obscuring the simultaneous involvement of that variable in other

weaker dependencies. In this case, there may not be two or more high VDPs

associated with the CI of the dominated (weaker) near dependency, but the

sum across the set of high Cis still will be greater than 0.50 (Belsley 1991, P

133).

2.6.5

As described for competing and dominating dependencies, if several near

dependencies are present, it is not always possible to determine which variables

are involved in which dependencies from an examination of the Cis and VDPs

alone. Though it is possible to determine which variables are involved in at

least one dependency, the nature of the involvement may be obscured due to

competing and dominating dependencies (Belsley 1991, p 144). In this

situation, the specific nature of the involvement can be clarified by performing

auxiliary regressions among the variables determined to be involved when the

65

Cis and VDPs were examined (See Belsley 1991, pp 144-147). These are

carried out by choosing Qilll variable known to be involved in u.d1 near

dependency. Then each of these variables is used as a "dependent variable"

in regressions involving the remaining variables as independent variables. The

t-tests of parameter estimates equal to zero can be used descriptively to show

the involvement of the variables. The way to choose "dependent variables" is

to start with the strongest near dependency (Cn and look for the largest VDPs.

If several are very large and have about the same value, pick the one that has

the remainder of its variance associated with more removed (smaller)

dependencies (Cis). This is done to avoid picking variables that may have VDPs

distorted by competing near dependencies (Belsley 1991, P 145). Continue in

this manner through each of the other near dependencies until a set of variables

has been picked. Then regress each of these variables separately on the

remaining variables and examine the results. When there are many

dependencies or several competing or dominating dependencies, the choice of

variables for the regressions may not be clear. Belsley (1991 , P 146) describes

extensions of this procedure for these situations.

..

2.6.6 Determine Uninvolved Variables

A variable is not involved in any near dependency if the total proportion

of its variance associated with the set of low scaled condition indexes exceeds

the threshold (0.50) (Belsley 1991, P 137).

66

..

2.7 Illustration of the Collinearity Diagnostics

The purpose of this section is to demonstrate the usage of the diagnostic

measures and procedure. This is accomplished by applying the GLUM

diagnostics and the MIXMOD diagnostics to appropriate subsets of the same

data set and comparing them.

2.7.1 Description of the Data

The data used in this example are from a subset of subjects in a study

of pulmonary function in children. The study has been described extensively

by Strope and Helms (1984) and by Fairclough and Helms (1984). A brief

description is provided here. Black and white children were selected prior to

birth to parents who were permanent residents of the Chapel Hill NC area.

Shortly after birth, the children were enrolled in the Frank Porter Graham Child

Development Center of the University of North Carolina at Chapel Hill. Their

respiratory illnesses and physiological development were studied as part of a

longitudinal investigation of social and cultural effects on development.

Pulmonary function testing began as early as two and a half years of age after

each child passed criteria for producing reliable results. Children were

scheduled to be studied at three month intervals; additional measurements of

pulmonary function were made during and one month following acute upper

and lower respiratory illnesses. The assessments made included (Fairclough

and Helms (1984):

FVC Forced Vital Capacity

the volume (liters) of gas expired after full inspiration, and with

67

expiration performed as rapidly and completely as possible;

FEV, Forced Expiration Volume (1 sec)

the volume (liters of gas that is exhaled in the first second during

the execution of a forced vital capacity;

PEF Peak Expiratory Flow (liters/sec)

V_&0'" Maximum Expiratory Flow (liters/sec)

measured when 50% of the FVC has been expired;

VlIla71'" Maximum Expiratory Flow (liters/sec)

measured when 75% of the FVC has been expired; and

FEF26.76'" Forced Expiratory Flow (liters/sec)

the mean flow rate during the middle half of the FVC.

In addition to measurements of height and weight, demographic variables (age,

race, sex) and presence of respiratory symptoms were recorded.

In this example, observations for a subject at a given assessment were

excluded for the following reasons:

1) age, weight, or height measurements were missing;

2) FVC measurement was missing;

3) Vmu:50% was less than Vmu:75%;

4) total time to complete the function test was greater than 4

seconds;

5) symptoms of lower respiratory illness were present.

After exclusions, the data file contained 85 children with 1207 pulmonary

function studies. Of the 66 black children, there were 34 females and 32

males; of the 19 white children, there were 10 females and 9 males.

68

Since the objective of this chapter is to illustrate the use of collinearity

diagnostics rather than perform a complete analysis of the data, only black

females were used for this purpose. Table 2.2 characterizes the data for black

females.

Table 2.2. Descriptive Statistics for 34 Black Female Children with 527 TotalPulmonary Function Studies

MEASURE MEAN S.D. MIN MAX

Number of Studies 16 8 1 30

Average Age (years)' 5.9 1.8 4.0 11.5Age at First Study 4.1 1.4 2.4 8.4Age at Last Study 8.3 3.0 4.0 15.6

Average Height (cm)' 116 14 99 151Height at First Study 103 12 88 131Height at Last Study 131 20 103 170

Average Weight (kg)' 25 11 15 63Weight at First Study 18 6 12 41Weight at Last Study 37 22 17 120

, Mean of individual means

2.7.2 GLUM Example and Diagnostics

For the GLUM analysis, the !M1 measurement of FVC is the response

variable and the three continuous variables, age, height, and weight at the last

measurement of FVC, are continuous regressors. The mean and standard

deviation of FVC at the last study for the 34 black females are 1.72 and 0.814,

respectively. The minimum and maximum values are 0.63 and 3.7,

respectively. Correlation among the regressors is given in Table 2.3 below.

The large correlation coefficients indicate that these three regressor variables

69

may be pairwise collinear. However, the diagnostics must be examined to

determine the degree of collinearity present and the extent of damage to

regression coefficients that might be caused by collinearity in these data.

Table 2.3 Correlation Coefficients of GLUMRegressors

AGEHEIGHTWEIGHT

AGE HEIGHT1.00 0.95

1.00

WEIGHT0.820.881.00

It is not necessary to fit a model first in order to assess collinearity in the

GLUM. However, for completeness and to compare with the MIXMOD results,

the following model is fit to these data:

y =X/J + e

where

(2.2.13)

measurement of FVC at the~ study for that subject;

design matrix of fixed effects: intercept and age, height, weight

at the last study;

vector of primary parameters;

vector of unobservable errors.

The results of fitting this model and the collinearity diagnostics are

presented in Table 2.4. Since the last measurement of FVC is the response

variable, the effects of age, height and weight are cross-sectional in nature.

A statistically significant impact on the prediction of FVC was found for both

age (p=0.015) and height (p=0.004), but not for weight (p=0.397). The

slopes for these two variables were positive, indicating that FVC increases with

70

Table 2.4 FVC: GLUM Results and Collinearity Diagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept -2.126 0.614 -3.46 0.002Age 0.101 0.039 2.59 0.015Height 0.022 0.007 3.09 0.004Weight 0.003 0.003 0.86 0.397

Covariance Matrix of Parameter Estimates of Fixed EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

0.377 0.018 -0.0041 0.0010.002 -0.0002 9.608E-6

0.00005 -0.000010.00001

(X'X)

A

fil = 0.04534 281

26194465

38705

598989

124112074

17559761735

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.814 1.95 1 0.000 0.001 0.000 0.0040.160 0.40 5 0.008 0.001 0.001 0.2150.025 0.16 12 0.019 0.305 0.000 0.4980.001 0.03 62 0.973 0.693 0.999 0.283

Correlation Matrix of Parameter Estimates of Fixed EffectsIntercept Age Height Weight VIF

Intercept 1.00 0.74 -0.98 0.58Age 1.00 -0.84 0.07 10Height 1.00 -0.54 14Weight 1.00 4

71

increasing age and height.

Using the procedure described in section 2.6 above, collinearity in these

data is assessed. There is one clear linear dependency and it is moderately

strong. This is indicated both by the one moderately large condition index, 62,

and by a gap of the first kind between 62 and 12, the next largest CI. The

values of the elements in the last row of the VDP matrix indicate that the

variables involved in the dependency are the intercept, age, and height; each

exceeds the threshold value of 0.50.

Although the value of the second largest CI, 12, is relatively small, it

may indicate a second weaker dependency; there is a smaller gap in the

progression between the Cis of 12 and 5. This situation here may be that of

a dominated dependency as was described in section 2.6. The VDP for weight

in the row associated with the second largest CI, 12, is 0.498 (- 0.50). And,

the proportion of the variance of weight associated with the set of Cis (62 and

12) is 0.781. Thus, the coefficients of all three independent variables and

the intercept may be degraded.

The exact nature of the relationships among the variables can be

determined through auxiliary regressions. For this example, height has the

largest VDP in the last row; it is chosen as a "dependent" variable associated

with the first dependency. Weight is chosen as the primary variable involved

in the second dependency. Thus, the auxiliary regressions feature height and

weight separately regressed on the intercept and age. The results are displayed

in Table 2.5.

From these results, we verify that the dominant relationship does involve

72

Table 2.5 GLUM Auxiliary Regressions for Pulmonary Data

Coefficients ofDependentVariablesHeight

Weight

Intercept

80.72«0.001 )

-13.42(0.048)

Age

6.13«0.001 )

6.05«0.001 )

0.90

0.67

CI

62

12

Numbers in parentheses are p-values for t-statistics.

the intercept, age and height. The weaker involves at least age and weight.

Thus, we conclude that each variable is involved in one or both of these near

dependencies and the coefficient of each is degraded to some degree.

The primary focus in this section is to illustrate the detection of

collinearity and the identification of collinear variables, rather than the

resolution of collinearity. However, at this point, one or more of the techniques

described in Chapter 1 might be employed to deal with the collinearity found

in these data. Specifically, one or more of the variables might be eliminated

from the model. This suggestion is made in spite of the cautions made by

several investigators because the relationships among the variables in these

data are easily known and understa.ndable.

Several measures that may aid in determining which variable(s) to delete

are considered. The first is the correlation of each variable with FVC. The

correlations are 0.95 for age, 0.96 for height and 0.86 for weight. The second

measure is a plot of FVC against each variable. These are provided in Figures

2.2, 2.3 and 2.4 for age, height and weight, respectively. The correlation of

73

age with height is 0.95 and are redundant variables. The physical reality is that

FVC is logically more related to height than to age. Thus, the current model

could be reduced to one containing only height and weight as independent

variables. The reduced model would be fit and the diagnostic procedure

repeated.

Pulmonary Function Study....~------------------,

..

to

+..rvc-

,......

f '+ • • •.,to. +. ...

..••

t J ••• 7' '''''''11''''''.InYean

Figure 2.2 Values of FVC Plotted Against Age

74

Pulmonary Function Study

• •

... •++

rvCUi

'M••

••... + +* +.+.+ f'

+.+

+ •+ ~

•

• • ... 1M ,. ,. ,. 1. ,. '70

Height .... 11m

Figure 2.3 Values of FVC Plotted against Height

Pulmonary Function Study

• +

•••

rvCUi ...

•• +;,.t. ..... t

'M

II • • • • • ,. • • 'Ill Ii' 'JO

Weight In kg

Figure 2.4 Values of FVC Plotted Against Weight

75

2.7.3 Mixed Model Examples and Diagnostics

In the MIXMOD analysis, illl measurements of FVC for each black female

constitute the response vector and the three continuous variables, age, height,

and weight at every measurement of FVC, are continuous regressors. The

number of pulmonary studies for each subject are given in Table 2.6 below.

Table 2.6 Number of Pulmonary Studies for Each Subject

SUBJECT NUMBER SUBJECT NUMBEROF OF

STUDIES STUDIES113 1 94 15

99 2 68 1629 6 70 1644 7 64 18

106 7 69 18110 8 75 1892 9 40 1993 9 81 21

101 9 43 2298 10 59 2276 11 19 2589 11 21 2677 12 18 2783 12 32 2753 13 39 2771 13 52 2782 13 28 30

2.7.3.1 A Simple Mixed Model

Prior to fitting the mixed model counterpart of the GLUM example just

described, a simple mixed model was fit to these data. The purpose was to

demonstrate the mixed model analysis and to interpret results for a situation in

which there was no collinearity. In contrast to a GLUM model, the mixed

model accounts for the correlation among observations for each subject. In this

case, FVC is the dependent variable and age is the independent variable. The

76

following model was fit:

v = X IJ + Z d + 8.

The model equation for the kth subject is

Vk = XklJ + Zk dk + 8k, k= 1, 2, ..., 34,

(2.2.14)

(2.2.15)

where

Vk (1\ x 1) is a vector of the nk observations of FVC for the kth black female,

k = 1,2, ..., 34;

V (527 xl) = [V, 1/ V2 // ... 1/ VK] is the vector of responses from all 34 black

females;

Xk (n k x2) is a known fixed effects design matrix for the kth black female:

the intercept and values of age at every study;

X (527 x 2) = [X, // X2 // ... / / Xd is the fixed effects design matrix for the

model;

Zk (nk x 2) is a known random effects design matrix for the kth black female:

the intercept and values of age at every study;

Z (527 x 68) = Diag(Z" Z2' ..., ZK) is the random effects design matrix for the

model.

The following assumptions are made:

IJ (2 xl) is a vector of fixed effect primary parameters;

dk (2 xl) is a random vector of unobservable random subject effects for the

kth black female;

d (68 x ,) = [d, // d2 // ... // dK] is a vector of random subject effects for the

model;

is an vector of unobservable within-subject random error terms;

77

8 (527 x 11

and

= [8, II 82 /I ... /I 8K] is a vector of random error terms for the

model.

The following additional assumptions are made:

4 (2x21

- NIDq (0, 4);

= V(dk) is the covariance matrix of the random effects (each dk

has the same covariance matrix);

is an unknown scalar within-subject error variance parameter;

= V(8k) is the covariance matrix of the random deviations about

the kth subject's random regression line.

is the V(Yk) = I.t = Zk 4 Zk' + fi2vk, a positive definite symmetric

covariance matrix for the kth black female. Here, we assume that

~ (527 x 5271 = Diag(~" ~2' "., ~) is the covariance matrix of the entire

response vector, Y.

The results of fitting this model are presented in Table 2.7 and

graphically, in Figure 2.5. Here, the fixed effects have the same interpretation

A

as do those for the GLUM. FVC increases significantly with i'lcreasing age (P2

= +0.219, p<0.001). For the mixed model, the random effects also are of

interest. The random effects indicate deviations of each subject's regression

from the estimated population regression. Hence, a subject's first random

78

Table 2.7 A Simple Mixed Model: FVC as a Function of Age

Parameter Estimates of Fixed EffectsVARIABLEInterceptAge

BETA STD ERR T-0.153 0.056 -2.730.219 0.010 21.90

P-VALUE0.007

<0.001A

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age

InterceptAge

0.0725 -0.00970.0020

Estimate of Correlation Matrix of the Random EffectsIntercept Age

InterceptAge

1.00 -0.811.00

A

02=0.020

Eigen­value

1214

Collinearity DiagnosticsSingular Condition Variance

Value Index DecompositionProportions

600040054

1.8600.140

, .360.37

14

79

INT AGE0.070 0.0700.930 0.930

Pulmonary Function Study

2 3 4 5 • 7 a 9 10 11 12 13 14 15 ,.

Age In Years

Figure 2.5 Values of FVC Predicted from Mixed Model with Age in X and in Z

80

...

..

..

coefficient is that subjects's deviation from the estimated population intercept;

the subject's second random coefficient is that subject's deviation from the

population slope with respect to age. The covariance matrix of theseA.

deviations is estimated by 4. The correlation matrix of the random effects, theA. A.

scaled version of 4, is often easier to interpret than 4. Here, the correlation

between the two random effects (the intercept and age) is -0.81. This means

that, overall, subjects with lower random intercept increments tend to have

higher slope random slope increments, and vice versa. This tendency can be

seen in Figure 2.5. The magnitude of this coefficient indicates that age and the

intercept are terms worthy of being retained as "significant effects" in the

random component of the model.

In addition to the modeling results, the collinearity diagnostics are

presented in Table 2.7. The highest condition index is 4, which in the GLUM

is "absolutely small" and indicates a lack of collinearity. Assuming that the

same is true for the mixed model, we can conclude that there is no collinearity

present. Both the intercept and age have variance decomposition proportions

of 0.930 which in the presence of a large CI would indicate collinearity. In this

case, however, collinearity is not indicated.

2.7.3.2 GLUM for Longitudinal Data

..Prior to fitting the mixed model, an ordinary least squares regression

model (GLUM) was fit to these data, ignoring the correlation among

observations for each subject, i.e., there are 527 observations from 34

subjects. This was done in order to have rough estimates of fixed effects and

81

GLUM collinearity diagnostics to compare with their mixed model counterparts.

The dependent variable is FVC; the independent variables are age, height and

weight. The trends in these GLUM results, shown in Table 2.8, are similar to

those found for the GLUM fit at the last measurement of FVC (Table 2.4).

Now, however, age and weight appear to be important factors in predicting

FVC, while height appears not to be a significant factor. The slopes are still

positive, but steeper for age and weight than those found at the last

measurement of FVC. Moreover, because the observations are correlated, the

p-values in the table are not meaningful.

In this example that ignores the correlation among the observations, the

patterns in the collinearity diagnostics are quite similar to those found for the

last measurement of FVC. As before, there is one moderately strong linear

dependency and perhaps, a weaker dependency. There is one gap of the first

kind between the two largest Cis, 68 and 14; there is a smaller gap between

the Cis of 14 and 5. The last row of the VDP matrix shows that again the

intercept, age, and height are involved in the strongest dependency. The total

proportion of the variance of weight associated with the set of largest Cis is

now 0.887, suggesting that the dependencies may involve all variables in a

manner similar to that found for the GLUM analysis.

82

..

Table 2.8 FVC: GLUM Results and Collinearity Diagnostics for 527Observations on 34 Subjects

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept -0.411 0.177 -2.32 0.021Age 0.152 0.014 11.07 <0.001Height 0.003 0.002 1.44 0.150Weight 0.012 0.001 7.89 <0.001

Covariance Matrix of Parameter Estimates of Fixed EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

0.031 0.002 -0.0004 0.00010.0002 -0.00003 8.4E-7

5.1E-6 -1.8E-62.2E-6

(X'X)

A

02 = 0.049527 3453

2650163820

445035

7927858

15298120527

2004029581948

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.830 1.96 1 0.000 0.001 0.000 0.0020.148 0.38 5 0.009 0.005 0.001 0.1110.021 0.14 14 0.009 0.286 0.000 0.6180.0008 0.03 68 0.982 0.708 0.999 0.269

Correlation Matrix of Parameter Estimates of Fixed EffectsIntercept Age Height Weight VIF

Intercept 1.00 0.78 -0.98 0.56Age 1.00 -0.84 0.04 15Height 1.00 -0.53 21Weight 1.00 6

83

2.7.3.3 Mixed Model with Multiple Independent Variables

(2.2.17)

After the preliminary analysis, a mixed model was fit to the pulmonary

function data, accounting for the correlation among observations for each

subject. The mixed model collinearity diagnostics were computed as described

in section 2.4. The following model was fit:

Y = X /l + Z d + e. (2.2.16)

The model equation for the kth subject is

Yk = Xk /l + Zk dk + ek , k =1, 2, ..., 34,

where

is a vector of the nk observations of FVC for the kth black female,

k = 1,2, ..., 34;

Y (527 x ') = [Y, II Y2 II ... II YK] is the vector of responses from all 34 black

females;

Xk (n.x4) is a known fixed effects design matrix for the kth black female:

the intercept and values of age, height, weight at every study;

X (527 x 4) = [X, II X2 II ... II XK] is the fixed effects design matrix for the

model;

Zk (nk x4) is a known random effects design matrix for the kth black female:

the intercept and values of age, height, weight at every study;

Z (527 x, 36) = Diag(Z" Z2, ..., ZK) is the random effects design matrix for the

model.

The following assumptions are made:

/l (4 x ') is a vector of fixed effect primary parameters;

84

..

dk (4 x 11 is a random vector of unobservable random subject effects for the

kth black female;

d (136 x 11 = [d, II d2 11·.. /1 dK] is a vector of random subject effects for the

model;

8 k (1'\ x 11 is an vector of unobservable within-subject random error terms;

and

8 (527 x 11 = [8, /I 82 II ... /I 8K] is a vector of random error terms for the

model.

The following additional assumptions are made:

dk (4x1) - NIDq (0, 4);

4 (4x4) = V(dk) is the covariance matrix of the random effects (each dk

has the same covariance matrix);

8 k (nk x 1) - NIDN (0, o2Vk ) independent of the dk; and for k' ¢ k,

Cov(dk·,dk) = 0; Cov(dk.,8k) = 0; and COV(8k.,8k) = 0;

02 is an unknown scalar within-subject error variance parameter;

Vk (nkxnk) = V(8k ) is the covariance matrix of the random deviations about

the kth subject's random regression line.

I k (nkxnk) is the V(Yk) = ~ = ~ 4 Zk' + o2Vk, a positive definite symmetric

covariance matrix for the kth black female. Here, we assume that

Vk=I...

I (527 x527) = Diag(I" 1 2, ... , I K) is the covariance matrix of the entire

response vector, Y.

The results of fitting this model are presented in Table 2.9. The fixed

effect results indicate that age and weight are significant predictors of FVC;

85

Table 2.9 FVC: MIXMOD Results and Collinearity Diagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUE

Intercept -0.127 0.359 -0.35 0.724Age 0.153 0.030 5.10 <0.001Height 0.00002 0.005 0.00 0.997Weight 0.016 0.004 4.00 <0.001

A-

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

2.3081 0.1801 -0.0313 0.00970.0154 -0.0024 0.0004

0.0004 -0.00010.0002

Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

1.00 0.96 -0.99 0.491.00 -0.95 0.27

1.00 -0.541.00

A-

02=0.0193903 16799

96970408783

1928606

44029265

72718395852

82161441712904

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.795 1.95 1 0.000 0.001 0.000 0.0020.176 0.42 5 0.005 0.020 0.000 0.0340.027 0.17 12 0.000 0.169 0.000 0.7550.006 0.02 78 0.995 0.810 0.999 0.209

86

height is not significant. The slopes of both are positive indicating that FVC

increases with increasing age and weight. These results are presented

graphically in Figures 2.6, 2.7 and 2.8 for age, height, and weight,

respectively. The population regression line was obtained using the fixed

A

effects (/1) and minimum and maximum values (over all studies) of age, height

and weight. The predicted values of FVC for each subject were obtained using

A

the fixed effects (/1) and the random effects (dk); the values of FVC for each

independent variable were computed at the mean (over all studies) of the other

two variables.

Portions of the correlation matrix of the random effects, and the Figures,

aid in interpreting the results. In contrast to the results for the simple model

containing only age in X and in Z, the correlation between the intercept and age

in this model is high, but positive (0.96). This indicates that subjects with

positive intercept increments also have higher than average random slopes and

vice versa. The correlation between the intercept and height is high and

negative (-0.99), indicating that subjects with negative intercept increments

tend to have higher than average random slopes and vice versa. The

correlation between the intercept and weight is positive (0.49), but not as

strong as the correlations for the intercept and age and the intercept and

height.

The other components of the correlation matrix of the random effects are

noteworthy. The correlation of age with height (-0.95) indicates that positive

deviations of the slopes for age tend to be associated with very negative

deviations of the slopes for height. However, positive deviations of the slopes

87

Pulmonary Function study5r-----------------------~

2 3 4 5 I 7 a I 10 11 11 13 14 15 1&

Age In Years

Figure 2.6 Values of FVC Predicted from Mixed Model with Age, Heightand Weight in X and in Z, Plotted Against Age

88

Pulmonary Function Study

10 100 110 1%0 130 140 150 180 170

Height in em

Figure 2.7 Values of FVC Predicted from Mixed Model with Age, Heightand Weight in X and in Z, Plotted against Height

89

Pulmonary Function Study

10 20 30 40 50 eo 70 80 80 100 110 120

Weight in kg

Figure 2.8 Values of FVC Predicted from Mixed Model with Age, Heightand Weight in X and in Z, Plotted Against Weight

90

..

..

for age tend to be associated with very little change in the random deviations

of the slopes for weight (0.27). The deviations of the slopes of height and

weight tend to vary inversely and moderately (-0.54). It is suspected that the

collinearity in these data may be affecting the estimation of 4. This will be

explored later.

The collinearity diagnostics for this model also are presented in Table

2.9. There is one clear near dependency and it is moderately strong, as

indicated by the one moderately large condition index, 78, and the gap of the

first kind between 78 and 12, the next largest CI. The values in the last row

of the VDP matrix indicate that the variables involved in the dependency are the

intercept, age, and height; each exceeds the threshold of 0.50.

There is also indication of a second weaker dependency. There is a gap

in the progression of Cis between 12 and 5. In addition, the total proportion

of the variance of weight associated with the set of Cis (78 and 12) is 0.964.

Thus the fixed effect coefficients of all three independent variables and the

intercept may be degraded. These results are similar to those found for the

GLUMs fit previously for last measurement of FVC (Table 2.4) and for all

measurements of FVC (Table 2.6). Thus, for analogous GLUM and MIXMOD

analyses, the patterning in the diagnostics is similar and the diagnostics appear

to function comparably.

In order to determine the exact nature of the relationships among these

variables, a mixed model counterpart of the auxiliary regressions could be

carried out. However, at this point the type of weighting that should be used

is not clear, Le., whether the regression should be computed from X or from

91

2.8 Factors Impacting Collinearity in the Mixed Model

As mentioned in section 2.4, collinearity in the mixed model is affected

by several factors other than collinear variables in the X matrix. The objective

of this section is to get a glimpse of what those factors might be so that they

can be addressed in Chapter 3. Both the GLUM and the mixed model involve

assessment of collinearity in the X matrix, even though the structure of these

matrices in the two models differs. However, for the mixed model, the

additional matrix I is involved in the assessment. Thus, the factors that might

impact collinearity in the mixed model all involve the structure and modeling of

I. The following components of the estimation of I are thought to be

important:

1) the number of variables in Z,

2) the presence and nature of the collinearity in Z,

3) the structure of A,

4) the structure of V, and

5) the value of cr.

In addition to these factors, it is thought that collinearity may have a different

impact for different response variables, even if they retain the same fixed and

random effects design matrices. This is due to the fact that I, the variance of

the response vector V, will be different for models with different responses.

This is in contrast to the GLUM in which collinearity for a given X matrix is the

92

..

same regardless of the response variable Y.

Three of the issues thought to impact collinearity in the mixed model are

explored in this section:

1) the number of variables in Z, of those thought to be collinear in X,

2) the structure of 4, and

3) the effect of a different response for the same X and Z matrices.

2.8.1 Number and Nature of Variables in Z

To explore the impact of the Z matrix on collinearity, the model described

in section 2.7.3.3 was refit with different subsets of variables in the Z matrix.

The results relative to the original model (See results in Table 2.9.) are reported

in 'this section.

2.8.1.1 Two variables, Pair-wise collinear

Three models were fit in which the Z matrix contained only two of the

three variables contained in the X matrix. In turn, each of the variables age,

height, and weight, were deleted from Z and the model was refit. The results

of fitting these mixed models are summarized in Table 2.10 (height and weight

in Z), Table 2.11 (age and weight in Z) and Table 2.12 (age and height in Z).

2.8.1.2 One Variable

Three models were fit in which the Z matrix contained only one of the

three variables contained in the X matrix. In turn, pairs of variables (height and

weight, age and weight, age and height) were deleted from Z and the model

was refit. The results of fitting these mixed models are summarized in Table

2.13 (age in Z), Table 2.14 (height in Z) and Table 2.15 (weight in Z).

93

2.8.1.3 Summary

The results of fitting the models with altered Z matrices are summarized

in Table 2.16. When only one variable was deleted from Z, the most dramatic

A

changes in fixed effects and in 4 occurred when age or height was deleted;

there was no substantial change when weight was deleted. When age was

deleted from Z, the diagnostics indicated that age was less involved in the

dependencies. The greatest change in the diagnostics occurred when height

was deleted from Z; the largest CI was reduced from 78 to 38 and age and

weight were less involved in the fixed effect dependencies.

In each model in which two variables were deleted from Z, the fixed

effect for height changed; the slope was negative and the p-value was smaller.

When the pairs (height and weight) and (height and age) were deleted from Z,

the magnitude of the correlation of the intercept and the remaining random

effect changed in magnitude and sign. In all cases of deleting two variables,

the Cis were reduced to 33-38 and age and weight were less involved in the

dependencies.

94

•

Table 2.10 FVC (Z = Int,Ht,Wt): MIXMOD Results and Collinearity Diagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.321 0.256 1.25 0.210Age 0.198 0.019 10.42 <0.001Height -0.006 0.004 -1.50 0.134Weight 0.017 0.004 4.25 <0.001

A

Ii, Estimate of Covariance Matrix of the Random EffectsIntercept Height Weight

InterceptHeightWeight

0.3123 -0.0050 0.00850.0001 -0.0001

0.0003

Estimate of Correlation Matrix of the Random EffectsIntercept Height Weight

InterceptHeightWeight

1.00 -0.99 -0.921.00 -0.96

1.00

A

02=0.0203076 13065

84756321369

1560357

35015410

61861365185

72165101692195

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.728 1.93 1 0.000 0.002 0.000 0.0020.239 0.49 4 0.009 0.045 0.001 0.0180.031 0.17 11 0.001 0.414 0.000 0.5990.002 0.04 50 0.989 0.539 0.999 0.380

95

Table 2.11 FVC (Z = Int,Age,Wt): MIXMOD Results and CollinearityDiagnostics

Parameter Estimates of Fixed Effects

VARIABLE BETA STD ERR T P-VALUEIntercept 0.370 0.257 1.44 0.151Age 0.198 0.022 9.00 <0.001Height -0.007 0.003 -2.33 0.020Weight 0.017 0.004 4.25 <0.001

A

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Weight

InterceptAgeWeight

0.0309 0.0021 -0.00110.0014 -0.0004

0.0001

Estimate of Correlation Matrix of the Random EffectsIntercept Age Weight

InterceptAgeWeight

1.00 0.31 -0.581.00 -0.89

1.00

A

a2=0.0201781 9133

72504199376

1192671

23552068

39136288969

49968261244657

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.705 1.92 1 0.001 0.002 0.000 0.0040.255 0.51 4 0.016 0.030 0.001 0.0410.038 0.19 10 0.000 0.308 0.002 0.8710.003 0.05 38 0.984 0.660 0.997 0.084

96

Table 2.12 FVC (Z = Int.Age.Ht): MIXMOD Results and Collinearity Diagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept -0.179 0.339 -0.53 0.598Age 0.158 0.035 4.51 <0.001Height 0.0007 0.005 0.14 0.889Weight 0.014 0.002 7.00 <0.001

A-

4. Estimate of Covariance Matrix of the Random EffectsIntercept Age Height

InterceptAgeHeight

2.0758 0.2171 -0.02890.0250 -0.0031

0.0004

Estimate of Correlation Matrix of the Random EffectsIntercept Age Height

InterceptAgeHeight

1.00 0.95 -1.001.00 -0.97

1.00

A

02=0.0204368 17613

84832447863

1907353

46714919

69519340746

75841751582084

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.758 1.94 1 0.000 0.001 0.000 0.0080.194 0.44 4 0.003 0.002 0.001 0.3690.047 0.22 9 0.005 0.133 0.000 0.6100.0006 0.03 77 0.991 0.865 0.999 0.013

97

Table 2.13 FVC (Z=lnt,Age): MIXMOD Results and Collinearity Diagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.016 0.262 0.06 0.951Age 0.186 0.023 8.09 <0.001Height -0.002 0.003 -0.67 0.505Weight 0.011 0.003 3.67 <0.001

A.

Ii, Estimate of Covariance Matrix of the Random EffectsIntercept Age

InterceptAge

0.0634 -0.00840.0016

Estimate of Correlation Matrix of the Random EffectsIntercept Age

InterceptAge

1.00 -0.841.00

A.

a2=0.0201604 8395

56393180034

1026087

20870058

33564228471

41652561096723

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.703 1.92 1 0.001 0.002 0.000 0.0090.233 0.48 4 0.014 0.007 0.002 0.2960.062 0.25 8 0.014 0.336 0.000 0.6030.003 0.05 36 0.971 0.655 0.998 0.092

98

Table 2.14 FVC (Z =Int.Ht): MIXMOD Results and Collinearity Diagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.173 0.277 0.62 0.533Age 0.204 0.021 9.67 <0.001Height -0.004 0.003 -1.33 0.183Weight 0.011 0.002 5.50 <0.001

A

A. Estimate of Covariance Matrix of the Random EffectsIntercept Height

InterceptHeight

0.4464 -0.00370.00003

Estimate of Correlation Matrix of the Random EffectsIntercept Height

InterceptHeight

1.00 -0.981.00

A

02=0.0201352 7268

54040152949910413

17918891

28388212641

36002141011044

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.643 1.91 1 0.001 0.003 0.000 0.0100.280 0.53 4 0.013 0.011 0.002 0.2600.074 0.27 7 0.009 0.345 0.000 0.6680.003 0.06 33 0.978 0.640 0.998 0.062

99

Table 2.15 FVC (Z=lnt,Wt): MIXMOD Results and Collinearity Diagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.303 0.253 1.20 0.232Age 0.191 0.020 9.55 <0.001Height -0.006 0.003 -2.00 0.046Weight 0.017 0.003 5.67 <0.001

A.

4, Estimate of Covariance Matrix of the Random EffectsIntercept Weight

InterceptWeight

0.0417 -0.00100.00004

Estimate of Correlation Matrix of the Random EffectsIntercept Weight

InterceptWeight

1.00 -0.781.00

A.

a%=0.0201741 9674

78005198969

1261025

23845622

38560288096

49222081183372

..

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.729 1.93 1 0.001 0.002 0.000 0.0050.217 0.47 4 0.020 0.038 0.001 0.0650.052 0.23 8 0.000 0.255 0.001 0.8040.003 0.05 38 0.979 0.705 0.998 0.126

100

Table 2.16 Summary of Results of Altering Z Matrix by Deleting Variables fromFVC Mixed Model· With All Three Variables in Z

Variables Impact onAlteration in Z Rxed Impa,t on Impact on

of Z (Table #) Effects 4 DiagnosticsIntercept Change in Cis: largest is 50, others are

Height and height magnitude the sameWeight have smaller and sign for

p-values, (int,wt); VDPs: age is less involved in(2.10) slope of change in dominant dependency

height is magnitudenegative for (ht,wt)Height is Dramatic Cis: largest is 38, others are

One Age significant change in the sameVariable Weight and slope is patterns, allDeleted negative correlations VDPs: age and weight are less

I(2.11 ) change in strongly involved in the

sign andlor dependenciesmagnitude

No change No change Cis: almost no changeAge in patterns in patterns

Height VDPs: age is more stronglyinvolved in the

(2.12) dominant dependency;weight is less stronglyinvolved in thedependencies

No change Magnitude Cis: largest is 36, others areAge in patterns, of (int,age) slightly smaller

but height is similar,(2.13) has smaller but sign is VDPs: age and weight are less

p-value and reversed strongly involved in theslope is dependenciesnegativeNo change No change Cis: largest is 33

Two Height in patterns, inVariables but height magnitude VDPs: age and weight are lessDeleted (2.14) has smaller or sign strongly involved in the

p-value and dependenciesslope isnegativeIntercept Magnitude Cis: largest is 38

Weight has smaller and sign ofp-value; (int,wt) VDPs: age is slightly less

(2.15) height is have strongly involved in thesignificant changed dependencyand slope isnegative

• Results from original model are in Table 2.9.

101

2.8.2

2.8.2.1

Structure of A

Constrained, One Off Diagonal Element Equal to Zero

To determine the effect of the matrix 4 on collinearity, one off diagonal

element of 4 was set to zero. The element was in the 2,4 [cov(age,weight)]

position. The results of fitting the original model, described in section 2.7.3,

with this change in 4 are reported in Table 2.17.

2.8.2.2 Constrained, All Off Diagonal Elements Equal to Zero

To further determine the effect of the matrix 4 on collinearity, all off

diagonal elements of 4 were set to zero. The results of fitting the original

model, described in section 2.7.3, with this change in 4 are reported in Table

2.18.

2.8.2.3 Summary ..A

The results of fitting the models with constrained 4 matrices areA

summarized in Table 2.19. When only one off diagonal element of 4 was set

to zero, the fixed effect for height changed, relative to the original results, from

highly non-significant to significant and the slope was negative. The patterns

A

among the remaining elements of 4 were similar to the original model, but

smaller in magnitude. A similar change in the fixed effects occurred when all

A A

diagonal elements of4 were set to zero. In both cases of constraining 4, the

diagnostics change dramatically. The largest CI was reduced from 78 to 23-24

and age and weight were less involved in the dependencies.

102

Table 2.17 FVC(One element 4 = 0): MIXMOD Results and CollinearityDiagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUE

Intercept 1.026 0.812 1.26 0.207Age 0.266 0.083 3.20 0.001Height -0.020 0.011 -1.82 0.070Weight 0.030 0.014 2.14 0.033

A

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

12.8782 1.1571 -0.1758 0.03300.1366 -0.0156 0.0000

0.0024 -0.00070.0036

Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

1.00 0.87 -0.99 0.151.00 -0.86 0.00

1.00 -0.241.00

A

02=0.020140 19

89010513

7578

840237

8031922

7423813557

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT2.661 1.63 1 0.000 0.005 0.001 0.0391.073 1.04 2 0.002 0.101 0.001 0.0260.261 0.51 3 0.002 0.141 0.003 0.9090.005 0.07 23 0.995 0.752 0.995 0.025

103

Table 2.18 FVC(AII off diag 4 = 0): MIXMOD Results and CollinearityDiagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.433 0.259 1.67 0.095Age 0.214 0.023 9.30 <0.001Height -0.007 0.003 -2.33 0.020Weight 0.014 0.003 4.67 <0.001

A

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

0.0206 0.0000 0.0000 0.00000.0001 0.0000 0.0000

3.7E-7 0.00003.0E-5

Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

1.00 0.00 0.00 0.001.00 0.00 0.00

1.00 0.001.00

A

a2=0.020945 2945

3374392358

446145

10139655

10896115121

1605670518648

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.154 1.78 1 0.001 0.005 0.001 0.0180.709 0.84 2 0.009 0.016 0.001 0.0890.132 0.36 5 0.002 0.203 0.001 0.8770.005 0.07 24 0.987 0.776 0.997 0.016

104

..

Table 2.19 Summary of Results of Altering Z MaVix by Constraining ACompared to FVC Mixed Model- With Unconstrained A

Elements Impact on5etto Impact on Remaining

Constraint Zero Fixed ElemlV'ts of Impact ononA (Table ') Effects A Diagnostics

Height Similar2,4 changes pattern, but Cis: dramatically reduced,

from highly smaller largest is 23(2.17) non- values for all

significant VDPs: age is slightly lessto highly involved in the

Off- significant dependencydiagonal and slope isElements negativeSet to InterceptZero All approaches Not Cis: dramatically reduced,

significance; applicable largest is 24(2.18) height (all set to

changes zero) VDPs: age and weight arefrom highly slightly less stronglynon- involved in thesignificant dependenciestosignificantand slope isnegative

• Results from original model are in Table 2.9.

2.8.3 Different Response, Same Fixed and Random Effects

In order to obtain an indication of what the impact of a different

response might be, a second mixed model was fit using the same independent

variables, in both X and Z, with a different dependent variable. For this model,

the response VMAX50% was fit using the same regressors, age, height and

weight, as were used for fitting the model for the response FVC. The

specifications are the same as those given in section 2.7.3, with VMAX50%

substituted for FVC in the description. The results of fitting this model are

presented in Table 2.20.

These results indicate that even though the same X and Z matrices were

105

used in this model as were used to model FVC, the collinearity diagnostics are

somewhat different. The general patterns found previously are also found in

this analysis. However, now the largest condition index is 38 and the next

largest is 7. This gap of the first kind indicates that one moderately strong

dependency is present, since 7 is -absolutely small. - The intercept, age, and

height are involved in the dependency. A second weaker dependency involving

weight may exist, as was seen for the original model.

106

..

Table 2.20 VM~o",: MIXMOD Results and Collinearity Diagnostics

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.429 0.725 0.59 0.553Age 0.075 0.068 1.10 0.271Height 0.005 0.009 0.56 0.579 .Weight 0.022 0.010 2.20 0.028

A

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

1.8769 0.1623 -0.0221 -0.01800.0221 -0.0018 -0.0032

0.0003 0.00020.0006

Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

1.00 0.80 -0.99 -0.521.00 -0.74 -0.84

1.00 0.441.00

A

02=0.165282 874

563427466

103742

2805154

387922073

44263798908

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.505 1.87 1 0.000 0.003 0.000 0.0080.428 0.65 3 0.006 0.025 0.001 0.0510.065 0.25 7 0.000 0.253 0.001 0.9400.002 0.05 38 0.993 0.719 0.997 0.001

107

1)

2.9 Summary and Implications of Results

In this chapter, collinearity diagnostic measures for the mixed model have

been proposed. Computation of the measures and application of the diagnostic

procedure, developed by Belsley (1991) for the GLUM, have been extended to

the mixed model and demonstrated for longitudinal data with collinear variables.

The purpose of this research was to obtain initial impressions of the behavior

of the collinearity diagnostics in the mixed model situation, with a view toward

refining the subsequent research.

2.9.1 Initial Conclusions

Several impressions have emerged from these analyses:

GLUM results and MIXMOD results for similar datil appear to be

comparable. Nearly identical results were found for the GLUM at the~

pulmonary study (n =34, Table 2.4) with age, height, and weight as

independent variables and the ordinary least squares model for the data

for .all visits (n = 527, Table 2.8) with age, height, and weight as

independent variables. The results indicated one strong near dependency

and a possible weaker near dependency; all variables appeared to be

involved. The mixed model results (n =527, Table 2.9), using age,

height, and weight as both fixed and random effects, were similar,

though a slightly greater degree of collinearity was indicated. Thus, we

believe that the diagnostics operate similarly for the GLUM and the

mixed model.

108

.,

2) Deletion of collinear variables from Z affects the degree of collinearity

present. More dramatic changes occurred in the condition indexes with

the deletion of two variables from Z than with the deletion of one

variable from Z. In each case, however the largest CI was smaller (38-

77) relative to the largest CI for the original model (78). The least

reduction in the largest CI (from 78 to 77) occurred when weight was

removed from Z. Weight not involved in the dominant dependency.

Thus, it appears that removing such a variable from Z has little impact

on the diagnostics. Contrastingly, when those variables (age and height)

more strongly involved in the dominant dependency were deleted from

Z, a dramatic reduction in the Cis was found (from 78 to 33-50). This

seems remarkable since all collinear variables were retained for the fixed

effects. Thus, we believe that deleting collinear variables from Z, while

retaining them in X, affects the degree of collinearity for the model.A.

3) The structure of 4 affects the degree of collinearity present. The issueA.

of constraining 4 and that of deleting variables from Z are related.

Usually, when the components of the covariance or the correlation of the

random effects are small, the corresponding variables are removed from

Z. However, for these examples, the variables were not removed.

Nevertheless, a dramatic reduction in the Cis (from 78 to 23-24) wasA. A.

found when one diagonal element of 4 or all off-diagonal elements of 4A.

were set to zero. Thus, we believe that smaller elements of 4, in the

presence of all collinear variables in both X and Z, affect the degree of

collinearity for the model.

109

4) Different dependent variables affect the degree of collinearity present.

When a mixed model with a different dependent variable, VMAX50 'l6' was

fit, a striking reduction in the largest CI (from 78 to 38) was found. The

significance of this is that the same variables (age, height, and weight)

that were used to model FVC as were used in X and Z in the VMAX50'l6

A.

model. This finding is not unexpected, since the variance of V, I, is

involved in the computation of the diagnostics. However, this condition

is unique to the mixed model. In the GLUM, the degree of collinearity

present for a given X is the same regardless of the dependent variable

in the model. Thus, we believe that a different dependent variable, with

the same collinear variables in both X and Z, impacts the degree of

collinearity for the model.

2.9.2 Implications for Subsequent Research

Initial impressions as to the types of factors impacting the diagnostics

have been reported in this chapter based on results from one data set with

three collinear variables. This has been useful as a point of departure in

designing the subsequent research. However, in order to fully characterize the

behavior of the diagnostics, an experimental approach is required. The

collinearity-impacting factors identified in this chapter, along with several

others, will be subjected to more rigorous scrutiny using simulated data. In this

way, the factors will be artifically controlled in known ways and the results will

be more definitively stated.

110

CHAPTER III

APPLICATION OF DIAGNOSTICS TO EXPERIMENTAL DATA

3. 1 Introduction

Because collinearity is data dependent, behavior of the diagnostics must

be determined empirically. Therefore in order to describe their behavior more

comprehensively, the diagnostics must be applied to data whose level of

collinearity has been predetermined. In addition, a range of severity of

collinearity must be examined. Then under known circumstances some

implications of collinearity can be determined and described, including the

viability of the diagnostics developed and the effects on the variances of

parameter estimates.

In this chapter, the diagnostics defined and used in Chapter 2 are applied

to contrived data. Several types of dependencies of varying levels are created.

Then the behavior of the diagnostics is assessed for carefully selected sets of

design matrices and parameter values. Initially, diagnostics are applied in the

context of the GLUM. The GLUM results then provide a background for

examining the diagnostics applied to the mixed model. For the GLUM, only an

X matrix (and X'X) are needed in order to evaluate the dependencies. For the

mixed model, however, the process has additional levels of complexity, arising

from the covariance structure and random effects design matrix.

In this chapter, experiments for the GLUM and the mixed model are

described. First, for the GLUM, the types of artificial dependencies in X and

the procedures for creating them are specified. Then, the results of applying

the collinearity diagnostics to the simulated X'X matrices for the GLUM are

portrayed. Next, for the mixed model, the procedures for creating the artificial

dependencies in X and Z are described. Then the procedures used to obtain the

matrices and parameters needed for the mixed model experiment are described.

Finally, the results of applying the collinearity diagnostics to the simulated

matrix (X'E"X) for the mixed model are summarized. Based on the results of

these experiments, some general guidelines for the behavior of the diagnostics

under the conditions examined are presented.

3.2 The GLUM Experiment

The procedure for using the collinearity diagnostics, described in

Chapters 1 and 2, was based on a series of experiments in which the behavior

of the diagnostics was determined empirically. The six sets of experiments are

described by Belsley (1991, Chapter 4) in which different types of near

dependencies with different types of variables are explored in the GLUM

context. Each experiment began with a "basic" data set of n = 24 to 30

observations on p = 3 to 5 variables. From the basic data sets, additional

collinear data series with increasingly tighter linear near dependencies with the

basic series were constructed (Belsley 1991, P 79). His basic data consisted

112

of actual economic time series or random series.

In this section, one of Belsley's experimental series was emulated.

However, in contrast to Belsley's experiments, the basic data used here were

totally random and an intercept term was added to the set of basic variables.

Data were chosen for the basic variables such that they represented three

continuous variables from a typical data set. Since the basic data were chosen

randomly, they were well-conditioned. Thus, the ill-conditioning assessed was

totally attributable to the contrived dependencies that augmented the basic

variables.

Dependencies for the experiment were created as described below. Then

the GLUM diagnostics were applied to them. The results, even with the

intercept term included, are similar to those obtained previously by Belsley and

are a basis of comparison for the subsequent mixed model experiment.

3.2.1 The Basic Data

The basic data set used in the experiment is

X == [lNT, aX1, aX2, aX3], (3.2.1 )

where INT is the intercept term and aX1, aX2, and aX3 are each generated

from a uniform integer distribution, on the interval 0-1. Sixty values of each

variable were generated. The range of potential values for aX1 and aX2 is

from 0 to 10; the range for aX3 is from 0 to 100.

3.2.2 The Dependency Sets

From the basic data, two dependency sets were created. In the first,

Wi = aX3 + Si,

113

i=O, ..., 4, (3.2.2)

j =0, ..., 4, (3.2.3)

with 8 j generated from a normal distribution with mean zero and variance

02 =10-j x [var(BX3)1]. In the second,

~ = 0.SBX1 + 0.2BX2 + 8j'

with 8 j generated from a normal distribution with mean zero and variance

02= 10-j x [var(0.SBX1 + 0.2BX2)1].

3.2.3 The Data Series

The ~ and ~ dependency sets were used to augment the basic data set

to produce three series of matrices:

X1{i} • [X Wi]

X2{j} • [X Zj]

X3{i,j} • [X Wi Zj]

i =0, ..., 4,

j =0, ..., 4,

i,j =0, ..., 4.

(3.2.4)

(3.2.5)

(3.2.6)

For each series, collinearity diagnostics were computed.

3.2.4 The Issues Addressed

The dependency created by the Wj is a simple relationship between two

variables that might be detectable through examination of a correlation matrix.

The dependency created by the Zj is a simple relationship among three

variables, but it is not easily detectable through examination of a correlation

matrix. These are sets of coexisting dependencies, Le., they are non­

overlapping. As the variance of the error term becomes smaller and smaller

with increasing i and j, the dependency becomes tighter and tighter.

3.2.5 Results

All of the diagnostics for the GLUM experiment are presented in

Appendix 1. The primary findings for the three data series are summarized

114

here.

3.2.5.1 Simple Dependency: Two Variables

•

For the X1{i} series, results are shown in Table 3.1. The variable in the

fifth column (labeled WO-W4) is related to the variable in the fourth column

(BX3) by (3.2.2). This is the only contrived dependency in this series. As

expected, there is one high condition index and a large proportion of the

variance of BX3 and the variances of WO-W4 are associated with it. As the

dependency between the two variables increases (from Set 1 to Set 5, i.e.,

from X1 {O} to X1 {4}), the pattern of the relationship becomes clearer. The one

condition index indicating the dependency becomes larger and the variance

decomposition proportions of the two variables becomes larger.

3.2.5.2 Simple Dependency: Three Variables

For the X2{j} series, results are shown in Table 3.2. The variable in the

fifth column (labeled ZO-Z4) is related to the variables in the columns 2 and 3

(BX1 and BX2) by (3.2.3). This is the only contrived dependency in this series.

As expected, there is one high condition index and a large proportion of the

variances of variables involved in the dependency are associated with it. As

the dependency between the three variables increases (from Set 6 to Set 10,

i.e., from X2{0} to X2{4}), the pattern of the relationship becomes clearer.

The one condition index indicating the dependency becomes larger and the

variance decomposition proportions of the two variables becomes larger. For

X2{0), the dependency is not yet apparent; for X2{1), the involvement of BX1

and Z1 is obvious. Thereafter, the dependency is clearly detectable.

115

3.2.5.3 Coexisting Dependency: Two Variables, Three Variables(Nonoverlapping)

The X3{i,j} series contains a combination of the two dependencies in

X1{i} and X2{j}; these are coexisting dependencies. Twenty-five sets of

matrices result from this series; selected results are shown in Table 3.3 for;= 2

and j=O, ..., 4. The dependency in (3.2.2) is fixed and strong while the

dependency in (3.2.3) varies. There are two high condition indexes indicating

the two dependencies, though for j = 0 the second dependency is dominated by

the first and is not yet apparent. The effects become separable when j = 1.

Thus the procedure can clearly detect two dependencies and the variables

involved in each. When the two dependencies are of nearly the same strength,

their effects become confounded. For X3{2,2}, the condition indexes are of

similar magnitude and the involvement of variables in the two dependencies is

somewhat obscured. However, it is possible to determine that there are two

dependencies present and that five variables are involved in them. As the

second dependency (BX1, BX2, ~) becomes strong relative to the first

dependency (BX3, W2), the effects are again separable.

The X3{i,j} series can be examined from the other direction by holding

j constant and varying;. Selected results are shown in Table 3.4 for ;=0, ...,

4 and j= 2. The dependency in 3.2.2 varies while the dependency in 3.2.3 is

held constant. For ;=0, the dependency involving BX3 and WO is relatively

weak; the one high condition index and the variance decomposition proportions

indicate the strong dependency in BX1, BX2, and Z2. When;= 1, the two

dependencies are distin~t. When;= 2, the two dependencies are of similar

116

strength and the variables involved in each are somewhat obscured. However,

when i =3, their separate identities emerge again and remain distinct for i =4.

Again, the procedure can clearly detect two dependencies and the variables

involved in each.

117

Table 3.1 The GLUM Experiment, Series X1{i}, ;=0, ..0' 4 (One Contrived NearDependency): Condition Indexes and Variance-Decomposition Proportions

118

Table 3.2 The GLUM Experiment, Series X2{j},j=O, ..., 4 (One Contrived NearDependency): Condition Indexes and Variance-Decomposition Proportions

D{O} Set 6: ZO

CI variance Proportions for Coefficients ofIHT aX1 aX2 aX3 ZO

1 0.005 0.006 0.011 0.010 0.0084 0.009 0.066 0.001 0.426 0.1564 0.003 0.030 0.905 0.100 0.0417 0.159 0.445 0.019 0.151 0.7448 0.824 0.453 0.063 0.313 0.051

D{l} Set 7: Zl

CI VarIance ProportIons for Coefficients ofINT aX1 aX2 aX3 Zl

1 0.005 0.001 0.007 0.009 0.0014 0.012 0.015 0.026 0.521 0.0075 0.002 0.027 0.523 0.006 0.0037 0.958 0.008 0.024 0.418 0.012

21 0.023 0.949 0.421 0.046 0.977

%2{2} Set 8: Z2

ofZ2

0.0000.0010.0000.0010.998

CI

1457

69

variance Proportions forINT aX1 aX2

0.005 0.000 0.0010.012 0.001 0.0030.003 0.002 0.0950.977 0.001 0.0050.003 0.995 0.896

CoefficientsaX3

0.0090.5120.0110.4570.010

D{3} Set 9: Z3

CI

1457

251

CI

1457

723

variance Proportions for Coefficients ofINT aX1 aX2 aX3 Z3

0.005 0.000 0.000 0.009 0.0000.012 0.000 0.000 0.503 0.0000.003 0.000 0.008 0.011 0.0000.978 0.000 0.000 0.443 0.0000.003 1.000 0.991 0.035 1.000

%2{4} Set 10: Z4

variance Proportions for coefficients ofINT aX1 aX2 aX3 Z4

0.005 0.000 0.000 0.009 0.0000.012 0.000 0.000 0.499 0.0000.003 0.000 0.001 0.011 0.0000.980 0.000 0.000 0.444 0.0000.000 1.000 0.999 0.037 1.000

119

Table 3.3 The GLUM Experiment, Series X3{i,j}, ;=2; j=O, ..., 4 (TwoContrived Near Dependencies): Condition Indexes and Variance-DecompositionProportions

D{2,0} Set 21: W2, ZO

CI Variance Proportions for Coefficients ofINT aXl aX2 aX3 W2 ZO

1 0.003 0.004 0.008 0.000 0.000 0.0053 0.000 0.042 0.031 0.001 0.002 0.0785 0.000 0.054 0.837 0.000 0.000 0.0977 0.282 0.285 0.047 0.000 0.000 0.7018 0.666 0.613 0.063 0.000 \ 0.001 0.119

67 0.049 0.002 0.013 0.998 0.997 0.001

D{2,1} Set 22: W2, Zl

CI variance Proportions for Coefficients ofINT aXl aX2 aX3 W2 Zl

1 0.003 0.001 0.004 0.000 0.000 0.0013 0.000 0.009 0.034 0.001 0.002 0.0055 0.001 0.028 0.506 0.000 0.000 0.0048 0.926 0.010 0.033 0.001 0.001 0.013

22 0.025 0.891 0.417 0.001 0.000 0.91470 0.045 0.061 0.006 0.997 0.997 0.064

D{2,2} Set 23: W2, Z2

CI Variance Proportions for Coefficients ofINT aXl aX2 aX3 W2 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.001 0.002 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.939 0.001 0.006 0.001 0.001 0.001

65 0.030 0.152 0.166 0.715 0.723 0.15479 0.026 0.843 0.731 0.283 0.274 0.844

X3{2,3} Set 24: W2, Z3

CI Variance Proportions for Coefficients ofINT aXl aX2 aX3 W2 Z3

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.002 0.002 0.0005 0.001 0.000 0.008 0.000 0.000 0.0008 0.945 0.000 0.001 0.001 0.001 0.000

68 0.048 0.000 0.000 0.996 0.997 0.000274 0.003 1.000 0.991 0.002 0.001 1.000

X3{2,4} Set 25: W2, Z4

CI Variance Proportions for Coefficients of •INT aXl aX2 aX3 W2 Z41 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.001 0.002 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.947 0.000 0.000 0.001 0.001 0.000

68 0.048 0.000 0.000 0.986 0.980 0.000795 0.000 1.000 0.999 0.012 0.017 1.000

120

Table 3.4 The GLUM Experiment, Series X3{i,j}, ;=0, ..., 4; j=2 (TwoContrived Near Dependencies): Condition Indexes and Variance-DecompositionProportions

D{0,2} Set 13: WO, Z2

CI VarIance Proportions for CoeffIcIents ofINT BX1 BX2 BX3 WO Z2

1 0.004 0.000 0.001 0.004 0.005 0.0004 0.000 0.001 0.004 0.087 0.127 0.0015 0.004 0.002 0.094 0.010 0.000 0.0007 0.359 0.001 0.000 0.190 0.603 0.0008 0.632 0.001 0.006 0.671 0.232 0.000

76 0.002 0.995 0.895 0.038 0.033 0.998

D{1,2} Set 18: W1, Z2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W1 Z2

1 0.003 0.000 0.001 0.001 0.001 0.0003 0.000 0.001 0.005 0.013 0.013 0.0015 0.001 0.003 0.092 0.000 0.000 0.0008 0.993 0.001 0.006 0.008 0.007 0.001

23 0.000 0.000 0.001 0.956 0.963 0.00076 0.002 0.995 0.896 0.023 0.016 0.998

D{2,2} Set 23: W2, Z2

CI variance Proportions for Coefficients ofINT BX1 BX2 BX3 W2 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.001 0.002 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.939 0.001 0.006 0.001 0.001 0.001

65 0 •.030 0.152 0.166 0.715 0.723 0.15479 0.026 0.843 0.731 0.283 0.274 0.844

X3{3,2} Set 28: W3, Z2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.000 0.000 0.0015 0.001 0.003 0.090 0.000 0.000 0.0008 0.978 0.001 0.007 0.000 0.000 0.001

75 0.003 0.939 0.879 0.001 0.001 0.954229 0.015 0.056 0.018 0.999 0.999 0.044

D{4,2} Set 33: W4, Z2

CI variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.000 0.000 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.989 0.001 0.006 0.000 0.000 0.001

76 0.003 0.982 0.890 0.000 0.000 0.989807 0.003 0.013 0.007 1.000 1.000 0.009

121

3.3 The Mixed Model Experiment

The primary goal of this dissertation is to determine the patterning of the

collinearity diagnostics in the mixed model context. In this section, the

procedures for creating the artificial dependencies in X and Z are described.

Then the approach used to obtain the matrices and parameters needed for the

mixed model simulations is summarized. Although the approach involves

varying some of the factors previously identified in Chapter 2, accomplishing

this directly in simulations is difficult. The problems encountered en route to

a final strategy are described; they pinpoint the difficulties of collinearity

assessment in the mixed model. Then the factors varied in the simulations are

described. Finally, the results of applying the collinearity diagnostics to the

simulated matrix (X'I-'X) for the mixed model are summarized. Based on the

results of these experiments, some general guidelines for the behavior of the

diagnostics under the conditions examined are presented.

3.3.1 The Basic Data

Analogous to the GLUM data, the basic data set used for the mixed

model simulations is

X .. [INT, BX1, BX2, BX3], (3.3.1)

where INT is the intercept term and BX1, BX2, and BX3 are each generated

from a uniform distribution. However, for the mixed simulations, data for 30

subjects with 10 observations each were generated, i.e., 300 values of each

variable were generated. The variable BX1 was made a "longitudinal variable"

within a given subject's data by sorting it in ascending order and rounding it to

122

one decimal place prior to generating the dependencies. The potential range of

values for BX1 is from 0.0 to 10.0; the potential range for BX2 is the set of

integers from 0 to 10; and the potential range for BX3 is the set of integers

from 0 to 100. The actual ranges for a specific subject may be shorter.

3.3.2 The Dependency Sets

From the basic data, the same two dependency sets used in the GLUM

experiment were created for the mixed model experiment. In the first,

Wi = BX3 + 8 i, i =0, ..., 4, (3.3.2)

with 8 j generated from a normal distribution with mean zero and variance

02 = 10-i X [var(BX3)1]. In the second,

Zj = 0.SBX1 + 0.2BX2 + 8;, j =0, ..., 4, (3.3.3)

with 8 j generated from a normal distribution with mean zero and variance

02 = 10·; x [var(0.SBX1 + 0.2BX2)1). These dependencies were created for the

entire data set (300 observations) rather than on a by-subject basis; i.e., the

overall theoretical population variance from the uniform distribution was used

for the variances of BX3 and (0.SBX1 + 0.2BX2), rather than the sample

variance of these variables, considered either overall or separately for each

subject.

3.3.3 The Data Series

As in the GLUM context, the Xi and Zj dependency sets were used to

augment the basic data set to produce three series of matrices:

X1{i} == [X Wi]

X2{j} == [X Zjl

i=O, ..., 4,

j =0, ..., 4,

123

(3.3.4)

(3.3.5)

X3{i,j} • [X Wi ~] i,j =0, ..., 4. (3.3.6)

The additional complexity in the mixed model case is that these dependencies

may exist in the fixed effects, in the random effects or in both. Thus, for each

series, collinearity diagnostics for the fixed effects of the mixed model were

computed separately for several cases of variate involvement in the random

effects.

3.3.4 The Issues Addressed

The basic issues addressed are the same as those addressed for the

GLUM. The behavior of the diagnostics is assessed for 1) a simple dependency

between two variables created by the Wi' 2) the simple dependency among

three variables created by the Zj' and 3) the sets of coexisting dependencies

created by considering the Wi and the Zi simultaneously.

The reconsideration of these issues in the mixed model context

constitutes the new territory of research for this dissertation. For the mixed

model, the behavior of the diagnostics for the fixed effects can be impacted by

dependencies both in the fixed effects (the X matrix in the model) and those in

the random effects (the Z matrix in the model, not to be confused with the

vectors of dependencies Zi,i:O,4). Thus, for each dependency series, collinearity

diagnostics for the fixed effects of the mixed model were computed as the

number of variables in Z varied from 0 to 6. There were 6 runs for each

dependency; for the sixth run in each set, all variables in X were also in Z. So,

for the sixth run, there were five variables in Z for the X1 {i} series and the

X2{j} series and six variables in Z for the X3{i,j} series. Therefore, instead of

124

the 35 simulation runs required for the GLUM experiment, 210 (35 x 6) runs

were required for the mixed model experiment. Table 3.5 contains a summary

of the variables involved in the mixed model runs.

125

Table 3.5 Variables in Fixed and Random Effects of Mixed Model Experiment

Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.

INT BX1 BX2 BX3 WO 1_0 1

INT BX1 BX2 BX3 WO INT 1_1 2

WOINT BX1 BX2 BX3 WO INT BX1 1_2 3

INT BX1 BX2 BX3 WO INT BX1 BX2 1_3 4

INT BX1 BX2 BX3 WO INT BX1 BX2 BX3 1_4 5

INT BX1 BX2 BX3 WO INT BX1 BX2 BX3 WO 1 5 6

INT BX1 BX2 BX3 W1 2_0 7

INT BX1 BX2 BX3 W1 INT 2 1 8

W1INT BX1 BX2 BX3 W1 INT BX1 2 2 9

INT BX 1 BX2 BX3 W1 INT BX1 BX2 2_3 10

INT BX1 BX2 BX3 W1 INT BX1 BX2 BX3 2 4 11

INT BX1 BX2 BX3 W1 INT BX1 BX2 BX3 W1 2_5 12

INT BX1 BX2 BX3 W2 3 0 13

INT BX1 BX2 BX3 W2 INT 3 1 14

W2INT BX1 BX2 BX3 W2 INT BX1 3_2 15

INT BX1 BX2 BX3 W2 INT BX1 BX2 3 3 16

INT BX1 BX2 BX3 W2 INT BX1 BX2 BX3 3_4 17

INT BX1 BX2 BX3 W2 INT BX1 BX2 BX3 W2 3_5 18

INT BX1 BX2 BX3 W3 4 0 19

INT BX1 BX2 BX3 W3 INT 4 1 20

W3 INT BX1 BX2 BX3 W3 INT BX1 4_2 21

INT BX1 BX2 BX3 W3 INT BX1 BX2 4 3 22

INT BX1 BX2 BX3 W3 INT BX1 BX2 BX3 44 23

INT BX1 BX2 BX3 W3 INT BX1 BX2 BX3 W3 4_5 24

INT BX1 BX2 BX3 W4 5_0 25

INT BX1 BX2 BX3 W4 INT 5_1 26

W4 INT BX1 BX2 BX3 W4 INT BX1 5_2 27

INT BX1 BX2 BX3 W4 INT BX1 BX2 5_3 28

INT BX1 BX2 BX3 W4 INT BX1 BX2 BX3 5 4 29

INT BX1 BX2 BX3 W4 INT BX1 BX2 BX3 W4 5 5 30

126

Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.

INT BX1 BX2 BX3 ZO 60 31

INT BX1 BX2 BX3 ZO INT 6_1 32

ZOINT BX1 BX2 BX3 ZO INT BX1 6 2 33

INT BX1 BX2 BX3 ZO INT BX1 BX2 6_3 34

INT BX1 BX2 BX3 ZO INT BX1 BX2 BX3 6_4 35

INT BX1 BX2 BX3 ZO INT BX1 BX2 BX3 ZO 6_5 36

INT BX1 BX2 BX3 Z1 7 0 37

INT BX1 BX2 BX3 Z1 INT 7 1 38

Z1INT BX1 BX2 BX3 Z1 INT BX1 7_2 39

INT BX1 BX2 BX3 Z1 INT BX1 BX2 7_3 40

INT BX1 BX2 BX3 Z1 INT BX1 BX2 BX3 7 4 41

INT BX1 BX2 BX3 Z1 INT BX1 BX2 BX3 Z1 7_5 42

INT BX1 BX2 BX3 Z2 8 0 43

INT BX1 BX2 BX3 Z2 INT 8_1 44

Z2INT BX1 BX2 BX3 Z2 INT BX1 8 2 45

INT BX1 BX2 BX3 Z2 INT BX1 BX2 8 3 46

INT BX1 BX2 BX3 Z2 INT BX1 BX2 BX3 8 4 47

INT BX1 BX2 BX3 Z2 INT BX1 BX2 BX3 Z2 8_5 48

INT BX1 BX2 BX3 Z3 90 49

INT BX1 BX2 BX3 Z3 INT 9_1 50

Z3INT BX1 BX2 BX3 Z3 INT BX1 9_2 51

INT BX1 BX2 BX3 Z3 INT BX1 BX2 9_3 52

INT BX1 BX2 BX3 Z3 INT BX1 BX2 BX3 94 53

INT BX1 BX2 BX3 Z3 INT BX1 BX2 BX3 Z3 9 5 54

INT BX1 BX2 BX3 Z4 10_0 55

INT BX1 BX2 BX3 Z4 INT 10 1 56

Z4INT BX1 BX2 BX3 Z4 INT BX1 10 2 57

INT BX1 aX2 aX3 Z4 INT aX1 aX2 10_3 58

INT ax1 BX2 aX3 Z4 INT ax1 BX2 aX3 10 4 59

INT BX1 BX2 BX3 Z4 INT ax1 BX2 BX3 Z4 10 5 60

127

Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.

INT BX1 BX2 BX3 WO ZO 11_0 61

INT BX1 BX2 BX3 WO ZO tNT 11_1 62

WOtNT BX1 BX2 BX3 WO ZO tNT BX1 11_2 63

ZO tNT BX1 BX2 BX3 WO ZO tNT BX1 BX2 11_3 64

tNT BX1 BX2 BX3 WO ZO tNT BX1 BX2 BX3 11_4 65

tNT BX1 BX2 BX3 WO ZO INT BX1 BX2 BX3 WO ZO 11_5 66

INT BX1 BX2 BX3 WO Z1 12 0 67

INT BX1 BX2 BX3 WO Z1 tNT 12_1 68

WO INT BX1 BX2 BX3 WO Z1 INT BX1 12_2 69

Z1 INT BX1 BX2 BX3 WO Z1 tNT BX1 BX2 12_3 70

INT BX1 BX2 BX3 WO Z1 tNT BX1 BX2 BX3 12 4 71

tNT BX1 BX2 BX3 WO Z1 INT BX1 BX2 BX3 WO Z1 12 5 72

tNT BX1 BX2 BX3 WO Z2 13 0 73

INT BX1 BX2 BX3 WO Z2 INT 13_1 74

WO INT BX1 BX2 BX3 WO Z2 INT BX1 13_2 75

Z2 INT BX1 BX2 BX3 WO Z2 INT BX1 BX2 13 3 76

INT BX1 BX2 BX3 WO Z2 INT BX1 BX2 BX3 13_4 77

INT BX1 BX2 BX3 WO Z2 INT BX1 BX2 BX3 WO Z2 13_5 78

INT BX1 BX2 BX3 WO Z3 14_0 79

INT BX1 BX2 BX3 WO Z3 tNT 14 1 80

WO INT BX1 BX2 BX3 WO Z3 INT BX1 14 2 81

Z3 INT BX1 BX2 BX3 WO Z3 INT BX1 BX2 14 3 82

INT BX1 BX2 BX3 WO Z3 tNT BX1 BX2 BX3 14 4 83

INT BX1 BX2 BX3 WO Z3 INT BX1 BX2 BX3 WO Z3 14_5 84

INT BX1 BX2 BX3 WO Z4 15 0 85

tNT BX1 BX2 BX3 WO Z4 INT 15 1 86

WOINT BX1 BX2 BX3 WO Z4 INT BX1 15 2 87

Z4 INT BX1 BX2 BX3 WO Z4 INT BX1 BX2 15 3 88

INT BX1 BX2 BX3 wO Z4 INT BX1 BX2 BX3 15_4 89

INT BX1 BX2 BX3 WO Z4 INT BX1 BX2 BX3 WO Z4 15_5 90

128

Collinear Fixed Effects Random Effects Set &. RunVariable Variables in X Variables in Z No. No.

INT BX1 BX2 BX3 W1 ZO 16_0 91

INT BX1 BX2 BX3 W1 ZO INT 16_1 92

W1INT BX1 BX2 BX3 W1 ZO INT BX1 16 2 93

ZO INT BX1 BX2 BX3 W1 ZO INT BX1 BX2 16_3 94

INT BX1 BX2 BX3 W1 ZO INT BX1 BX2 BX3 16 4 95

INT BX1 BX2 BX3 W1 ZO INT BX1 BX2 BX3 W1 ZO 16_5 96

INT BX1 BX2 BX3 W1 Z1 17_0 97

INT BX1 BX2 BX3 W1 Z1 INT 17_1 98

W1 INT BX1 BX2 BX3 W1 Z1 INT BX1 17_2 99

Z1 INT BX1 BX2 BX3 W1 Z1 INT BX1 BX2 173 100

INT BX1 BX2 BX3 W1 Z1 INT BX1 BX2 BX3 17_4 101

INT BX1 BX2 BX3 W1 Z1 INT BX1 BX2 BX3 W1 Z1 17_5 102

INT BX1 BX2 BX3 W1 Z2 18 0 103

INT BX1 BX2 BX3 W1 Z2 INT 18 1 104

W1INT BX1 BX2 BX3 W1 Z2 INT BX1 18 2 105

Z2 INT BX1 BX2 BX3 W1 Z2 INT BX1 BX2 18 3 106

INT BX1 BX2 BX3 W1 Z2 INT BX1 BX2 BX3 18 4 107

INT BX1 BX2 BX3 W1 Z2 INT BX1 BX2 BX3 W1 Z2 18_5 108

INT BX1 BX2 BX3 W1 Z3 19 1 109

INT BX1 BX2 BX3 W1 Z3 INT 19 1 110

W1INT BX1 BX2 BX3 W1 Z3 INT BX1 19_2 111

Z3 INT BX1 BX2 BX3 W1 Z3 INT BX1 BX2 19_3 112

INT BX1 BX2 BX3 W1 Z3 INT BX1 BX2 BX3 19_4 113

INT BX1 BX2 BX3 W1 Z3 INT BX1 BX2 BX3 W1 Z3 19_5 114

INT BX1 BX2 BX3 W1 Z4 20 1 115

INT BX1 BX2 BX3 W1 Z4 INT 20 1 116

W1INT BX1 BX2 BX3 W1 Z4 INT BX1 20 2 117

Z4 INT BX1 BX2 BX3 W1 Z4 INT BX1 BX2 20 3 118

INT BX1 BX2 BX3 W1 Z4 INT BX1 BX2 BX3 20 4 119

INT BX1 BX2 BX3 W1 Z4 INT BX1 BX2 BX3 W1 Z4 20 5 120

129

Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.

INT BX1 BX2 BX3 W2 ZO 21_0 121

INT BX1 BX2 BX3 W2 ZO INT 21_1 122

W2INT BX1 BX2 BX3 W2 ZO INT BX1 21_2 123

ZO INT BX1 BX2 BX3 W2 ZO INT BX1 BX2 21_3 124

INT BX1 BX2 BX3 W2 ZO INT BX1 BX2 BX3 21_4 125

INT BX1 BX2 BX3 W2 ZO INT BX1 BX2 BX3 W2 ZO 21_5 126

INT BX1 BX2 BX3 W2 Z1 22_0 127

INT BX1 BX2 BX3 W2 Z1 INT 22_1 128

W2INT BX1 BX2 BX3 W2 Z1 INT BX1 22 2 129

Z1 INT BX1 BX2 BX3 W2 Z1 INT BX1 BX2 22 3 130

INT BX1 BX2 BX3 W2 Z1 INT BX1 BX2 BX3 22_4 131

INT BX1 BX2 BX3 W2 Z1 INT BX1 BX2 BX3 W2 Z1 22_5 132

INT BX1 BX2 BX3 W2 Z2 23_0 133

INT BX1 BX2 BX3 W2 Z2 INT 23_1 134

W2INT BX1 BX2 BX3 W2 Z2 INT BX1 23 2 135

Z2 INT BX1 BX2 BX3 W2 Z2 INT BX1 BX2 23 3 136

INT BX1 BX2 BX3 W2 Z2 INT BX1 BX2 BX3 23_4 137

INT BX1 BX2 BX3 W2 Z2 INT BX1 BX2 BX3 W2 Z2 23_5 138

INT BX1 BX2 BX3 W2 Z3 24_0 139

INT BX1 BX2 BX3 W2 Z3 INT 24_1 140

W2INT BX1 BX2 BX3 W2 Z3 INT BX1 24_2 141

Z3 INT BX1 BX2 BX3 W2 Z3 INT BX1 BX2 24_3 142

INT BX1 BX2 BX3 W2 Z3 INT BX1 BX2 BX3 24 4 143

INT BX1 BX2 BX3 W2 Z3 INT BX1 BX2 BX3 W2 Z3 24 5 144

INT BX1 BX2 BX3 W2 Z4 25_0 145

INT BX1 BX2 BX3 W2 Z4 INT 25 1 146

W2 INT BX1 BX2 BX3 W2 Z4 INT BX1 25 2 147

Z4INT BX1 BX2 BX3 W2 Z4 INT BX1 BX2 25_3 148

INT BX1 BX2 BX3 W2 Z4 INT BX1 BX2 BX3 25_4 149

INT BX1 BX2 BX3 W2 Z4 INT BX1 BX2 BX3 W2 Z4 25_5 150

130

Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.

INT BX1 BX2 BX3 W3 ZO 26 0 151

INT BX1 BX2 BX3 W3 ZO INT 26 1 152

W3INT BX1 BX2 BX3 W3 ZO INT BX1 26_2 153

ZO INT BX1 BX2 BX3 W3 ZO INT BX1 BX2 26_3 154

INT BX1 BX2 BX3 W3 ZO INT BX1 BX2 BX3 26_4 155

INT BX1 BX2 BX3 W3 ZO INT BX1 BX2 BX3 W3 ZO 26_5 156

INT BX1 BX2 BX3 W3 Z1 27_0 157

INT BX1 BX2 BX3 W3 Z1 INT 27 1 158

W3INT BX1 BX2 BX3 W3 Z1 INT BX1 27 2 159

Z1 INT BX1 BX2 BX3 W3 Z1 INT BX1 BX2 27_3 160

INT BX1 BX2 BX3 W3 Z1 INT BX1 BX2 BX3 27_4 161

INT BX1 BX2 BX3 W3 Z1 INT BX1 BX2 BX3 W3 Z1 27_5 162

INT BX1 BX2 BX3 W3 Z2 28_0 163

INT BX1 BX2 BX3 W3 Z2 INT 28_1 164

W3INT BX1 BX2 BX3 W3 Z2 INT BX1 28 2 165

Z2 INT BX1 BX2 BX3 W3 Z2 INT BX1 BX2 28 3 166

INT BX1 BX2 BX3 W3 Z2 INT BX1 BX2 BX3 28 4 167

INT BX1 BX2 BX3 W3 Z2 INT BX1 BX2 BX3 W3 Z2 28_5 168

INT BX1 BX2 BX3 W3 Z3 29_0 169

INT BX1 BX2 BX3 W3 Z3 INT 29 1 170

W3INT BX1 BX2 BX3 W3 Z3 INT BX1 29 2 171

Z3 INT BX1 BX2 BX3 W3 Z3 INT BX1 BX2 29 3 172

INT BX1 BX2 BX3 W3 Z3 INT BX1 BX2 BX3 29 4 173

INT BX1 BX2 BX3 W3 Z3 INT BX1 BX2 BX3 W3 Z3 29_5 174

INT BX1 eX2 eX3 W3 Z4 30_0 175

INT ex1 eX2 eX3 W3 Z4 INT 30_1 176

W3INT ex1 eX2 eX3 W3 Z4 INT BX1 30 2 177

Z4 INT ex1 eX2 eX3 W3 Z4 INT eX1 eX2 30 3 178

INT ex1 eX2 BX3 W3 Z4 INT ex1 BX2 eX3 30 4 179

INT ex1 eX2 eX3 W3 Z4 INT ex1 eX2 eX3 W3 Z4 30 5 180

131

Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.

INT BX1 BX2 BX3 W4 ZO 31_0 181

INT BX1 BX2 BX3 W4 ZO INT 31 1 182

INT BX1 BX2 BX3 W4 ZO INT BX1 31_2 183W4ZO INT BX1 BX2 BX3 W4 ZO INT BX1 BX2 31 3 184

INT BX1 BX2 BX3 W4 ZO INT BX1 BX2 BX3 31 4 185

INT BX1 BX2 BX3 W4 ZO INT BX1 BX2 BX3 W4 ZO 31_5 186

INT BX1 BX2 BX3 W4 Z1 32_0 187

INT BX1 BX2 BX3 W4 Z1 INT 32_1 188

W4INT BX1 BX2 BX3 W4 Z1 INT BX1 32 2 189

Z1 INT BX1 BX2 BX3 W4 Z1 INT BX1 BX2 32 3 190

INT BX1 BX2 BX3 W4 Z1 INT BX1 BX2 BX3 32_4 191

INT BX1 BX2 BX3 W4 Z1 INT BX1 BX2 BX3 W4 Z1 32 5 192

INT BX1 BX2 BX3 W4 Z2 33_0 193

INT BX1 BX2 BX3 W4 Z2 INT 33_1 194

W4INT BX1 BX2 BX3 W4 Z2 INT BX1 33 2 195

Z2 INT BX1 BX2 BX3 W4 Z2 INT BX1 BX2 33 3 196

INT BX1 BX2 BX3 W4 Z2 INT BX1 BX2 BX3 33_4 197

INT BX1 BX2 BX3 W4 Z2 INT BX1 BX2 BX3 W4 Z2 33_5 198

INT BX1 BX2 BX3 W4 Z3 34_0 199

INT BX1 BX2 BX3 W4 Z3 INT 34 1 200

W4INT BX1 BX2 BX3 W4 Z3 INT BX1 34_2 201

Z3 INT BX1 BX2 BX3 W4 Z3 INT BX1 BX2 34_3 202

INT BX1 BX2 BX3 W4 Z3 INT BX1 BX2 BX3 34_4 203

INT BX1 BX2 BX3 W4 Z3 INT BX1 BX2 BX3 W4 Z3 34_5 204

INT BX1 BX2 BX3 W4 Z4 35_0 205

INT BX1 BX2 BX3 W4 Z4 INT 35 1 206

W4INT BX1 BX2 BX3 W4 Z4 INT BX1 35 2 207

Z4 INT BX1 BX2 BX3 W4 Z4 INT BX1 BX2 35 3 208

INT BX1 BX2 BX3 W4 Z4 INT BX1 BX2 BX3 35 4 209

INT BX1 BX2 BX3 W4 Z4 INT BX1 BX2 BX3 W4 Z4 35 5 210

132

•

3.3.5 Mixed Model Matrices and Parameters

..In practice, the assessment of collinearity in the mixed model context

must be carried out after model parameters are estimated because the

A.

assessment involves the estimated matrix E. This was the procedure used in

Chapter 2 for the initial exploration of the mixed model diagnostics. However,A.

fitting a model in order to produce the matrix E greatly complicates the

experiment because it requires a Y. Recall that calculation of the GLUM

diagnostics did not require a Y.

Initially, the following strategy was conceived to generate the Y, fit theA.

model using the Y and the simulated variables in X and Z, and obtain E:

1. Values for 11 and a2 would be selected and used to compute E. Recall

that

definite symmetric covariance matrix for the kth

subject, and

E (NxN) = Diag(E" E2, "., I.r.) is the covariance matrix of the

entire response vector, Y.

2. A value for P would be selected and values for dk, and ek would be

generated from appropriate normal distributions (N(O,I1) and N(O,u2I),

respectively) and used to compute Yk' Recall that

is a random vector of unobservable random subject

effects for the kth subject;

is an vector of unobservable within-subject random

133

3.

error terms; and

The Vk would be concatenated to obtain V. ..A- A- A-

4. The mixed models would be fit and values for Il, 4, 02, I: would be

obtained.A-

S. The collinearity diagnostics would be obtained using (X'r'X).

These steps would have to be repeated for~ of the 210 simulation runs.

After several trials and some reflection, it became apparent that although

A-

this procedure was working, a different V, and hence a different 4, was

generated for every run. The simulations call for changing the number of

A-

variables in Z, which then changes 4; both of these changes also impact I:.

Thus the effect on the diagnostics due to changes in Z would be confounded

with the effects due to changes in V, making it impossible to separate their

effects. So, one problem with this strategy was obtaining the V and another

was retaining the same V for every simulation run.

In order to reduce the complications of the initial approach, a more direct

strategy was devised:

1. A 4 and 02 were specified and used to compute ~:

A-

2. Then r', instead of r' was used to compute

K

(X' r -,Xl = L X: r;'XII + crVII .k-1

3. The collinearity diagnostics were obtained from the matrix

134

3.3.6 Values for 4, tr, and Vk

To implement the direct approach described above, values for 4, u2, and

Vk were chosen. With this approach, the matrix E changed necessarily with

each run, but not because Y changed; no Y was involved. It was because the

variables in Z changed. Selecting a subset of the columns of Z required the

selection of a corresponding submatrix of 4. This procedure ensured

comparability of results from one model (X, Z, 4) to another because the only

differences between models were essential differences. This approach was

implemented by specifying an overall 4, called 4(1), and selecting rows and

columns of the matrix that corresponded to the number of variables in Z. The

overall 4 was

1 0.6 0 0 0 00.6 1 o 0 0 0

4(1) = 0 0 100 0 (3.3.7)0 0 o 1 0 00 0 001 00 0 000 1

So, for example, when the first two variables (lNT, aX1) were in Z, 4 was the

(2 x 2) matrix

4 = [1 0.6]0.6 1

When all six variables were in Z, 4 =4(1).

(3.3.8)

This particular matrix for 4(1) was chosen because we were interested in

the situation in which all of the random effects in the model, except the first

135

two, were uncorrelated. We began with a simple case in which the variances

were all equal. For the same reason, the same values of rr =1 and V" = I".

were used in each calculation.

For the special case of no random effects, i.e., no variables in Z, the

computations were modified slightly to obtain the diagnostics. In this situation,

I = IN' These calculations were included as "baseline" runs in order to more

clearly assess the impact of adding variables to Z. In addition, the "baseline"

runs were more immediately comparable to the corresponding GLUM

simulations.

3.3.7 Results

The mixed model experiment in this chapter addresses several issues:

1) the behavior of the diagnostics for Data Series X1 {i} • [X Wi] i =0, , 4;

2) the behavior of the diagnostics for Data Series X2{j} .. [X Zj] j = 0, , 4;

3) the behavior of the diagnostics for Data Series X3{i,j} • [X Wi Zj] i,j = 0,

..., 4; and 4) the effect on the diagnostics of varying the number and nature of

the variables in the random effects matrix, Z.

Following the results of the GLUM diagnostics shown in Appendix 1, the

results for the mixed model are presented completely in Appendices 2-3.

Appendix 2 contains the baseline diagnostics (35 runs); Appendix 3 contains

the diagnostics for models with 1-6 variables in the random effects. Selected

results from the experiment are presented in detail in this section in a certain

pattern. For each of the X1{i}, X2{j} and X3{i,j} data series, the baseline

results are presented first and the results of adding variables to the random

effects are presented second.

136

3.3.7.1 Simple Dependency: Two Variables

Baseline

For the data series X1{i} • [X Wi]' i =0, ..., 4, recall that the variable

BX3 is related to Wi by (3.3.2). This is only contrived dependency in this

series. The results (shown in Appendix 2, Sets 1-5) of the baseline simulations

(no variables in Z) are nearly identical to those found for the GLUM simulations

(shown in Appendix 1, Sets 1-5). As found previously, there is one high

condition index and a large proportion of the variance of BX3 and the variances

of WO-W4 are associated with it. As the dependency between the two

variables increases, i.e., as Wi increases, the pattern of the relationship

becomes clearer. The one condition index indicating the dependency becomes

larger and the variance decomposition proportions of the two variables become

larger. The diagnostics clearly point to the variables involved in the

dependency.

Adding Variables to Random Effects

For the data series X1{i} - [X Wi]' i =0, ..., 4, the results of adding

variables to the random effects can be seen in Appendix 3 (Sets 1-5) and

graphically, in Figure 3.1. The baseline diagnostics are shown in row one

(NVARS = 0). As Wi increases, the condition index increases, reaching a value

of 633 for W 4 • The patterns in the~ indicating one, two, or three variables

in Z are similar to that in the baseline row, but the values are smaller.

However, when four or five variables are in the random effects, the condition

index remains about the same (1-3) as Wi increases.

137

o

Value of Condition Index for BX3 and Wiby Level of Wi and

Number of Variables in Random Effects

WI

COlUNEAR VARIABLES: BX3. WI and lXI, BX2, ZI

Figure 3.1 Condition Index for BX3 and Wi' by Number of Random EffectsVariables

The lowest dependency is shown in column one for WOo As the number

of variables in Z increases, the condition index decreases, falling to a value of

1 when five variables are in Z. For each Wi (column), the pattern is similar;

there is a reduction in the largest condition index as variables are added to Z.

However, the reduction becomes more dramatic as the dependency increases.

The most dramatic change occurs for W4; the condition index is reduced from

271, when three variables are in Z, to 3, when four or five variables are in Z.

There appears to be a relationship between the number and the nature

of the variables in Z and the indication of collinearity by these diagnostics.

138

When three variables are in the random effects, the variables are INT, BX1, and

BX2, none of which is involved in the constructed dependency. The fourth and

fifth variables added to Z are the two collinear variables, BX3 and Wi' Thus,

it appears that when one or both collinear variables are included in Z, the

dependency in X, that clearly remains, no longer appears to adversely impactA A

the asymptotic variance of P; i.e., V(JJ) is not ill-conditioned. The condition

index no longer indicates the presence of collinearity even though the variance

decomposition proportions (see Appendix 3, Sets 1-5) still point to the variables

involved in the dependency in X. According to Belsley (1991 ), both diagnostics

must indicate collinearity for the collinearity to be degrading.

3.3.7.2

Baseline

Simple Dependency: Three Variables

For the data series X2{j} • [X Zj]' j =0, ..., 4, recall that the variables

BX1 and BX2 are related to Zj by (3.3.3). This is only contrived dependency

in this series. The results (shown in Appendix 2, Sets 6-10) of the baseline

simulations (no variables in Z) are very similar to those found for the GLUM

simulations (shown in Appendix 1, Sets 6-10). As found previously, there is

one high condition index and a large proportion of the variances of BX1, BX2

and the variances of ZO-Z4 are associated with it. As the dependency among

the three variables increases, Le., as Zj increases, the pattern of the relationship

becomes clearer. The one condition index indicating the dependency becomes

larger and the variance decomposition proportions of the three variables

become larger. The diagnostics clearly point to the variables involved in the

139

dependency.

Adding Variables to Random Effects

For the data series X2{i} • [X ~], j =0, ..., 4, the results of adding

variables to the random effects can be seen in Appendix 3 (Sets 6-10) and

graphically, in Figure 3.2. The baseline diagnostics are shown in row one

Value of Condition Index for ex 1, BX2 and Ziby Level of Zi and

Number of Variables in Random Effects

ZI

COlUNEAR VAR1A81.£S: aX3, WI CIIId IXI. 1X2. ZI

Figure 3.2 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables

(NVARS =0). As Zj increases, the condition index increases reaching a value

of 640 for Z4' The patterns in the!2M indicating one to five variables in Z are

similar to that in the baseline row, but the values are smaller. When two or

more variables are in the random effects, the condition index values are

140

substantially smaller as ~ increases.

The lowest dependency is shown in column one for Zo. As the number

of variables in Z increases, the condition index decreases, falling to a value of

1 when five variables are in Z. For each ~ (column), the pattern is similar;

there is a reduction in the largest condition index as variables are added to Z.

However, the reduction becomes more dramatic as the dependency increases.

The most dramatic change occurs for Z.; the condition index is reduced from

271, when one variable is in Z, to 63, when two variables are in Z. When

three or more variables are in Z, the condition index decreases to 27 or less.

Again there appears to be a relationship between the number and the

nature of the variables in Z and the indication of collinearity by these

diagnostics. When only one variable, INT, is in the model, the dependency is

detectable for Z2 or tighter. When two variables are in the model, one of them,

BX1, is involved in the constructed dependency; the dependency is detectable

for Z3 or tighter. When the second collinear variable, BX2, is in Z along with

INT and BX1, the condition indexes are reduced further and detectable only at

Z.. There is not much further reduction, however, when the last collinear

variable Zj is included. For the tightest dependency, Z., but not for the weaker

dependencies, the presence of collinearity is still detectable when all five

variables are in the model. Again, except for the tightest dependency, it

appears that when collinear variables are included in Z, the dependency in X no

A

longer appears to adversely impact the asymptotic variance of p. The condition

index no longer indicates the presence of collinearity. However, the variance

decompositions (see Appendix 3, Sets 6-10) do point to at least two of the

141

involved variables when the condition indexes are as low as 2. When the

values of the condition index are at least 7, the variance decomposition

proportions behave as expected.

3.3.7.3

Baseline

Coexisting Dependency: Two Variables, Three Variables(Nonoverlapping)

The X3{i,j} series contains a combination of the two dependencies in

X1{i} and X2{j}; these are coexisting dependencies. The results of the baseline

simulations (no variables in Z) are again quite similar to those found for the

GLUM simulations. The 25 sets of diagnostics produced from this series for

the baseline runs are shown in Appendix 2 (Sets 11-35). The diagnostics are

summarized in Figures 3.3 (for the relationship between BX3 and Wi) and in

Figure 3.4 (for the relationship among BX1, BX2, and Zj). Selected sets are

discussed here.

In Appendix 2, Sets 21-25, the dependency in (3.3.2) is fixed (i=2) and

strong while the dependency in (3.3.3) varies V=O, ..., 4). These results can

be seen also in Figure 3.3 (plot of CI for BX3 and Wi) and in Figure 3.4 (plot of

CI for BX1, BX2, and ~); look at the W2 column in both. There are two high

condition indexes indicating the two dependencies, though for i = 0 the second

dependency is dominated by the first and is not yet apparent. The effects

become separable when i= 1 (W2 and Z,). Thus the procedure can clearly

detect two dependencies and the variables involved in each. When the two

dependencies are of nearly the same strength, their effects become

confounded. For X3{2,2} (W2 and Z2)' the condition indexes are of similar

142

Value of Condition Index for 8X3 and Wiby Level of Wi and Zj

No Variables are in Random Effects

o

WI

COLUHEAR VARIASUS: SX3. Wi and BX1, BX2. Zi

Figure 3.3 Condition Index for BX3 and Wi by Levels of Wi and Zj, NoRandom Effects Variables

magnitude (66 and 71) and the involvement of variables in the two

dependencies is somewhat obscured. However, it is possible to determine that

there are two dependencies present and that five variables are involved in

them. As the second dependency (BX1, BX2, Zj) becomes strong (at Z3)

relative to the first dependency (BX3, W2 ), the effects are again separable.

The X3{i,j} series can be examined from the other direction by holding

j constant and varying i. These results are shown in Appendix 2 in Sets 13,

18,23,28, and 33 for i=O, ..., 4 andj=2. These results can be seen also in

Figure 3.3 (plot of CI for BX3 and Wi) and in Figure 3.4 (plot of CI for BX1,

BX2, and Zj); look at the Z2 row in both. The dependency in 3.3.2 varies while

143

Value of Condition Index for 8X1 , 8X2 and Ziby Level of Wi and Zj

No Variables are in Random Effects

..

o

WI

COLLINEAR VARIAaL£S: aX3. Wi and BX1, BX2, Zj

Figure 3.4. Condition Index for BX1, BX2 and Zj by Levels of Wi and Zit NoRandom Effects Variables

the dependency in 3.3.3 is held constant. For ;=0, the dependency involving

BX3 and WO is relatively weak; the one high condition index and the variance

decomposition proportions indicate the strong dependency in BX1, BX2, and

Z2. When;= 1, the two dependencies are distinct. When;= 2, the two

dependencies are of similar strength and the variables involved in each are

somewhat obscured. However, when ;= 3, their separate identities emerge

again and remain distinct for ;=4. Again, the procedure can clearly detect two

dependencies and the variables involved in each.

Overall, the pattern for the dependency between BX3 and Wi is similar,

regardless of the level o'f Zj; the condition index increases from 9 at Wo to

144

nearly 700 at W•. Also as the Wi increases, the first dependency dominates

the second at lower levels of Zj. Similarly, the pattern for the dependency

between BX1, BX2, and ~ is the same, regardless of the level of Wi; the

condition index increases from 8-9 at Zo to about 700 at Z.. Also as the Zj

increases, the second dependency dominates the first at lower levels of Wi'

Adding Variables to Random Effects

For the data series X3{i,j} • [X Wi ~], i,j =0, ..., 4, the 25 sets (Sets

11-35) of diagnostics resulting from adding variables to the random effects can

be seen in Appendix 3. In Figures 3.5-3.9, the largest condition index

attributable to the dependency between BX3 and Wi is plotted by level of Wi

and number of variables included in the random effects, separately for each

level of Zj' In Figures 3.10-3.14, the largest condition index attributable to the

dependency between BX1, BX2, and ~ is plotted by level of Zj and number of

variables included in the random effects, separately for each level of Wi'

145

o

Value of Condition Index for BX3 and Wiby Ley.. of wr and

Number of Variables in Random Effectsat ZO

WI

Figure 3.5 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Zo

Value of Condition Index for BX3 end Wiby Lev.1 of WI and

Number of Variables In Random Effectsot Zl

o

WI

Figure 3.6 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z,

146

o

Value of Condition Index for BX3 and Wiby Lev" of WI and

Number of Variables in Random Effectsat Z2

Figure 3.7 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z2

Value of Condition Index for 8X3 and Wiby Lev•• of WI and

Number of Variables In Random EffectsCIt Z3

o

WI

Figure 3.8 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z3

147

o

Value of Condition Index for BX3 and Wiby Level of WI and

Number of Variables in Random EffectsalU

WI

Figure 3.9 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z4

Value of Condition Index for BX1. BX2 and Zjby Leve' of II and

Number of Variables In Random EffectsafWO

o

cow~V_liS: 113. WI _ 1I1.1IlCZ, Zl

Figure 3.10 Condition Index for aX1, aX2, and Zj by Number of RandomEffects Variables, at Wo

148

o

Value of Condition Index for ex 1. eX2 and Zjby Level of ZI and

Number Df Variables in RandDm Effects01 W1

..

Figure 3.11 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W 1

Value of Condition Index for eX1, eX2 and Zjby Level of ZI and

Number of Variables In Random EffectscllW2

..

o

Figure 3.12 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W 2

149

o

Value of Condition Index for ex 1, BX2 and Zjby Level of II and

Number of Variables in Random Eff.cts0lW3

zr

..

Figure 3.13 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W 3

Value of Condition Index for BX1, BX2 and Zjby Level of II and

Number of Variables In Random Effects01 W4

o

a1W_"_10: 10, WI _II Dl,lX2. ZI

Figure 3.14 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W 4

150

"

The fact that the two dependencies were designed to be nonoverlapping

is again confirmed by examining these figures. In Figures 3.5-3.9, the pattern

and values of the condition indexes are nearly the same in each plot, i. e.,

regardless of the level of~. Further, they are very similar to the values

obtained when this dependency was considered independently in the Xl {i}

series; this can be seen in Figure 3.1. Again, the addition of variables to the

random effects, especially collinear ones, diminishes the impact of the

collinearity in X. Thus, the effect of adding variables to the random effects

when the dependencies are coexisting is the same as the effect when the

dependencies are considered separately.

Similarly, in Figures 3.10-3.14, the pattern and values of the condition

indexes are nearly the same in each plot, i. e., regardless of the level of Wi'

Further, they are very similar to the values obtained when this dependency was

considered independently in the X2{j} series; this can be seen in Figure 3.2.

And again, the addition of variables to the random effects, especially collinear

ones, diminishes the impact of the collinearity in X. Once more, we conclude

that the effect of adding variables to the random effects when the

dependencies are coexisting is the same as the effect when the dependencies

are considered separately.

For the case when variables are added to the model composed of

coexisting dependencies, an interesting result is found regarding the competing

dependencies. In the baseline runs, it was found that when the two

dependencies were of nearly equal strength, e.g., for X3{2,2} (W2 and Z2),

their effects became confounded. The condition indices for the two

151

dependencies with no variables in the random effects were 66 for the

dependency between BX3 and W 2 and 71 for the dependency among BX1,

BX2, and Z2' However, when variables were added to the model, the

competition disappeared. This can be seen in Appendix 3, Set 23 and in Figure

3.7 (W2 column) and Figure 3.12 (Z2 column). When only one variable was in

Z, the competition was still present; the Cis were 29 and 31. However, when

two or three variables were in Z, there was only one high CI (27-28) indicating

the presence of the dependency between BX3 and Wi' Thus, it appears that

increasing the tightness of the dependency in ~ or adding variables to the

model diminishes the competition between the two dependencies. However,

the fact that the variables added are those involved in the dependency among

BX1, BX2, and Zj may confound this conclusion. As has been established,

when variables are added, the condition indexes decrease. For these particular

dependencies for fixed Wi and Zit the second one disappears faster than the

first as variables are added.

These same competing dependencies can now be viewed from the other

direction. An examination of Sets 13,18,23,28, and 33 (i=0, ..., 4) andj=2)

for a given number of variables in the model reveals that increasing Wi also

diminishes the competition. However, when two or more variables are in Z, the

competition disappears anyway due to the additional variables. Then we can

surmise that any of the three actions affects the diagnostics.

152

3.4 Effect of Adding Variables to Random Effects

As noted repeatedly, the collinearity in the fixed effects (X), that was

present in baseline runs, disappeared as more variables were added to the

random effects. It appeared that this was especially true when the variables

added were those involved in one of the two dependencies. To explore further

whether the disappearance was due to the number of variables added or their

nature, two sets of models were rerun varying the set of variables added to the

random effects Z; all possible combinations of variables were examined. This

was carried out only for the X1 {4} series and the X2{4} series since each had

the tightest dependencies and effects of changing variables in the random

effects might be more easily seen for them. The results for X1 {4}are

summarized in Table 3.6; the results for X2{4} are summarized in Table 3.7.

153

Table 3.6 Impact of Collinearity in Fixed Effects for Different Combinationsof Variables in Random Effects for X 1{4}

Fixed Effects: Variebles in X Random Effect.: Varieble. in Z Highest CI

INT BX1 BX2 BX3 W4 833

INT BX1 BX2 BX3 W4 INT 298

INT BX1 BX2 BX3 W4 BX1 361

INT BX1 BX2 BX3 W4 BX2 388

INT BX1 BX2 BX3 W4 BX3 3

INT BX1 BX2 BX3 W4 W4 3

INT BX1 BX2 BX3 W4 INT BX1 290

INT BX1 BX2 BX3 W4 INT BX2 278

INT BX1 BX2 BX3 W4 INT BX3 3

INT BX1 BX2 BX3 W4 INTW4 3

INT BX1 BX2 BX3 W4 BX1 BX2 298

INT BX1 BX2 BX3 W4 BX1 BX3 3

INT BX1 BX2 BX3 W4 BX1 W4 3

INT BX1 BX2 BX3 W4 BX2 BX3 3

INT BX1 BX2 BX3 W4 BX2W4 3

INT BX1 BX2 BX3 W4 BX3W4 3

INT BX1 BX2 BX3 W4 INT BX1 BX2 271

INT ax 1 aX2 aX3 W4 INT aX1 aX3 3

INT BX1 BX2 BX3 W4 INT BX1 W4 3

INT BX 1 BX2 BX3 W4 INT BX2 BX3 3

INT aX1 aX2 BX3 W4 INT aX2 W4 3

INT BX1 BX2 BX3 W4 INT BX3 W4 2

INT BX1 BX2 BX3 W4 BX1 BX2 BX3 3

INT BX1 BX2 BX3 W4 BX1 BX2 W4 3

INT BX1 BX2 BX3 W4 BX1 BX3 W4 2

INT ax1 BX2 BX3 W4 BX2 BX3 W4 2

INT BX1 BX2 BX3 W4 INT BX1 BX2 BX3 3

INT BX 1 BX2 BX3 W4 INT BX1 BX2 W4 3

INT BX1 BX2 BX3 W4 INT BX1 BX3 W4 2

INT BX 1 BX2 BX3 W4 INT BX2 BX3 W4 2

INT BX 1 BX2 BX3 W4 BX 1 BX2 BX3 W4 2

INT BX 1 BX2 BX3 W4 INT BX1 BX2 BX3 W4 2

154

Table 3.7 Impact of Collinearity in Fixed Effects for Different Combinationsof Variables in Random Effects for X2{4}

Fixed Effects: Veriabln in X Random Effects: Variables in Z Highest CI

INT BX1 BX2 BX3 Z4 640

INT BX1 BX2 BX3 Z4 INT 271

INT BX1 BX2 BX3 Z4 BX1 84

INT BX1 BX2 BX3 Z4 BX2 315

INT BX1 BX2 BX3 Z4 BX3 327

INT BX1 BX2 BX3 Z4 Z4 68

INT BX1 BX2 BX3 Z4 INT BX1 83

INT BX1 BX2 BX3 Z4 INT BX2 228

INT BX1 BX2 BX3 Z4 INT BX3 243

INT BX1 BX2 BX3 Z4 INTZ4 59

INT BX1 BX2 BX3 Z4 BX1 BX2 24

INT BX1 BX2 BX3 Z4 BX1 BX3 72

INT BX1 BX2 BX3 Z4 BX1 Z4 34

tNT BX1 BX2 BX3 Z4 BX2 BX3 242

INT BX1 BX2 BX3 Z4 BX2 Z4 29

INT BX1 BX2 BX3 Z4 BX3 Z4 81

INT BX1 BX2 BX3 Z4 INT BX1 BX2 27

INT BX 1 BX2 BX3 Z4 INT BX1 BX3 60

INT BX 1 BX2 BX3 Z4 INT BX1 Z4 38

INT BX 1 BX2 BX3 Z4 INT BX2 BX3 212

tNT BX 1 BX2 BX3 Z4 INT BX2 Z4 30

INT BX1 BX2 BX3 Z4 INT BX3 Z4 59

INT BX1 BX2 BX3 Z4 BX1 BX2 BX3 25

INT BX 1 BX2 BX3 Z4 BX1 BX2 Z4 18

INT BX1 BX2 BX3 Z4 BX1 BX3 Z4 34

INT BX 1 BX2 BX3 Z4 BX2 BX3 Z4 31

INT BX1 BX2 BX3 Z4 INT BX1 BX2 BX3 27

INT BX1 BX2 BX3 Z4 INT BX1 BX2 Z4 20

INT BX1 BX2 BX3 Z4 INT BX1 BX3 Z4 37

INT BX1 BX2 BX3 Z4 INT BX2 BX3 Z4 31

INT BX1 BX2 BX3 Z4 BX1 BX2 BX3 Z4 19

INT BX1 BX2 BX3 Z4 INT BX1 BX2 BX3 Z4 21

155

An examination of the highest condition indexes resulting from these

additional models reveals several patterns. For the case of X1 {4}, there are

two variables involved in the dependency, BX3 and W4. As shown in Table

3.6, the highest condition index (633) occurred when there were no variables

in the random effects; the lowest condition indexes (2-3) occurred when either

or both of these two variables were in the random effects. When neither of

these variables was present in Z, the values were very high (271-351), but

were reduced from the model without any variables in the random effects.

Thus, we conclude that for this two-variable dependency, adding any variables

to the random effects reduces the impact of collinearity. However, the impact

is still enormous. In contrast, adding collinear variables to the random effects

causes a huge reduction in the impact of the collinearity in the fixed effects.

For the case of X2{4}, there are three variables involved in the

dependency, BX1, BX2 and Z4. As shown in Table 3.7, the highest condition

index (640) occurred when there were no variables in the random effects; the

lowest condition index (18) occurred when all three ofthe variables (BX1, BX2,

Z4) were in the random effects without any other variables. Similar low values

(19-21) were found when these three variables occurred along with the

intercept, BX3 or both. In addition, when two of the three collinear variables

were in Z, with or without other uninvolved variables, the values ranged from

24-38. When only one of the three collinear variables was present in Z along

with an uninvolved variable, the values ranged from 59-72 if the variable was

BX1 or Z4; they ranged from 212-228 if the variable was BX2. For each

collinear variable alone in Z, values were 84 for BX1, 66 for Z4, and 31 5 for

156

aX2.

The conclusion for the X{2} series is not as clear. It appears that for this

three-variable dependency, adding any variables to the random effects also

reduces the impact of collinearity. However, as for the two-variable

dependency, the impact is still enormous. In contrast, adding collinearvariables

to the random effects causes a great reduction in the impact of the collinearity

in the fixed effects. For this dependency, however, the reduction is greatest

when all collinear variables are present in the random effects.

3.5 Summary of Results

The results in this chapter confirm that collinearity diagnostics behave

similarly for the GLUM and for the mixed model. The important new discovery

is that adding variables, especially collinear variables, to the random effects of

the mixed model can greatly attenuate the impact of the collinearity in the fixed

effects.

157

CHAPTER IV

IMPACT OF RANDOM EFFECTS COVARIANCE ON COLLINEARITY

4. 1 Introduction

The exploratory analyses in Chapter 2 revealed that changing the

A-

structure of the covariance matrix of the random effects, 4, affected the

collinearity diagnostics. This was determined by deliberately constraining one

A-

of the elements of 4 in the estimation procedure, fitting the same model

(except for the constraint on 4) twice, and examining the results. In Chapter

3, the effect of a changing 4 also was seen. However, the only way in which

the matrix 4 changed was a result of a change in the number of variables used

in the model for a given set. Even then, the elements of 4 corresponding to

any specific pair of Z-variables used did not change. Over sets containing the

same variables in Z, but at different degrees of dependency (e. g., Wi' i =0,

...4), the same covariance matrix A was used. The was done in order to be able

to attribute changes in effects of collinearity to the tightening dependency

alone, rather to than a changing covariance as well. Thus, the only changes

in 4 in the experiment reported in Chapter 3 were essential changes that

resulted from changing the variables in the random effects; they were not

changes in the basic structure of 4.

In order to truly determine the effect of a different covariance structure

on the diagnostics, the entire experiment described in Chapter 3 was repeated

using a completely different 4; the same X and Z data were used in both

experiments. Then the effect of the change in the structure of 4 can be seen

by contrasting corresponding models in the two experiments.

In this chapter, the results of replicating the experiment with a different

4 are reported. Since all other aspects of the second experiment are the same,

they are not described again here. Only the results and a comparison to those

in Chapter 3 are provided.

4.2 The Mixed Model Experiment

In order to investigate the effect of a different covariance structure for

4 on the collinearity in the fixed effects of the mixed model, a different overall

4, analogous to 4(1) defined in (3.3.7), was specified and called 4 121• Then for

the second experiment, the same steps described in Chapter 3 in section 3.3.5

were repeated using the same values of rr =1 and V. = In. ' but with

1 0.8 0.7 0.6 0.5 0.40.8 1 0.7 0.6 0.5 0.40.7 0.7 1 0.7 0.6 0.50.6 0.6 0.7 1 0.7 0.60.5 0.5 0.6 0.7 1 0.70.4 0.4 0.5 0.6 0.7 1

(4.2.1 )

The comparison of corresponding models in the two experiments reveals the

behavior of the diagnostics with a different 4. 4 121 was chosen to be very

different from 4 111 so that if effects required a drastic change in order to be

159

seen, they would be. The difference between the two matrices is in the

off-diagonal elements; there was only one nonzero off-diagonal element in AI1I,

whereas all off-diagonal elements of A(2) are nonzero and are fairly large.

4.3 Results

The results of the second mixed model experiment using the new A (AI2»)

are shown in Appendix 4. They are directly comparable to the results in

Appendix 3 that were obtained using the original A (AI1I). (See Table 3.5 for the

menu of models.) This presentation of results follows the order of Chapter 3,

except that the baseline results are not re-presented per se since they are the

same for both experiments.

4.3.1 Simple Dependency: Two Variables

4.3.1.1 Adding Variables to Random Effects

For the data series X1{i} !5 [X Wi]' i =0, .u, 4, the results of adding

variables to the random effects can be seen in Appendix 4 (Sets 1-5) and

graphically, in Figure 4.1 (Compare to Appendix 3 (Sets 1-5) and Figure 3.1).

The baseline diagnostics are shown in row one; this row is the same in both

figures. The patterns found with A(2) are the same are those found previously

for A(1). As Wi increases, the condition index increases, reaching a value of

633 for W4 • The patterns in the !QM indicating one, two, or three variables

in Z are similar to that in the baseline row, but again, the values are smaller.

As before, when four or five variables are in the random effects, the condition

index remains about the same (3-5) as Wi increases.

160

..

Value of Condition Index for BX3 and Wiby Level of Wi and

Number of Variables in Random Effects

WI

COlUNEAR VAR....BLES: aX3, WI CII'Id BlCI. BX2. ZI

Figure 4.1 Condition Index for BX3 and Wi by Number of Random EffectsVariables

The lowest dependency is shown in column one for WOo As the number

of variables in Z increases, the condition index decreases, falling to a value of

3 when five variables are in Z. For each Wi (column), the pattern is similar;

there is a reduction in the largest condition index as variables are added to Z.

However, the reduction becomes more dramatic as the dependency increases.

The most dramatic change occurs for W4 ; the condition index is reduced from

277, when three variables are in Z, to 3 and 5, when four or five variables are

in Z.

Again, there appears to be a relationship between the number and the

161

nature of the variables in Z and the indication of collinearity by these

diagnostics. As found for 411), when variables involved in the constructed

dependency are included in the random effects, the impact of collinearity is

diminished.

4.3.1.2 Comparison to Experiment 1

A comparison of Figure 3.1 and Figure 4.1 reveals that for the X1 {i}

series the patterns found for 4 12) are the same as those found for 411). There

is a slight difference in the magnitude of the values; those in Figure 4.1 for

4 12) are somewhat higher. The higher values occur mainly after the introduction

of the collinear variables and at the tightest dependency, W 4 • However, the

differences are negligible given the magnitude of the condition indexes; i.e.,

there is very little actual difference between the value of 271 for three variables

in random effects in Figure 3.1 and the corresponding value of 277 in Figure

4.1.

4.3.2 Simple Dependency: Three Variables

4.3.2.1 Adding Variables to Random Effects

For the data series X2{i} !II! [X ~], j = 0, ..., 4, the results of adding

variables to the random effects can be seen in Appendix 4 (Sets 6-10) and

graphically, in Figure 4.2 (Compare to Appendix 3 (Sets 6-10) and Figure 3.2).

The baseline diagnostics are shown in row one. The patterns found with 4 12)

are the same found previously for 411). As Zj increases, the condition index

increases reaching a value of 640 for Z4' The patterns in the~ indicating

one to five variables in Z are similar to that in the baseline row, but the values

162

Value of Condition Index for BX 1, BX2 and Ziby Level of Zi and

Number of Voriables in Random Effects

o

Z\

COlUNEAR VARIABl£S: BX3. WI CNld BXl. BX2. ZI

Figure 4.2 Condition Index for aX1, aX2, and Zj by Number of RandomEffects Variables

are smaller. When two or more variables are in the random effects, the

condition index values are substantially smaller as Zj increases.

The lowest dependency is shown in column one for Zo. As the number

of variables in Z increases, the condition index decreases, falling to a value of

3 when five variables are in Z. For each Zj (column), the pattern is similar;

there is a reduction in the largest condition index as variables are added to Z.

However, the reduction becomes more dramatic as the dependency increases.

The most dramatic change occurs for Z4; the condition index is reduced from

271, when one variable is in Z, to 66, when two variables are in Z. When

163

three or more variables are in Z, the condition indexes range from 30 to 37.

Again there appears to be a relationship between the number and the

nature of the variables in Z and the indication of collinearity by these

diagnostics. As was found for 4(1), when variables involved in the constructed

dependency are included in the random effects, the impact of collinearity is

attenuated.

4.3.2.2 Comparison to Experiment 1

A comparison of Figure 3.2 and Figure 4.2 reveals that for the X2{j}

series the patterns found for 4 121 are the same as those found for 4(1). There

is a slight difference in the magnitude of the condition indexes; those in Figure

4.2 for 4 121 are somewhat higher. The higher values occur mainly after the

introduction of the collinear variables and at the tightest dependencies. For Z4'

the condition indexes are larger for the second experiment when three or more

variables are in the random effects. For 4 121 values range from 30-36;

corresponding values for 4(1) range from 21-27. The differences are not great

enough to change any conclusions however.

4.3.3

4.3.3.1

Coexisting Dependency: Two Variables, Three Variables(Nonoverlapping)

Adding Variables to Random Effects

For the data series X3{i,j} iii [X Wi ~], i,j =0, ..., 4, the 25 sets (Sets

11-35) of diagnostics resulting from adding variables to the random effects can

be seen in Appendix 4. In Figures 4.3-4.7 (Compare to Figures 3.5-3.9.), the

largest condition index attributable to the dependency between BX3 and Wi is

plotted by level of Wi and number of variables included in the random effects,

164

..

..

separately for each level of~. In Figures 4.8-4.12 (Compare to Figures 3.10­

3.14.), the largest condition index attributable to the dependency between

BX1, BX2, and ~ is plotted by level of Zj and number of variables included in

the random effects, separately for each level of Wi'

165

o

Value of Condition Index for BX3 and Wiby Lev" of WT and

Number of Variables in Random Effectsat ZO

WI

Figure 4.3 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Zo

Value of Condition Index for 8X3 and Wiby Lev.1 of WT Gnd

Number of Variables In Random Effects<It Zl

..

o

WI

..

Figure 4.4 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z,

166

o

Value of Condition Index for BX3 and Wiby Ley.. of WI and

Number of Variables in Random Effectsat Z2

Figure 4.5 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z2

Value of Condition Index for BX3 and Wiby Level of WT and

Number of Variables Tn Random Effectsat Z3

o

WI

Figure 4.6 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z3

167

o

Value of Condition Index for BX3 and WIby Level of WT and

Number of Variables in Random Effec:tsatU

Figure 4.7 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z4

Value of Condition Index for BX1. 8X2 and Zjby Level of II and

Number of Vanables In Random Effecisat WO

o

Z1

CCWNrMl VAIIAlIl.IS: 113. WI _ 111. 112. ZI

Figure 4.8 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at Wo

168

o

Value of Condition Index for eX1, eX2 and Zjby Level of ZI and

Number of Variables in Random Effectsaf W1

ClllalI:AIl V.wAIIIS, 1ltJ. WI _ lin. IlC2, Zl

Figure 4.9 Condition Index for aX1, aX2, and Zj by Number of RandomEffects Variables, at W 1

Velue of Condition Index for ex 1, eX2 end Zjby Level of ZI and

Number of Variables Tn Random EffectsatW2

o

Z1

COWIIL\Ilv-.s: 1ltJ. WI _ 1111. Ill2, Zl

Figure 4.10 Condition Index for aX1, aX2, and Zj by Number of RandomEffects Variables, at W 2

169

o

Value of Condition Index for 8X1, 8X2 and Zjby Leve' of ZI Clnd

Number of Variables in Random Effectsat W3

zr

CllllaIrAIl VAllAlllS, 113. WI _ IXl. IX2. Zl

Figure 4.1 1 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W3

Value of Condition Index for 8X1. 8X2 and Zjby Level of ZI Clnd

Number of Variables In Random EffectsCIt W4

o

Z1

Clll.lJIlfM V_LD, 113. WI _ IXI. ilia, ZI

Figure 4.12 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W4

170

The patterns in these plots are similar to those found previously for 4 111•

The nonoverlapping nature of the dependencies is again confirmed. In Figures

4.3-4.7, the pattern and values of the condition indexes are nearly the same in

each plot, i. e., regardless of the level of~. Further, they are very similar to

the values obtained when this dependency was considered independently in the

X1 {i} series; this can be seen in Figure 4.1. Again, the addition of variables to

the random effects, especially collinear ones, diminishes the impact of the

collinearity in X. Thus, the effect of adding variables to the random effects

when the dependencies are coexisting is the same as the effect when the

dependencies are considered separately.

Similarly, in Figures 4.8-4.12, the pattern and values of the condition

indexes are nearly the same in each plot, i. e., regardless of the level of Wi'

Further, they are very similar to the values obtained when this dependency was

considered independently in the X2{j} series; this can be seen in Figure 4.2.

And again, the addition of variables to the random effects, especially collinear

ones, diminishes the impact of the collinearity in X. Once more, we conclude

that the effect of adding variables to the random effects when the

dependencies are coexisting is the same as the effect when the dependencies

are considered separately.

4.3.3.2 Comparison to Experiment 1

A comparison of Figures 4.3-4.7 and Figures 3.5-3.9 reveals that for the

X3{i,j} series the patterns found for 4 (2) are again similar to those for 4 111 • For

both dependencies, the condition indexes for 4 (2) are slightly higher than for 4 111

171

when at least two variables are in the random effects. The discrepancies

between the series are greater for the tighter dependencies (W. and Z.).

Figures 3.9 and 4.7 give the condition indexes for the dependency

between BX3 and Wi at Z. for the two experiments. For Wo, there is very little

difference in the results; the CI decreases from 9 to 1 in Figure 3.9 and from

9 to 4 in Figure 4.7. For W., the CI decreases from 691 to 3 in Figure 3.9 and

from 691 to 4 in Figure 4.7. The main difference is for W3 and W. when two

or three variables are in the model; the Cis in Figure 4.7 are from 3 to 17 units

higher than those in Figure 3.9. However, none of these differences is large

enough to change any conclusions about the collinearity.

Figures 3.14 and 4.12 give the condition indexes for the dependency

between BX1, BX2, and ~ at W. for the two experiments. For lo, there is very

little difference in the results; the CI decreases from 9 to 1 in Figure 3.14 and

from 9 to 4 in Figure 4.12. For Z., the CI decreases from 703 to 21 in Figure

3.14 and from 703 to 34 in Figure 4.12. The main differences is for Z3 and Z.

when two or more variables are in the model; the Cis in Figure 4.12 are from

1 to 13 units higher than those in Figure 3.14. Again, none of these

differences is large enough to change any conclusions about the collinearity.

4.4 Summary of Results

The impetus for changing the covariance structure of A and repeating the

experiment came from the exploratory analyses in Chapter 2. Recall that actual

A

data were analyzed and that A was deliberately constrained. When either one

172

..

or all off-diagonal elements were set to zero, the condition index was reduced

dramatically. However, it was still high enough to indicate collinearity and the

variance deomposition proportions still pointed to the involved variables. InA

Chapter 2 with the actual data, 4 was estimated. In addition, all elements ofA

4 changed for any constraint imposed, but in unpredictable ways. Thus, when

the structure of 4 was changed and models were compared, it was not a pure

comparison. Because the same V was used, at least one of the models was

misspecified. However, the exploratory research reported in Chapter 2 did

determine that changing the covariance structure impacted the effect of

collinearity.

A major goal in this chapter was to compare "models" that were identical

except for their covariance structure 4, i.e., to compare models reported in this

chapter to those in Chapter 3. This was possible using the direct approach that

specified the required parameters with only 4 changed and did not involve a

dependent variable Y. In this case, the result was that the collinearity

diagnostics for the new covariance structure 4 121 are virtually identical to those

found for 4 111 •

173

CHAPTER V

DATA EXAMPLES OF THE IMPACT OF RANDOM EFFECTSON COLLINEARITV

5.1 Introduction

Analyses in Chapters 3 and 4 determined the behavior of the collinearity

diagnostics under controlled conditions. The advantage of this approach was

that effects of the contrived dependencies could be readily seen without their

being contaminated by changes in other parameters (A, I) of the mixed model,

parameters that would actually be expected to change when actual data are

analyzed. While this approach provided the information needed to clearly

assess the dependencies themselves, it nevertheless, by design, did not provide

a complete inquiry. The purpose of the analyses in this chapter is to combine

the knowledge gained from the previous inquires with an analysis of actual

data. In Chapter 2, components of the mixed model thought to impact

collinearity were explored using actual data. Now we revisit those same data

equipped with additional information.

The specific goal of the analyses is to investigate the impact of varying

the number and nature of the random effects in a model, while retaining the

same fixed effects. This was undertaken in two basic inquiries. First, using a

subset of the variables in the data set and a deliberately derived collinear

..

variable, a dependency was created in the spirit of the dependency of the data

series X1 of the experimental data, Le., one simple dependency involving two

variables. Second, by using another subset of the data and the deliberately

derived collinear variable, two overlapping dependencies were created. Recall

that the two dependencies of the X2 experimental series were non-overlapping.

The purpose of this was to determine whether the techniques used for simple

dependencies would hold in this new case; this is new territory that was not

explored previously in the experiment.

There are several advantages to fitting models and using actual data in

these analyses. Since the complete dynamics of the mixed model are

operating, the other components of the mixed model that may be impacted by

(or may impact) collinearity can be examined. These include changes in J1 and

A A

changes in IJ as well as the impact on V(JJ), as indicated by the diagnostics.

Further, the time to convergence of the model may be seen under different

conditions.

The only constraints remaining in the investigation as a whole are those

naturally associated with an examination of only one data set. 1) This data set

is unique with respect to the particular level of collinearity present. 2) Only

three basic continuous variables are in the model. 3) When the artificial

dependency is used, the data contain the basic underlying collinearity of the

three variables in addition to that of the created dependency. This is in

contrast to the experimental data of Chapter 3 and 4; since they were

generated randomly, there was no underlying collinearity. 4) Since J1 is

modelled from the data, it is an estimate and one that changes as variables in

175

the random effects are changed. This is in contrast to the fixed 4 used in the

experimental data. (However, both estimated and fixed 4's are explored in this

chapter.) 5) The number of subjects and the number of observations per

subject are specific to this data set. While these constraints are enumerated

as qualifications for the results, they are also reminders of the breadth of this

problem. No data set can serve all inquiries. Having previously examined the

behavior of this general type of data experimentally, we now determine

whether those findings apply to actual data of the same type.

5.2 The Data

The data for the examples in this chapter are the same as those used in

Chapter 2; see Section 2.7.1 for a full description. Recall that the dependent

variable was forced vital capacity (FVC) and that age, height, and weight were

independent variables. Because there is some question as to whether a straight

line is the best fit for these data, the square of height used in these analyses

as well. This increases the collinearity in the data.

5.3 Example 1: One Simple Dependency

In the experimental analyses of Chapters 3 and 4, two of the factors

determined to impact the diagnostics were 1) the number and nature of

variables in Z and 2) the structure of 4. It is of particular interest here to

determine whether, in actual data, the collinearity in the fixed effects

disappears as more variables are added to the random effects. The derived

176

variable, height2, was used in models along with the intercept and height.

Models were fit in which all three variables were retained as fixed effects and

all combinations of variables were used as random effects. In the estimation

process, the starting values for 4 for each pair of variables in Z were the same

for each run, but final values are not necessarily the same because they are

estimated. From model to model, 4 always changes as Z changes, but we are

not deliberately constraining 4.

For this experiment with the actual data, seven models were fit and are

summarized in Table 5.1. When only the intercept is in the random effects

component of the model, the impact of the collinearity in the fixed effects is

strong; the highest condition index is 164. When either collinear variable

(height or heighe) is in the random effects, the condition index is greatly

reduced. When they are included as either as single variables or as the only

variable along with the intercept, the highest condition index ranges from 47­

52. When they are both included with or without the intercept, the condition

index is 78. Thus, we see that for a simple dependency involving two

variables, the pattern observed in the experimental data is only partially borne

out in the actual data. We would have expected the least collinearity when all

variables are in the random effects. The number of iterations and time to

convergence is also shown in Table 5.1; these appear to be positively

correlated with the degree of collinearity present in the models.

177

Table 5.1 Impact of Collinearity in Fixed Effects for Different Combinationsof Variables in Random Effects for Example 1

FIxed Effec:ta: v..... In X AMdom Effec:ta: v..... In Z High.. ItenltioneCI (Mlnutu)

INTERCEPT HEIGHT HEIGHT2 INTERCEPT 184 179(113)

INTERCEPT HEIGHT HEIGHT2 HEIGHT 47 18(18)

INTERCEPT HEIGHT HEIGHT2 HEIGHT2 61 18(12)

INTERCEPT HEIGHT HEIGHT2 INTERCEPT HEIGHT 48 97(74)

INTERCEPT HEIGHT HEIGHT2 INTERCEPT HEIGHT2 62 136(93)

INTERCEPT HEIGHT HEIGHT2 HEIGHT HEIGHT2 78 37, (29)

INTERCEPT HEIGHT HEIGHT2 INTERCEPT HEIGHT HEIGHT2 78 106(80)

While the objective of this analysis was to compare the behavior of the

diagnostics in actual data to those of the experiment, the results were

somewhat surprising. In order to put the actual data in the context of the

experiment, another approach was taken. Constraints were put on A and the

analyses just described was repeated; Le., the number of variables in Z was

varied. In this case the normal dynamics of the model are not allowed to

operate. Comparing the results of the two approaches permits one to see the

extent to which constraining A in actual data changes the impact of

collinearity.

For this approach, collinearity diagnostics were computed using the

procedures of Chapters 3 and 4, as described in Section 3.3.5. Recall that no

dependent variable was involved and that values for A, 02 and Vk were

A

specified and used to compute ~ = Zk A Zk' +o2vk. Then r', instead of r' was

178

K

used to compute (X'I -'X) = Ex: I;'X. + u2V.*-,

diagnostics were obtained from the matrix (X'F'X).

Finally, the collinearity

To implement the approach described above for this data example, the

values for 4 and 0'- used were the actual estimates obtained from the model in

which all three independent variables (intercept, height, and height2) were in the

random effects. When subsets of the variables were in Z, the corresponding

submatrices of the 4 were used. Thus, as with the experimental data, the

variation due to having a dependent variable and fitting a model is removed.

Using this approach, the collinearity diagnostics for the same seven

models were computed; results are summarized in Table 5.2 along with the

results produced by actual fitting of models. The estimated 4 and the

"constrained" 4 are necessarily the same for the case when all variables are in

the random effects. Thus, the cOllinearity for those models is the same

(CI = 78). Since the submatrices of 4 for the models with height and height2

are so similar to the 4 of the full model, the collinearity for those models also

is approximately the same (CI = 78). The condition index for the models with

only the intercept in the random effects is also the same for both the

constrained and estimated 4 and quite high (CI =164). However, when height

or height2 was included singly or with the intercept, the condition indexes for

the constrained models are much lower. These findings are similar to those

found for the experimental data for data series X1. Lack of reduction in all

models is probably due to the fact that the "constrained" matrix used is the

same as that for the actual 4 for the case when all variables are in the random

179

Table 5.2 Impact of Constraining Covariance of Random Effects 4 for Model with Intercept, Height, and Height2 inFixed Effects

Z Actual Coveri--=- CI. carwtr.ined Covariance CI.

INT 0.0000012154 164 0.000000715 164

HT 0.000001433 47 0.0000174 13

HT2 0.0000000000957 51 o.0000000013846 21

INT 0.00000074779 -0.0000002171 48 0.000000715 0.000000052049 13HT -0.0000002171 0.0000014259 0.000000052049 0.0000174

INT 0.00000083244 -0.000000003325 52 0.000000715 -0.000000000537 21HT 2 -0.000000003325 0.00000000009508 -0.000000000537 0.0000000013846

HT 0.0000172 -0.0000001478 78 0.0000174 -0.00000015 78HT 2 -0.0000001478 0.00000000136 -0.00000015 0.0000000013846

INT 0.000000715 0.000000052049 -0.000000000537 78 0.000000715 0.00000oo52049 -O. OOOOOOOOO537 78HT 0.000000052049 0.0000174 -0.00000015 0.000000052049 0.0000174 -0.00000015

HT 2 -0.000000000537 -0.00000015 0.0000000013846 -0.000000000537 -0.00000015 0.0000000013846

..

..

effects. In this case, it is not possible to show the expected reduction.

5.4 Example 2: Overlapping Dependencies

One type of dependency not explored in the experiments of Chapters 3

and 4 was that of overlapping dependencies. Recall that when two

dependencies occurred together in the experimental data, they were coexisting

(non-overlapping); this data set does not have non-overlapping dependencies.

Thus, the heretofore unexplored type of dependency in the mixed model was

examined in a manner similar to that for Analysis 1 above. In this analysis, the

new variable HEIGHT2 was used in models along with the intercept, age and

height. Models were fit in which all four variables were retained as fixed

effects and all combinations of variables were used as random effects. As for

Example 1, the starting values for Ii. for each pair of variables in Z were the

same for each run, but final values are not necessarily the same, due to the

estimation process.

For this second analysis, 15 models were fit and are summarized in Table

5.3. For each model, the degree of collinearity present is about the same.

There appear to be two overlapping dependencies, probably involving all the

variables. Thus, there was only slight or no reduction of collinearity when

subsets of variables were included in the random effects component of the

model. It is not known whether there would be reduction if additional

noncollinear variables were also in the model.

181

Table 5.3 Impact of Collinearity in Fixed Effects for Different Combinationsof Variables in Random Effects for Example 2

Fixed Effects: v...... In X ......... Effects: v...... InZ High.. h.......CI. (MInut..)

INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT 18,66 136(82)

INTERCEPT AGE HEIGHT HEIGHT2 AGE 18,81 8(81

INTERCEPT AGE HEIGHT HEIGHT2 HEIGHT 16,49 18(141

INTERCEPT AGE HEIGHT HEIGHT2 HEIGHT2 16,64 8(81

INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT AGE 18,81 121(87)

INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT HEIGHT 16,49 101(79)

INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT HEIGHT2 16,63 118(88)

INTERCEPT AGE HEIGHT HEIGHT2 AGE HEIGHT 18, 71 18(14)

INTERCEPT AGE HEIGHT HEIGHT2 AGE HEIGHT2 20, 66 12(12)

INTERCEPT AGE HEIGHT HEIGHT2 HEIGHT HEIGHT2 16, 71 22(11 )

INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT AGE HEIGHT 16, 71 94(76)

INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT AGE HEIGHT2 20, 68 109(841

INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT HEIGHT HEIGHT2 16, 71 86(71)

INTERCEPT AGE HEIGHT HEIGHT2 AGE HEIGHT HEIGHT2 18, 74 86(721

INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT AGE HEIGHT HEIGHT2 18, 76 78(68)

The procedure used in Example 1 to constrain 11 and determine its effect

was repeated for Example 2. For this example, values for 11 and if- were the

actual estimates obtained from the model in which all four independent

variables (intercept, age, height, and height2) were in the random effects.

When subsets of the variables were in Z, the corresponding submatrices of the

182

A were used. Thus, as with the experimental data, the variation due to having

a dependent variable and fitting a model is removed.

Using this approach, collinearity diagnostics for the same 15 models

were computed; results are summarized in Table 5.4 along with the results

produced by actual fitting of models. As before, the estimated A and the

"constrained" A are necessarily the same for the case when all variables are in

the random effects. Thus, the collinearity for those models is the same

(Cis = 18 and 75) (The computation for the constrained A produced Cis of 19

and 75). Also, since the submatrices for the models with age, height, and

heighe are so similar to that of the matrix for the full model, the collinearity for

those models also is similar. In contrast to the previous experiment, the Cis for

the model with only the intercept in the random effects, were much higher than

those based on the estimated A, 32 and 188 compared to 18 and 55. For all

other combinations of variables in the random effects, both Cis based on the

"constrained" A were greatly reduced compared to those based on the

estimated A. In fact, only one dependency is indicated. This finding is

intriguing since it is similar to the results found in the experiments for the non­

overlapping dependencies. Perhaps further research will confirm that the

cancellation of collinearity found in the experimental data holds for overlapping

dependencies as well.

183

Table 5.4 Impact of Constraining Covariance of Random Effects 4 for Model with Intercept, Age, Height, and Height2 in Fixed Effects

Z Actuel COv8rtance Cia Conatr8tned COv8rtence Cia

INT 0.0235966 18 0.0000026843 3255 188

AGE 0.0005593 16 0.0046687 861 37

HT 0.0000016193 15 0.0000069018 949 22

HT2 0.0000000001091 15 0.0000000009589 654 24

INT 0.0000029204 -0.000008384 16 0.0000026843 0.0000105 8AGE -0.000008384 0.0005643 61 0.0000105 0.0046687 37

INT 0.0000029466 -0.0000006566 15 0.0000026843 -0.000000005879 9HT -0.0000006566 0.0000016382 49 -0.000000005879 0.0000069018 22

INT 0.000003105 -0.000000006503 15 0.0000026843 -0.000000005353 6Hy2 -0.000000006503 0.0000000001109 53 -0.000000005353 0.0000000009589 24

AGE 0.0025303 -0.000118 16 0.0046687 -0.000073 7HT -0.000118 0.0000071368 71 -0.000073 0.0000069018 34

AGE 0.0119744 -0.000004829 20 0.0046687 -0.000001416 8Hy2 -0.000004829 0.0000000020548 66 -0.000001416 0.0000000009589 35

HT 0.0000174 -0.0000001319 16 0.0000069018 -0.0000000272 6HT2 -0.0000001319 0.0000000011142 71 -0.0000000272 0.0000000009589 23

INT 0.0000026592 0.0000034256 -0.0000002325 16 0.0000026843 0.0000105 -0.000000005879 7AGE 0.0000034256 0.0025261 -0.000118 71 0.0000105 0.0046687 ·0.000073 34HT -0.0000002325 -0.000118 0.00000715 -0.000000005879 -0.000073 0.0000069018

INT 0.0000027295 -0.000014 0.0000000031676 20 0.0000026843 0.0000105 -0.000000005353 8AGE -0.000014 0.0119576 -0.000004825 66 0.0000105 0.0046687 -0.000001416 35Hy2 0.0000000031676 -0.000004825 0.0000000020545 -0.000000005353 -0.000001416 0.0000000009589

INT 0.0000026684 0.0000003939 -0.000000004052 16 0.0000026843 -0.000000005879 -0.000000005353 6HT 0.0000003939 0.0000175 -0.0000001329 71 -0.000000005879 0.0000069018 -0.0000000272 23Hy2 -0.000000004052 -0.0000001329 0.0000000011223 -0.000000005353 -0.0000000272 0.0000000009589

AGE 0.0046454 -0.000074 -0.000001394 18 0.0046687 -0.000073 -0.000001416 19HT -0.000074 0.000006831 -0.0000000261 74 -0.000073 0.0000069018 -0.0000000272 75HT2 -0.000001394 -0.0000000261 0.0000000009403 -0.000001416 -0.0000000272 0.0000000009589

INT 0.0000026843 0.0000105 -0.000000005879 -0.000000005353 18 0.0000026843 0.0000105 -0.000000005879 -0.000000005353 19AGE 0.0000105 0.0046687 -0.000073 -0.000001416 75 0.0000105 0.0046687 -0.000073 -0.000001416 75HY -0.000000005879 -0.000073 0.0000069018 -0.0000000272 -0.000000005879 -0.000073 0.0000069018 -0.0000000272HT2 -0.000000005353 -0.000001416 -0.0000000272 0.0000000009589 -0.000000005353 -0.000001416 -0.0000000272 0.0000000009589

,,.

5.5 Summary

In actual data models, the impact of collinearity in the fixed effects was

diminished when variables were added to the random effects. However the

patterns were not always the same as those seen previously in the

experimental data of Chapters 3 and 4. There are several reasons for the

departure of these findings from those found with the experimental data. The

first reason stems from the differing types of collinearity in the artificial and real

data. The basic variables in the experimental data set were free of underlying

collinearity; one dependency was created and five variables were in the fixed

effects of each model. Only two or three variables were collinear. In contrast,

in the actual data set, the basic variables appeared to be involved in one or

more dependencies; one additional dependency was created and three or four

variables were in the fixed effects for these models. So all variables in a model

were collinear. Thus, even though the dependencies in the actual data were

similar to those in the experimental data, the actual models were different with

respect to the numbers of variables included and the levels of collinearity

present.

Another difference between the experimental models and the actual

models was the structure of the covariance matrix A. The covariance in the

experimental models had equal diagonal elements and only two nonzero off­

diagonal elements. In contrast, the covariance in actual data does not have

equal diagonal elements and the off-diagonal elements are not only not zero,

but are a mixture of positive and negative values.

185

A third reason for the differences found is related to the estimation

process. The experimental ·models· were not estimated, i.e., a dependent

variable was not involved and the same pair-wise elements of the covariance

were used for each model. In contrast, parameters for the actual data were

estimated; 4 was different for each model. The ·constrained· 4 subsets

examined the extent to which this was an issue. Thus, the dynamics of the

estimation process and the additional variability of the dependent variable

obscure the patterning in the actual data. However, it was determined that for

a dependency similar to that found in the experiment, adding collinear variables

to the random effects does reduce the impact the collinearity, but not as much

as was found in the experiments. Further research is needed to clarify the

findings for the overlapping dependencies, though the diagnostics tended to

behave in a manner similar to those for the simple dependencies examined.

186

CHAPTER VI

EXAMPLE MIXED MODEL DATA ANALYSISUSING COLLINEARITY DIAGNOSTICS

6.1 Introduction

Until now, this dissertation has focused on determining the behavior of

the diagnostics in the presence of several types of dependencies. Performance

of the diagnostics has been characterized extensively for two types of

dependencies (simple, two and three variables involved and coexisting, five

variables involved) in experimental data. Performance of the diagnostics in

actual data has been examined for a simple dependency (three variables

involved) similar to the simple dependency in the experimental data and for an

overlapping dependency (four variables involved). It has been demonstrated

that the diagnostics can be used confidently to detect the presence of

collinearity in the mixed model context. Even though the particular impact of

adding variables to the random effects may be specific to a given data set, the

general pattern of behavior has been established in both experimental and

actual data.

The purpose of this chapter to suggest and illustrate a strategy for

incorporating the diagnostics into an overall approach for model fitting. These

analyses go beyond the diagnostic behavior to demonstrate a realistic data

analysis in which the collinearity diagnostics are included in the process of

refining the model.

6.2 Model Fitting Strategy

A general strategy for mixed model fitting has been proposed previously

by Helms (1993). That strategy is expanded here by using the collinearity

diagnostics in decisions about variable inclusion and deletion. The steps

involved in the procedure, which are described below, begin with variable

selection for the fixed effects. This is because an error in specifying the

covariance structure has relatively little impact on the fixed effects while the

reverse is not true.

Step 1

a) Put all variables of interest in the fixed effects.

b) Assume a simplified covariance structure for 4, e.g., only have random

subject intercepts as a random effect.

c) Fine tune the fixed effects. Delete nonsignificant variables, keeping the

significance level liberal (0.10 or 0.15). Alternatively, delete highly

collinear variables. (Recall that collinearity shows up best when only the

intercept is in the random effects of the model.)

Step 2

a) Use the fixed effects decided on from Step 1.

b) Fine tune the random effects. Add variables one-by-one to the random

effects. Take the difference in -2 log likelihood between two models

188

with the same fixed effects; this indicates the significance of the

additional parameters in the covariance 4. The -2 log likelihood ratio has

approximately a Chi-square distribution with degrees of freedom equal

to the difference between the number of parameters in the two models.

(This is not formal hypothesis testing, but rather is used as a guide to

variable selection.)

Step 3

a) Use the random effects from Step 2.

b) Fine tune the fixed effects. Consider deleting variables that have

become nonsignificant in Step 2 or those that contribute greatly to

collinearity. Also consider inclusion of additional variables, as logic

suggests.

Step 4

a) Compare results from Steps 1 and 3.

b) If there is a big difference in conclusions, repeat Steps 2 and 3, with

obvious modifications.

6.3 Model Fitting Example

Once again, the data for this example are the same as those used in

Chapter 2; see Section 2.7.1 for a full description. Recall that the dependent

variable was forced vital capacity (FVC) and that age, height, and weight were

independent variables in both the fixed and random effects. The additional

variable height2 also was used for this example. The strategy described above

189

was applied to these data. Each step in the process is described in detail;

tables giving the details of· each model are provided. The results are

summarized in Table 6.10.

The first model considered in Step 1 included all variables (intercept, age,

height, weight, height2) in the fixed effects and only the intercept in the random

effects. Table 6.1 provides the model parameter estimates and collinearity

diagnostics. The objective of fitting this model was to determine important

fixed effects. The results show that all variables except weight were highly

significant (p <0.001); weight was nonsignificant (p =0.296). The two small

eigenvalues (0.010 and 0.0008) of the scaled version of the (>rr'X) and the

two high condition indexes (21 and 74) indicate the presence of collinearity in

these data. There are two dependencies with the intercept, age, height and

height2 involved in one or both. This is determined by summing over the

variance decomposition proportions for the two highest condition indexes.

The choice of a second model to fit could be based on significance levels

for fixed effects; weight would be deleted and the model refit. Alternatively,

the choice could be made with a view toward eliminating the collinearity; one

of the collinear variables would be deleted and the model refit. In this case, the

decision was based on the collinearity since it was considered more serious;A

the variance of the /l seemed quite unstable. Heighe was deleted since it is

certainly collinear with height.

The second model fit in Step 1 included the intercept, age, height, and

weight in the fixed effects and only the intercept in the random effects. Table

6.2 provides the model parameter estimates and collinearity diagnostics. The

190

..

Table 6.1 Results for Model 1 in Step 1 of Model Fitting

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 2.586 0.401 6.46 <0.001Age 0.198 0.020 9.92 <0.001Height -0.047 0.006 -7.30 <0.001Weight -0.002 0.002 -1.05 0.296Height2 0.0002 0.00003 7.57 <0.001

A

4, Estimate of Covariance Matrix of the Random Effects

Estimate of Correlation Matrix of the Random EffectsI Intercept

...tr=0.025

h223 7256114722

1425481320527

19876447

31381 16977317555641 223457457

6153540 0.00000000283167543 . 0.0000000011

0.000000000045

Collinearity Diagnostics

Eigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT HT2

4.285 2.07 1 0.000 0.001 0.000 0.003 0.0000.630 0.79 3 0.004 0.003 0.000 0.025 0.0000.075 0.27 8 0.004 0.096 0.000 0.467 0.0020.010 0.10 21 0.047 0.821 0.008 0.113 0.1450.0008 0.03 74 0.944 0.079 0.992 0.393 0.853

significant fixed effects in this model are age and weight. Now, there is only

one small eigenvalue (0.003) and one high condition index (34). Thus, as

expected, one of the dependencies has been eliminated. It appears, from the

191

variance decomposition proportions, that the intercept, age and height are

involved in the remaining dependency. The choice of a third model to fit was

based on both the significance levels and the collinearity. Height was deleted•

from the fixed effects since it was both nonsignificant and collinear with age.

Table 6.2 Results for Model 2 in Step 1 of Model Fitting

Parameter Estimates of Fixed EffectsVARIABLE BETA STD T P-VAlUE

ERRIntercept 0.052 0.239 0.22 0.827Age 0.203 0.020 9.98 <0.001Height -0.00288 0.003 -0.95 0.344Weight 0.010 0.002 6.41 <0.001

A.

4, Estimate of Covariance Matrix of the Random Effects

Intercept 0.0195I Intercept

Estimate of Correlation Matrix of the Random EffectsI InterceptIntercept 1.00

A.

a2=0.0281533 9128

119312178830

1496219

23842583

39460570122

68801373164673

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.401 1.84 1 0.001 0.002 0.000 0.0090.534 0.73 3 0.010 0.005 0.001 0.0770.063 0.25 7 0.010 0.159 0.001 0.8760.003 0.05 34 0.979 0.834 0.998 0.038

192

..

The third model fit in Step 1 included the intercept, age, and weight in

the fixed effects and only the intercept in the random effects. Table 6.3

provides the model parameter estimates and collinearity diagnostics. For this

model, all fixed effects are significant. No collinearity is indicated since the

highest condition index is 6, though there are two high VDPs associated with

the condition index (for age and weight). Now, both criteria (significance level

Table 6.3 Results for Model 3 in Step 1 of Model Fitting

Parameter Estimates of Fixed EffectsVARIABLE BETA STD T P-

ERR VALUEIntercept -0.171 0.035 -4.88 <0.001Age 0.185 0.009 20.85 <0.001Weight 0.010 0.002 6.39 <0.001

A.

4, Estimate of Covariance Matrix of the Random EffectsI Intercept

Estimate of Correlation Matrix of the Random Effects

I Intercept

A.

02=0.0281601 9542

12176941252

580751

3211191

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE WT2.469 1.57 1 0.055 0.016 0.0190.471 0.69 2 0.764 0.014 0.0630.060 0.24 6 0.181 0.970 0.918

193

and lack of collinearity) are satisfied and we assume that the intercept, age and

weight are the best variables for the fixed effects of this model. This

concludes Step 1 of the model fitting.

For Step 2, the fixed effects from Step 1 are used and the random

effects are fine tuned. Any of the original variables could have been added to

the random effects component at this stage, however it seemed logical to

chose from those already in the fixed effects. Age was chosen first, though

weight could have been. Thus, the first model fit in Step 2 contained the

intercept, age, and weight in the fixed effects and the intercept and age in the

random effects. Table 6.4 provides the model parameter estimates and

collinearity diagnostics. All fixed effects are still significant and no collinearity

is present (the highest condition index is 7). The variance decomposition

proportion for weight associated with the highest condition index has

diminished, possibly indicating any impact of dependency between age and

weight has diminished. The difference between the -2 log likelihood statistics

for this and the previous model (123.6) indicates that age is an important

random effect ("p" <0.001). The statistic has two degrees of freedom due to

the additional covariance parameters for age and the covariance of the intercept

and age. The conclusion based on these results was to retain age and add

weight to the random effects.

194

..

Table 6.4 Results for Model 1 In Step 2 of Model Fitting

Parameter Estimates of Fixed Effects

• VARIABLE BETA STD ERR T P­VALUE

InterceptAgeWeight

-0.1410.1740.011

0.0530.0140.002

-2.6512.79

4.51

0.008<0.001<0.001

A

/i, Estimate of Covariance Matrix of the Random EffectsIntercept Age

InterceptAge

0.0639 -0.00850.0016

Estimate of Correlation Matrix of the Random EffectsIntercept Age

InterceptAge

1.00 -0.851.00

A

a2=O.0201643 8682

5846134731

236933

1132662

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE WT2.742 1.66 1 0.025 0.012 0.0180.198 0.44 4 0.633 0.006 0.3090.060 0.25 7 0.342 0.982 0.673

The second model fit in Step 2 contained the intercept, age, and weight

in the fixed effects and the intercept, age, and weight in the random effects.

Table 6.5 provides the model parameter estimates and collinearity diagnostics.

All fixed effects are still significant and no collinearity is present (the highest

condition index is 9). The variance decomposition proportion for weight

195

Table 6.5 Results for Model 2 in Step 2 of Model Fitting

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T p­

VALUEInterceptAgeWeight

-0.1500.1590.015

0.0450.0140.004

-3.3511.494.32

0.001<0.001<0.001

A

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Weight

InterceptAgeWeight

0.0351 -0.0005 -0.00070.0015 -0.0003

0.0001

Estimate of Correlation Matrix of the Random EffectsIntercept Age Weight

InterceptAgeWeight

1.00 -0.06 -0.361.00 -0.84

1.00

A.

a2=O.0201796 9640

7374839012

284365

1182490

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE WT2.766 1.66 1 0.031 0.009 0.0080.197 0.44 4 0.963 0.063 0.0470.037 0.19 9 0.006 0.928 0.944

associated with the highest condition index has increased, possibly indicating

an increase in the dependency between age and weight. The difference

between the -2 log likelihood statistics for this and the previous model (8.7)

indicates that weight is a moderately important random effect ("p" =0.034).

196

•

The statistic has three degrees of freedom due to the additional covariance

parameters for weight and the covariances of the intercept and weight and of

age and weight. The conclusion based on these results was to retain both age

and weight in the random effects. This concludes Step 2 of the model fitting.

For Step 3, the random effects from Step 2 are used and the fixed

effects are revisited. Since all fixed effects were significant in the previous

model (Table 6.5), none were deleted. Of the two original variables remaining

as candidates for the fixed effects, it seemed logical to add height before

height2 • Thus, the first model fit in Step 3 contained the intercept, age,

weight, and height in the fixed effects and the intercept, age, and weight in the

random effects. Table 6.6 provides the model parameter estimates and

collinearity diagnostics. In this model, all fixed effects except the intercept are

significant. (Recall that for Model 2 in Step 1 when these four variables were

included as fixed effects and only the intercept was a random effect, height

was not significant.) However, the addition of height has created collinearity.

There is one moderately high condition index (38) and one small condition index

(10), indicating two possible dependencies. The corresponding variance

decomposition proportions indicate that all variables are involved in one or both

dependencies. Based on these results, it is not logical to add heighe to the

fixed effects. Thus, Step 3 is of the model fitting is concluded.

197

Table 6.6 Results for Model 1 in Step 3 of Model Fitting

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T p-

VALUEIntercept 0.370 0.257 1.44 0.153Age 0.198 0.022 8.88 <0.001Height -0.007 0.003 -2.05 0.043Weight 0.017 0.004 4.62 <0.001

A

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Weight

•

InterceptAgeWeight

0.0309 0.0021 -0.00110.0014 -0.0004

0.0001

Estimate of Correlation Matrix of the Random EffectsIntercept Age Weight

InterceptAgeWeight

1.00 0.31 -0.581.00 -0.89

1.00

A

02=0.0201781 9138

72561199416

1193333

23559335

39147289150

49987701245103

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.704 1.92 1 0.001 0.002 0.000 0.0040.255 0.51 4 0.016 0.030 0.001 0.0410.037 0.19 10 0.000 0.308 0.002 0.8720.003 0.05 38 0.984 0.660 0.997 0.084

For Step 4, previous results are re-examined and additional models fit as

believed to be necessary. The results in Step 3 indicate that height probably

198

should not have been added to the fixed effects. Thus, one option would be

to stop the process at this point and consider the Model 2 of Step 2 to be the

final model. However, we have learned from the results in Chapters 3 and 4

that adding a collinear variable to the random effects may alleviate the impact

of collinearity in the fixed effects. Because of this finding, height was added

to the random effects. Thus, the first model fit in Step 4 contained the

intercept, age, weight, and height in the fixed effects and the intercept, age,

weight, and height in the random effects. Table 6.7 provides the model

parameter estimates and collinearity diagnostics. In this model, height is no

longer a significant fixed effect. However, height is an important random

effect. The difference in -2 log likelihoods between this model and the previous

model is 34.5 with 4 degrees of freedom (due to additional covariance

parameters) ("p" <0.001). The conclusion based on these results was to delete

height as a fixed effect, but to retain it as a random effect.

199

Table 6.7 Results for Model 1 in Step 4 of Model Fitting

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T p-

VALUE

Intercept -0.127 0.359 -0.35 0.724Age 0.153 0.030 5.10 <0.001Height 0.00002 0.005 0.00 0.997Weight 0.016 0.004 4.00 <0.001

A.

A, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight

..

InterceptAgeHeightWeight

2.3081 0.1801 -0.0313 0.00970.0154 -0.0024 0.0004

0.0004 -0.00010.0002

Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

1.00 0.96 -0.99 0.491.00 -0.95 0.27

1.00 -0.541.00

•

A.

02=0.0193903 16799

96970408783

1928606

44029265

72718395852

82161441712904

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT WT3.795 1.95 1 0.000 0.001 0.000 0.0020.176 0.42 5 0.005 0.020 0.000 0.0340.027 0.17 12 0.000 0.169 0.000 0.7550.006 0.02 78 0.995 0.810 0.999 0.209

200

The second model fit in Step 4 contained the intercept, age, and weight in the

fixed effects and the intercept, age, height, and weight in the random effects.

Table 6.8 provides the model parameter estimates and collinearity diagnostics.

Table 6.8 Results for Model 2 in Step 4 of Model Fitting

Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept -0.127 0.035 -3.59 <0.001Age 0.153 0.014 11.33 <0.001Weight 0.016 0.004 4.62 <0.001

A

4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

2.3245 0.1817 -0.0315 0.00960.0155 -0.0024 0.0004

0.0004 -0.00010.0002

Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight

InterceptAgeHeightWeight

1.00 0.96 -0.99 0.491.00 -0.95 0.27

1.00 -0.531.00

A

x'r'xA

02=0.0193756 16267

9519670128

386556

1665503

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE WT2.813 1.68 1 0.024 0.007 0.0060.160 0.40 4 0.899 0.082 0.0290.027 0.17 10 0.076 0.911 0.965

201

All fixed effects are still significant. Height is an important random variable.

The difference in -2 log likelihoods between this model and Model 2 of Step 2

(Table 6.5) is 37.0 with 4 degrees of freedom (due to additional covariance

parameters related to height) ("p" <0.001). The conclusion was to stop the

model fitting process; criteria based on both significance levels and collinearity

are met.

Two routes toward obtaining a final model seemed apparent after fitting

Model 1 in Step 1. Recall that the alternative chosen involved deleting a

collinear variable. The road not taken involved eliminating weight as a fixed

effect because it was not significant. To determine the results had that choice

been made, weight was deleted and the model refit. The reduced model

contained the intercept age, height, and height2 as fixed effects and only the

intercept as a random effect. Table 6.9 provides the model parameter

estimates and collinearity diagnostics. All fixed effects are significant, however

there are two collinearities present; the highest condition indexes are 18 and

54. Thus, had this path been taken, the next step would have been to delete

one of the collinear variable, height2 • This probably would have led us back to

the model ultimately chosen. A summary of the entire model fitting strategy

is presented in Table 6.10.

202

..

•

Table 6.9 Results for Alternative Model 2 in Step 1

Parameter Estimates of Fixed EffectsVARIABLE BETA STD T p-

ERR VALUEIntercept 2.364 0.346 6.82 <0.001Age 0.195 0.199 9.82 <0.001Height -0.042 0.005 -8.18 <0.001Height2 0.0002 0.00002 9.99 <0.001

A

4, Estimate of Covariance Matrix of the Random EffectsI Intercept

Estimate of Correlation Matrix ,of the Random EffectsI Intercept

A

X'I"'X

A

a2=0.0251294 7682

117093150849

1369265

20842101

17969880229132500

0.0000000030.00000000005

Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions

INT AGE HT HT2

3.542 1.88 1 0.000 0.001 0.000 0.0000.445 0.67 3 0.008 0.014 0.000 0.0010.012 0.11 18 0.028 0.876 0.003 0.2570.0012 0.03 54 0.964 0.108 0.996 0.741

203

Table 6.10 Summary of Model Fitting

-2 logFixed Random Uke-

Step Model Effects Effects lihood Results/Conclusions

1 1 INT- INT -364.5 ResultsAGE-HEIGHT- 1) WEIGHT is not significant.WEIGHTHEIGHT2

• 2) INT, AGE, HEIGHT and HEIGHT2 areinvolved in one or more dependencies.

Conclusions

1) Delete WEIGHT from fixed effects;retain INT in random effects (ignorescollinearity) .

2) Presume HEIGHT2 is culprit and delete itfrom fixed effects (ignores nonsignificanceof WEIGHT); retain INT in random effects.[This option was chosen.]

1 2 INT" INT -311.8 ResultsAGE-HEIGHT 1) HEIGHT is not significant.WEIGHT-

2) HEIGHT is collinear with AGE and INT.

Conclusions

Delete HEIGHT from fixed effects; retainINT in random effects.

1 3 INT- INT -310.9 ResultsAGE-WEIGHT" 1) All fixed effects are significant.

2) No collinearity is present. Even thoughtwo variables have high VDPs, there is nohigh CI.

Conclusions

Retain all fixed effects; add variables torandom effects. AGE is picked next,though WEIGHT could have been.

204

•

"

..

Table 6.10 Summary of Model Fitting

-2 logFixed Random Like-

Step Model Effects Effects lihood Results/Conclusions

2 1 INT· INT -434.5 ResultsAGE· AGEWEIGHT· 1) All fixed effects are still significant.

2) No collinearity is present. The VDP forweight has decreased, perhaps indicatingless collinearity between age and weightthan for previous model. Still there is nohigh CI.

3) AGE is an important random effect(difference in likelihoods between previousmodel and this one is 123.6 with 2 df,P<O.OOl) (df due to 4 parameters AGEand covarllNT, AGE)).

Conclusions

Retain AGE and add WEIGHT to randomeffects.

2 2 INT· INT -443.2 ResultsAGE· AGEWEIGHT" WEIGHT 1) All fixed effects are still significant.

2) No collinearity is present. The VDP forweight has increased, perhaps indicatingmore collinearity between age and weightthan for previous model. Still there is nohigh CI.

3) WEIGHT is a moderately importantrandom effect (difference in likelihoodsbetween previous model and this one is8.7 with 3 df, P=O.034) (df due to 4parameters WEIGHT, covarllNT, WEIGHT),and covar (AGE, WEIGHT)).

Conclusions

Retain all random effects; go back toexamination of fixed effects.

205

Table 6.10 Summary of Model Fitting

-2 logFixed Random Uke-

Step Model Effects Effects Iihood Results/Conclusions

3 1 INT- INT -446.1 ResultsAGE- AGEHEIGHT- WEIGHT 1) All fixed effects, except the intercept,WEIGHT- are significant.

2) INT, AGE and HEIGHT are involved inone dependency; high CI is 38. There ispossibly another dependency indicated bythe second CI (1 0). Both Cis are lowerthan found previously.

Conclusions

Retain all fixed effects. Fine tune randomeffects again. Add HEIGHT to the randomeffects.

4 1 INT- INT -480.6 ResultsAGE- AGEHEIGHT WEIGHT 1) INT and HEIGHT are not significantWEIGHT- HEIGHT fixed effects.

2) HEIGHT is an important random effect(difference in likelihoods between previousmodel and this one is 34.5 with 4 df,P<0.001) (df due to 4 parametersHEIGHT and covar(lNT, HEIGHT),covar(AGE, HEIGHT), and covar(WEIGHT,HEIGHT)).

Conclusions

Delete HEIGHT as a fixed effect; retainHEIGHT as a random effect.

206

,

..

Table 6.10 Summary of Model Fitting

-2 logFixed Random Like-

Step Model Effects Effects lihood Results/Conclusions

4 2 INT- INT -480.2 ResultsAGE- AGEWEIGHT- WEIGHT 1) All fixed effects are significant.

HEIGHT2) All random effects are important.HEIGHT is still important random effecteven when HEIGHT is not in the model asa fixed effect (difference in likelihoodsbetween Model 2 of Step 2 and this modelis 37 with 4 df, P<0.0011 (df due to 4parameters HEIGHT and covar(lNT,HEIGHT), covar(AGE, HEIGHT), andcovar(WEIGHT, HEIGHT)).

Conclusions

Stop the process. This is the final model.

• p<0.05

Applying the model fitting strategy to these data has resulted in the

selection of a final model, shown in Table 6.8, that includes the intercept, age

and weight in the fixed effects and the intercept, age, height and weight in the

random effects. All effects are important predictors of the variation in forced

vital capacity (FVC) in these black females aged 2-15 years and weighing 15-

120 kg over their entire time in the study. The slopes of both age (0.153) and

weight (0.016) are positive indicating that FVC increases with increasing age

(for fixed weight) and weight (for fixed age). These results are presented

graphically in Figures 6.1 and 6.2 for age and weight, respectively. Height is

not a fixed effect, thus its line has zero slope, as shown in Figure 6.3. The

lines in the graphs were produced using the following equations:

207

+ 11-zSge

A

For the graphs of ElY) versus age

Estimated population regression line:

Estimated individual regression line:

( 11, + 113weigh f)Yk (age,welght,helght)] =

+ (ak1 + ak4welght) + auhelght) + ak:Pge

A

For the graphs of ElY) versus weight

Estimated population regression line:

Estimated individual regression line:

Yk (age,weight,helght)] =(11, + 11~ge)

A

For the graphs of ElY) versus height

Estimated population regression line:

208

Estimated individual regression line:

'fir (age,welght,height)] =( ~, + ~~ge + ~3welght)

+ (air 1 + a/r2age + alr4welght) + a/r3height

Portions of the correlation matrix of the random effects, and the Figures,

aid in interpreting the results. The correlation between the random intercepts

and ages in this model is high and positive (0.96). This indicates that subjects

with positive intercept increments also have higher than average random slopes

for age and vice versa. In other words, subjects who start with high values of

FVC tend to have even higher values as they age. The correlation between the

intercept and weight is positive (0.49). This indicates that subjects with

greater than average intercept increments have higher than average random

slopes for weight. In other words, subjects who start with high values of FVC

tend to have even higher values as they gain weight. The correlation between

the intercept and height is very high, but negative (-0.99). This indicates that

subjects with negative intercept increments have higher than average random

slopes for height and vice versa. In other words, subjects who start with low

values of FVC tend to "catch up" as they grow taller.

209

Pulmonary Function StudyI~--------------------'

~l

j~

JiILl

2 I 4 • • 7 • • .. l' 12 II .4 II II

Age In Yearw

Figure 6.1 Values of FVC Predicted from Final Mixed Model withIntercept, Age and Weight in X and in Z at Mean Weight, PlottedAgainst Age

Pulmonary Function StudyI.r---------------------,

Ie • • • • • ~ • • _ ,.. I.

Figure 6.2 Values of FVC Predicted from Final Mixed Model withIntercept, Age and Weight in X and in Z at Mean Weight, PlottedAgainst Weight

210

Pulmonary Function Study.~------------------,

~I

J~

Jia.. 1

• • I. ,. •• t. '. 1. •• IJII

H"llhf 'n om

Figure 6.3 Values of FVC Predicted from Final Mixed Model withIntercept, Age and Weight in X and in Z at Mean Age, PlottedAgainst Height

The collinearity diagnostics shown in Table 6.8 indicate that this model

is free from serious effects of collinearity in the fixed effects. The highest

condition index is 10, indicating that only a weak dependency exists. The

variance decomposition proportions for both age (O.911) and weight (O.965)

are high. However, as noted repeatedly, Belsley (1991) states that a

"degrading" collinearity is indicated by both a high condition index and

corresponding high variance decomposition proportions for two or more

variables. High variance decomposition proportions in the absence of high

condition indexes may indicate that collinearity in the fixed effects exists, but

that it does not adversely impact the model.

As described in Chapter 2 (Section 2.7.1), the data set used in this

dissertation is a subset of a data from larger investigation that was analyzed

211

previously by Strope and Helms (1984) and Fairclough and Helms (1984). Both

of the earlier analyses were larger in scope and addressed different issues than

those focused on here, though both are directed at modeling lung function

using longitudinal parameters, primarily height. Since collinearity diagnostics

have only now been developed in this dissertation, they were not available as

tools in the earlier modeling. Clearly, they have been useful in the analysis

described in this chapter.

6.4 Summary

In this chapter, it has been demonstrated that collinearity diagnostics can

be incorporated reasonably into an existing mixed model fitting procedure. A

clearly collinear set of variables was used in the initial model. When collinear

variables were removed prior to assessing the significance of the fixed effects,

an interim model was found that was free of collinearity. When random effects

were fine tuned and the fixed effects revisited, it was found that adding more

fixed effects again produced collinearity. A final model was obtained that was

both free of collinearity and in which all fixed and random effects were

significant. Thus, use of the diagnostics in model fitting has improved the

process.

212

•

.'

CHAPTER VII

SUMMARY AND RECOMMENDATIONS FORFUTURE RESEARCH

7.1 Summary of Research

It is well known that collinearity in the independent variables of the

General Linear Univariate Model (GLUM) causes variances of the least squares

parameter estimates to be unduly large. This can affect estimation, hypothesis

testing, and prediction. Methods for detection of collinearity in the GLUM are

well established. However, no previously published research has been directed

toward extending these methods for use in the mixed model. In the mixed

A

model, collinearity in the fixed effects arises from the ill-conditioning of (r' /2X)

A A A A

and (X'r'x), leading to inflated elements of V(JJ) = (X'r'xr'.

The objective of this dissertation was to develop a method for assessing

collinearity in the fixed effects of the mixed model by expanding a strategy

currently used for the GLUM. In Chapter 1, the literature on both collinearity

and the mixed model was reviewed in order to provide a background for this

new research on the combination of these topics. In Chapter 2, mixed model

diagnostics were defined and a procedure for their use was specified. GLUM

and mixed model diagnostics were applied to a data set with three collinear

variables; they appeared to perform similarly. For the mixed model, initial

impressions were that variation in two factors, the number of variables in the

random effects and constraints on the covariance of the random effects,

produced different collinearity diagnostics for models with the same variables

in the fixed effects. Since these findings were specific to this data set, they

were used as a point of departure for designing the subsequent research.

In order to generalize the behavior of the diagnostics, they were applied

to experimental data with known predetermined collinearities with increasingly

tighter dependencies. In Chapter 3, two types of dependencies were created,

a simple dependency involving two variables and a coexisting dependency

involving three variables. For comparison, GLUM and mixed model diagnostics

were computed for the same dependencies, though the data for the two types

of models were necessarily different.

The focus of Chapter 3 was the behavior of the diagnostics when

different random effects were in models containing the same fixed effects, for

a given type and level of dependency and a specified covariance matrix A. In

practice, the assessment of collinearity would be made after model fitting since

.....it requires the estimated matrix J:. However, actual model fitting in this case

would have confounded the experiment. Thus a A and 02 were specified and

used to compute ~ which was used to compute r' and (X'r'x), from which

the diagnostics were computed directly. This enabled a pure assessment of the

impact of the varying the random effects, Le., A was always the same for each

pair of variables in the experiment. Eliminated was the "noise" that might have

been introduced due to the other component of the variance of V, crvk , and the

manner in which the estimation process would necessarily change the Ii. for

214

..

each model fit. The results indicated that adding variables, especially collinear

variables, to the random effects of the mixed model can greatly attenuate the

impact of the collinearity in the fixed effects.

The initial results of Chapter 2 also indicated that constraints on the

covariance of the random effects produced different collinearity diagnostics for

models with the same variables in the fixed effects. In order to determine the

effect of a different covariance structure on the diagnostics, the entire

experiment of Chapter 3 was repeated in Chapter 4 using a different covariance

matrix 4. The results indicated that the diagnostics, and therefore, the

conclusions, were virtually identical for these two covariance structures.

Several findings emerged from these experiments involving two types of

dependencies (simple, two variables involved and coexisting, three variables

involved). First, it was demonstrated that the diagnostics could be used with

confidence to pinpoint designed collinearities, Le., the procedure works in the

mixed model context for these dependencies. This was an important finding

because it is not general knowledge. Second, it was discovered that adding

variables to the random effects essentially cancelled the collinearity in the fixed

effects. This was unexpected and somewhat counterintuitive and also was not

previously known. Third, these results held for a different covariance structure

4. However, it is not yet known whether the pattern will hold for a variety of

covariance structures.

Analyses in Chapters 3 and 4 determined the behavior of the collinearity

diagnostics under controlled conditions. The advantage was that effects of the

contrived dependencies could be readily seen without being "contaminated" by

215

the estimation process. The disadvantage was that the full dynamics of the

model fitting process were not allowed to operate. Therefore, in Chapter 5, the

experimental results were explored in models fit to actual data. Using the data

originally analyzed in Chapter 2, one dependency similar to a dependency in the

experimental data was created. While retaining the same fixed effects, the

number of random effects was varied. The pattern of diminished collinearity

was seen when variables were added to the random effects, though not to the

same degree nor in exactly the same pattern as was seen for the experimental

data. The difference in findings was attributed to the difference in

dependencies in the two data sets, the effect of the estimation process (the l1.

changes for each model) and the structure of the covariance matrix in the

actual data, which was quite different than either of the matrices used in the

experiments. In Chapter 5, another dependency that was overlapping in nature

was also explored. The results suggested that the same cancellation of

collinearity as seen for simple dependencies, in both actual and experimental

data, might occur for this type as well. Overall, the results in Chapter 5

suggested that even though the behavior of the diagnostics is specific to an

actual data set, the general pattern will be similar to that seen in experimental

data; collinearity will be diminished when variables are added to the random

effects.

In Chapter 6, the focus of the dissertation shifted from the behavior of

the diagnostics to their practical use as a part of a model fitting strategy. Both

the strategy and the diagnostics were applied to a set of collinear independent

variables (age, height, weight and height2 ) that were candidates for a model to

216

•

..

..

assess the longitudinal variation of forced vital capacity (FVC) in children. A

final model was obtained that was both free of serious collinearity and in which

all fixed and random effects were significant. It was demonstrated that using

the diagnostics had improved the model fitting procedure.

7.2 Directions for Future Research

The experiments and the data for this research were necessarily limited

in scope. The investigation has focused on simple models with 3-5 continuous

time-dependent independent variables in both actual and experimental data.

The behavior of collinearity diagnostics was examined for two types of

dependencies that were created in the experimental data. One of the two

types was also examined in the actual data. The results are dependent on the

experimental data generated and the type of dependencies created and cannot

be generalized to other types with absolute certainty. Results also depend on

the sample manifestations for the series that were used to generate the

dependencies. And of course, the findings for the actual data may be specific

to the data set used. However, the weight of the evidence over all situations

examined supports the findings and provides an excellent basis for subsequent

research.

Future research can proceed in several directions, involving aspects of

both collinearity and of the mixed model. First, the viability of the findings of

this investigation might be examined under a variety of other conditions. For

example, using the same types of dependencies, several other factors that

217

might impact the results could be examined. For these same experiments, the

number of "subjects" and the number of observations per subject could be

varied. In these experiments, there were 30 subjects with 10 observations

each. Also, the degree to which "missingness" or incomplete data affects the

results could be studied; no observations were missing in these experiments.

These are all subject-related aspects.

Several variable-related aspects also could be pursued for these same

experiments. These experiments was comprised of three to five continuous

time-dependent covariates. When dichotomous, ordinal or time-invariant

covariates are included with these same independent variables, different

diagnostic behavior might result. For example, a continuous variable might be

collinear with a dichotomous term if most of the data falls into one of the two

categories. This would be similar to a dependency with the intercept, which

is another aspect that could be pursued. In these experiments, collinearities

with the intercept were noted, but not particularly dealt with. The number of

noncollinear variables in models also could be varied.

There are several avenues of specific collinearity-related research that

could be pursued, as suggested by the literature review. The effect of

centering the data could be studied since this is a controversial and seemingly

unsettled topic. Remedial measures, other than variable deletion, could be

examined, such as mixed model analogs of biased regression techniques used

in the GLUM. In actual data, the degree to which perturbation of the data

impacts conditioning and the related concept of collinearity-influential

observations could be examined. A systematic study of how collinearity

218

...

impacts mixed model parameter estimates of both collinear and noncollinear

variables could be undertaken.

Another obvious pursuit is the investigation of the behavior of the

collinearity diagnostics for more complicated, perhaps overlapping,

dependencies. With more complex dependencies, the condition indexes may

indicate the number of dependencies present. However, it may be difficult to

determine the particular variables involved from the variance decomposition

proportions alone. Thus, it would also be of interest to determine how to use

auxiliary regressions in the mixed model context. Whether they would be

comparable to the GLUM analogs is uncertain.

An investigation of certain model-related aspects would be crucial to

generalizing the characterization of these diagnostic measures. The effect of

the covariance matrix seems to be particularly important. In this research, fairly

simple covariance structures were used. One was essentially an identity

matrix, except for two non-zero off-diagonal elements; the other had one's on

the diagonal and all off-diagonal elements were positive and ranged from 0.4

to 0.8. It is especially important to look at other covariance structures,

particularly those with diagonal elements not equal to one. In addition, a range

of values might be chosen for u2 and a variety of structures, for Vk •

Finally, a theoretical aspect could be pursued. Although the experiments

showed repeatedly that collinearity in the fixed effects was virtually cancelled

when collinear variables were added to the random effects, the theoretical basis

for this cancellation has not yet been determined. If shown, this result would

provide a rationale for the empirical findings.

219

APPENDIX 1

GENERAL LINEAR UNIVARIATE MODEL (GLUM)COLLINEARITV DIAGNOSTICS

*** The W8 -*

Set 1: WO

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX! wo

1 0.005 0.009 0.011 0.006 0.0074 0.005 0.133 0.1n 0.097 0.1475 0.021 0.356 0.735 0.003 0.0157 0.237 0.249 0.000 0.317 0.6658 0.732 0.253 0.083 0.577 0.166

Set 2: W1

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W1

1 0.005 0.009 0.011 0.001 0.0013 0.004 0.108 0.192 0.014 0.0145 0.016 0.424 0.717 0.000 0.0007 0.975 0.458 0.078 0.007 0.006

21 0.000 0.001 0.002 0.978 0.979

Set 3: W2

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX! W2

1 0.005 0.009 0.011 0.000 0.0003 0.005 0.105 0.179 0.002 0.0025 0.016 0.417 0.716 0.000 0.0007 0.926 0.469 0.080 0.001 0.001

62 0.048 0.001 0.014 0.998 0.997

Set 4: W3

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W3

1 0.005 0.008 0.011 0.000 0.0003 0.004 0.095 0.186 0.000 0.0005 0.015 0.403 0.683 0.000 0.0007 0.957 0.425 0.082 0.000 0.000

204 0.018 0.070 0.039 1.000 1.000

Set 5: W4

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX! W4

..1 0.005 0.008 0.011 0.000 0.0003 0.004 0.100 0.191 0.000 0.0005 0.016 0.416 0.714 0.000 0.0007 0.970 0.445 0.083 0.000 0.000

731 0.004 0.030 0.001 1.000 1.000

..

-- The Zs --

set 6: ZO

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO

1 0.005 0.006 0.011 0.010 O.OOB4 0.009 0.066 0.001 0.426 0.1564 0.003 0.030 0.905 0.100 0.0417 0.159 0.445 0.019 0.151 0.744a 0.824 0.453 0.063 0.313 0.OS1

set 7: Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1

1 0.005 0.001 0.007 0.009 0.0014 0.012 0.015 0.026 0.521 0.0075 0.002 0.027 0.523 0.006 0.0037 0.958 O.OOB 0.024 0.418 0.012

21 0.023 0.949 0.421 0.046 0.977

Set 8: Z2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z2

1 0.005 0.000 0.001 0.009 0.0004 0.012 0.001 0.003 0.512 0.0015 0.003 0.002 0.095 0.011 0.0007 0.977 0.001 0.005 0.457 0.001

69 0.003 0.995 0.896 0.010 0.998

Set 9: Z3

Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z3

1 O.OOS 0.000 0.000 0.009 0.0004 0.012 0.000 0.000 0.503 0.0005 0.003 0.000 O.OOB 0.011 0.0007 0.978 0.000 0.000 0.443 0.000

251 0.003 1.000 0.991 0.035 1.000

Set 10: Z4

Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4

1 0.005 0.000 0.000 0.009 0.0004 0.012 0.000 0.000 0.499 0.0005 0.003 0.000 0.001 0.011 0.0007 0.980 0.000 0.000 0.444 0.000

723 0.000 1.000 0.999 0.037 1.000

221

Append i x 1: GLl.I4

Appendi x 1: GUM

... The Ws 8nd Zs ...

set 11: wa, ZO

CI V.rienee Proportions for Coefficients oflilT 1X1 1X2 IX3 wa ZO

1 0.004 0.004 0.008 0.004 0.005 0.0054 0.000 0.OS8 0.021 0.094 0.118 0.0845 0.000 0.043 0.866 0.009 0.006 0.0836 0.201 0.090 0.026 0.104 0.346 0.3238 0.209 0.7'90 0.005 0.OS8 0.221 0.4369 0.586 0.015 0.074 0.731 0.304 0.069

set 12: wa, Z1

CI V.ri8nCe Proportions for Coefficients oflilT IX1 IX2 IX3 WO Z1

1 0.004 0.001 O.OOS 0.004 0.005 0.0014 0.000 0.012 0.024 0.089 0.138 0.0075 0.003 0.025 0.522 0.010 0.000 0.0037 0.384 0.011 0.000 0.184 0.598 0.0068 0.587 0.002 0.031 0.67'9 0.258 0.008

22 0.022 0.948 0.418 0.034 0.001 0.976

Set 13: wa, Z2

CI Veri8nCe Proportions for Coefficients oflilT IX1 IX2 IX3 WO Z2

1 0.004 0.000 0.001 0.004 0.005 0.0004 0.000 0.001 0.004 0.087 0.127 0.0015 0.004 0.002 0.094 0.010 0.000 0.0007 0.359 0.001 0.000 0.190 0.603 0.0008 0.632 0.001 0.006 0.671 0.232 0.000

76 0.002 0.995 0.895 0.038 0.033 0.998

Set 14: wa, Z3

CI V.ri8nCe Proportions for Coefficients oflilT IX1 IX2 IX3 WO Z3

1 0.004 0.000 0.000 0.004 0.005 0.0004 0.000 0.000 0.000 0.085 0.131 0.0005 0.004 0.000 0.008 0.011 0.000 0.0007 0.365 0.000 0.000 0.188 0.613 0.0008 0.626 0.000 0.001 0.663 0.236 0.000

275 0.002 1.000 0.991 0.049 0.016 1.000

Set 15: wa, Z4

CI V.ri8nCe Proportions for Coefficients oflilT IX1 IX2 IX3 wa Z4

1 0.004 0.000 0.000 0.004 0.005 0.0004 0.000 0.000 0.000 0.089 0.129 0.0005 0.004 0.000 0.001 0.011 0.000 0.0007 0.365 0.000 0.000 0.197 0.608 0.0008 0.628 0.000 0.000 0.696 0.234 0.000

795 0.000 1.000 0.999 0.002 0.025 1.000

222

Appendix 1: GLlJ4

*** The \Is end Zs ***set 16: W1, ZO

CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 W1 ZO

1 0.004 0.004 0.008 0.001 0.001 0.0053 0.000 0.043 0.029 0.012 0.012 0.0825 0.000 0.OS2 0.858 0.001 0.000 0.0887 0.242 0.346 0.035 0.005 0.002 0.7188 0.753 0.548 0.067 0.004 O.OOS 0.076

23 0.002 0.006 0.004 0.978 0.980 0.031

set 17: W1, Z1

CI Variance Proportions for Coefficients oflilT IX1 IX2 IX3 W1 Z1

1 0.003 0.001 0.004 0.001 0.001 0.0014 0.000 0.010 0.034 0.013 0.013 0.0065 0.001 0.029 0.507 0.000 0.000 0.0048 0.975 0.011 0.031 0.008 0.007 0.013

22 0.017 0.671 0.274 0.200 0.250 0.67524 0.003 0.277 0.150 0.778 0.730 0.302

Set 18: W1, Z2

CI Variance Proportions for Coefficients oflilT IX1 aX2 aX3 W1 Z2

1 0.003 0.000 0.001 0.001 0.001 0.0003 0.000 0.001 0.005 0.013 0.013 0.0015 0.001 0.003 0.092 0.000 0.000 0.0008 0.993 0.001 0.006 0.008 0.007 0.001

23 0.000 0.000 0.001 0.956 0.963 0.00076 0.002 0.995 0.896 0.023 0.016 0.998

Set 19: W1, Z3

CI Variance Proportions for Coefficients oflilT aX1 aX2 aX3 W1 Z3

1 0.003 0.000 0.000 0.001 0.001 0.0003 0.000 0.000 0.000 0.013 0.013 0.0005 0.001 0.000 0.008 0.000 0.000 0.0008 0.992 0.000 0.001 0.008 0.007 0.000

23 0.000 0.000 0.000 0.978 0.974 0.000275 0.003 1.000 0.991 0.000 0.005 1.000

Set 20: W1, Z4

CI Variance Proportions for Coefficients oflilT aX1 aX2 aX3 W1 Z4

1 0.003 0.000 0.000 0.001 0.001 0.0003 0.000 0.000 0.000 0.013 0.013 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.995 0.000 0.000 0.008 0.007 0.000

23 0.000 0.000 0.000 0.974 0.963 0.000795 0.000 1.000 0.999 O.OOS 0.017 1.000

223

Appendi x 1: GLlJ4

- The wa end Zs -

set 21: 112, ZO

CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 112 ZO

1 0.003 0.004 0.008 0.000 0.000 0.0053 0.000 0.042 0.031 0.001 0.002 0.0785 0.000 0.054 0.837 0.000 0.000 0.0977 0.282 0.285 0.047 0.000 0.000 0.7018 0.666 0.613 0.063 0.000 0.001 0.119

67 0.049 0.002 0.013 0.998 0.997 0.001

Set 22: 112, Z1

CI Variance Proportions for Coefficients oflilT IX1 IX2 IX3 112 Z1

1 0.003 0.001 0.004 0.000 0.000 0.0011 0.000 0.009 0.034 0.001 0.002 0.0055 0.001 0.028 0.506 0.000 0.000 0.0048 0.926 0.010 0.033 0.001 0.001 0.013

22 0.025 0.891 0.417 0.001 0.000 0.91470 0.045 0.061 0.006 0.997 0.997 0.064

Set 23: 112, Z2

C[ Variance Proportions for Coefficients of[liT IX1 IX2 IX3 112 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.001 0.002 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.939 0.001 0.006 0.001 0.001 0.001

65 0.030 0.152 0.166 0.715 0.723 0.15479 0.026 0.843 0.731 0.283 0.274 0.844

Set 24: 112, 23

C[ Variance Proportions for Coefficients of[liT IX1 IX2 IX3 112 Z3

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.002 0.002 0.0005 0.001 0.000 0.008 0.000 0.000 0.0008 0.945 0.000 0.001 0.001 0.001 0.000

68 0.048 0.000 0.000 0.996 0.997 0.000214 0.003 1.000 0.991 0.002 0.001 1.000

Set 25: 112, Z4

C[ Variance Proportions for Coefficients of[liT IX1 IX2 IX3 W2 Z4

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.001 0.002 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.947 0.000 0.000 0.001 0.001 0.000

68 0.048 0.000 0.000 0.986 0.980 0.000795 0.000 1.000 0.999 0.012 0.017 1.000

224

Appendi x 1: GlLM

*** The lis end Zs --

set 26: 113, ZO

CI V.riance Proportions for Coefficients oflIT &X1 &X2 IlG 113 ZO

1 0.003 0.004 0.008 0.000 0.000 0.0053 0.000 0.040 0.033 0.000 0.000 0.0795 0.000 0.052 0.811 0.000 0.000 0.0997 0.273 0.292 0.045 0.000 0.000 0.7108 0.706 0.560 0.066 0.000 0.000 0.107

222 0.017 0.051 0.037 1.000 1.000 0.000

Set 27: 113, Z1

CI V.riance Proportions for Coefficients oflIT IX1 &X2 IlG 113 Z1

1 0.003 0.001 0.004 0.000 0.000 0.0014 0.000 0.009 0.036 0.000 0.000 0.0055 0.001 0.029 0.499 0.000 0.000 0.0048 0.955 0.010 0.035 0.000 0.000 0.013

22 0.021 0.917 0.418 0.000 0.000 0.969225 0.019 0.033 0.009 1.000 1.000 0.008

set 28: 113, Z2

CI V.riance Proportions for Coefficients oflIT IX1 IX2 IlG 113 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.000 0.000 0.0015 0.001 0.003 0.090 0.000 0.000 0.0008 0.978 0.001 0.007 0.000 0.000 0.001

75 0.003 0.939 0.879 0.001 0.001 0.954229 0.015 0.056 0.018 0.999 0.999 0.044

Set 29: 113, Z3

CI V.riance Proportions for Coefficients oflIT IX1 IX2 IX3 113 Z3

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.000 0.000 0.0005 0.001 0.000 0.008 0.000 0.000 0.0008 0.974 0.000 0.001 0.000 0.000 0.000

223 0.015 0.021 0.030 0.946 0.948 0.024276 0.006 0.978 0.961 0.054 0.051 0.976

Set 30: 113, Z4

CI V.riance Proportions for Coefficients oflIT IX1 IX2 IlG 113 Z4

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.000 0.000 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.9n 0.000 0.000 0.000 0.000 0.000

224 0.018 0.000 0.000 0.991 0.992 0.000792 0.000 1.000 0.999 0.009 0.008 1.000

225

Appendi x 1: GLIJII

*** The wa end Zs *-Set 31: Wit, ZO

CI Varience Proportions for Coefficients ofJIlT IX1 8X2 8X3 Wit ZO

1 0.004 0.004 0.008 0.000 0.000 0.0053 0.000 0.042 0.034 0.000 0.000 0.0795 0.000 0.054 0.845 0.000 0.000 0.0977 0.274 0.306 0.045 0.000 0.000 0.7058 0.716 0.586 0.067 0.000 0.000 0.105

799 0.006 0.007 0.001 1.000 1.000 0.009

Set 32: Wit. Z1

CI Varience Proportions for Coefficients ofINT IX1 IX2 8X3 Wit Z1

1 0.003 0.001 0.004 0.000 0.000 0.0014 0.000 0.010 0.035 0.000 0.000 0.0055 0.001 0.030 0.501 0.000 0.000 0.0048 0.970 0.011 0.033 0.000 0.000 0.013

22 0.022 0.947 0.418 0.000 0.000 0.964809 0.004 0.002 0.009 1.000 1.000 0.013

Set 33: Wit. Z2

CI Veriance Proportions for Coefficients ofINT IX1 IX2 IX3 Wit Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.000 0.000 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.989 0.001 0.006 0.000 0.000 0.001

76 0.003 0.982 0.890 0.000 0.000 0.989 •807 0.003 0.013 0.007 1.000 1.000 0.009

Set 34: Wit. Z3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Wit Z3

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.000 0.000 0.0005 0.001 0.000 O.ooa 0.000 0.000 0.0008 0.989 0.000 0.001 0.000 0.000 0.000

274 0.003 0.979 0.968 0.000 0.000 0.977812 0.004 0.021 0.023 1.000 1.000 0.023

Set 35: Wit, Z4

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Wit Z4

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.000 0.000 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.991 0.000 0.000 0.000 0.000 0.000

764 0.002 0.565 0.567 0.353 0.351 0.567832 0.002 0.435 0.432 0.647 0.649 0.433

226

APPENDIX 2

MIXED MODEL BASELINECOLLINEARITY DIAGNOSTICS

- The wa-

Set 1: WO

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 WO

1 0.005 0.011 0.011 0.006 0.0083 0.015 0.080 0.105 0.040 0.2014 0.000 0.484 0.478 0.001 0.0046 0.215 0.309 0.188 0.319 0.4458 0.104 0.111 0.218 0.634 0.336

Set 2: '11

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 '11

1 0.005 0.010 0.010 0.001 0.0013 0.013 0.069 0.185 0.013 0.0174 0.000 0.515 0.392 0.001 0.0001 0.915 0.345 0.405 0.002 0.004

20 0.007 0.000 0.008 0.983 0.977

Set 3: W2

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W2

1 0.005 0.010 0.010 0.000 0.0003 0.013 0.078 0.184 0.002 0.0024 0.000 0.551 0.409 0.000 0.0001 0.980 0.345 0.391 0.000 0.000

62 0.001 0.010 0.000 0.998 0.998

Set 4: W3

CI Variance Proportions for Coefficients ofINT aX1 BX2 BX3 W3

1 0.005 0.010 0.010 0.000 0.0003 0.013 0.016 0.191 0.000 0.0004 0.000 0.568 0.402 0.000 0.0001 0.980 0.345 0.391 0.000 0.000

183 0.001 0.000 0.000 1.000 1.000

Set 5: W4

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4

1 0.005 0.010 0.010 0.000 0.0003 0.013 0.076 0.191 0.000 0.0004 0.000 0.569 0.400 0.000 0.0008 0.919 0.345 0.396 0.000 0.000

633 0.002 0.000 0.003 1.000 1.000

Appendix 2: MIxeD Besel ine

-- The Z.·- ..set 6: ZO

CI Verience Proportions for Coefficients oflilT IX1 BX2 BX3 ZO

1 0.005 0.006 0.010 0.011 0.0074 0.010 0.096 0.064 0.278 0.1404 0.000 0.035 0.514 0.335 0.0017 0.392 0.199 0.029 0.316 0.5048 0.592 0.664 0.383 0.060 0.348

Set 7: 21

CI Verience Proportions for Coefficients oflilT BX1 BX2 BX3 Z1

1 0.005 0.001 0.007 0.011 0.0014 0.008 0.030 0.140 0.161 0.0114 0.000 0.000 0.314 0.548 0.0017 0.987 0.006 0.233 0.280 0.007

20 0.000 0.963 0.305 0.000 0.980

Set 8: Z2

CI Verience Proportions for Coefficients oflilT BX1 BX2 BX3 22

1 0.005 0.000 0.001 0.011 0.0004 0.008 0.004 0.026 0.184 0.0014 0.000 0.000 0.067 0.520 0.0007 0.985 0.001 0.045 0.285 0.001

63 0.002 0.996 0.861 0.000 0.998

Set 9: Z3

CI Verience Proportions for Coefficients oflilT BX1 BX2 BX3 Z3

1 0.005 0.000 0.000 0.011 0.0004 0.008 0.000 0.003 0.184 0.0004 0.000 0.000 0.007 0.516 0.0007 0.986 0.000 0.005 0.284 0.000

210 0.001 1.000 0.985 0.006 1.000

Set 10: 24

CI Veriance Proportions for Coefficients oflilT BX1 BX2 BX3 24

1 0.005 0.000 0.000 0.010 0.0004 O.OOS 0.000 0.000 0.183 0.0004 0.000 0.000 0.001 0.511 0.0007 0.985 0.000 0.001 0.281 0.000

640 0.002 1.000 0.998 0.014 1.000

228

Appendix 2: MIXED Baseline

-- The wa end Zs *-Set 11: WO end ZO

CI Variance Proportiona for Coefficients oflilT BX1 BX2 BX3 WO ZO

1 0.004 0.004 0.007 0.004 0.006 0.0053 0.002 0.031 0.016 0.044 0.168 0.0544 0.012 0.102 0.507 0.002 0.012 0.0466 0.263 0.040 0.098 0.102 0.283 0.2948 0.002 0.493 0.051 0.483 0.356 0.4799 0.717 0.330 0.320 0.365 0.176 0.122

set 12: WO end Z1

CI Variance Proportiona for Coefficients oflilT BX1 BX2 BX3 WO Z1

1 0.004 0.001 0.005 0.004 0.005 0.0013 0.003 0.008 0.013 0.044 0.182 0.0054 0.010 0.021 0.399 0.003 0.009 0.0057 0.341 0.007 0.152 0.253 0.427 0.0069 0.643 0.001 0.124 0.696 0.376 0.003

22 0.000 0.963 0.307 0.000 0.002 0.980

Set 13: WO end Z2

CI Variance Proportions for Coefficients oflilT BX1 BX2 BX3 WO Z2

1 0.004 0.000 0.001 0.004 0.005 0.0003 0.003 0.001 0.003 0.044 0.182 0.0004 0.009 0.003 0.082 0.002 0.007 0.0007 0.348 0.001 0.030 0.242 0.415 0.0019 0.635 0.000 0.024 0.706 0.385 0.000

68 0.001 0.996 0.860 0.002 0.006 0.998

Set 14: WO end Z3

CI Variance Proportions for Coefficients oflilT BX1 BX2 BX3 WO Z3

1 0.004 0.000 0.000 0.004 0.005 0.0003 0.003 0.000 0.000 0.044 0.183 0.0004 0.009 0.000 0.009 0.002 0.007 0.0007 0.347 0.000 0.003 0.243 0.420 0.0009 0.636 0.000 0.003 0.704 0.385 0.000

226 0.001 1.000 0.985 0.003 0.000 1.000

Set 15: WO Z4

CI Variance Proportions for Coefficients oflilT BX1 BX2 BX3 WO Z4

1 0.004 0.000 0.000 0.004 0.005 0.0003 0.003 0.000 0.000 0.044 0.183 0.0004 0.009 0.000 0.001 0.002 0.007 0.0007 0.347 0.000 0.000 0.241 0.419 0.0009 0.635 0.000 0.000 0.699 0.385 0.000

689 0.002 1.000 0.998 0.010 0.001 1.000

229

Appendix 2: MIXED Baseline

.- The Ws n Zs'-

Set 16: W1 n ZO

CI Variance Proportions for Coefficients ofliT BX1 BX2 IX3 W1 ZO

1 0.004 0.004 0.007 0.001 0.001 0.0053 0.001 0.033 0.023 0.012 0.015 0.0714 0.010 0.102 0.513 0.001 0.001 0.0427 0.435 0.162 0.064 0.004 0.005 0.5039 0.544 0.699 0.387 0.000 0.001 0.379

22 0.007 0.000 0.006 0.983 0.977 0.001

set 17: W1 n Z1

CI Variance Proportions for Coefficients oflifT IX1 IX2 IX3 W1 Z1

1 0.004 0.001 0.005 0.001 0.001 0.0013 0.002 0.009 0.021 0.013 0.017 0.0064 0.009 0.021 0.398 0.000 0.001 0.0058 0.980 0.006 0.268 0.003 0.005 0.008

22 0.002 0.598 0.230 0.349 0.341 0.60822 0.004 0.365 0.078 0.634 0.636 0.3n

Set 18: W1 n Z2

CI Variance Proportions for Coefficients ofliT IX1 IX2 IX3 W1 12

1 0.004 0.000 0.001 0.001 0.001 0.0003 0.002 0.001 0.005 0.013 0.017 0.0014 0.009 0.003 0.081 0.000 0.001 0.0008 0.978 0.001 0.052 0.003 0.005 0.001

22 0.006 0.000 0.001 0.981 0.976 0.00069 0.002 0.996 0.860 0.002 0.001 0.998

Set 19: W1 and Z3

CI Variance Proportions for Coefficients oflifT IX1 IX2 BX3 W1 Z3

1 0.004 0.000 0.000 0.001 0.001 0.0003 0.002 0.000 0.001 0.013 0.017 0.0004 0.009 0.000 0.009 0.000 0.001 0.0008 0.979 0.000 0.006 0.003 0.005 0.000

22 0.007 0.000 0.000 0.982 0.977 0.000229 0.001 1.000 0.985 0.001 0.000 1.000

Set 20: W1 n Z4

CI Variance Proportions for Coefficients ofliT IX1 IX2 BX3 W1 Z4

1 0.004 0.000 0.000 0.001 0.001 0.0003 0.002 0.000 0.000 0.013 0.016 0.0004 0.009 0.000 0.001 0.000 0.001 0.0008 0.977 0.000 0.001 0.003 0.005 0.000

22 0.007 0.000 0.000 0.949 0.951 0.000T06 0.002 1.000 0.998 0.035 0.026 1.000

230

ApPendix 2: MIXED lasel ine

-* The Ws and Zs -

set 21: W2 and ZO

CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 W2 ZO

1 0.004 0.004 0.007 0.000 0.000 0.0053 0.001 0.037 0.019 0.001 0.001 0.0754 0.010 0.098 0.531 0.000 0.000 0.0377 0.433 0.160 0.058 0.000 0.001 0.5069 0.551 0.694 0.385 0.000 0.000 0.376

67 0.002 0.007 0.000 0.998 0.998 0.000

set 22: W2 and Z1

CI Variance Proportions for Coefficients oflilT IX1 IX2 III W2 Z1

1 0.003 0.001 0.005 0.000 0.000 0.0013 0.001 0.010 0.017 0.002 0.002 0.0074 0.009 0.020 0.413 0.000 0.000 0.0048 0.984 0.006 0.260 0.000 0.000 0.008

22 0.000 0.959 0.305 0.000 0.000 0.97968 0.002 0.004 0.000 0.998 0.998 0.001

Set 23: W2 and Z2

CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 W2 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.001 0.001 0.004 0.002 0.002 0.0014 0.009 0.002 0.083 0.000 0.000 0.0008 0.983 0.001 0.050 0.000 0.000 0.001

66 0.003 0.346 0.311 0.586 0.584 0.35771 0.000 0.650 0.551 0.411 0.414 0.641

Set 24: W2 and Z3

CI Variance Proportions for Coefficients oflilT IX1 IX2 III W2 Z3

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.002 0.002 0.0004 0.009 0.000 0.009 0.000 0.000 0.0008 0.984 0.000 0.005 0.000 0.000 0.000

68 0.002 0.000 0.000 0.998 0.998 0.000229 0.001 1.000 0.985 0.000 0.000 1.000

Set 25: W2 and Z4

CI Variance Proportions for Coefficients oflilT IX1 aX2 BX3 W2 Z4

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.002 0.002 0.0004 0.009 0.000 0.001 0.000 0.000 0.0008 0.983 0.000 0.001 0.000 0.000 0.000

68 0.002 0.000 0.000 0.998 0.998 0.000698 0.002 1.000 0.998 0.000 0.000 1.000

231

Appendix 2: MIXED lasel ine

-- The wa rd zs *-set 26: W3 rd ZO

CI Veri~e Proportions for Coefficients oflilT 1X1 IX2 BX3 W3 ZO

1 0.004 0.004 0.001 0.000 0.000 0.0053 0.001 0.031 0.020 0.000 0.000 0.0114 0.010 0.099 0.530 0.000 0.000 0.0311 0.435 0.163 0.058 0.000 0.000 0.5059 0.550 0.698 0.385 0.000 0.000 0.311

199 0.001 0.000 0.000 1.000 1.000 0.000

Set 21: W3 rd Z1

CI Veriance Proportions for Coefficients oflilT IX1 IX2 IX3 W3 Z1

1 0.003 0.001 0.005 0.000 0.000 0.0013 0.001 0.010 0.011 0.000 0.000 0.0014 0.009 0.020 0.412 0.000 0.000 0.0048 0.984 0.006 0.260 0.000 0.000 0.008

22 0.000 0.961 0.305 0.000 0.000 0.918201 0.002 0.002 0.000 1.000 1.000 0.002

Set 28: W3 end Z2

CI Variance Proportions for Coefficients oflilT IX1 IX2 IX3 W3 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.001 0.001 0.004 0.000 0.000 0.0014 0.009 0.002 0.083 0.000 0.000 0.0008 0.983 0.001 0.050 0.000 0.000 0.001

69 0.002 0.996 0.861 0.000 0.000 0.998201 0.001 0.000 0.000 1.000 1.000 0.000

Set 29: W3 end Z3

CI Variance Proportions for Coefficients oflilT IX1 aX2 aX3 W3 Z3

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.000 0.000 0.0004 0.009 0.000 0.009 0.000 0.000 0.0008 0.984 0.000 0.005 0.000 0.000 0.000

201 0.001 0.001 0.001 0.999 0.999 0.001230 0.001 0.999 0.984 0.001 0.001 0.999

Set 30: W3 end Z4

CI Veriance Proportions for Coefficients ofINT IX1 aX2 IX3 W3 Z4

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.000 0.000 0.0004 0.009 0.000 0.001 0.000 0.000 0.0008 0.983 0.000 0.001 0.000 0.000 0.000

201 0.001 0.000 0.000 0.995 0.995 0.000100 0.002 1.000 0.998 0.005 0.004 1.000

232

Appendix 2: MIXED Basel ine

*** The Ws and Zs -*Set 31: W4 and ZO

CI Variance Proportions for Coefficients ofINT 1X1 IX2 BX3 W4 ZO

1 0.004 0.004 0.007 0.000 0.000 0.0053 0.001 0.037 0.020 0.000 0.000 0.0774 0.010 0.099 0.530 0.000 0.000 0.0377 0.434 0.163 0.058 0.000 0.000 0.5059 0.550 0.697 0.384 0.000 0.000 0.376

688 0.002 0.000 0.002 1.000 1.000 0.001

Set 32: W4 and Z1

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z1

1 0.003 0.001 0.005 0.000 0.000 0.0013 0.001 0.010 0.017 0.000 0.000 0.0074 0.009 0.020 0.410 0.000 0.000 0.0048 0.984 0.006 0.259 0.000 0.000 0.008

22 0.000 0.961 0.304 0.000 0.000 0.978695 0.002 0.002 0.005 1.000 1.000 0.002

Set 33: W4 end Z2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z2

1 0.003 0.000 0.001 0.000 0.000 0.0003 0.001 0.001 0.004 0.000 0.000 0.0014 0.009 0.002 0.083 0.000 0.000 0.0008 0.982 0.001 0.050 0.000 0.000 0.001

69 0.002 0.990 0.854 0.000 0.000 0.993697 0.002 0.005 0.008 1.000 1.000 0.005

Set 34: W4 and Z3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z3

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.000 0.000 0.0004 0.009 0.000 0.009 0.000 0.000 0.0008 0.984 0.000 0.005 0.000 0.000 0.000

230 0.001 0.993 0.978 0.000 0.000 0.993697 0.002 0.007 0.008 1.000 1.000 0.006

Set 35: W4 Z4

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z4

1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.000 0.000 0.0004 0.009 0.000 0.001 0.000 0.000 0.0008 0.982 0.000 0.001 0.000 0.000 0.000

691 0.000 0.353 0.354 0.631 0.630 0.353703 0.004 0.647 0.644 0.369 0.370 0.647

233

APPENDIX 3

MIXED MODEL EXPERIMENT 1COLLINEARITV DIAGNOSTICS

- The Ws-

Set 1: WO

Veriebl.. in Z: INT

CI Veriance Proportions for Coefficients ofINT IX1 IX2 IX3 WO

1 0.052 0.039 0.039 0.037 0.0352 0.059 0.143 0.132 0.063 0.1112 0.000 0.461 0.505 0.001 0.0003 0.781 0.357 0.303 0.003 0.0534 0.109 0.000 0.021 0.897 0.802

Veriebles in Z: INT IX1

CI Veriance Proportions for Coefficients ofINT IX1 IX2 IX3 we

1 0.057 0.035 0.044 0.040 0.0381 0.051 0.175 0.110 0.061 0.0972 0.003 0.546 0.446 0.002 0.0012 0.775 0.239 0.381 0.002 0.0653 0.114 0.004 0.019 0.894 0.798

Veriebles in Z: INT IX1 IX2

CI Verience Proportions for Coefficients ofINT IX1 IX2 IX3 we

1 0.077 0.034 0.002 0.066 0.0641 0.116 0.396 0.059 0.031 0.0591 0.005 0.054 0.931 0.000 0.0012 0.730 0.514 0.008 0.003 0.0453 o.on 0.001 0.000 0.900 0.830

Veriebles in Z: INT IX1 IX2 IX3

CI Verienee Proportions for Coefficients ofINT IX1 IX2 IX3 we

0.324 0.309 0.025 0.000 0.0130.000 0.001 0.081 0.465 0.4380.001 0.013 0.614 0.314 0.0590.004 0.040 0.265 0.221 0.4840.671 0.637 0.015 0.000 0.006

Veriebles in Z: INT IX1 IX2 IX3 weCI Verienee Proportions for Coefficients of

INT IX1 IX2 IX3 we0.337 0.322 0.033 0.000 0.0000.000 0.000 0.001 0.474 0.5250.000 0.000 0.001 0.524 0.4750.004 0.050 0.945 0.002 0.0000.659 0.628 0.020 0.000 0.000

Appendix 3: MIXED Experiment 1

. - The wa-

set 2: \11

Vari8bl.. in Z: INT

Cl Variance Proportions for Coefficients ofINT IX1 IX2 BX3 WO

1 0.044 0.033 0.029 0.006 0.0062 0.062 0.119 0.197 0.011 0.0132 0.000 0.532 0.434 0.000 0.0003 0.889 0.315 0.330 0.002 0.0029 0.004 0.000 0.010 0.981 0.979

Vari8blea in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO

1 0.048 0.028 0.032 0.007 0.0071 0.056 0.169 0.143 0.010 0.0122 0.003 0.576 0.418 0.000 0.0003 0.889 0.228 0.396 0.002 0.0029 0.003 0.000 0.011 0.981 0.979

Vari8bles in Z: INT IX1 IX2

Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO

1 0.062 0.024 0.001 0.011 0.0111 0.118 0.424 0.080 0.006 0.0062 0.007 0.070 0.911 0.000 0.0002 0.810 0.482 0.008 0.003 0.0048 0.004 0.000 0.000 0.980 0.979

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO

0.325 0.312 0.026 0.000 0.0080.001 0.005 0.002 0.490 0.4570.004 0.038 0.948 0.010 0.0000.000 0.003 0.010 0.500 0.5330.669 0.641 0.014 0.000 0.003

Variables in Z: INT IX1 IX2 IX3 \11

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO

0.333 0.319 0.031 0.000 0.0000.000 0.000 0.000 0.491 0.5060.000 0.001 0.003 0.507 0.4920.004 0.046 0.947 0.001 0.0030.663 0.635 0.019 0.000 0.000

235

Appendix 3: MIXED Experilllent 1

.- The wa'-

set 3: W2

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2

1 0.043 0.031 0.029 0.001 0.0012 0.063 0.138 0.183 0.001 0.0012 0.000 0.500 0.465 0.000 0.0003 0.893 0.318 0.322 0.000 0.000

28 0.001 0.013 0.001 0.998 0.998

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2

1 0.047 0.026 0.032 0.001 0.0012 0.058 0.185 0.140 0.001 0.0012 0.003 0.559 0.437 0.000 0.0003 0.892 0.230 0.389 0.000 0.000

27 0.000 0.000 0.001 0.998 0.998

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2

1 0.060 0.023 0.001 0.001 0.0011 0.119 0.429 0.075 0.001 0.0012 0.007 0.067 0.915 0.000 0.0002 0.812 0.481 0.008 0.000 0.000

26 0.003 0.000 0.000 0.998 0.998

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients of(NT IX1 IX2 IX3 W2

0.328 0.318 0.027 0.000 0.0000.001 0.000 0.001 0.427 0.4300.004 0.037 0.957 0.002 0.0000.002 0.003 0.001 0.568 0.5640.665 0.642 0.015 0.002 0.006

Variables in Z: INT IX1 IX2 IX3 W2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2

0.335 0.321 0.032 0.000 0.0000.000 0.000 0.000 0.486 0.4870.003 0.045 0.934 0.008 0.0090.000 0.004 0.014 0.504 0.5030.661 0.630 0.020 0.001 0.000

236

..

Appendix 3: MIXED Experiment

*** The ... ***set 4: W3

V.riabl.. in Z: INT

CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 W3

1 0.043 0.032 0.029 0.000 0.0002 0.063 0.134 0.190 0.000 0.0002 0.000 0.515 0.456 0.000 0.0003 0.892 0.319 0.325 0.000 0.000

84 0.002 0.000 0.000 1.000 1.000

V.riables in Z: INT IX1

CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 W3

1 0.046 0.026 0.032 0.000 0.0002 0.058 0.183 0.140 0.000 0.0002 0.003 0.560 0.436 0.000 0.0003 0.890 0.230 0.391 0.000 0.000

82 0.003 0.000 0.000 1.000 1.000

V.riables in z: INT IX1 IX2

CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 W3

1 0.059 0.023 0.001 0.000 0.0001 0.119 0.430 0.076 0.000 0.0002 0.007 0.067 0.915 0.000 0.0002 0.809 0.481 0.008 0.000 0.000

78 0.006 0.000 0.000 1.000 1.000

V.riables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3

1 0.051 0.039 0.003 0.243 0.2621 0.275 0.278 0.024 0.059 0.0351 0.004 0.037 0.959 0.000 0.0001 0.601 0.617 0.014 0.046 0.0322 0.069 0.029 0.001 0.652 0.671

V.riables in Z: INT IX1 IX2 IX3 W3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3

0.333 0.316 0.037 0.002 0.0000.000 0.000 0.000 0.412 0.4150.004 0.055 0.941 0.000 0.0000.003 0.016 0.000 0.575 0.5760.660 0.614 0.022 0.011 0.009

237

Appendix 3: MIXED ExperiMnt

-- The wa--

Set 5: W4

Variebles in Z: INT

CI Variance Proportions for Coefficient. ofINT IX1 IX2 BX3 W4

1 0.043 0.031 0.029 0.000 0.0002 0.063 0.134 0.190 0.000 0.0002 0.000 0.515 0.455 0.000 0.0003 0.892 0.318 0.324 0.000 0.000

298 0.002 0.001 0.002 1.000 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4

1 0.046 0.026 0.032 0.000 0.0002 0.058 0.183 0.140 0.000 0.0002 0.003 0.560 0.436 0.000 0.0003 0.891 0.230 0.390 0.000 0.000

290 0.002 0.000 0.002 1.000 1.000

Variebles in Z: INT IX1 IX2

CI

1122

271

Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4

0.059 0.023 0.001 0.000 0.0000.119 0.430 0.077 0.000 0.0000.007 0.068 0.914 0.000 0.0000.815 0.479 O.ooa 0.000 0.0000.000 0.000 0.000 1.000 1.000

Variebles in Z: INT IX1 BX2 BX3

•

CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 W4

1 0.000 0.000 0.000 0.086 0.0861 0.328 0.318 0.027 0.000 0.0001 0.004 0.038 0.959 0.000 0.0002 0.667 0.644 0.015 0.000 0.0003 0.001 0.000 0.000 0.914 0.914

Variables in Z: INT BX1 BX2 BX3 W4

CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 W4

1 0.000 0.000 0.000 0.152 0.1521 0.329 0.318 0.027 0.000 0.0001 0.004 0.039 0.957 0.000 0.0002 0.665 0.643 0.015 0.000 0.0002 0.002 0.000 0.000 0.848 0.848

238

Appendix 3: MIXED Experillll!nt

••• The Zs -

Set 6: ZO

Veriables in Z: INT

CI VerilnCe Proportions for Coefficients ofINT IX1 IX2 8X3 ZO

1 0.050 0.036 0.038 0.037 0.0382 0.041 0.111 0.104 0.249 0.0912 0.000 0.023 0.509 0.384 0.0043 0.806 0.002 0.133 0.330 0.0983 0.102 0.828 0.216 0.000 0.768

Veriables in Z: INT IX1

CI VerilnCe Proportions for Coefficients ofINT IX1 IX2 IX3 ZO

1 0.090 0.060 0.083 0.069 0.0241 0.010 0.157 0.064 0.027 0.4892 0.000 0.344 0.028 0.612 0.0072 0.027 0.276 0.451 0.085 0.4632 0.873 0.163 0.375 0.207 0.018

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO

1 0.194 0.141 0.007 0.147 0.0051 0.005 0.061 0.236 0.032 0.5811 0.005 0.020 0.742 0.068 0.1681 0.002 0.455 0.012 0.368 0.2332 0.795 0.323 0.004 0.385 0.012

Variables in Z: INT IX1 IX2 aX3

CI Variance Proportions for Coefficients ofINT aX1 ax2 aX3 zo

0.297 0.317 0.018 0.000 0.0310.029 0.012 0.421 0.000 0.4760.000 0.000 0.001 0.999 0.0000.061 0.010 0.554 0.000 0.4150.613 0.662 0.006 0.000 0.078

Variables in Z: INT aX1 IX2 IX3 ZO

CI Variance Proportions for Coefficients ofINT IX1 aX2 IX3 ZO

0.337 0.325 0.037 0.000 0.0030.000 0.014 0.273 0.000 0.7010.000 0.000 0.001 0.999 0.0000.011 0.036 0.673 0.001 0.2920.651 0.626 0.016 0.000 0.004

239

Apeendix 3: MIXED Experiment 1

... The Zs ...

Set 7: Z1

Variables in Z: INT

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX! Z1

1 0.043 0.007 0.023 0.031 0.D072 0.045 0.022 0.131 0.193 0.0112 0.000 0.001 0.353 0.504 O.ODO3 0.912 0.002 0.176 0.272 0.0059 0.000 0.968 0.317 0.000 0.976

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z1

1 0.080 0.035 0.072 0.060 0.0361 0.016 0.215 0.038 0.028 0.2352 0.000 0.215 0.060 0.675 0.0022 0.769 0.445 0.002 0.214 0.1632 0.134 0.089 0.828 0.023 0.564

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z1

1 0.157 0.146 0.003 0.107 0.0501 0.050 0.070 0.113 0.143 0.3541 0.007 0.022 0.848 0.086 0.0232 0.130 0.225 0.037 0.454 0.3872 0.656 0.538 0.000 0.210 0.186

Variables in Z: INT IX1 IX2 IX!

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1

1 0.142 0.239 0.000 0.000 0.1631 0.142 0.000 0.610 0.001 0.1371 0.000 0.000 0.001 0.998 0.0001 0.410 0.001 0.384 0.001 0.2562 0.307 0.760 0.004 0.000 0.443

Variables in Z: INT IX1 IX2 IX! Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 8X! . Z1

0.237 0.291 0.021 O.ODO 0.1010.054 0.004 0.539 0.001 0.3360.000 O.DOO 0.002 0.998 0.0000.207 0.010 0.439 0.001 0.4010.502 0.695 0.000 0.000 0.162

240

" Appendix 3: MIXED Expe~;.ent 1

*** The Zs ***set 8: Z2

V.~i~l .. in Z: INT

CI V.~ianc. P~opo~t;ona fo~ Coefficients ofINT IX1 IX2 IX3 Z2

1 0.042 0.001 O.OOS 0.030 0.0012 0.047 0.002 0.023 0.212 0.0012 0.000 0.000 0.071 0.476 0.0003 0.910 0.000 0.032 0.282 0.001

28 0.001 0.997 0.869 0.000 0.998

V.~i~l.. in Z: INT IX1

Cl V.~;ance P~opo~tions fo~ Coefficients oflNT IX1 IX2 IX3 Z2

1 0.060 0.Q08 0.015 0.041 0.0111 0.039 0.107 0.005 0.056 0.0252 0.000 0.124 0.009 0.653 0.0002 0.899 0.146 0.018 0.249 0.0027 0.002 0.616 0.953 0.001 0.962

V.~i~les in Z: lNT IX1 IX2

Cl V.~iance P~opo~tiona fo~ Coefficients ofINT IX1 IX2 IX3 Z2

1 0.074 0.062 0.001 0.037 0.0591 0.114 0.015 0.212 0.245 0.0451 0.015 0.030 0.646 0.183 0.0052 0.776 0.009 0.031 0.535 0.0223 0.021 0.884 0.111 0.001 0.869

V.~iables in Z: lNT IX1 IX2 IX3

CI V.~iance P~opo~tions fo~ Coefficients ofINT IX1 IX2 IX3 Z2

1 0.065 0.076 0.002 0.000 0.0751 0.148 0.001 0.661 0.001 0.0121 0.000 0.000 0.001 0.999 0.0002 0.769 0.033 0.214 0.000 0.0283 0.019 0.890 0.122 0.000 0.885

V.~iabl .. in Z: INT IX1 IX2 IX3 Z2

CI V.~iance P~opo~tions fo~ Coefficients ofINT IX1 IX2 IX3 Z2

1 0.071 0.127 0.001 0.000 0.1191 0.132 0.000 0.627 0.001 0.0471 0.000 0.000 0.001 0.999 0.0001 0.770 0.032 0.224 0.001 0.0282 0.028 0.841 0.147 0.000 0.807

241

Appendix 3: MIXED Experilllent 1

*** The Zs ***Set 9: 13

Variebl.. in Z: lIT

CI Variance Proportions for Coefficients oflIT IX1 IX2 8X3 13

1 0.041 0.000 0.001 0.030 0.0002 0.041 0.000 0.003 0.211 0.0002 0.000 0.000 0.008 0.413 0.0003 0.911 0.000 0.004 0.280 0.000

91 0.000 1.000 0.986 0.001 1.000

Variebl.. in Z: lIT IX1

CI Variance Proportions for Coefficients oflIT IX1 IX2 IX3 13

1 0.058 0.001 0.002 0.039 0.0011 0.041 0.016 0.001 0.058 0.0022 0.000 0.019 0.001 0.649 0.0002 0.901 0.022 0.002 0.245 0.000

21 0.000 0.943 0.995 0.009 0.996

Variables in Z: lIT IX1 IX2

CI Variance Proportions for Coefficients oflIT IX1 IX2 IX3 13

1 0.058 0.008 0.001 0.021 0.0081 0.128 0.002 0.115 0.254 0.0042 0.016 0.004 0.341 0.184 0.0002 0.793 0.002 0.015 0.525 0.002

10 0.005 0.984 0.522 0.010 0.986

Variables in Z: lIT IX1 IX2 IX3

CI Variance Proportions for Coefficients oflIT IX1 IX2 BX3 13

1 0.052 0.009 0.002 0.000 0.0081 0.150 0.000 0.329 0.001 0.0011 0.000 0.000 0.000 0.999 0.0002 0.796 0.005 0.099 0.000 0.002

10 0.002 0.981 0.570 0.000 0.988

Variables in Z: INT BX1 BX2 BX3 13

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13

1 0.045 0.016 0.000 0.000 0.0151 0.140 0.000 0.319 0.000 0.0041 0.000 0.000 0.000 0.999 0.0002 0.812 0.005 0.098 0.001 0.0028 0.002 0.979 0.583 0.000 0.979

242

Appendix 3: MIXED Experiment 1

*** The Zs ***set 10: Z4

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 &X2 IX3 Z4

1 0.041 0.000 0.000 0.030 0.0002 0.047 0.000 0.000 0.210 0.0002 0.000 0.000 0.001 0.472 0.0003 0.911 0.000 0.000 0.278 0.000

271 0.000 1.000 0.998 0.010 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4

1 0.057 0.000 0.000 0.039 0.0001 0.042 0.002 0.000 0.058 0.0002 0.000 0.002 0.000 0.651 0.0002 0.901 0.003 0.000 0.245 0.000

63 0.000 0.993 0.999 0.008 1.000

Variables in z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4

1 0.057 0.001 0.000 0.025 0.0011 0.130 0.000 0.024 0.259 0.0012 0.015 0.001 0.075 0.181 0.0002 0.798 0.000 0.003 0.525 0.000

27 0.001 0.998 0.897 0.010 0.998

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4

1 0.051 0.001 0.000 0.000 0.0011 0.149 0.000 0.071 0.001 0.0001 0.000 0.000 0.000 0.999 0.0002 0.799 0.001 0.021 0.000 0.000

27 0.000 0.998 0.907 0.000 0.998

Variables in Z: INT IX1 IX2 IX3 Z4

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4

1 0.044 0.002 0.000 0.000 0.0021 0.140 0.000 0.069 0.000 0.0011 0.000 0.000 0.000 0.999 0.0002 0.816 0.001 0.021 0.001 0.000

21 0.000 0.997 0.910 0.000 0.997

243

Appendix 3: MIXED Experiment 1

-- The Ws lIIld Zs --

set 11: WO lIIld ZO •Vari8bles in Z: INT

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO ZO

1 0.038 0.024 0.028 0.021 0.019 0.0251 0.001 0.054 0.007 0.070 0.098 0.0632 0.035 0.092 0.550 0.009 0.012 0.0192 0.716 0.001 0.187 0.000 0.068 0.1224 0.153 0.760 0.228 0.045 0.036 0.6904 0.056 0.071 0.000 0.855 0.766 0.081

Vari8bles in Z: INT IX1

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO ZO

1 0.055 0.031 0.043 0.037 0.035 0.0111 0.016 0.012 0.138 0.053 o.on 0.2162 0.035 0.363 0.002 0.012 0.027 0.2792 0.051 0.419 0.339 0.001 0.007 0.4582 0.729 0.170 0.463 0.003 0.061 0.0363 0.113 0.005 0.015 0.895 0.798 0.000

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO zo

1 0.077 0.034 0.002 0.066 0.064 0.0001 0.067 0.326 0.001 0.022 0.042 0.2251 0.020 0.009 0.701 0.005 0.008 0.1972 0.060 0.092 0.291 0.006 0.014 0.5502 0.704 0.539 0.005 0.002 0.043 0.0283 o.on 0.002 0.000 0.899 0.830 0.001

Variables in z: INT IX1 IX2 IX3

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO ZO

0.294 0.309 0.017 0.000 0.011 0.0300.028 0.013 0.412 0.000 0.003 0.4820.000 0.003 0.017 0.510 0.450 0.0040.001 0.003 0.052 0.482 0.454 0.0190.060 0.017 0.495 0.007 0.078 0.3870.617 0.655 0.007 0.000 0.005 0.077

Variables in Z: INT IX1 IX2 IX3 WO ZO

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO zo

0.345 0.329 0.046 0.000 0.000 0.0050.002 0.019 0.283 0.001 0.000 0.6810.000 0.000 0.001 0.4n 0.527 0.0000.000 0.000 0.003 0.524 0.473 0.0000.015 0.043 0.643 0.002 0.000 0.3060.638 0.604 0.023 0.000 0.000 0.007

244

Appendix 3: MIXED Experiment 1

-- The Ws end Zs --

Set 12: YO end Z1

Veriables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 YO Z1

1 D.035 0.005 0.018 0.018 0.015 0.0052 0.000 0.011 0.000 0.069 0.101 0.0092 0.043 0.013 0.430 0.013 0.017 0.0033 0.822 0.003 0.219 0.001 0.062 0.0074 0.101 0.000 0.012 0.897 0.799 0.0009 0.000 0.968 0.320 0.002 0.005 0.976

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 YO Z1

1 0.053 0.022 0.039 0.034 0.031 0.0171 0.001 0.031 0.095 0.032 0.042 0.2122 0.047 0.347 0.009 0.036 0.061 0.0412 0.783 0.390 0.089 0.001 0.062 0.0463 0.008 0.210 0.736 0.009 0.007 0.6783 0.108 0.001 0.032 0.889 0.797 0.007

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z1

1 0.076 0.036 0.001 0.063 0.061 0.0061 0.014 0.215 0.006 0.022 0.029 0.273

• 1 0.033 0.014 0.827 0.005 0.012 0.0312 0.325 0.040 0.165 0.009 0.047 0.3842 0.486 0.695 0.001 0.001 0.019 0.3043 0.067 0.000 0.000 0.900 0.832 0.003

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 YO Z1

1 0.143 0.237 0.000 0.000 0.002 0.1611 0.143 0.000 0.524 0.000 0.068 0.1481 0.000 0.001 0.065 0.4n 0.445 0.0001 0.001 0.001 0.068 0.525 0.413 0.0021 0.406 0.000 0.339 0.002 0.071 0.2472 0.307 0.760 0.004 0.000 0.000 0.442

Variables in Z: INT IX1 IX2 IX3 YO Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 YO Z1

0.216 0.290 0.021 0.000 0.000 0.1320.074 0.005 0.558 0.001 0.000 0.2860.000 0.000 0.001 0.483 0.515 0.0000.000 0.000 0.001 0.514 0.485 0.0000.302 0.005 0.418 0.001 0.000 0.3480.408 0.701 0.001 0.000 0.000 0.234

245

246

Appendix 3: MIXED Experflllent 1

Appendix 3: MIXED Experiment 1

*** The wa end Zs *-set 14: WO end 13

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 &X2 BX3 WO 13

1 0.034 0.000 0.000 0.017 0.015 0.0002 0.000 0.000 0.000 0.072 0.102 0.0002 0.042 0.000 0.009 0.012 0.015 0.0003 0.819 0.000 0.004 0.001 0.064 0.0004 0.105 0.000 0.000 0.896 0.803 0.000

95 0.000 1.000 0.986 0.003 0.000 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO 13

1 0.045 0.001 0.001 0.026 0.024 0.0011 0.003 0.005 0.001 0.034 0.037 0.0032 0.055 0.024 0.000 0.042 0.073 0.0002 0.787 0.027 0.002 0.001 0.066 0.0004 0.110 0.001 0.000 0.893 0.800 0.000

22 0.000 0.943 0.995 0.004 0.000 0.996

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO 13

1 0.055 O.OOS 0.000 0.026 0.023 0.0051 0.004 0.006 0.009 0.062 0.074 0.0062 0.031 0.001 0.428 O.OOS 0.010 0.0002 0.836 0.004 0.040 0.007 0.061 0.0033 0.069 0.000 0.000 0.896 0.832 0.000

10 0.005 0.984 0.522 0.004 0.000 0.986

Vari ables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO 13

1 0.052 0.009 0.002 0.000 0.001 O.ooa1 0.153 0.000 0.311 0.000 0.034 0.0011 0.000 0.000 0.010 0.504 0.456 0.0001 0.001 0.000 0.012 0.496 0.485 0.0002 0.7'92 0.005 0.095 0.001 0.023 0.002

10 0.003 0.987 0.570 0.000 0.001 0.988

Variables in Z: INT IX1 IX2 IX3 WO 13

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO 13

1 0.041 0.015 0.000 0.000 0.000 0.0141 0.161 0.000 0.292 0.001 0.000 0.0031 0.000 0.000 0.000 0.535 0.464 0.0001 0.000 0.000 0.000 0.463 0.536 0.0002 0.7'96 0.005 0.103 0.001 0.000 0.0018 0.002 0.981 0.604 0.000 0.000 0.981

247

Appendix 3: MIXED Experilller'lt 1

*** The ... end Zs ***

set 15: WI) end Z4

Vari~les in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 WI) Z4

1 0.034 0.000 0.000 0.011 0.015 0.0002 0.000 0.000 0.000 0.011 0.102 0.0002 0.042 0.000 0.001 0.012 0.015 0.0003 0.819 0.000 0.000 0.001 0.064 0.0004 0.105 0.000 0.000 0.891 0.802 0.000

284 0.000 1.000 0.998 0.009 0.001 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z4

1 0.045 0.000 0.000 0.025 0.024 0.0001 0.003 0.001 0.000 0.035 0.031 0.0002 0.055 0.003 0.000 0.041 0.013 0.0002 0.781 0.003 0.000 0.001 0.065 0.0004 0.110 0.000 0.000 0.891 0.7'99 0.000

61 0.000 0.993 0.999 0.001 0.001 1.000

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z4

1 0.055 0.001 0.000 0.025 0.022 0.0011 0.004 0.001 0.002 0.063 0.014 0.0012 0.031 0.000 0.092 0.005 0.010 0.0002 0.839 0.001 O.ooa 0.001 0.062 0.0003 0.069 0.000 0.000 0.891 0.830 0.000

28 0.001 0.998 0.891 0.010 0.002 0.998

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z4

1 0.051 0.001 0.000 0.000 0.001 0.0011 0.153 0.000 0.061 0.000 0.036 0.0001 0.000 0.000 0.002 0.502 0.456 0.0001 0.001 0.000 0.003 0.498 0.482 0.0002 0.795 0.001 0.020 0.001 0.024 0.000

21 0.000 0.998 0.901 0.000 0.002 0.998

Variables in Z: INT IX1 IX2 IX3 WI) Z4

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z4

1 0.040 0.002 0.000 0.000 0.000 0.0021 0.161 0.000 0.058 0.001 0.000 0.0001 0.000 0.000 0.000 0.536 0.463 0.0001 0.000 0.000 0.000 0.462 0.531 0.0002 0.7'99 0.001 0.020 0.001 0.000 0.000

22 0.000 0.991 0.922 0.000 0.000 0.998

248

Appendix 3: MIXED Experiment 1

- The wa ..:I Zs *-set 16: W1 ..:I ZO

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT 1X1 IX2 IX3 W1 ZO

1 0.035 0.021 0.023 0.004 0.004 0.0211 0.003 0.050 0.020 0.012 0.013 0.0692 0.035 0.099 0.535 0.001 0.001 0.0233 0.822 0.002 0.200 0.003 0.003 0.1194 0.100 0.828 0.215 0.000 0.000 0.769

10 0.004 0.001 0.008 0.981 0.979 0.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 ZO

1 0.048 0.025 0.032 0.007 0.007 0.0071 0.022 0.019 0.151 0.008 0.009 0.2022 0.032 0.380 0.002 0.003 0.003 0.2912 0.046 0.401 0.350 0.000 0.001 0.4753 0.849 0.175 0.455 0.002 0.002 0.0269 0.003 0.000 0.009 0.981 0.979 0.000

VariabLes in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 ZO

1 0.062 0.024 0.001 0.011 0.011 0.0001 0.067 0.350 0.002 0.004 0.004 0.2361 0.022 0.011 0.699 0.001 0.001 0.1992 0.061 0.109 0.292 0.001 0.001 0.5382 0.785 0.506 0.006 0.003 0.003 0.0278 0.004 0.000 0.000 0.980 0.979 0.000

VariabLes in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 ZO

0.295 0.312 0.018 0.000 0.007 0.0300.016 0.015 0.297 0.099 0.153 0.3560.012 0.000 0.119 0.402 0.304 0.1190.001 0.001 0.056 0.473 0.505 0.0100.061 0.013 0.504 0.026 0.030 0.4070.615 0.659 0.006 0.000 0.002 0.078

VariabLes in Z: INT IX1 IX2 IX3 W1 ZO

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 ZO

0.342 0.324 0.051 0.000 0.000 0.0080.001 0.016 0.312 0.002 0.000 0.6540.000 0.000 0.001 0.487 0.509 0.0000.000 0.000 0.001 0.510 0.491 0.0010.017 0.058 0.611 0.000 0.000 0.3340.640 0.602 0.024 0.000 0.000 0.003

249

Appendix 3: MIXED Experiment 1

*** The Ws 8nd zs ***set 17: W1 8nd Z1

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W1 Z1

1 0.032 0.004 0.015 0.003 0.003 0.0042 0.001 0.011 0.006 0.012 0.013 0.0102 0.044 0.014 0.426 0.001 0.002 0.0033 0.919 0.003 0.234 0.002 0.003 0.0069 0.000 0.916 0.318 0.035 0.032 0.926

10 0.004 0.051 0.002 0.947 0.947 0.050

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1

1 0.046 0.019 0.030 0.006 0.006 0.0121 0.005 0.011 0.112 0.005 0.007 0.1932 0.046 0.380 0.007 0.005 0.006 0.0613 0.823 0.464 0.028 0.002 0.002 0.1163 0.077 0.127 0.821 0.000 0.000 0.6149 0.003 0.000 0.004 0.981 0.979 0.003

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1

1 0.061 0.024 0.001 0.010 0.011 0.0031 0.018 0.228 0.004 0.003 0.003 0.2822 0.029 0.014 0.852 0.001 0.001 0.0252 0.338 0.060 0.142 0.004 0.005 0.4122 0.551 0.673 0.000 0.001 0.001 0.2758 0.003 0.000 0.000 0.980 0.979 0.004

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1

1 0.142 0.239 0.000 0.000 0.000 0.1631 0.122 0.000 0.339 0.047 0.244 0.1211 0.019 0.000 0.219 0.481 0.221 0.0181 0.006 0.000 0.155 0.455 0.417 0.0051 0.412 0.002 0.282 0.016 0.114 0.2442 0.299 0.759 0.004 0.000 0.004 0.449

Variables in Z: INT IX1 IX2 IX3 W1 Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1

0.212 0.284 0.021 0.000 0.000 0.1330.073 0.005 0.564 0.004 0.000 0.2790.000 0.000 0.001 0.488 0.507 0.0000.000 0.000 0.005 0.506 0.492 0.0000.310 0.005 0.409 0.002 0.000 0.3490.406 0.707 0.001 0.000 0.000 0.239

250

Appendix 3: MIXED Experiment'

.- The Ws and Zs --

set 18: W1 and Z2

Variebles in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z2

1 0.031 0.000 0.003 0.003 0.003 0.0002 0.001 0.001 0.001 0.012 0.013 0.0012 0.043 0.002 0.083 0.001 0.002 0.0003 0.920 0.000 0.043 0.002 0.003 0.001

10 0.004 0.000 0.002 0.980 0.978 0.00030 0.001 0.997 0.868 0.001 0.001 0.998

Variebles in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W1 Z2

1 0.043 0.007 0.007 0.005 0.005 0.0041 0.000 0.016 0.017 0.005 0.006 0.0272 0.056 0.186 0.002 0.007 0.007 0.0043 0.896 0.176 0.021 0.002 0.003 0.0037 0.002 0.615 0.953 0.000 0.000 0.9619 0.003 0.001 0.000 0.980 0.979 0.001

Variebles in Z: INT BX1 BX2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W1 Z2

1 0.057 0.022 0.000 0.007 0.007 0.0201 0.000 0.064 0.014 0.007 0.007 0.0752 0.028 0.009 0.803 0.001 0.001 0.0022 0.889 0.020 0.072 0.004 0.005 0.0344 0.021 0.883 0.111 0.000 0.000 0.8679 0.004 0.002 0.000 0.980 0.979 0.002

Variables in Z: INT BX1 BX2 BX3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W1 Z2

1 0.065 0.076 0.002 0.000 0.000 0.0751 0.131 0.000 0.500 0.036 0.153 0.0111 0.017 0.000 0.135 0.474 0.315 0.0011 0.000 0.000 0.040 0.488 0.511 0.0002 0.769 0.032 0.201 0.002 0.018 0.0273 0.018 0.890 0.122 0.000 0.003 0.885

Variables in Z: INT BX1 BX2 BX3 W1 Z2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W1 Z2

1 0.064 0.117 0.001 0.000 0.000 0.1101 0.147 0.000 0.602 0.001 0.000 0.041, 0.000 0.000 0.000 0.495 0.502 0.000, 0.000 0.000 0.001 0.504 0.498 0.000, 0.764 0.029 0.236 0.001 0.000 0.0203 0.024 0.854 0.160 0.000 0.000 0.828

251

Appendix 3: MIXED Expe,.illlent 1

.- The Ws n Zs'-

set 19: '11 n Z3

Va,.iebl.. in Z: INT

CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 BX3 '11 Z3

1 0.031 0.000 0.000 0.003 0.003 0.0002 0.001 0.000 0.000 0.012 0.013 0.0002 0.043 0.000 0.009 0.001 0.002 0.0003 0.920 0.000 0.005 0.002 0.003 0.000

10 0.004 0.000 0.000 0.980 0.979 0.00097 0.000 1.000 0.986 0.001 0.000 1.000

Va,.iebl.. in Z: INT IX1

CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 IX3 W1 Z3

1 0.042 0.001 0.001 0.005 0.005 0.0001 0.000 0.003 0.002 0.005 0.006 0.0032 0.056 0.028 0.000 0.007 0.008 0.0003 0.898 0.026 0.002 0.002 0.003 0.0009 0.003 0.000 0.000 0.975 0.976 0.000

23 0.000 0.942 0.995 0.006 0.003 0.996

Va,.iebl.. in Z: INT IX1 IX2

CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 IX3 W1 Z3

1 0.051 0.004 0.000 0.006 0.006 0.0041 0.000 0.007 0.008 0.009 0.009 0.0082 0.029 0.001 0.433 0.001 0.001 0.0002 0.910 0.004 0.036 0.004 0.005 0.0039 0.003 0.019 0.008 0.935 0.944 0.019

11 0.006 0.965 0.515 0.045 0.035 0.967

Va,.iables in Z: INT IX1 IX2 IX3

CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 IX3 W1 Z3

1 0.052 0.009 0.001 0.000 0.001 0.0081 0.134 0.000 0.280 0.025 0.096 0.0011 0.014 0.000 0.041 0.479 0.366 0.0001 0.000 0.000 0.009 0.495 0.525 0.0002 0.797 0.005 0.096 0.001 0.006 0.002

10 0.003 0.987 0.5n 0.000 0.006 0.989

Va,.iabl.. in Z: INT IX1 IX2 IX3 W1 Z3

CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 IX3 W1 Z3

1 0.043 0.014 0.000 0.000 0.000 0.0131 0.154 0.000 0.286 0.001 0.000 0.0031 0.000 0.000 0.000 0.497 0.500 0.0001 0.000 0.000 0.000 0.502 0.500 0.0002 0.799 0.004 0.098 0.001 0.000 0.0018 0.004 0.982 0.616 0.000 0.000 0.982

252

Appendix 3: MIXED Experiment 1

- The WII lind Zs -

set 20: "1 lind Z4

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG 111 Z4

1 0.031 0.000 0.000 0.003 0.003 0.0002 0.001 0.000 0.000 0.012 0.013 0.0002 0.043 0.000 0.001 0.001 0.002 0.0003 0.920 0.000 0.001 0.002 0.003 0.000

10 0.004 0.000 0.000 0.947 0.953 0.000294 0.000 1.000 0.998 0.034 0.027 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG 111 Z4

1 0.042 0.000 0.000 0.005 0.005 0.0001 0.000 0.000 0.000 0.005 0.006 0.0002 0.057 0.003 0.000 0.006 0.007 0.0003 0.898 0.003 0.000 0.002 0.003 0.0009 0.003 0.000 0.000 0.946 0.950 0.000

69 0.000 0.993 0.999 0.035 0.029 1.000

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG 111 Z4

1 0.051 0.000 0.000 0.005 0.005 0.0001 0.001 0.001 0.002 0.008 0.009 0.0012 0.029 0.000 0.090 0.001 0.001 0.0002 0.915 0.001 0.007 0.004 0.005 0.0009 0.004 0.000 0.000 0.926 0.934 0.000

30 0.001 0.998 0.901 0.055 0.046 0.998

Variables in Z: INT IX1 IX2 IlG

CI Variance Proportions for Coefficients ofINT Ix1 IX2 IlG 111 Z4

1 0.051 0.001 0.000 0.000 0.000 0.0011 0.134 0.000 0.059 0.025 0.094 0.0001 0.014 0.000 0.009 0.479 0.354 0.0002 0.000 0.000 0.002 0.495 0.508 0.0002 0.800 0.001 0.020 0.001 0.006 0.000

28 0.000 0.998 0.910 0.000 0.038 0.998

Variables in Z: INT IX1 IX2 IlG 111 Z4

Cl Variance Proportions for Coefficients ofINT IX1 IX2 IlG 111 Z4

1 0.042 0.002 0.000 0.000 0.000 0.0021 0.153 0.000 0.055 0.001 0.000 0.0001 0.000 0.000 0.000 0.497 0.500 0.0001 0.000 0.000 0.000 0.502 0.500 0.0002 0.805 0.001 0.018 0.001 0.000 0.000

23 0.000 0.998 0.926 0.000 0.000 0.998

253

Appendix 3: MIXED ExperiMent 1

-* The wa 8nd Zs -*set 21: W2 8nd ZD

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO

1 0.035 0.019 0.023 0.000 0.000 0.0202 0.003 0.055 0.016 0.001 0.001 0.0112 0.033 0.094 0.554 0.000 0.000 0.0203 0.826 0.001 0.190 0.000 0.000 0.1214 0.103 0.820 0.216 0.000 0.000 0.161

30 0.001 0.010 0.001 0.998 0.998 0.001

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 \,/2 ZO

1 0.046 0.024 0.032 0.001 0.001 0.0011 0.021 0.017 0.155 0.001 0.001 0.2192 0.035 0.388 0.001 0.000 0.000 0.2152 0.046 0.395 0.361 0.000 0.000 0.4n3 0.852 0.176 0.450 0.000 0.000 0.026

27 0.000 0.000 0.001 0.998 0.998 0.001

Variables in Z: INT IX1 Ix2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO

1 0.059 0.023 0.001 0.001 0.001 0.0001 0.067 0.349 0.002 0.000 0.000 0.2412 0.022 0.012 0.710 0.000 0.000 0.1882 0.061 0.111 0.282 0.000 0.000 0.5432 0.787 0.505 0.005 0.000 0.000 0.026

26 0.003 0.000 0.000 0.998 0.998 0.002

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO

0.296 0.316 0.018 0.000 0.000 0.0320.003 0.000 0.015 0.352 0.413 0.0660.025 0.012 0.407 0.092 0.010 0.3950.053 0.012 0.550 0.049 0.006 0.3660.021 0.007 0.004 0.503 0.555 0.0530.601 0.653 0.006 0.004 0.016 0.088

Variables in Z: INT IX1 IX2 IX3 W2 ZO

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO

0.344 0.325 0.047 0.001 0.000 0.0020.000 0.000 0.000 0.483 0.485 0.0000.001 0.021 0.256 0.003 0.003 0.7020.013 0.044 0.671 0.000 0.001 0.2860.000 0.007 0.002 0.509 0.510 0.0030.642 0.603 0.024 0.003 0.002 O.OOS

254

Appendix 3: MIXED Experiment 1

*** The Ws end Zs ***-.set 22: W2 end Z1

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.001 0.001 0.0102 0.042 0.013 0.438 0.000 0.000 0.0033 0.925 0.003 0.226 0.000 0.000 0.0079 0.000 0.965 0.316 0.000 0.000 0.976

31 0.001 0.003 0.000 0.998 0.998 0.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

1 0.044 0.018 0.029 0.001 0.001 0.0121 0.004 0.017 0.112 0.001 0.001 0.2072 0.049 0.375 0.010 0.001 0.001 0.0503 0.826 0.462 0.027 0.000 0.000 0.1133 0.076 0.128 0.821 0.000 0.000 0.618

28 0.000 0.001 0.000 0.998 0.998 0.001

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

1 0.059 0.023 0.001 0.001 0.001 0.0021 0.019 0.228 0.004 0.000 0.000 0.2832 0.027 0.014 0.859 0.000 0.000 0.0242 0.342 0.061 0.135 0.000 0.000 0.4172 0.550 0.674 0.000 0.000 0.000 0.274

26 0.002 0.000 0.000 0.998 0.998 0.001

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

1 0.140 0.237 0.000 0.000 0.002 0.1641 0.012 0.001 0.002 0.411 0.425 0.0031 0.136 0.000 0.605 0.015 0.002 0.1341 0.225 0.000 0.257 0.267 0.177 0.1351 0.183 0.002 0.131 0.307 0.392 0.1192 0.304 0.759 0.004 0.000 0.001 0.445

Variables in Z: INT IX1 IX2 IX3 W2 Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

0.215 0.287 0.019 0.000 0.000 0.1290.073 0.004 0.555 0.005 0.001 0.2860.000 0.000 0.004 0.481 0.483 0.0010.002 0.000 0.006 0.508 0.513 0.0010.290 0.005 0.415 0.006 0.003 0.3530.420 0.703 0.001 0.001 0.000 0.231

255

Appendix 3: MIXED Expe";.."t 1

-ThewanZ.*- ..set 23: W2 lind Z2

Variables in Z: INT

CI Variance Proportions for Coefficient. ofINT IX1 IX2 IX3 W2 Z2

1 D.D31 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.001 0.001 0.0012 0.041 0.002 0.085 0.000 0.000 0.0003 0.926 0.000 0.042 0.000 0.000 0.001

29 0.001 0.816 0.729 0.142 0.142 0.82631 0.000 0.180 0.140 0.856 0.856 0.1n

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2

1 0.041 0.006 0.007 0.001 0.001 0.0041 0.000 0.018 0.017 0.001 0.001 0.0282 0.057 0.184 0.002 0.001 0.001 0.0033 0.899 '0.176 0.021 0.000 0.000 0.0037 0.002 0.615 0.953 0.000 0.000 0.960

28 0.000 0.002 0.000 0.998 0.998 0.002

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2

1 0.056 0.021 0.000 0.001 0.001 0.0181 0.001 0.065 0.014 0.001 0.001 0.0762 0.027 0.009 0.805 0.000 0.000 0.0022 0.894 0.020 0.070 0.000 0.000 0.0344 0.020 0.884 0.111 0.000 0.000 0.869

27 0.003 0.001 0.000 0.998 0.998 0.000

Variables in z: INT IX1 Ix2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2

1 0.065 0.076 0.002 0.000 0.000 0.0751 0.000 0.000 0.003 0.426 0.428 0.0001 0.148 0.001 0.658 0.002 0.001 0.0122 0.020 0.001 0.004 0.559 0.547 0.0012 0.748 0.031 0.210 0.013 0.022 0.0283 0.019 0.890 0.121 0.000 0.001 0.884

Variables in Z: INT IX1 IX2 IX3 W2 Z2

CI Variance Proportions for Coefficients ofINT IX1 BX2 IX3 W2 Z2

1 0.060 0.110 0.001 0.000 0.000 0.1041 0.155 0.000 0.590 0.002 0.000 0.0371 0.000 0.000 0.001 0.486 0.485 0.0001 0.003 0.000 0.005 0.506 0.512 0.0001 0.757 0.026 0.235 0.006 0.003 0.0173 0.024 0.863 0.169 0.000 0.000 0.842

256

Appendix 3: MIXED Experiment 1

.- The wa lind Z• .-10.

set 24: W2 rd 13

Variebles in z: INT

CI Variance Proportions for Coefficient. ofINT IX1 IX2 BX3 W2 13

1 0.031 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.001 0.001 0.0002 0.041 0.000 0.009 0.000 0.000 0.0003 0.927 0.000 0.005 0.000 0.000 0.000

31 0.001 0.000 0.000 0.996 0.997 0.00098 0.000 1.000 0.986 0.002 0.001 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 13

1 0.040 0.001 0.001 0.001 0.001 0.0001 0.000 0.003 0.002 0.001 0.001 0.0032 0.057 0.028 0.000 0.001 0.001 0.0003 0.901 0.026 0.002 0.000 0.000 0.000

23 0.000 0.936 0.990 0.002 0.003 0.99028 0.000 0.006 0.005 0.996 0.995 0.006

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 13

1 0.050 0.003 0.000 0.001 0.001 0.0031 0.000 0.007 0.008 0.001 0.001 0.0082 0.028 0.001 0.434 0.000 0.000 0.0002 0.913 0.004 0.035 0.000 0.000 0.003

10 0.005 0.981 0.521 0.000 0.000 0.98328 0.003 0.003 0.001 0.998 0.998 0.003

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W2 13

1 0.052 0.009 0.002 0.000 0.000 0.0081 0.000 0.000 0.002 0.426 0.428 0.0001 0.150 0.000 0.327 0.002 0.001 0.0012 0.019 0.000 0.002 0.560 0.547 0.0002 0.776 0.005 0.097 0.012 0.021 0.002

10 0.002 0.987 0.571 0.000 0.002 0.988

Variables in Z: INT BX1 BX2 BX3 W2 13

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W2 13

1 0.042 0.014 0.000 0.000 0.000 0.0141 0.159 0.000 0.291 0.001 0.000 0.0031 0.000 0.000 0.000 0.487 0.485 0.0001 0.003 0.000 0.002 0.505 0.512 0.0002 0.794 0.004 0.100 0.006 0.003 0.0018 0.002 0.981 0.607 0.000 0.000 0.982

257

Appendix 3: MIXED Experi.ent 1

*- The W8 8nd Zs *-set 25: W2 8nd Z4

Variables in Z: INT

CI VarilnCe Proportiona for Coefficients ofINT IX1 IX2 IX3 W2 Z4

1 0.031 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.001 0.001 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.927 0.000 0.001 0.000 0.000 0.000

31 0.001 0.000 0.000 0.998 0.998 0.000291 0.000 1.000 0.998 0.000 0.000 1.000

Variables in Z: INT IX1

CI VarilnCe Proportiona for Coefficients ofINT IX1 IX2 IX3 W2 Z4

1 0.040 0.000 0.000 0.001 0.001 0.0001 0.000 0.000 0.000 0.001 0.001 0.0002 0.058 0.003 0.000 0.001 0.001 0.0003 0.901 0.003 0.000 0.000 0.000 0.000

28 0.000 0.000 0.000 0.998 0.998 0.00069 0.000 0.993 0.999 0.000 0.000 1.000

Variables in Z: INT IX1 IX2

CI Variance Proportiona for Coefficients ofINT IX1 IX2 IX3 W2 Z4

1 0.050 0.000 0.000 0.001 0.001 0.0001 0.000 0.001 0.002 0.001 0.001 0.0012 0.028 0.000 0.094 0.000 0.000 0.0002 0.918 0.001 0.007 0.000 0.000 0.000

28 0.003 0.048 0.042 0.940 0.936 0.04829 0.000 0.950 0.856 0.058 0.062 0.950

Variables in Z: INT IX1 BX2 BX3

CI VarilnCe Proportions for Coefficients ofINT BX1 BX2 BX3 W2 Z4

1 0.051 0.001 0.000 0.000 0.000 0.0011 0.000 0.000 0.000 0.427 0.429 0.0001 0.150 0.000 0.071 0.001 0.001 0.0002 0.023 0.000 0.001 0.558 0.543 0.0002 0.776 0.001 0.021 0.014 0.026 0.000

27 0.000 0.998 0.907 0.000 0.001 0.998

Variables in Z: INT IX1 IX2 IX3 W2 24

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W2 Z4

1 0.041 0.002 0.000 0.000 0.000 0.002 ..1 0.158 0.000 0.061 0.001 0.000 0.0001 0.000 0.000 0.000 0.487 0.485 0.0001 0.004 0.000 0.000 0.505 0.512 0.0002 0.797 0.001 0.021 0.007 0.003 0.000

22 0.000 0.997 0.918 0.000 0.000 0.997

258

Appendix 3: MIXED Experiment 1

- The Ws and Z. -

set 26: W3 and ZO

V.riables in Z: INT

CI V.riance Proportions for Coefficient. ofINT IX1 IX2 1lC3 W3 zo

1 0.034 0.020 0.023 0.000 0.000 0.0202 0.003 0.055 0.017 0.000 0.000 0.0712 0.034 0.096 0.552 0.000 0.000 0.0203 0.826 0.001 0.192 0.000 0.000 0.1194 0.101 0.828 0.216 0.000 0.000 0.767

89 0.001 0.001 0.000 1.000 1.000 0.002

V.riables in Z: INT IX1

CI V.riance Proportions for Coefficients ofINT IX1 IX2 1lC3 W3 ZO

1 0.045 0.024 0.032 0.000 0.000 0.0071 0.022 0.018 0.154 0.000 0.000 0.2152 0.034 0.387 0.001 0.000 0.000 0.2772 0.046 0.394 0.362 0.000 0.000 0.4713 0.850 0.177 0.450 0.000 0.000 0.025

82 0.003 0.000 0.001 1.000 1.000 0.004

Variables in Z: INT IX1 IX2

CI V.riance Proportions for Coefficients ofINT IX1 IX2 1lC3 W3 zo

1 0.059 0.022 0.001 0.000 0.000 0.0001 0.067 0.352 0.002 0.000 0.000 0.2382 0.022 0.012 0.704 0.000 0.000 0.1942 0.061 0.110 0.288 0.000 0.000 0.5412 0.785 0.504 0.005 0.000 0.000 0.026

78 0.006 0.000 0.000 1.000 1.000 0.001

V.riables in Z: INT IX1 IX2 1lC3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 ZO

1 0.057 0.046 0.003 0.235 0.255 0.0011 0.238 0.270 0.015 0.064 0.042 0.0321 0.029 0.011 0.421 0.002 0.000 0.4741 0.060 0.010 0.555 0.003 0.000 0.4101 0.543 0.630 0.005 0.047 0.036 0.0822 0.073 0.032 0.001 0.648 0.667 0.000

V.riables in Z: INT IX1 IX2 1lC3 W3 ZO

CI Variance Proportions for Coefficients ofINT IX1 IX2 1lC3 W3 ZO

0.338 0.315 0.053 0.004 0.000 0.0020.000 0.001 0.000 0.401 0.406 0.0000.001 0.021 0.235 0.000 0.000 0.n90.014 0.053 0.686 0.000 0.000 0.2620.024 0.067 0.001 0.541 0.544 0.0010.623 0.543 0.024 0.053 0.049 0.006

259

Appendix 3: MIXED Experiment

- The wa end Zs -*...

Set 27: W3 end Z1

Variabl.. in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1

1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.000 0.000 0.0102 0.043 0.013 0.437 0.000 0.000 0.0033 0.923 0.003 0.227 0.000 0.000 0.0069 0.000 0.967 0.316 0.000 0.000 0.975

92 0.002 0.001 0.000 1.000 1.000 0.001

Variabl.. in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1

1 0.044 0.018 0.029 0.000 0.000 0.0121 0.004 0.016 0.113 0.000 0.000 0.2052 0.049 0.377 0.010 0.000 0.000 0.0523 0.824 0.462 0.027 0.000 0.000 0.1133 0.075 0.128 0.821 0.000 0.000 0.618

83 0.003 0.000 0.000 1.000 1.000 0.001

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1

1 0.059 0.022 0.001 0.000 0.000 0.0021 0.019 0.228 0.004 0.000 0.000 0.2832 0.027 0.013 0.859 0.000 0.000 0.0242 0.340 0.061 0.135 0.000 0.000 0.4182 0.549 0.674 0.000 0.000 0.000 0.273

78 0.006 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1

1 0.143 0.233 0.001 0.004 0.007 0.1551 0.000 0.005 0.000 0.291 0.291 0.0101 0.140 0.000 0.607 0.006 0.000 0.1361 0.394 0.001 0.387 0.014 0.002 0.2482 0.014 0.001 0.001 0.685 0.700 0.0102 0.309 0.759 0.004 0.001 0.001 0.440

Variables in Z: INT IX1 IX2 IX3 W3 Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1

0.217 0.282 0.025 0.001 0.000 0.1310.000 0.000 0.000 0.404 0.405 0.0000.069 0.006 0.558 0.000 0.001 0.2890.282 0.004 0.399 0.036 0.040 0.3180.018 0.014 0.019 0.552 0.548 0.0460.414 0.693 0.000 0.007 0.006 0.217

260

Af?pendix 3: MIXED Experilllent

-* The \Is end Zs *-• set 28: W3 end Z2

Variables in Z: INT

Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z2

1 0.030 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.000 0.000 0.0012 0.041 0.002 0.085 0.000 0.000 0.0003 0.925 0.000 0.042 0.000 0.000 0.001

30 0.001 0.995 0.868 0.000 0.000 0.99692 0.001 0.002 0.002 1.000 1.000 0.002

Variables in Z: INT IX1

Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z2

1 0.041 0.006 0.007 0.000 0.000 0.0041 0.000 0.017 0.017 0.000 0.000 0.0282 0.057 0.184 0.002 0.000 0.000 0.0033 0.897 0.176 0.021 0.000 0.000 0.0037 0.002 0.614 0.950 0.000 0.000 0.957

85 0.003 0.003 0.003 1.000 1.000 0.004

Variables in Z: INT BX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 W3 Z2

1 0.055 0.021 0.000 0.000 0.000 0.0181 0.001 0.065 0.014 0.000 0.000 0.0762 0.027 0.009 0.805 0.000 0.000 0.0022 0.890 0.020 0.070 0.000 0.000 0.0344 0.020 0.881 0.111 0.000 0.000 0.864

81 0.006 0.003 0.000 1.000 1.000 0.006

Variables in Z: INT BX1 BX2 BX3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z2

1 0.065 0.075 0.002 0.000 0.002 0.0741 0.000 0.001 0.000 0.299 0.295 0.0011 0.147 0.001 0.660 0.002 0.000 0.0122 0.746 0.033 0.214 0.012 0.004 0.0272 0.022 0.000 0.002 0.686 0.693 0.0013 0.020 0.890 0.122 0.001 0.005 0.884

Variables in Z: INT BX1 BX2 BX3 W3 Z2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z2

1 0.065 0.123 0.001 0.000 0.000 0.1151 0.000 0.000 0.000 0.412 0.410 0.0001 0.155 0.000 0.591 0.000 0.001 0.0441 0.484 0.024 0.178 0.182 0.190 0.0111 0.274 0.006 0.073 0.404 0.396 0.0093 0.022 0.847 0.157 0.002 0.002 0.822

261

Appendix 3: MIXED Experiment 1

*** The wa Md Zs ***Set 29: W3 Md Z3 •

Variabl.. in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3

1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.042 0.000 0.009 0.000 0.000 0.0003 0.925 0.000 0.005 0.000 0.000 0.000

92 0.002 0.000 0.000 1.000 1.000 0.00098 0.000 1.000 0.986 0.000 0.000 1.000

Variabl.. in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3

1 0.040 0.001 0.001 0.000 0.000 0.0001 0.000 0.003 0.002 0.000 0.000 0.0032 0.057 0.028 0.000 0.000 0.000 0.0003 0.899 0.026 0.002 0.000 0.000 0.000

23 0.000 0.942 0.994 0.000 0.000 0.99685 0.003 0.000 0.001 1.000 1.000 0.000

Variabl.. in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3

1 0.050 0.003 0.000 0.000 0.000 0.0031 0.000 0.007 0.008 0.000 0.000 0.0082 0.028 0.001 0.434 0.000 0.000 0.0002 0.910 0.004 0.035 0.000 0.000 0.003

10 0.005 0.983 0.522 0.000 0.000 0.98583 0.006 0.001 0.000 1.000 1.000 0.001

Variabl.. in Z: INT IX1 IX2 8X3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3

1 0.052 0.009 0.002 0.000 0.001 0.0081 0.001 0.000 0.000 0.297 0.298 0.0001 0.148 0.000 0.328 0.003 0.000 0.0012 0.761 O.OOS 0.099 0.020 0.007 0.0022 0.035 0.000 0.001 0.680 0.694 0.000

10 0.002 0.987 0.570 0.000 0.001 0.988

Variables in Z: INT IX1 IX2 IX3 W3 Z3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3

1 0.042 0.015 0.000 0.000 0.000 0.0141 0.000 0.000 0.000 0.414 0.412 0.0001 0.164 0.000 0.290 0.000 0.002 0.0042 0.503 0.003 0.081 0.17'9 0.188 0.0012 0.290 0.001 0.027 0.407 0.398 0.0008 0.001 0.981 0.602 0.000 0.000 0.981

262

Appendix 3: MIXED Experiment 1

*** The ... lind Zs ***

set 30: W3 lind Z4

Variebl.. in Z: INT

CI Variance Proportions for Coefficients oflilT 1X1 1X2 IX3 W3 Z4

1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.925 0.000 0.001 0.000 0.000 0.000

92 0.002 0.000 0.000 0.996 0.997 0.000292 0.000 1.000 0.998 0.003 0.003 1.000

Variables in Z: lilT IX1

CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 W3 Z4

1 0.040 0.000 0.000 0.000 0.000 0.0001 0.000 0.000 0.000 0.000 0.000 0.0002 0.058 0.003 0.000 0.000 0.000 0.0003 0.899 0.003 0.000 0.000 0.000 0.000

69· 0.000 0.971 0.978 0.009 0.010 0.97885 0.003 0.022 0.022 0.991 0.990 0.022

Variables in Z: lilT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z4

1 0.050 0.000 0.000 0.000 0.000 0.0001 0.000 0.001 0.002 0.000 0.000 0.0012 0.028 0.000 0.093 0.000 0.000 0.0002 0.914 0.001 0.007 0.000 0.000 0.000

29 0.001 0.995 0.895 0.000 0.000 0.99583 0.007 0.003 0.003 1.000 1.000 0.003

Variables in Z: INT 1X1 IX2 BX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z4

1 0.051 0.001 0.000 0.000 0.001 0.0011 0.001 0.000 0.000 0.297 0.297 0.0001 0.148 0.000 0.071 0.003 0.000 0.0002 0.766 0.001 0.021 0.019 0.007 0.0002 0.033 0.000 0.000 0.681 0.694 0.000

27 0.000 0.998 0.907 0.000 0.001 0.998

Variables in Z: INT IX1 IX2 IX3 W3 Z4

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z4

1 0.040 0.002 0.000 0.000 0.000 0.0021 0.000 0.000 0.000 0.414 0.412 0.0001 0.165 0.000 0.062 0.000 0.002 0.0012 0.507 0.000 0.017 0.177 0.185 0.0002 0.286 0.000 0.006 0.409 0.401 0.000

21 0.001 0.997 0.915 0.000 0.000 0.997

263

Appendix 3: MIXED Experilllent 1

*** The Ws end Zs ***set 31: W4 end ZO

Vari~les in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 &X2 8X3 W4 ZO

1 0.034 0.020 0.023 0.000 0.000 0.0202 0.003 0.055 0.017 0.000 0.000 0.0712 0.034 0.096 0.552 0.000 0.000 0.0203 0.825 0.001 0.192 0.000 0.000 0.1194 0.102 0.828 0.216 0.000 0.000 0.765

318 0.002 0.001 0.001 1.000 1.000 0.005

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 8X3 W4 ZO

1 0.045 0.024 0.032 0.000 0.000 0.0071 0.021 0.018 0.155 0.000 0.000 0.2162 0.034 0.387 0.001 0.000 0.000 0.2772 0.046 0.394 0.362 0.000 0.000 0.4n3 0.852 0.177 0.450 0.000 0.000 0.025

293 0.002 0.000 0.001 1.000 1.000 0.004

Variables in Z: INT IX1 IX2

Cl Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 W4 ZO

1 0.059 0.022 0.001 0.000 0.000 0.0001 0.067 0.351 0.002 0.000 0.000 0.2392 0.022 0.012 0.706 0.000 0.000 0.1902 0.061 O. "1 0.286 0.000 0.000 0.5372 0.790 0.503 0.005 0.000 0.000 0.026

2n 0.000 0.000 0.000 1.000 1.000 0.008

Variables in Z: INT IX1 BX2 IX3

Cl Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 W4 ZO

1 0.001 0.000 0.000 0.085 0.085 0.0011 0.296 0.316 0.018 0.000 0.000 0.0301 0.029 0.012 0.421 0.000 0.000 0.4741 0.061 0.010 0.555 0.000 0.000 0.4122 0.612 0.661 0.006 0.000 0.000 0.0783 0.002 0.000 0.000 0.914 0.915 0.005

Variables in Z: INT IX1 IX2 IX3 W4 ZO

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 ZO

1 0.000 0.000 0.000 0.135 0.135 0.0001 0.337 0.325 0.037 0.000 0.000 0.0021 0.000 0.014 0.2n 0.000 0.000 0.7011 0.011 0.036 0.674 0.000 0.000 0.2922 0.648 0.625 0.016 0.000 0.000 0.0043 0.003 0.000 0.000 0.865 0.865 0.000

264

Appendix 3: MIXED Experiment

*** The Ws end Zs ***•

set 32: W4 end Z1

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1

1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.000 0.000 0.0102 0.043 0.013 0.436 0.000 0.000 0.0033 0.923 0.003 0.226 0.000 0.000 0.0069 0.000 0.967 0.316 0.000 0.000 0.975

325 0.002 0.001 0.003 1.000 1.000 0.001

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1

1 0.044 0.018 0.029 0.000 0.000 0.0121 0.004 0.016 0.113 0.000 0.000 0.2052 0.049 0.377 0.010 0.000 0.000 0.0523 0.826 0.462 0.027 0.000 0.000 0.1133 0.076 0.127 0.819 0.000 0.000 0.618

295 0.001 0.000 0.002 1.000 1.000 0.000

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1

1 0.059 0.022 0.001 0.000 0.000 0.0021 0.019 0.228 0.004 0.000 0.000 0.283

" 2 0.027 0.014 0.860 0.000 0.000 0.0242 0.342 0.062 0.134 0.000 0.000 0.4192 0.553 0.673 0.000 0.000 0.000 0.2n

271 0.000 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1

1 0.001 0.000 0.000 0.086 0.086 0.0001 0.141 0.238 0.000 0.000 0.000 0.1631 0.141 0.000 0.611 0.000 0.000 0.1371 0.410 0.001 0.384 0.000 0.000 0.2562 0.306 0.760 0.004 0.000 0.000 0.4433 0.001 0.000 0.000 0.914 0.914 0.000

Variables in Z: INT IX1 IX2 IX3 W4 Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1

1 0.000 0.000 0.000 0.144 0.144 0.0001 0.234 0.291 0.021 0.000 0.000 0.1041 0.056 0.004 0.541 0.000 0.000 0.3301 0.215 0.009 0.438 0.000 0.000 0.3952 0.492 0.695 0.000 0.000 0.000 0.1702 0.002 0.001 0.000 0.855 0.855 0.001

265

Appendix 3: MIXED Experiment 1

- The Ws end Zs --

set 33: W4 end Z2 •Varieblea in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IIG W4 Z2 •

1 0.030 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.000 0.000 0.0012 0.041 0.002 0.084 0.000 0.000 0.0003 0.925 0.000 0.041 0.000 0.000 0.001

30 0.001 0.986 0.858 0.000 0.000 0.987328 0.002 0.011 0.013 1.000 1.000 0.011

Variebles in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z2

1 0.041 0.006 0.007 0.000 0.000 0.0041 0.000 0.017 0.017 0.000 0.000 0.0282 0.057 0.184 0.002 0.000 0.000 0.0033 0.898 0.175 0.021 0.000 0.000 0.0037 0.002 0.613 0.944 0.000 0.000 0.954

301 0.001 0.005 0.009 1.000 1.000 0.008

Varieblea in Z: INT IX1 IX2

CI Variance Proportions for Coefficients of •INT IX1 IX2 IIG W4 Z2

1 0.055 0.021 0.000 0.000 0.000 0.0181 0.001 0.065 0.014 0.000 0.000 0.0762 0.027 0.009 0.805 0.000 0.000 0.0022 0.896 0.020 0.070 0.000 0.000 0.0344 0.020 0.883 0.111 0.000 0.000 0.867

281 0.000 0.002 0.000 1.000 1.000 0.003

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IIG W4 Z2

1 0.065 0.076 0.002 0.000 0.000 0.0741 0.000 0.000 0.000 0.085 0.085 0.0001 0.147 0.001 0.662 0.000 0.000 0.0122 0.768 0.032 0.214 0.000 0.000 0.0283 0.020 0.764 0.102 0.108 0.103 0.7513 0.000 0.127 0.020 0.806 0.812 0.134

Variables in Z: INT IX1 IX2 IIG W4 Z2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IIG W4 Z2

1 0.070 0.125 0.001 0.000 0.000 0.1171 0.000 0.000 0.000 0.149 0.149 0.0001 0.134 0.000 0.624 0.000 0.000 0.0451 0.769 0.032 0.225 0.000 0.000 0.0262 0.014 0.220 0.037 0.601 0.602 0.2083 0.013 0.623 0.113 0.249 0.249 0.603 •

266

Appendix 3: MIXED Experilllent 1

.. *** The WI and Zs ***set 34: W4 and Z3

Vari8bl.. in Z: INT

CI Varience Proportions for Coefficients ofINT IX1 8X2 IX3 W4 Z3

1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.009 0.000 0.000 0.0003 0.925 0.000 0.005 0.000 0.000 0.000

98 0.000 0.993 0.979 0.000 0.000 0.993321 0.002 0.001 0.001 1.000 1.000 0.006

Vari8bles in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z3

1 0.040 0.001 0.001 0.000 0.000 0.0001 0.000 0.003 0.002 0.000 0.000 0.0032 0.051 0.028 0.000 0.000 0.000 0.0003 0.900 0.026 0.002 0.000 0.000 0.000

23 0.000 0.939 0.990 0.000 0.000 0.993301 0.001 0.004 0.004 1.000 1.000 0.004

.: Vari8bles in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W4 Z3

1 0.050 0.003 0.000 0.000 0.000 0.0031 0.000 0.001 0.008 0.000 0.000 0.0082 0.028 0.001 0.435 0.000 0.000 0.0002 0.916 0.004 0.035 0.000 0.000 0.003

11 0.005 0.984 0.522 0.000 0.000 0.986286 0.000 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z3

1 0.052 0.009 0.002 0.000 0.000 0.0081 0.000 0.000 0.000 0.086 0.085 0.0001 0.149 0.000 0.329 0.000 0.000 0.0012 0.795 0.005 0.099 0.000 0.000 0.0024 0.001 0.000 0.000 0.913 0.912 0.000

10 0.003 0.981 0.510 0.002 0.002 0.988

Variables in Z: INT IX1 IX2 IX3 W4 Z3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z3

1 0.045 0.015 0.000 0.000 0.000 0.0151 0.000 0.000 0.000 0.151 0.151 0.0001 0.142 0.000 0.314 0.000 0.000 0.0042 0.809 0.005 0.098 0.000 0.000 0.0013 0.002 0.000 0.000 0.841 0.841 0.0008 0.002 0.980 0.588 0.001 0.001 0.980

267

Apeendix 3: MIXED Experilllllnt 1

*** The wa end Z. --

Set 35: W4 end Z4

Variabl.. in Z: INT

CI Varience Proportions for Coefficient. ofINT IX1 IX2 BX3 W4 Z4

1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.925 0.000 0.001 0.000 0.000 0.000

291 0.000 0.997 0.995 0.002 0.002 0.997327 0.002 0.003 0.003 0.998 0.998 0.003

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z4

1 0.040 0.000 0.000 0.000 0.000 0.0001 0.000 0.000 0.000 0.000 0.000 0.0002 0.058 0.003 0.000 0.000 0.000 0.0003 0.900 0.003 0.000 0.000 0.000 0.000

69 0.000 0.992 0.998 0.000 0.000 0.998301 0.002 0.001 0.001 1.000 1.000 0.001

Variables in Z: INT IX1 IX2

'.CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z4

1 0.050 0.000 0.000 0.000 0.000 0.0001 0.000 0.001 0.002 0.000 0.000 0.0012 0.028 0.000 0.094 0.000 0.000 0.0002 0.920 0.001 0.007 0.000 0.000 0.000

29 0.001 0.997 0.897 0.000 0.000 0.998286 0.000 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z4

1 0.051 0.001 0.000 0.000 0.000 0.0011 0.000 0.000 0.000 0.086 0.085 0.0001 0.149 0.000 0.071 0.000 0.000 0.0002 0.798 0.001 0.021 0.000 0.000 0.0004 0.001 0.000 0.000 0.913 0.912 0.000

27 0.000 0.998 0.907 0.001 0.002 0.998

Variables in Z: INT IX1 IX2 aX3 W4 Z4

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W4 Z4

1 0.043 0.002 0.000 0.000 0.000 0.0021 0.000 0.000 0.000 0.151 0.151 0.0001 0.143 0.000 0.068 0.000 0.000 0.0012 0.812 0.001 0.021 0.000 0.000 0.0003 0.002 0.000 0.000 0.847 0.847 0.000

21 0.000 0.997 0.911 0.001 0.001 0.997

268

APPENDIX 4

MIXED MODEL EXPERIMENT 2COLLINEARITY DIAGNOSTICS

- The WI!-

set 1: WO

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we

1 0.052 0.039 0.039 0.037 0.0352 0.059 0.143 0.132 0.063 0.1112 0.000 0.461 0.505 0.001 0.0003 0.781 0.357 0.303 0.003 0.0534 0.109 0.000 0.021 0.897 0.802

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we

1 0.042 0.038 0.039 0.031 0.0292 0.037 0.111 0.095 0.069 0.1152 0.002 0.396 0.603 0.001 0.0003 0.743 0.434 0.232 0.020 0.1054 0.175 0.021 0.031 0.880 0.751

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we

1 0.065 0.016 0.000 0.064 0.0631 0.000 0.130 0.164 0.006 0.0072 0.427 0.025 0.073 0.027 0.0943 0.340 0.769 0.713 0.052 0.0873 0.169 0.060 0.049 0.851 0.748

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we

1 0.030 0.069 0.123 0.063 0.0011 0.105 0.149 0.009 0.160 0.0031 0.191 0.022 0.019 0.026 0.5681 0.363 0.017 0.024 0.034 0.4283 0.312 0.743 0.825 0.718 0.000

Variables in Z: INT IX1 IX2 IX3 WO

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we

1 0.027 0.062 0.090 0.072 0.0291 0.020 0.081 0.026 0.066 0.1191 0.526 0.012 0.085 0.003 0.0251 0.180 0.191 0.020 0.068 0.1893 0.247 0.654 0.780 0.792 0.639

ApPendix 4: MIXED ExperiMent 2

-- The In--

Set 2: 111

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 111

1 0.044 0.033 0.029 0.006 0.0062 0.062 0.119 0.197 0.011 0.0132 0.000 0.532 0.434 0.000 0.0003 0.889 0.315 0.330 0.002 0.0029 0.004 0.000 0.010 0.981 0.979

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 111

1 0.037 0.032 0.031 0.005 0.0052 0.040 0.111 0.125 0.012 0.0142 0.002 0.429 0.565 0.000 0.0003 0.918 0.428 0.268 0.002 0.0029 0.003 0.000 0.011 0.981 0.979

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 111

1 0.053 0.010 0.000 0.010 0.0101 0.000 0.136 0.163 0.001 0.0012 0.469 0.030 0.082 0.008 0.0093 0.474 0.824 0.755 0.000 0.0008 0.004 0.000 0.000 0.981 0.979

Variables in Z: INT aX1 aX2 aX3

CI Variance Proportions for Coefficients ofINT aX1 BX2 aX3 111

1 0.029 0.065 0.121 0.064 0.0091 0.093 0.158 0.006 0.155 0.0151 0.266 0.011 0.037 0.006 0.5021 0.300 0.023 0.012 0.057 0.4743 0.312 0.743 0.825 0.718 0.000

Variables in Z: INT aX1 BX2 aX3 111

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 111

1 0.034 0.075 0.097 0.061 0.0191 0.016 0.067 0.017 0.079 0.1301 0.498 0.013 0.090 0.004 0.0291 0.194 0.188 0.016 0.069 0.1893 0.258 0.657 0.780 0.787 0.634

270

•

Appendix 4: MIXED Experilllent 2

- The Ws-

set 3: W2

Variables in z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W2

1 0.043 0.031 0.029 0.001 0.0012 0.063 0.138 0.183 0.001 0.0012 0.000 0.500 0.465 0.000 0.0003 0.893 0.318 0.322 0.000 0.000

28 0.001 0.013 0.001 0.998 0.998

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2

1 0.035 0.031 0.031 0.001 0.0012 0.042 0.121 0.124 0.001 0.0012 0.002 0.418 0.582 0.000 0.0003 0.920 0.430 0.262 0.000 0.000

29 0.000 0.000 0.002 0.998 0.998

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2

1 0.052 0.010 0.000 0.001 0.0011 0.000 0.136 0.164 0.000 0.0002 0.411 0.031 0.081 0.001 0.0013 0.415 0.824 0.155 0.000 0.000

21 0.002 0.000 0.000 0.998 0.998

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W2

1 0.024 0.049 0.111 0.016 0.0251 0.055 0.158 0.000 0.100 0.1361 0.509 0.002 0.060 0.001 0.1601 0.101 0.052 0.000 0.101 0.6163 0.310 0.139 0.824 0.716 0.003

Variables in Z: INT IX1 IX2 IX3 W2

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W2

1 0.036 0.091 0.112 0.043 0.0041 0.016 0.042 0.004 0.118 0.1601 0.451 0.020 0.081 0.004 0.0551 0.230 0.110 0.006 0.071 0.1193 0.260 0.670 0.191 0.751 0.603

271

Appendix 4: MIXED Experiment 2

- The wa-

set 3: W3

Vari8bl.. in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! W3

1 0.043 0.032 0.029 0.000 0.0002 0.063 0.134 0.190 0.000 0.0002 0.000 0.515 0.456 0.000 0.0003 0.892 0.319 0.325 0.000 0.000

84 0.002 0.000 0.000 1.000 1.000

Vari8bl.. in Z: INT IX1

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX! W3

1 0.035 0.031 0.031 0.000 0.0002 0.042 0.120 0.124 0.000 0.0002 0.002 0.419 0.582 0.000 0.0003 0.919 0.430 0.263 0.000 0.000

87 0.002 0.000 0.000 1.000 1.000

Vari8bles in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! W3

1 0.051 0.010 0.000 0.000 0.0001 0.000 0.136 0.164 0.000 0.0002 0.469 0.030 0.081 0.000 0.0003 0.473 0.824 0.755 0.000 0.000

79 0.006 0.000 0.000 1.000 1.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3

1 0.012 0.006 0.064 0.080 0.1061 0.052 0.210 0.029 0.028 0.0471 0.565 0.002 0.079 0.000 0.0262 0.088 0.072 0.033 0.176 0.7883 0.283 0.710 0.794 0.716 0.033

Variables in Z: INT IX1 IX2 IX! W3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3

1 0.035 0.082 0.122 0.032 0.0291 0.125 0.105 0.020 0.224 0.0011 0.003 0.038 0.001 0.056 0.5091 0.551 0.048 0.032 0.086 0.0083 0.286 0.727 0.826 0.602 0.453

272

Appendix 4: MIXED ExperiJRent 2

*** The wa ***set 5: W4

Vari8bles in Z: INT

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 W4

1 0.043 0.031 0.029 0.000 0.0002 0.063 0.134 0.190 0.000 0.0002 0.000 0.515 0.455 0.000 0.0003 0.892 0.318 0.324 0.000 0.000

298 0.002 0.001 0.002 1.000 1.000

Vari8bl.. in Z: INT IX1

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 W4

1 0.035 0.031 0.030 0.000 0.0002 0.042 0.120 0.124 0.000 0.0002 0.002 0.419 0.581 0.000 0.0003 0.919 0.430 0.263 0.000 0.000

310 0.002 0.000 0.002 1.000 1.000

Variables in Z: INT IX1 IX2

..

•

CI

1123

277

Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4

0.051 0.010 0.000 0.000 0.0000.000 0.136 0.164 0.000 0.0000.472 0.031 0.081 0.000 0.0000.476 0.824 0.755 0.000 0.0000.000 0.000 0.000 1.000 1.000

Variables in Z: INT IX1 IX2 IX3

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 W4

1 0.003 0.000 0.037 0.027 0.0311 0.066 0.208 0.049 0.003 0.0051 0.602 0.007 0.073 0.001 0.0013 0.275 0.688 0.724 0.005 0.1465 0.054 0.097 0.117 0.963 0.817

Variables in Z: INT IX1 IX2 IX3 W4

Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4

1 0.005 0.001 0.050 0.073 0.0761 0.069 0.216 0.035 0.016 0.0181 0.599 0.010 0.071 0.007 0.0033 0.000 0.005 0.007 0.644 0.8063 0.327 0.768 0.837 0.261 0.097

273

Appendix 4: MIXED Experilllent 2

*** The Zs ***set 6: ZD

Variabl.. in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO

1 0.050 0.036 0.038 0.037 0.0382 0.041 0.1" 0.104 0.249 0.0912 0.000 0.023 0.509 0.384 0.0043 0.806 0.002 0.133 0.330 0.0983 0.102 0.828 0.216 0.000 0.768

Variabl.. in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO

1 0.062 0.059 0.064 0.057 0.0141 0.007 0.070 0.064 0.028 0.5792 0.003 0.202 0.037 0.743 0.0062 0.001 0.292 0.598 0.007 0.4003 0.926 0.377 0.238 0.165 0.002

Variables in Z: INT IX1 IX2

CI

11123

Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO

0.065 0.124 0.065 0.054 0.0180.133 0.015 0.113 0.178 0.0060.000 0.004 0.018 0.002 0.9600.314 0.038 0.060 0.742 0.0060.488 0.819 0.744 0.024 0.010

Variables in z: INT IX1 IX2 IX3

..

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO

1 0.029 0.074 0.120 0.055 0.0071 0.046 0.115 0.010 0.155 0.1211 0.499 0.000 0.027 0.004 0.2521 0.112 0.065 0.019 0.072 0.6153 0.314 0.745 0.824 0.713 0.005

Variables in Z: INT IX1 IX2 IX3 ZO

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO

1 0.030 0.082 0.121 0.050 0.0051 0.007 0.070 0.003 0.107 0.1681 0.583 O.OOS 0.064 0.001 0.0351 0.128 0.172 0.011 0.067 0.3613 0.251 0.668 0.801 0.776 0.432

274

Appendix 4: MIXED Experiment 2

*** The Zs ***

set 7: Z1

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1

1 0.043 0.007 0.023 0.031 0.0072 0.045 0.022 0.131 0.193 0.0112 0.000 0.001 0.353 0.504 0.0003 0.912 0.002 0.176 0.272 0.0059 0.000 0.968 0.317 0.000 0.976

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1

1 0.058 0.044 0.055 0.053 0.0211 0.010 0.113 0.053 0.023 0.3002 0.003 0.155 0.046 0.759 0.0013 0.023 0.197 0.789 0.005 0.6423 0.906 0.490 0.057 0.161 0.037

Variables in Z: INT IX1 IX2

CI

11223

Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1

0.030 0.095 0.063 0.019 0.0850.166 0.002 0.066 0.214 0.0110.002 0.012 0.151 0.049 0.7140.346 0.041 0.017 0.701 0.0950.457 0.851 0.702 0.017 0.095

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 Z1

1 0.028 0.097 0.064 0.004 0.0991 0.001 0.017 0.063 0.192 0.0801 0.659 0.000 0.033 0.010 0.0262 0.000 0.126 0.035 0.121 0.7463 0.312 0.760 0.805 0.673 0.049

Variables in Z: INT IX1 IX2 IX3 Z1

CI Variance Proportions for Coefficients ofINT BX1 BX2 Ix3 Z1

1 0.020 0.119 0.037 0.020 0.1151 0.009 0.007 0.097 0.140 0.0411 0.646 0.003 0.047 0.001 0.0162 0.044 0.256 0.005 0.065 0.5393 0.281 0.616 0.813 0.774 0.289

275

Appendix 4: MIXED Experiment 2

*** The Zs ***set 8: Z2

Variabl.. in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z2

1 0.042 0.001 0.005 0.030 0.0012 0.047 0.002 0.023 0.212 0.0012 0.000 0.000 0.071 0.476 0.0003 0.910 0.000 0.032 0.282 0.001

28 0.001 0.997 0.869 0.000 0.998

Variabl.. in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z2

1 0.048 0.015 0.012 0.042 0.0071 0.023 0.061 0.009 0.040 0.0352 0.003 0.090 0.006 0.748 0.0003 0.925 0.206 0.022 0.169 0.0017 0.001 0.628 0.952 0.000 0.958

Variables in Z: INT IX1 IX2

CI

11224

Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z2

0.028 0.032 0.029 0.015 0.0430.150 0.004 0.088 0.198 0.0000.094 0.002 0.178 0.627 0.0500.636 0.001 0.248 0.159 0.2070.093 0.962 0.457 0.001 0.700

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z2

1 0.025 0.037 0.015 0.003 0.0481 0.000 0.000 0.101 0.176 0.0061 0.651 0.003 0.036 0.024 0.0013 0.223 0.005 0.308 0.592 0.2935 0.100 0.955 0.539 0.204 0.652

Variables in Z: INT IX1 IX2 IX! Z2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z2

1 0.012 0.050 0.015 0.017 0.0661 0.003 0.004 0.116 0.137 0.0071 0.707 0.001 0.024 0.012 0.0003 0.096 0.103 0.095 0.427 0.8204 0.183 0.843 0.750 0.407 0.108

276

Appendix 4: MIXED Experiment 2

-TheZs-

Set 9: 13

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 13

1 0.041 0.000 0.001 0.030 0.0002 0.047 0.000 0.003 0.211 0.0002 0.000 0.000 0.008 0.473 0.0003 0.911 0.000 0.004 0.280 0.000

91 0.000 1.000 0.986 0.007 1.000

Variables in Z: INT BX1

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13

1 0.047 0.002 0.001 0.040 0.0011 0.024 0.010 0.001 0.041 0.0032 0.003 0.014 0.001 0.742 0.0003 0.926 0.031 0.003 0.168 0.000

22 0.001 0.944 0.995 0.009 0.996

Variables in Z: INT BX1 BX2

•

CI

1123

13

Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13

0.024 0.004 0.014 0.012 0.0050.152 0.000 0.047 0.198 0.0000.128 0.000 0.091 0.677 0.0030.683 0.003 0.231 0.104 0.0170.014 0.992 0.617 0.010 0.975

Variables in Z: INT BX1 BX2 BX3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13

1 0.023 0.004 0.006 0.004 0.0051 0.001 0.000 0.052 0.174 0.0002 0.651 0.000 0.018 0.025 0.0003 0.318 0.002 0.265 0.787 0.016

14 0.008 0.993 0.659 0.009 0.978

Variables in Z: INT BX1 BX2 BX3 13

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13

1 0.011 0.006 0.006 0.015 0.0081 0.001 0.000 0.060 0.139 0.0012 0.705 0.000 0.011 0.015 0.0003 0.276 0.004 0.273 0.826 0.047

11 0.006 0.989 0.650 0.005 0.944

277

Appendix 4: MIXED Experiment 2

... The Zs *-Set 10: Z4

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 &X2 IX3 Z4

1 0.041 0.000 0.000 0.030 0.0002 0.047 0.000 0.000 0.210 0.0002 0.000 0.000 0.001 0.4n 0.0003 0.911 0.000 0.000 0.278 0.000

211 0.000 1.000 0.998 0.010 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4

1 0.041 0.000 0.000 0.040 0.0001 0.025 0.001 0.000 0.041 0.0002 0.003 0.002 0.000 0.143 0.0003 0.926 0.004 0.000 0.161 0.000

66 0.000 0.993 0.999 0.008 1.000

Variables in Z: INT BX1 BX2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 Z4

1 0.024 0.001 0.003 0.012 0.0011 0.152 0.000 0.010 0.198 0.0002 0.124 0.000 0.020 0.610 0.0003 0.699 0.000 0.051 0.110 0.002

36 0.002 0.999 0.916 0.010 0.991

Variables in Z: INT BX1 BX2 BX3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 Z4

1 0.023 0.001 0.001 0.004 0.0011 0.001 0.000 0.011 0.174 0.0002 0.650 0.000 0.004 0.025 0.0003 0.326 0.000 0.059 0.796 0.002

31 0.001 0.999 0.925 0.001 0.991

Variables in Z: INT BX1 BX2 aX3 Z4

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 Z4

1 0.010 0.001 0.001 0.014 0.0011 0.001 0.000 0.013 0.138 0.0002 0.706 0.000 0.002 0.015 0.0003 0.282 0.001 0.061 0.831 0.006

30 0.001 0.998 0.922 0.001 0.993

278

279

Appendix 4: MIXED Experiment 2

Appendix 4: M(XED Experiment 2

-* The Ws end Zs .-

set 12: WO end Z1

Vari~l .. in Z: INT

CI Variance Proportions for Coefficients ofINT ax1 IX2 IX3 WO Z1

1 0.035 0.005 0.018 0.018 0.015 0.0052 0.000 0.011 0.000 0.069 0.101 0.0092 0.043 0.013 0.430 0.013 0.017 0.0033 0.822 0.003 0.219 0.001 0.062 0.0074 0.101 0.000 0.012 0.897 0.799 0.0009 0.000 0.968 0.320 0.002 0.005 0.976

Variables in Z: INT aX1

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 WO Z1

1 0.039 0.028 0.033 0'.027 0.025 0.0111 0.001 0.013 0.094 0.029 0.041 0.2402 0.041 0.228 0.004 0.042 0.077 0.0693 0.162 0.024 0.821 0.016 0.038 0.4533 0.593 0.698 0.008 0.006 0.064 0.2214 0.164 0.009 0.040 0.879 0.755 0.006

Variables in Z: INT aX1 aX2

C( Variance Proportions for Coefficients ofINT aX1 aX2 aX3 WO Z1

1 0.059 0.027 0.004 0.053 0.053 0.0171 0.004 0.069 0.101 0.017 0.018 0.0812 0.252 0.001 0.161 0.021 0.060 0.1832 0.200 0.042 0.024 0.005 0.037 0.6153 0.297 0.787 0.645 0.076 0.109 0.1033 0.188 0.075 0.065 0.828 0.723 0.001

Variables in Z: (NT aX1 aX2 aX3

CI Variance Proportions for Coefficients of(NT aX1 aX2 aX3 WO Z1

1 0.028 0.097 0.064 0.004 0.000 0.0991 0.001 0.017 0.063 0.192 0.001 0.0801 0.343 0.000 0.021 0.003 0.423 0.0141 0.317 0.000 0.013 0.007 0.576 0.0112 0.000 0.126 0.035 0.121 0.000 0.7463 0.311 0.760 0.805 0.673 0.000 0.049

Variables in Z: (NT aX1 aX2 aX3 WO Z1

C( Variance Proportions for Coefficients of(NT aX1 aX2 ax3 WO Z1

1 0.017 0.093 0.037 0.002 0.037 0.0871 0.000 0.010 0.043 0.182 0.007 0.0541 0.013 0.045 0.084 0.009 0.149 0.0341 0.705 0.000 0.037 0.003 0.000 0.0082 0.049 0.351 0.064 0.005 0.092 0.4404 0.215 0.500 0.736 0.800 0.716 0.377

280

Appendix 4: MIXED ExperiMent 2

- The \Is 8nd Zs -

Set 13: WO 8nd 12

Variebl.. in Z: INT

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO 12

1 0.034 0.001 0.004 0.011 0.015 0.0012 0.000 0.001 0.000 0.012 0.102 0.0012 0.042 0.002 0.084 0.011 0.015 0.0003 0.811 0.000 0.041 0.001 0.064 0.0014 0.101 0.000 0.003 0.895 0.796 0.000

29 0.000 0.991 0.869 0.004 0.008 0.998

variebles in Z: INT IX1

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z2

1 0.036 0.011 0.001 0.023 0.022 0.0041 0.002 0.020 0.015 0.026 0.026 0.0342 0.041 0.104 0.002 0.050 0.095 0.0043 0.138 0.226 0.020 0.018 0.103 0.0014 0.116 0.008 0.005 0.812 0.142 0.0008 0.000 0.631 0.950 0.011 0.012 0.958

Variebles in Z: INT IX1 IX2

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z2

1 0.038 0.022 0.014 0.020 0.011 0.031• 1 0.022 0.013 0.053 0.050 0.055 0.013

2 0.255 0.000 0.166 0.024 0.011 0.0222 0.543 0.002 0.311 0.001 0.020 0.2293 0.031 0.015 0.011 0.868 0.796 0.0085 0.105 0.948 0.444 0.035 0.034 0.698

Variables in Z: INT IX1 IX2 IX3

CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z2

1 0.026 0.031 0.015 0.003 0.001 0.0481 0.000 0.000 0.101 0.116 0.000 0.0061 0.268 0.002 0.019 0.011 0.511 0.0001 0.385 0.001 0.011 0.012 0.474 0.0013 0.220 0.004 0.311 0.596 0.004 0.2905 0.101 0.955 0.536 0.201 0.005 0.655

Variables in Z: INT IX1 IX2 IX3 WO Z2

CI Varience Proportions for Coefficients ofINT IX1 BX2 BX3 WO Z2

1 0.012 0.042 0.014 0.001 0.018 0.0501 0.001 0.001 0.043 0.174 0.032 0.0111 0.023 0.006 0.138 0.002 0.164 0.0041 0.746 0.000 0.001 0.002 0.036 0.0024 0.052 0.181 0.034 0.331 0.368 0.8584 0.161 0.110 0.111 0.484 0.382 0.076

281

Appendix 4: MIXED Experiment 2

*** The Ws lind Zs ***Set 14: WO end Z3

Variabl.. in Z: INT

CI Variance Proportions for Coefficients ofINT ax1 IX2 IX3 WO Z3

1 0.034 0.000 0.000 0.017 0.015 0.0002 0.000 0.000 0.000 0.072 0.102 0.0002 0.042 0.000 0.009 0.012 0.015 0.0003 0.819 0.000 0.004 0.001 0.064 0.0004 0.105 0.000 0.000 0.896 0.803 0.000

95 0.000 1.000 0.986 0.003 0.000 1.000

V.riables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT ax1 IX2 IX3 WO Z3

1 0.035 0.002 0.001 0.023 0.022 0.0001 0.003 0.003 0.002 0.025 0.027 0.0032 0.047 0.016 0.000 0.051 0.095 0.0003 0.742 0.033 0.003 0.018 0.104 0.0004 0.173 0.002 0.000 0.879 0.753 0.000

24 0.001 0.944 0.995 0.004 0.000 0.996

Variables in Z: INT IX1 IX2

CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z3

1 0.031 0.003 0.008 0.015 0.013 0.0041 0.027 0.001 0.027 0.056 0.059 0.0012 0.282 0.000 0.087 0.028 0.082 0.0023 0.558 0.003 0.260 0.001 0.030 0.0183 0.089 0.000 0.001 0.897 0.815 0.000

13 0.014 0.992 0.617 0.004 0.000 0.975

V.riables in Z: INT IX1 IX2 IX3

CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z3

1 0.023 0.004 0.006 0.004 0.000 0.0051 0.001 0.000 0.052 0.175 0.001 0.0011 0.276 0.000 0.009 0.009 0.517 0.0002 0.376 0.000 0.009 0.016 0.481 0.0003 0.317 0.002 0.265 0.788 0.000 0.016

14 0.008 0.993 0.660 0.009 0.001 0.978

V.riables in Z: INT IX1 IX2 IX3 WO Z3

CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z3

1 0.011 0.005 0.006 0.001 0.014 0.0061 0.001 0.000 0.020 0.171 0.039 0.0011 0.045 0.001 0.070 0.002 0.155 0.0002 0.725 0.000 0.000 0.003 0.052 0.0004 0.213 0.005 0.245 0.820 0.738 0.040

12 0.005 0.989 0.659 0.004 0.002 0.952

282

Appendix 4: MIXED ExperiMnt 2

*** The WS end Zs -*set 15: WO end Z4

Variables in Z: INT

CI VarilnCe Proportions for Coefficients ofINT IX1 IX2 IlG WO Z4

1 0.034 0.000 0.000 0.017 0.015 0.0002 0.000 0.000 0.000 0.071 0.102 0.0002 0.042 0.000 0.001 0.012 0.015 0.0003 0.819 0.000 0.000 0.001 0.064 0.0004 0.105 0.000 0.000 0.891 0.802 0.000

284 0.000 1.000 0.998 0.009 0.001 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we Z4

1 0.035 0.000 0.000 0.023 0.021 0.0001 0.003 0.000 0.000 0.025 0.027 0.0002 0.047 0.002 0.000 0.051 0.095 0.0003 0.743 0.004 0.000 0.018 0.103 0.0004 0.172 0.000 0.000 0.877 0.752 0.000

71 0.000 0.993 0.999 0.007 0.001 1.000

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG WO Z4

1 0.030 0.000 0.002 0.014 0.012 0.001• 1 0.028 0.000 0.006 0.056 0.060 0.000

2 0.277 0.000 0.019 0.027 0.081 0.0003 0.571 0.000 0.057 0.001 0.033 0.0023 0.092 0.000 0.000 0.891 0.812 0.000

37 0.002 0.999 0.916 0.010 0.002 0.997

Variables in Z: INT IX1 IX2 IlG

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG we Z4

1 0.023 0.001 0.001 0.004 0.000 0.0011 0.001 0.000 0.011 0.174 0.001 0.0001 0.275 0.000 0.002 0.009 0.519 0.0002 0.376 0.000 0.002 0.016 0.478 0.0003 0.325 0.000 0.059 0.796 0.000 0.002

37 0.001 0.999 0.925 0.001 0.002 0.997

Variables in Z: INT IX1 IX2 IlG WO Z4

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG WO Z4

1 0.011 0.001 0.001 0.001 0.014 0.0011 0.001 0.000 0.004 0.170 0.039 0.0001 0.048 0.000 0.014 0.002 0.153 0.0002 0.723 0.000 0.000 0.003 0.054 0.0004 0.217 0.001 0.050 0.824 0.739 0.005

33 0.000 0.998 0.931 0.000 0.000 0.994

283

Appendix 4: MIXED Experiment 2

.- The \Is 8nd Zs .-

set 16: 111 8nd ZO

Vari~l .. in Z: INT

CI Variance Proportions for Coefficients ofINT 1X1 IX2 IX3 111 ZO

1 0.035 0.021 0.023 0.004 0.004 0.0211 0.003 0.050 0.020 0.012 0.013 0.0692 0.035 0.099 0.535 0.001 0.001 0.0233 0.822 0.002 0.200 0.003 0.003 0.1194 0.100 0.828 0.215 0.000 0.000 0.769

10 0.004 0.001 0.008 0.981 0.979 0.000

Vari~les in Z: INT IX1

CI Variance Proportions for Coefficients ofINT 1X1 IX2 IX3 111 ZO

1 0.036 0.030 0.030 0.005 0.005 0.0052 0.014 0.017 0.128 0.008 0.010 0.2132 0.032 0.223 0.001 0.004 0.004 0.3822 0.001 0.324 0.565 0.000 0.000 0.3973 0.914 0.406 0.268 0.002 0.002 0.002

10 0.003 0.000 0.009 0.981 0.979 0.000

Vari~l .. in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 111 ZO

1 0.053 0.010 0.000 0.010 0.010 0.0011 0.000 0.128 0.152 0.001 0.001 0.0242 0.002 0.003 0.025 0.000 0.000 0.9432 0.470 0.031 0.071 0.008 0.009 0.0223 0.470 0.828 0.752 0.000 0.000 0.0108 0.004 0.000 0.000 0.981 0.979 0.000

Variables in Z: INT IX1 IX2 BX3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 111 ZO

1 0.029 0.070 0.118 0.057 0.008 0.0061 0.042 0.120 0.007 0.147 0.014 0.1201 0.397 0.000 0.036 0.001 0.249 0.1051 0.082 0.003 0.000 0.002 0.640 0.2601 0.137 0.062 0.016 0.081 0.089 0.5033 0.313 0.745 0.824 0.713 0.000 0.005

Variables in Z: INT BX1 BX2 IX3 111 ZO

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 111 ZO

1 0.021 0.083 0.069 0.022 0.045 0.0481 0.014 0.040 0.058 0.000 0.098 0.0921 0.007 0.065 0.004 0.185 0.008 0.0541 0.694 0.000 0.049 0.004 0.001 0.0062 0.070 0.251 0.097 0.001 0.124 0.2644 0.194 0.561 0.n2 0.787 0.n4 0.536

284

Appendix 4: MIXED Experiment 2

-- The Ws 8nd Za --

Set 17: W1 8nd Z1

Var;8blea ;n Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W1 Z1

1 0.032 0.004 0.015 0.003 0.003 0.0042 0.001 0.011 0.006 0.012 0.013 0.0102 0.044 0.014 0.426 0.001 0.002 0.0033 0.919 0.003 0.234 0.002 0.003 0.0069 0.000 0.916 0.318 0.035 0.032 0.926

10 0.004 0.051 0.002 0.947 0.947 0.050

Vari8bles ;n Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1

1 0.035 0.024 0.026 0.005 0.005 0.0081 0.003 0.003 0.107 0.005 0.007 0.2132 0.041 0.250 0.003 0.007 0.008 0.0983 0.035 0.168 0.810 . 0.000 0.000 0.6253 0.883 0.554 0.051 0.002 0.002 0.052

10 0.003 0.000 0.004 0.981 0.979 0.003

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1

1 0.052 0.013 0.000 0.010 0.010 0.006• 1 0.000 0.082 0.102 0.001 0.001 0.092

2 0.190 0.000 0.186 0.004 0.004 0.3102 0.310 0.047 0.005 0.004 0.005 0.4933 0.444 0.857 0.707 0.000 0.000 0.0968 0.003 0.000 0.000 0.981 0.979 0.003

Vari8bles in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1

1 0.028 0.096 0.064 0.005 0.000 0.0971 0.001 0.017 0.057 0.182 0.027 0.0811 0.418 0.000 0.035 0.000 0.311 0.0131 0.242 0.002 0.004 0.024 0.654 0.0102 0.000 0.125 0.036 0.117 0.006 0.7493 0.310 0.760 0.804 0.673 0.001 0.049

Variables in Z: INT IX1 IX2 IX3 W1 Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1

1 0.018 0.093 0.034 0.001 0.036 0.0891 0.000 0.009 0.047 0.185 0.007 0.0481 0.012 0.044 0.084 0.009 0.152 0.0321 0.697 0.000 0.038 0.003 0.000 0.0092 0.050 0.357 0.060 0.004 0.093 0.4474 0.222 0.497 0.736 0.796 0.713 0.374

285

Appendix 4: MIXED Experiment 2

-- The \Is lind Zs --

Sat 18: .,1 lind Z2

Variables in Z: INT

CI Variance Proportiona for Coefficients of •INT IX1 IX2 BX3 .,1 Z2

1 0.031 0.000 0.003 0.003 0.003 0.0002 0.001 0.001 0.001 0.012 0.013 0.0012 0.043 0.002 0.083 0.001 0.002 0.0003 0.920 0.000 0.043 0.002 0.003 0.001

10 0.004 0.000 0.002 0.980 0.978 0.00030 0.001 0.997 0.868 0.001 0.001 0.998

Variables in Z: INT IX1

CI Variance Proportiona for Coefficients ofINT IX1 IX2 IX3 .,1 Z2

1 0.033 0.010 0.006 0.004 0.004 0.0031 0.000 0.010 0.017 0.004 0.005 0.0322 0.049 0.125 0.001 0.009 0.010 0.0063 0.913 0.227 0.024 0.002 0.002 0.0018 0.001 0.628 0.952 0.000 0.000 0.957

10 0.003 0.000 0.000 0.981 0.979 0.001

Variables in Z: INT BX1 BX2

CI Variance Proportiona for Coefficients ofINT BX1 BX2 BX3 .,1 Z2

1 0.040 0.015 0.006 0.005 0.005 0.0231 0.009 0.021 0.061 0.006 0.006 0.0212 0.277 0.000 0.181 0.007 O.ooa 0.0252 0.576 0.001 0.295 0.001 0.002 0.2305 0.094 0.961 0.457 0.000 0.000 0.6989 0.004 0.002 0.000 0.980 0.979 0.003

Variables in Z: INT BX1 BX2 BX3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 .,1 Z2

1 0.025 0.037 0.015 0.003 0.000 0.0481 0.001 0.000 0.096 0.170 0.019 0.0061 0.332 0.001 0.034 0.002 0.435 0.0001 0.319 0.002 0.007 0.029 0.544 0.0013 0.224 0.005 0.309 0.590 0.000 0.2925 0.099 0.955 0.538 0.205 0.002 0.653

Variables in Z: INT BX1 BX2 BX3 .,1 Z2

CI Variance Proportiona for Coefficients ofINT BX1 BX2 BX3 .,1 Z2

1 0.014 0.043 0.014 0.001 0.018 0.0511 0.001 0.000 0.045 0.176 0.031 0.0101 0.020 0.006 0.136 0.003 0.167 0.0041 0.737 0.000 0.001 0.002 0.037 0.0023 0.046 0.200 0.025 0.307 0.346 0.8764 0.183 0.750 0.779 0.510 0.402 0.057

286

Appendix 4: MIXED"Experiment 2

-* The Ws end Zs -*set 19: "1 8nd Z3

Variables in Z: INT,CI Variance Proportions for Coefficients of

INT IX1 IX2 IlG "1 Z31 0.031 0.000 0.000 0.003 0.003 0.0002 0.001 0.000 0.000 0.012 0.013 0.0002 0.043 0.000 0.009 0.001 0.002 0.0003 0.920 0.000 0.005 0.002 0.003 0.000

10 0.004 0.000 0.000 0.980 0.979 0.00091 0.000 1.000 0.986 0.001 0.000 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG "1 Z3

1 0.033 0.001 0.001 0.004 0.004 0.0001 0.000 0.002 0.002 0.004 0.005 0.0032 0.049 0.019 0.000 0.009 0.010 0.0013 0.914 0.034 0.003 0.002 0.002 0.000

10 0.003 0.000 0.000 0.916 0.916 0.00024 0.001 0.944 0.995 0.005 0.002 0.995

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG "1 Z3

1 0.034 0.002 0.005 0.004 0.004 0.003• 1 0.014 0.002 0.029 0.001 O.ooa 0.002

2 0.298 0.000 0.091 0.001 O.ooa 0.0023 0.631 0.003 0.252 0.001 0.001 0.0189 0.003 0.003 0.001 0.966 0.910 0.002

13 0.014 0.990 0.616 0.015 0.009 0.9n

Variables in Z: INT IX1 IX2 IlG

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG "1 Z3

1 0.023 0.004 0.006 0.004 0.000 0.0051 0.001 0.000 0.049 0.110 0.016 0.0001 0.319 0.000 0.011 0.003 0.446 0.0002 0.333 0.000 0.004 0.026 0.531 0.0003 0.316 0.002 0.264 0.788 0.001 0.016

14 O.ooa 0.993 0.661 0.009 0.006 0.979

Variables in Z: INT IX1 IX2 IlG "1 Z3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG "1 Z3

1 0.012 0.005 0.005 0.001 0.014 0.0061 0.001 0.000 0.021 0.113 0.038 0.0011 0.042 0.001 0.067 0.002 0.158 0.0002 0.117 0.000 0.000 0.003 0.053 0.0004 0.221 O.OOS 0.238 0.818 0.135 0.038

12 0.007 0.989 0.668 0.003 0.002 0.955

287

Appendix 4: "MIXED Experiment 2

-- The lis 8nd Zs --•

set 20: 111 8nd Z4

Vari~les in Z: INT

CI Variance Proportions for Coefficients of •INT IX1 IX2 IX3 111 Z41 0.031 0.000 0.000 0.003 0.003 0.0002 0.001 0.000 0.000 0.012 0.013 0.0002 0.043 0.000 0.001 0.001 0.002 0.0003 0.920 0.000 0.001 0.002 0.003 0.000

10 0.004 0.000 0.000 0.947 0.953 0.000294 0.000 1.000 0.998 0.034 0.027 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 111 Z4

1 0.033 0.000 0.000 0.004 0.004 0.0001 0.000 0.000 0.000 0.004 0.005 0.0002 0.050 0.002 0.000 0.009 0.010 0.0003 0.914 0.004 0.000 0.002 0.002 0.000

10 0.003 0.000 0.000 0.945 0.950 0.00073 0.000 0.993 0.999 0.036 0.030 1.000

Variables in Z: INT aX1 aX2

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 111 Z4

1 0.034 0.000 0.001 0.003 0.003 0.0001 0.015 0.000 0.006 0.007 0.007 0.0002 0.293 0.000 0.021 0.007 0.008 0.0003 0.653 0.000 0.053 0.001 0.001 0.0029 0.003 0.000 0.000 0.930 0.937 0.000

38 0.002 0.999 0.919 0.051 0.043 0.997

Vari~les in Z: INT aX1 aX2 aX3

CI Variance Proportions for Coefficients ofINT aX1 IX2 aX3 111 Z4

1 0.023 0.001 0.001 0.004 0.000 0.0011 0.001 0.000 0.010 0.169 0.016 0.0001 0.317 0.000 0.004 0.003 0.435 0.0002 0.334 0.000 0.001 0.026 0.512 0.0003 0.324 0.000 0.057 0.7'96 0.001 0.002

37 0.001 0.999 0.927 0.001 0.037 0.997

Variables in Z: INT aX1 aX2 aX3 111 Z4

CI Variance Proportions for Coefficients ofINT aX1 IX2 IX3 111 Z4

1 0.012 0.001 0.001 0.001 0.014 0.0011 0.001 0.000 0.004 0.172 0.038 0.0001 0.046 0.000 0.013 0.002 0.156 0.0002 0.714 0.000 0.000 0.003 0.056 0.0004 0.227 0.001 0.047 0.821 0.735 0.005

34 0.000 0.998 0.935 0.000 0.001 0.994

288

Appendix 4: MIXED Experillllnt 2

- The wa .-.:I Za -

set 21: W2 .-.:I ZO

Vari~les in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG W2 ZO

1 0.035 0.019 0.023 0.000 0.000 0.0202 0.003 0.055 0.016 0.001 0.001 0.0712 0.033 0.094 0.554 0.000 0.000 0.0203 0.826 0.001 0.190 0.000 0.000 0.1214 0.103 0.820 0.216 0.000 0.000 0.767

30 0.001 0.010 0.001 0.998 0.998 0.001

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO

1 0.035 0.029 0.029 0.001 0.001 0.0052 0.013 0.015 0.131 0.001 0.001 0.2362 0.034 0.230 0.000 0.000 0.000 0.3592 0.001 0.318 0.576 0.000 0.000 0.3973 0.917 0.408 0.262 0.000 0.000 0.002

29 0.000 0.000 0.001 0.998 0.998 0.001

Variables in Z: INT aX1 aX2

CI Variance Proportions for Coefficients ofINT aX1 aX2 BX3 W2 ZO

1 0.051 0.010 0.000 0.001 0.001 0.0001 0.000 0.128 0.152 0.000 0.000 0.0242 0.004 0.003 0.026 0.000 0.000 0.9372 0.471 0.032 0.070 0.001 0.001 0.0263 0.471 0.827 0.752 0.000 0.000 0.010

27 0.002 0.000 0.000 0.998 0.998 0.002

Variables in Z: INT aX1 aX2 aX3

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W2 ZO

1 0.025 0.054 0.115 0.069 0.023 0.0051 0.036 0.135 0.000 0.107 0.100 0.0681 0.562 0.001 0.052 0.002 0.076 0.0371 0.004 0.003 0.009 0.002 0.317 0.6011 0.061 0.067 0.001 0.110 0.481 0.2843 0.312 0.740 0.823 0.710 0.002 0.005

Variables in Z: INT aX1 aX2 aX3 W2 ZO

CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W2 ZO

1 0.029 0.119 0.101 0.017 0.014 0.0181 0.009 0.001 0.023 0.012 0.112 0.1531 0.035 0.068 0.003 0.198 0.045 0.0071 0.645 0.001 0.057 0.015 0.010 0.0042 0.085 0.236 0.084 0.001 0.126 0.2843 0.196 0.575 0.732 0.757 0.692 0.535

289

Appendix 4: MIXED Experillll!nt 2

-- The ... n Zs"-

set 22: W2 n Z1

Variables in Z: INT

CI Variance Proportions for Coefficients of •INT IX1 IX2 IX3 W2 Z1

1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.001 0.001 0.0102 0.042 0.013 0.438 0.000 0.000 0.0033 0.925 0.003 0.226 0.000 0.000 0.0079 0.000 0.965 0.316 0.000 0.000 0.976

31 0.001 0.003 0.000 0.998 0.998 0.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

1 0.034 0.023 0.026 0.001 0.001 0.0081 0.002 0.006 0.108 0.001 0.001 0.2312 0.044 0.249 0.005 0.001 0.001 0.0813 0.036 0.167 0.813 0.000 0.000 0.6273 0.884 0.554 0.048 0.000 0.000 0.051

29 0.001 0.001 0.000 0.998 0.998 0.001

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

1 0.051 0.013 0.000 0.001 0.001 0.0051 0.000 0.083 0.102 0.000 0.000 0.0932 0.190 0.000 0.186 0.000 0.000 0.3112 0.313 0.048 0.005 0.000 0.000 0.4943 0.444 0.857 0.707 0.000 0.000 0.096

27 0.002 0.000 0.000 0.998 0.998 0.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

1 0.028 0.098 0.063 0.004 0.000 0.1001 0.000 0.013 0.047 0.155 0.098 0.0601 0.576 0.000 0.050 0.000 0.089 0.0061 0.086 0.002 0.005 0.037 0.793 0.0702 0.000 0.131 0.031 0.131 0.017 0.7143 0.310 0.756 0.803 0.6n 0.003 0.050

Variables in Z: INT IX1 IX2 IX3 W2 Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1

1 0.016 0.098 0.029 0.000 0.038 0.0971 0.004 0.002 0.108 0.128 0.009 0.0561 0.030 0.047 0.017 0.090 0.175 0.0051 0.673 0.000 0.044 0.010 0.002 0.0072 0.049 0.339 0.053 0.006 0.095 0.465 ..4 0.229 0.514 0.748 0.766 0.681 0.370

290

Appendix 4: MIXED Experiment 2

-- The WS 8nd Zs --

Set 23: W2 8nd Z2

Vari8bles in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! W2 Z2

1 0.031 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.001 0.001 0.0012 0.041 0.002 0.085 0.000 0.000 0.0003 0.926 0.000 0.042 0.000 0.000 0.001

29 0.001 0.816 0.729 0.142 0.142 0.82631 0.000 0.180 0.140 0.856 0.856 0.1n

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2

1 0.032 0.010 0.006 0.000 0.000 0.0031 0.000 0.011 0.017 0.000 0.000 0.0342 0.051 0.125 0.001 0.001 0.001 0.0053 0.916 0.226 0.023 0.000 0.000 0.0018 0.001 0.627 0.952 0.000 0.000 0.956

30 0.000 0.002 0.001 0.998 0.998 0.002

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 22.. 1 0.039 0.015 0.006 0.001 0.001 0.021

1 0.008 0.021 0.061 0.001 0.001 0.0222 0.276 0.000 0.182 0.001 0.001 0.0262 0.582 0.001 0.294 0.000 0.000 0.2305 0.092 0.962 0.457 0.000 0.000 0.700

28 0.002 0.001 0.000 0.998 0.998 0.001

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2

1 0.025 0.037 0.014 0.004 0.003 0.0491 0.002 0.000 0.085 0.149 0.068 0.0041 0.469 0.002 0.057 0.002 0.196 0.0002 0.187 0.002 0.000 0.045 0.n1 0.0063 0.217 0.005 0.305 0.601 0.012 0.2905 0.100 0.955 0.539 0.199 0.000 0.652

Variables in Z: INT IX1 IX2 IX3 W2 Z2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2

1 0.012 0.040 0.012 0.002 0.020 0.0491 0.000 0.000 0.083 0.176 0.007 0.0091 0.005 0.006 0.089 0.027 0.229 0.0011 0.752 0.000 0.005 0.004 0.024 0.0014 0.077 0.103 0.084 0.415 0.444 0.7874 0.153 0.850 0.n8 0.377 0.277 0.152

291

Appendix 4: MIXED Experiment 2

- The wa end Zs -

Set 24: W2 rd Z3

Variebl.. in Z: INT

CI Variance Proportions for Coefficients of "INT IX1 IX2 1X3 W2 Z31 0.031 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.001 0.001 0.0002 0.041 0.000 0.009 0.000 0.000 0.0003 0.927 0.000 0.005 0.000 0.000 0.000

31 0.001 0.000 0.000 0.996 0.997 0.00098 0.000 1.000 0.986 0.002 0.001 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z3

1 0.031 0.001 0.001 0.000 0.000 0.0001 0.000 0.002 0.002 0.000 0.000 0.0032 0.051 0.019 0.000 0.001 0.001 0.0003 0.916 0.034 0.003 0.000 0.000 0.000

24 0.000 0.939 0.991 0.001 0.002 0.99130 0.001 0.005 0.004 0.996 0.996 0.004

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z3

1 0.034 0.002 0.005 0.000 0.000 0.0031 0.013 0.002 0.029 0.001 0.001 0.0022 0.299 0.000 0.097 0.001 0.001 0.0023 0.638 0.003 0.252 0.000 0.000 0.018

13 0.013 0.988 0.615 0.000 0.000 0.97129 0.003 0.004 0.001 0.998 0.998 0.004

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z3

1 0.022 0.004 0.005 0.005 0.003 0.0051 0.003 0.000 0.044 0.147 0.067 0.0001 0.469 0.000 0.028 0.002 0.195 0.0002 0.185 0.000 0.000 0.046 0.n5 0.0003 0.313 0.002 0.263 0.792 0.008 0.016

14 0.008 0.993 0.660 0.008 0.002 0.979

Variables in Z: INT IX1 IX2 IX3 W2 Z3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z3

1 0.011 0.005 0.005 0.002 0.016 0.0061 0.000 0.000 0.038 0.178 0.012 0.0011 0.021 0.001 0.049 0.020 0.222 0.0002 0.738 0.000 0.000 0.004 0.040 0.0004 0.224 0.006 0.249 0.794 0.706 0.039 ..

12 0.006 0.989 0.658 0.002 0.003 0.953

292

Appendix 4: MIXED Experiment 2

- The wa 8nd Zs -*Set 25: W2 8nd Z4

Variebles in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4

1 0.031 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.001 0.001 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.927 0.000 0.001 0.000 0.000 0.000

31 0.001 0.000 0.000 0.998 0.998 0.000291 0.000 1.000 0.998 0.000 0.000 1.000

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4

1 0.031 0.000 0.000 0.000 0.000 0.0001 0.001 0.000 0.000 0.000 0.000 0.0002 0.051 0.002 0.000 0.001 0.001 0.0003 0.917 0.004 0.000 0.000 0.000 0.000

30 0.000 0.000 0.000 0.998 0.998 0.00073 0.000 0.993 0.999 0.000 0.000 1.000

... Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4

1 0.034 0.000 0.001 0.000 0.000 0.0001 0.013 0.000 0.006 0.001 0.001 0.0002 0.295 0.000 0.021 0.001 0.001 0.0003 0.654 0.000 0.055 0.000 0.000 0.002

29 0.002 0.001 0.001 0.994 0.993 0.00138 0.002 0.997 0.915 0.004 0.005 0.995

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4

1 0.022 0.001 0.001 0.005 0.003 0.0011 0.003 0.000 0.010 0.146 0.067 0.0001 0.470 0.000 0.006 0.002 0.194 0.0002 0.183 0.000 0.000 0.048 0.n9 0.0003 0.321 0.000 0.059 0.798 0.007 0.002

31 0.001 0.999 0.925 0.001 0.001 0.997

Variables in Z: INT IX1 IX2 IX3 W2 Z4

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4

1 0.011 0.001 0.001 0.002 0.016 0.0011 0.000 0.000 0.008 0.177 0.012 0.0001 0.022 0.000 0.010 0.020 0.221 0.0002 0.738 0.000 0.000 0.004 0.042 0.0004 0.229 0.001 0.053 0.196 0.109 0.005

32 0.000 0.998 0.921 0.000 0.000 0.994

293

Appendix 4: MIXED ExperiMent 2

*** The Ws IInCI Zs -*Set 26: W3 IInCI ZO

Variabl.. in Z: INT

CI Variance Proportions for Coefficients ofINT 8X1 8X2 8X3 W3 ZO

1 0.034 0.020 0.023 0.000 0.000 0.0202 0.003 0.055 0.017 0.000 0.000 0.0712 0.034 0.096 0.552 0.000 0.000 0.0203 0.826 0.001 0.192 0.000 0.000 0.1194 0.101 0.828 0.216 0.000 0.000 0.767

89 0.001 0.001 0.000 1.000 1.000 0.002

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W3 ZO

1 0.035 0.029 0.029 0.000 0.000 0.0052 0.014 0.016 0.131 0.000 0.000 0.2312 0.034 0.229 0.001 0.000 0.000 0.3622 0.001 0.317 0.576 0.000 0.000 0.3963 0.915 0.409 0.262 0.000 0.000 0.002

aa 0.002 0.000 0.001 1.000 1.000 0.004

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 ZO

1 0.051 0.010 0.000 0.000 0.000 0.0011 0.000 0.128 0.152 0.000 0.000 0.0242 0.003 0.003 0.026 0.000 0.000 0.9412 0.470 0.032 0.071 0.000 0.000 0.0243 0.470 0.827 0.752 0.000 0.000 0.010

79 0.006 0.000 0.000 1.000 1.000 0.001

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 ZO

1 0.012 0.005 0.063 0.081 0.107 0.0001 0.039 0.183 0.027 0.023 0.037 0.0731 0.564 0.000 0.060 0.000 0.013 0.0661 0.015 0.025 0.026 0.004 0.034 0.8472 0.086 0.075 0.030 0.181 o.m 0.0103 0.285 0.712 0.794 0.711 0.033 0.005

Variables in Z: INT IX1 IX2 IX3 W3 ZO

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 ZO

1 0.005 0.000 0.027 0.072 O.OU 0.0891 0.035 0.161 0.090 0.000 0.004 0.0251 0.461 0.009 0.057 0.062 0.032 0.0151 0.232 0.027 0.004 0.257 0.139 0.0152 0.049 0.181 0.055 0.004 0.192 0.333 ..3 0.218 0.621 0.769 0.605 0.546 0.523

294

Appendix 4: MIXED Experilnent 2

-* The wa 8M Zs .-

set 27: W3 8M Z1

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT ax1 IX2 IX3 W3 Z1

1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.000 0.000 0.0102 0.043 0.013 0.437 0.000 0.000 0.0033 0.923 0.003 0.227 0.000 0.000 0.0069 0.000 0.967 0.316 0.000 0.000 0.975

92 0.002 0.001 0.000 1.000 1.000 0.001

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1

1 0.034 0.023 0.026 0.000 0.000 O.OOS1 0.002 0.006 0.1OS 0.000 0.000 0.2282 0.043 0.250 0.005 0.000 0.000 0.0843 0.036 0.167 0.813 0.000 0.000 0.6273 0.882 0.555 0.048 0.000 0.000 0.051

88 0.003 0.000 0.000 1.000 1.000 0.001

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W3 Z1

1 0.050 0.012 0.000 0.000 0.000 0.005.: 1 0.000 0.083 0.102 0.000 0.000 0.094

2 0.189 0.000 0.186 0.000 0.000 0.3122 0.313 0.048 0.005 0.000 0.000 0.4933 0.442 0.857 0.707 0.000 0.000 0.096

80 0.006 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 BX2 IX3 W3 Z1

1 0.012 0.006 0.065 0.079 0.104 0.0001 0.017 0.106 0.014 0.017 0.023 0.1471 0.598 0.000 0.065 0.000 0.016 0.0042 0.036 0.044 0.071 0.001 0.214 0.6382 0.052 0.114 0.006 0.230 0.611 0.1633 0.285 0.730 0.780 0.673 0.031 0.047

Variables in Z: INT IX1 IX2 IX3 W3 Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1

1 0.004 0.057 0.000 0.044 0.065 0.1091 0.026 0.069 0.137 0.045 0.021 0.0001 0.567 0.005 0.036 0.028 0.042 0.0081 0.124 0.017 O.OOS 0.265 0.212 0.0042 0.028 0.286 0.030 0.002 0.127 0.5203 0.250 0.566 0.788 0.615 0.534 0.359

295

Appendix 4: MIXED ExperiMent 2

- The Ws and Zs -*Set 28: Y3 and Z2

Verfebl.. in Z: INT

CI Veriance Proportions for Coefficients ofINT 1X1 IX2 BX3 Y3 Z2

1 0.030 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.000 0.000 0.0012 0.041 0.002 0.085 0.000 0.000 0.0003 0.925 0.000 0.042 0.000 0.000 0.001

30 0.001 0.995 0.868 0.000 0.000 0.99692 0.001 0.002 0.002 1.000 1.000 0.002

Variebl.. in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Y3 Z2

1 0.032 0.010 0.006 0.000 0.000 0.0031 0.000 0.011 0.017 0.000 0.000 0.0332 0.051 0.125 0.001 0.000 0.000 0.0053 0.914 0.226 0.023 0.000 0.000 0.0018 0.001 0.627 0.949 0.000 0.000 0.954

90 0.002 0.002 0.003 1.000 1.000 0.004

Variables in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Y3 Z2

1 0.039 0.014 0.006 0.000 0.000 0.0211 0.008 0.021 0.061 0.000 0.000 0.0222 0.275 0.000 0.182 0.000 0.000 0.026 '/2 0.57'9 0.001 0.294 0.000 0.000 0.2295 0.092 0.960 0.457 0.000 0.000 0.697

84 0.006 0.003 0.000 1.000 1.000 0.005

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT 1X1 IX2 BX3 Y3 Z2

1 0.020 0.036 O.ooa 0.010 O.ooa 0.0491 0.010 0.001 0.056 0.083 0.109 0.0001 0.538 0.002 0.07'9 0.000 0.032 0.0002 0.174 0.001 0.064 0.059 o.no 0.0353 0.157 0.006 0.254 0.676 0.131 0.2655 0.100 0.954 0.539 0.171 0.000 0.650

Variables in Z: INT IX1 IX2 IX3 Y3 Z2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z2

1 O.ooa 0.038 0.004 0.019 0.030 0.0531 0.006 0.011 0.136 0.089 0.038 0.0011 0.698 0.000 0.017 0.001 0.041 0.0002 0.035 0.002 O.ooa 0.237 0.315 0.0003 0.101 0.063 0.130 0.390 0.361 0.7874 0.151 0.886 0.706 0.263 0.214 0.159

296

Appendix 4: MIXED Experiment 2

.- The Ws 8nd Zs *-Set 29: W3 8nd 13

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13

1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.042 0.000 0.009 0.000 0.000 0.0003 0.925 0.000 0.005 0.000 0.000 0.000

92 0.002 0.000 0.000 1.000 1.000 0.00098 0.000 1.000 0.986 0.000 0.000 1.000

Variebles in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13

1 0.031 0.001 0.001 0.000 0.000 0.0001 0.000 0.002 0.002 0.000 0.000 0.0032 0.051 0.019 0.000 0.000 0.000 0.0003 0.915 0.034 0.003 0.000 0.000 0.000

24 0.001 0.944 0.994 0.000 0.000 0.99590 0.002 0.000 0.000 1.000 1.000 0.000

Variebles in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13

1 0.034 0.002 0.004 0.000 0.000 0.0031 0.013 0.002 0.029 0.000 0.000 0.0022 0.298 0.000 0.098 0.000 0.000 0.0023 0.636 0.003 0.252 0.000 0.000 0.018

13 0.014 0.992 0.616 0.000 0.000 0.97485 0.006 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13

1 0.018 0.004 0.003 0.010 0.009 0.0051 0.011 0.000 0.029 0.081 0.107 0.0001 0.542 0.000 0.039 0.000 0.031 0.0002 0.162 0.000 0.030 0.079 0.775 0.0023 0.260 0.002 0.239 0.819 0.078 0.015

14 0.007 0.993 0.660 0.009 0.000 0.978

Variables in Z: INT IX1 IX2 IX3 W3 13

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13

1 0.008 0.005 0.002 0.015 0.024 0.0061 0.003 0.001 0.068 0.098 0.041 0.0002 0.705 0.000 0.007 0.000 0.043 0.0002 0.030 0.000 0.004 0.232 0.321 0.0004 0.250 0.006 0.271 0.651 0.571 0.042

12 0.004 0.988 0.649 0.004 0.001 0.951

297

Appendix 4: MIXED Experiment 2

*** The \Is and Zs ***set 30: W3 end Z4

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W3 Z4

1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.925 0.000 0.001 0.000 0.000 0.000

92 0.002 0.000 0.000 0.996 0.997 0.000292 0.000 1.000 0.998 0.003 0.003 1.000

Variables in Z: INT BX1

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z4

1 0.031 0.000 0.000 0.000 0.000 0.0001 0.000 0.000 0.000 0.000 0.000 0.0002 0.051 0.002 0.000 0.000 0.000 0.0003 0.915 0.004 0.000 0.000 0.000 0.000

n 0.000 0.973 0.980 0.008 0.009 0.97990 0.002 0.020 0.020 0.992 0.991 0.020

Variables in Z: INT BX1 BX2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z4

1 0.033 0.000 0.001 0.000 0.000 0.0001 0.013 0.000 0.006 0.000 0.000 0.0002 0.294 0.000 0.021 0.000 0.000 0.0003 0.652 0.000 0.055 0.000 0.000 0.002

37 0.002 0.995 0.913 0.000 0.000 0.99386 0.007 0.003 0.003 1.000 1.000 0.003

Variables in Z: INT BX1 BX2 BX3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z4

1 0.018 0.001 0.001 0.010 0.009 0.0011 0.011 0.000 0.006 0.081 0.107 0.0001 0.539 0.000 0.009 0.000 0.032 0.0002 0.165 0.000 0.007 0.079 o.m 0.0003 0.266 0.000 0.053 0.829 0.079 0.002

37 0.001 0.999 0.925 0.000 0.001 0.997

Variables in Z: INT BX1 BX2 BX3 W3 Z4

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z4

1 0.008 0.001 0.000 0.015 0.024 0.0011 0.003 0.000 0.015 0.098 0.041 0.0002 0.705 0.000 0.001 0.000 0.043 0.0002 0.030 0.000 0.001 0.231 0.323 0.0004 0.253 0.001 0.060 0.656 0.569 0.006

32 0.001 0.998 0.923 0.000 0.001 0.993

298

Appendix 4: MIXED Experiment 2

*** The wa end Zs ***Set 31: W4 end ZO

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT 8X1 IX2 BX3 W4 ZO

1 0.034 0.020 0.023 0.000 0.000 0.0202 0.003 0.055 0.017 0.000 0.000 0.0712 0.034 0.096 0.552 0.000 0.000 0.0203 0.825 0.001 0.192 0.000 0.000 0.1194 0.102 0.828 0.216 0.000 0.000 0.765

318 0.002 0.001 0.001 1.000 1.000 0.005

Variables in Z: INT 8X1

CI Varience Proportions for Coefficients ofINT IX1 8X2 8X3 W4 ZO

1 0.035 0.029 0.029 0.000 0.000 0.0052 0.014 0.016 0.131 0.000 0.000 0.2322 0.034 0.230 0.001 0.000 0.000 0.3612 0.001 0.318 0.576 0.000 0.000 0.3963 0.916 0.408 0.262 0.000 0.000 0.002

312 0.002 0.000 0.001 1.000 1.000 0.003

Variables in Z: INT BX1 8X2..CI Variance Proportions for Coefficients of

INT BX1 8X2 8X3 W4 ZO1 0.051 0.010 0.000 0.000 0.000 0.0011 0.000 0.128 0.152 0.000 0.000 0.0242 0.003 0.003 0.026 0.000 0.000 0.9342 0.4n 0.032 0.071 0.000 0.000 0.0233 0.473 0.827 0.752 0.000 0.000 0.010

278 0.001 0.000 0.000 1.000 1.000 0.008

Variables in Z: INT BX1 BX2 BX3

CI Variance Proportions for Coefficients ofINT 8X1 BX2 BX3 W4 ZO

1 0.003 0.000 0.036 0.027 0.031 0.0021 0.051 0.184 0.047 0.002 0.003 0.0591 0.548 0.000 0.042 0.000 0.000 0.1562 0.068 0.029 0.036 0.003 0.003 0.7753 0.275 0.686 0.719 0.004 0.146 0.0035 0.056 0.101 0.119 0.963 0.816 0.005

Variables in z: INT BX1 BX2 BX3 W4 ZO

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 ZO

1 0.001 0.001 0.012 0.041 0.042 0.0431 0.039 0.164 0.096 0.000 0.000 0.0152 0.625 0.001 0.066 0.000 0.000 0.0112 0.080 0.1n 0.029 0.021 0.010 0.4224 0.000 0.010 0.016 0.612 0.876 0.0314 0.255 0.652 0.781 0.326 0.073 0.478

299

Appendix 4: MIXED Experilllent 2

*** The WI and Zs ***set 32: W4 and Z1

Variables in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 W4 Z1

1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.000 0.000 0.0102 0.043 0.013 0.436 0.000 0.000 0.0033 0.923 0.003 0.226 0.000 0.000 0.0069 0.000 0.961 0.316 0.000 0.000 0.915

325 0.002 0.001 0.003 1.000 1.000 0.001

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 W4 Z1

1 0.034 0.023 0.025 0.000 0.000 0.0081 0.002 0.006 0.108 0.000 0.000 0.2292 0.043 0.250 0.005 0.000 0.000 0.0843 0.036 0.161 0.811 0.000 0.000 0.6213 0.884 0.554 0.049 0.000 0.000 0.051

314 0.002 0.000 0.002 1.000 1.000 0.001

Variables in Z: INT IX1 IX2»

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1

1 0.051 0.012 0.000 0.000 0.000 0.0051 0.000 0.083 0.102 0.000 0.000 0.0942 0.181 0.000 0.186 0.000 0.000 0.3162 0.311 0.048 0.004 0.000 0.000 0.4893 0.445 0.857 0.701 0.000 0.000 0.096

218 0.000 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT IX1 IX2 IX3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1

1 0.002 0.000 0.032 0.028 0.032 0.0051 0.027 0.111 0.038 0.000 0.001 0.1311 0.645 0.000 0.050 0.001 0.000 0.0112 0.005 0.014 0.080 0.006 0.012 0.n43 0.266 0.722 0.681 0.003 0.142 0.0185 0.056 0.092 0.120 0.962 0.813 0.001

Variables in Z: INT IX1 IX2 IX3 W4 Z1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1

1 0.000 0.012 0.005 0.043 0.044 0.0501 0.034 0.120 0.110 0.004 0.002 0.0192 0.644 0.001 0.056 0.000 0.000 0.0062 0.034 0.248 0.010 0.022 0.014 0.5883 0.000 0.008 0.010 0.627 0.839 0.0114 0.288 0.611 0.809 0.304 0.101 0.326

300

Appendix 4: MIXED Experiment 2

• .- The wa end Zs *-set 33: W4 end Z2

Vari~les in Z: INT

CI Vari.nce Proportions for Coefficients ofINT IX1 IX2 BX3 W4 Z2

1 0.030 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.000 0.000 0.0012 0.041 0.002 0.084 0.000 0.000 0.0003 0.925 0.000 0.041 0.000 0.000 0.001

30 0.001 0.986 0.858 0.000 0.000 0.987328 0.002 0.011 0.013 1.000 1.000 0.011

Vari~les in Z: INT IX1

CI Vari~e Proportions for Coefficients ofINT IX1 IX2 IX3 .W4 Z2

1 0.032 0.010 0.006 0.000 0.000 0.0031 0.000 0.011 0.017 0.000 0.000 0.0332 0.051 0.125 0.001 0.000 0.000 0.0053 0.915 0.225 0.023 0.000 0.000 0.0018 0.001 0.625 0.943 0.000 0.000 0.950

319 0.002 0.005 0.010 1.000 1.000 0.008

Vari~les in Z: INT IX1 IX2

CI Vari.nce Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z2

1 0.040 0.014 0.006 0.000 0.000 0.021, 1 0.008 0.022 0.061 0.000 0.000 0.0222 0.276 0.000 0.182 0.000 0.000 0.0262 0.583 0.001 0.294 0.000 0.000 0.2295 0.092 0.961 0.457 0.000 0.000 0.698

292 0.000 0.002 0.000 1.000 1.000 0.003

Variables in Z: INT IX1 IX2 IX3

CI Vari.nce Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z2

1 0.001 0.009 0.008 0.022 0.025 0.0191 0.028 0.028 0.048 O.OOS 0.005 0.0272 0.606 0.002 0.065 0.001 0.001 0.0003 0.254 0.000 0.355 0.017 0.083 0.2325 0.036 0.779 0.311 0.064 0.204 0.6685 0.075 0.182 0.213 0.892 0.682 0.054

Variables in Z: INT BX1 BX2 BX3 W4 Z2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z2

1 0.002 0.018 0.000 0.030 0.032 0.0331 0.019 0.033 0.104 0.020 0.018 0.0102 0.688 0.000 0.036 0.003 0.001 0.000

.~ 3 0.134 0.014 0.210 0.023 0.358 0.6603 0.027 0.001 0.021 0.870 0.525 0.0534 0.131 0.933 0.628 0.053 0.065 0.244

301

Appendix 4: MIXED Experiment 2

-- The wa 8nd Zs --

set 34: W4 8nd Z3

Variebl.. in Z: INT

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG W4 Z3

1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 D.OOO 0.000 0.0002 0.041 0.000 0.009 0.000 0.000 0.0003 0.925 0.000 O.OOS 0.000 0.000 0.000

98 0.000 0.993 0.979 0.000 0.000 0.993321 0.002 0.001 0.001 1.000 1.000 0.006

Varieblea in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG W4 Z3

1 0.031 0.001 0.001 0.000 0.000 0.0001 0.000 0.002 0.002 0.000 0.000 0.0032 0.051 0.019 0.000 0.000 0.000 0.0003 0.915 0.034 0.003 0.000 0.000 0.000

24 0.001 0.941 0.991 0.000 0.000 0.992318 0.002 0.003 0.004 1.000 1.000 0.004

Varieblea in Z: INT IX1 IX2

CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG W4 Z3

1 0.034 0.002 0.004 0.000 0.000 0.0031 0.013 0.002 0.029 0.000 0.000 0.0022 0.300 0.000 0.098 0.000 0.000 0.0023 0.640 0.003 0.252 0.000 0.000 0.018

13 0.014 0.992 0.611 0.000 0.000 0.914298 0.000 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT IX1 IX2 IlG

CI Variance Proportions for coefficients ofINT IX1 IX2 IlG W4 Z3

1 0.004 0.002 0.001 0.016 0.019 0.0031 0.024 0.003 0.026 0.010 0.011 0.0022 0.606 0.000 0.032 0.001 0.001 0.0003 0.321 0.002 0.251 0.020 0.111 0.0155 0.038 0.000 0.023 0.944 0.855 0.002

14 O.ooa 0.993 0.661 0.009 0.002 0.918

Variables in Z: INT IX1 IX2 IlG W4 Z3

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z3

1 0.002 0.003 0.000 0.026 0.021 0.004 ..1 0.016 0.003 0.050 0.026 0.024 0.0012 0.684 0.000 0.011 0.004 0.002 0.0004 0.026 0.001 0.044 0.351 0.941 0.0014 0.266 0.005 0.246 0.582 0.006 0.038

13 0.006 0.981 0.641 0.005 0.000 0.950

302

Appendix 4: MIXED Experiment 2

... The lis end Z. *-Set 35: W4 end Z4

V.riables in Z: INT

CI Vari8nCe Proportions for Coefficients ofINT IX1 IX2 BX3 W4 Z4

1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.925 0.000 0.001 0.000 0.000 0.000

291 0.000 0.997 0.995 0.002 0.002 0.997327 0.002 0.003 0.003 0.998 0.998 0.003

Variables in Z: INT IX1

CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z4

1 0.031 0.000 0.000 0.000 0.000 0.0001 0.000 0.000 0.000 0.000 0.000 0.0002 0.051 0.002 0.000 0.000 0.000 0.0003 0.916 0.004 0.000 0.000 0.000 0.000

73 0.000 0.992 0.999 0.000 0.000 0.999318 0.002 0.001 0.001 1.000 1.000 0.001

Variables in Z: INT IX1 BX2

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z4

1 0.034 0.000 0.001 0.000 0.000 0.0001 0.013 0.000 0.006 0.000 0.000 0.0002 0.295 0.000 0.021 0.000 0.000 0.0003 0.656 0.000 0.055 0.000 0.000 0.002

38 0.002 0.999 0.916 0.000 0.000 0.996298 0.000 0.000 0.000 1.000 1.000 0.000

Variables in Z: INT BX1 BX2 BX3

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z4

1 0.004 0.000 0.000 0.016 0.018 0.0001 0.024 0.000 0.006 0.010 0.012 0.0002 0.604 0.000 0.007 0.001 0.001 0.0003 0.327 0.000 0.056 0.019 0.114 0.0025 0.041 0.000 0.006 0.950 0.853 0.000

38 0.001 0.999 0.925 0.003 0.002 0.997

Variablell in Z: INT BX1 BX2 BX3 W4 Z4

CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z4

1 0.003 0.000 0.000 0.025 0.027 0.0011 0.016 0.000 0.011 0.027 0.025 0.0002 0.684 0.000 0.004 0.004 0.002 0.0004 0.032 0.000 0.011 0.332 0.943 0.0014 0.265 0.001 0.053 0.609 0.002 0.005

34 0.001 0.998 0.921 0.002 0.001 0.993

303

REFERENCES

Beckman, R. J., Nachtsheim, C. J., and Cook, D. R. (1987), "Diagnostics forMixed-Model Analysis of Variance," Technometrics, 29 (4), 413-426.

Belsley, D. A., Kuh, E., and Welsch, R. E. (1980), Regression Diagnostics:Identifying Influential Data and Sources of Collinearity, New York: JohnWiley.

Belsley, D. A. (1984), "Demeaning Conditioning Diagnostics ThroughCentering," The American Statistician, 38, 73-77.

Belsley, D. A. (1987), "Comment: Well-conditioned Collinearity Indices?"Statistical Science, 2, 86-91.

Belsley, D. A. (1991), Conditioning Diagnostics: Collinearity and Weak Datain Regression, New York: John Wiley.

Callanan, T. P. and Harville, D. A. (1991), "Some New Algorithms forComputing Restricted Maximum Likelihood Estimates of VarianceComponents," Journal of Statistical Computation and Simulation, 38,239-259.

Chatterjee, S. and Hadi, A. S. (1988), Sensitivity Analysis in Linear Regression,New York: John Wiley.

Chatterjee, S. and Price, B. (1991), Regression Analysis by Example, NewYork: John Wiley.

Christensen, R., Pearson, L. M. and Johnson, W. (1992), "Case-DeletionDiagnostics for Mixed Models," Technometrics, 34 (1), 38-45.

Cook, R. D. (1984), Comment on "Demeaning Conditioning DiagnosticsThrough Centering," The American Statistician, 38, 78-79.

Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), "Maximum Likelihoodfrom Incomplete Data via the EM algorithm (with Discussion)," Journalof the Royal Statistical Society (8),39,1-38.

.\<

Diggle, P. J. 1988), "An Approach to the Analysis of Repeated Measures,"Biometrics, 44, 959-971.

Fairclough, D. L. and Helms, R. W. (1984), "Mixed Effects Model Analyses ofIncomplete Longitudinal Pulmonary Function Measurements in Children, "Institute of Statistics Mimeo Series, University of North Carolina atChapel Hill.

Fairclough, D. L. and Helms, R. W. (1986), "A Mixed Model with LinearCovariance Structure: A Sensitivity Analysis of the Maximum LikelihoodEstimators, " JournalofStatistical Computation andSimulation, 25, 205­236.

Grady, J. J. and Helms, R. W. (1992), "Structured Covariance Matrices forIncomplete Longitudinal Data," Institute of Statistics Mimeo Series,University of North Carolina at Chapel Hill.

Gunst, R. A. (1984), "Toward a Balanced Assessment of CollinearityDiagnostics (Comment)," The American Statistician, 38, 79-82.

Hadi, A. S. and Velleman, P. F. (1987), "Comment: Diagnosing NearCollinearities in Least Squares Regression," Statistical Science, 2, 93-98.

Hamilton, D. (1987), "Sometimes R2 >r~, + r~.: Correlated Variables Are Not

Always Redundant," The American Statistician, 41, 129-132.

Harville, D. A. (1977), "Maximum Likelihood Approaches to VarianceComponent Estimation and to Related Problems," Journal of theAmerican Statistical Association, 72, 320-340.

Helms, R. W. (1992), The Joy of Modeling.

Helms, R. W. (1993), Personal Communication.

Holditch-Davis, D., Helms, R. W., and Edwards, L. J. (1992), "A Model ofDevelopment of Sleeping and Waking in Preterm Infants,"

Jennrich, R. I. and Schluchter, M. D. (1986), "Unbalanced Repeated-MeasuresModels with Structured Covariance Matrices," Biometrics, 42, 805-820.

Kleinbaum, D. G., Kupper, L. L., and Muller, K. E. (1988), Applied RegressionAnalysis and Other Multivariable Methods, Boston, MA: PWS-KentPublishing Company.

Laird, N. M. (1988), "Missing Data in Longitudinal Studies," Statistics inMedicine, 7, 305-315.

305

Laird, N. M. and Ware, J. H. (1982), "Random-Effects Models for LongitudinalData," Biometrics, 38, 963-974.

Laird, N. M., Lange, N, and Stram, D. (1987), "Maximum LikelihoodComputations With Repeated Measures: Application of the EMAlgorithm, " Journalof the American Statistical Association, 82, 97-105.

Lindstrom, M. J. and Bates, D. M. (1988), "Newton-Raphson and EMAlgorithms for Linear Mixed-Effects Models for Repeated MeasuresData," Journal of the American Statistical Association, 83, 1014-1022.

Louis, T. A. (1988), "General Methods for Analysing Repeated Measures,"Statistics in Medicine, 7, 29-45.

Mandel, J. (1982), "Use of the Singular Value Decomposition in RegressionAnalysis," The American Statistician, 36, 15-24.

Mansfield, E.R. and Helms, B. P. (1982), "Detecting Multicol/inearities," TheAmerican Statistician, 36, 158-60.

Marquardt, D. W. (1980), "You Should Standardize the Predictor Variables inYour Regression Models, " Journalof the American StatisticalAssociation75, 74-103.

Marquardt, D. W. (1987), "Comment on Collinearity and Least SquaresRegression," Statistical Science, 1, 84-85.

Mason, R. L. and Gunst, R. F. (1985), "Outlier-induced Col/inearities,"Technometrics 27, 401-407.

Myers, R. H. (1986), Classical and Modern Regression with Applications,Boston: Duxbury Press.

Neter, J., Wasserman, W., and Kutner, M. H. (1985), Applied Linear StatisticalModels, Homewood, 11/: Richard D. Irwin, Inc.

Oman, S.D. (1982), "Shrinking Towards Subspaces in Multiple LinearRegression," Technometrics, 24, 307-311.

Rawlings, J. O. (1988), Applied Regression Analysis: A Research Tool,Belmont, CA: Wadsworth, Inc.

SAS Institute Inc (1988), SASIIML™ User's Guide, Release 6.03 Edition, Cary,NC: SAS Institute Inc.

Searle, S. R. (1988), "Mixed Models and Unbalanced Data: Wherefrom,Whereat, Whereto?, " Communications in Statistics-Theory andMethods,

306

)

J

•

17, 935-968.

Schindler, J. S. (1986), "Regression Diagnostics: Mechanical and StructuralAspects of Collinearity," Institute of Statistics Mimeo Series, Universityof North Carolina at Chapel Hill.

Smith, G. and Campbell, F., (1980), "A Critique of Some Ridge RegressionMethods," Journal of the American Statistical Association, 75, 74-81.

Snee, R. D. and Marquardt, D. W. (1984), "Collinearity Diagnostics Depend onthe Domain of Prediction, the Model, and the Data (Comment)," TheAmerican Statistician, 38, 83-87.

Stewart, G. W. (1987), "Collinearity and Least Squares Regression, " StatisticalScience 1, 68-100.

Strang G. (1988), Linear Algebra and Its Applications, New York: AcademicPress.

Strope, G. L. and Helms, R. W. (1984) A Longitudinal Study of Spirometry inYoung Black and White Children. manuscript

Swamy, P. A. V. B, Mehta, J. S., Thurman, S. S., and Iyengar, N. S. (1985),"A Generalized Multicollinearity index for Regression Analysis, " Sankhya:The Indian Journal of Statistics, 47, 401-431.

Thisted, R. A. (1987), "Comment on Collinearity and Least SquaresRegression," Statistical Science, 1, 91-93.

Velleman, P. F. and Welsch, R. E. (1981), "Efficient Computing of RegressionDiagnostics," The American Statistician, 35, 234-242.

Ware, J. H. (1985), "Linear Models for the Analysis of Longitudinal Studies,"The American Statistician, 39, 95-101.

Willan, A. R. and Watts, D. C., (1978), "Meaningful MulticollinearityMeasures," Technometrics, 20, 407-412.

Wood, F. S. (1984), "Effect of Centering on Collinearity and Interpretation ofthe Constant (Comment)," The American Statistician, 38, 88-90.

307