Matrix Algebra Matrix algebra is a means of

expressing large numbers of calculations made upon ordered sets of numbers.

Often referred to as Linear Algebra

Why use it? Matrix algebra is used primarily to

facilitate mathematical expression. Many equations would be

completely intractable if scalar mathematics had to be used. It is also important to note that the scalar algebra is under there somewhere.

Definitions - scalar scalar - a number

denoted with regular type as is scalar algebra

[1] or [a]

Definitions - vector vector - a single row or column of

numbers denoted with bold small letters row vector

column vector

54321

5

4

3

2

1

Definitions - Matrix A matrix is a set of rows and

columns of numbers

Denoted with a bold Capital letter All matrices (and vectors) have an

order - that is the number of rows x the number of columns. Thus A is

654

321

32654

321

x

Matrix Equality Thus two matrices are equal iff (if

and only if) all of their elements are identical

Note: your data set is a matrix.

Matrix Operations Addition and Subtraction Multiplication Transposition Inversion

Addition and Subtraction Two matrices may be added iff

they are the same order. Simply add the corresponding

elements

3231

2221

1211

3231

2221

1211

3231

2221

1211

cc

cc

cc

bb

bb

bb

aa

aa

aa

Addition and Subtraction (cont.)

Where

Hence

129

107

85

64

64

64

65

43

21

323232

313131

222222

212121

121212

111111

cba

cba

cba

cba

cba

cba

Matrix Multiplication To multiply a scalar times a matrix,

simply multiply each element of the matrix by the scalar quantity

2221

1211

2221

1211

22

222

aa

aa

aa

aa

Matrix Multiplication (cont.) To multiply a matrix times a

matrix, we write A times B as AB

This is pre-multiplying B by A, or post-multiplying A by B.

Matrix Multiplication (cont.) In order to multiply matrices, they

must be conformable (the number of columns in A must equal the number of rows in B.)

an (mxn) x (nxp) = (mxp) an (mxn) x (pxn) = cannot be done a (1xn) x (nx1) = a scalar (1x1)

Matrix Multiplication (cont.) Thus

where

3231

2221

1211

3231

2221

1211

333231

232221

131211

cc

cc

cc

bb

bb

bb

x

aaa

aaa

aaa

32332232123132

31332132113131

32232222122122

31232122112121

32132212121112

31132112111111

bababac

bababac

bababac

bababac

bababac

bababac

Matrix Multiplication- an example Thus

where

9642

8136

6630

63

52

41

963

852

741

3231

2221

1211

cc

cc

cc

x

96695643

42392613

81685542

36382512

66675441

30372411

32

31

22

21

12

11

***

***

***

***

***

***

c

c

c

c

c

c

Matrix multiplication is not Commutative AB does not necessarily equal BA (BA may even be an impossible

operation)

Yet matrix multiplication is Associative A(BC) = (AB)C

Special matrices There are a number of special

matrices Diagonal Null Identity

Diagonal Matrices A diagonal matrix is a square matrix

that has values on the diagonal with all off-diagonal entities being zero.

44

33

22

11

000

000

000

000

a

a

a

a

Identity Matrix An identity matrix is a diagonal

matrix where the diagonal elements all equal one. It is used in a fashion analogous to multiplying through by "1" in scalar math.

1000

0100

0010

0001

Null Matrix A square matrix where all elements

equal zero.

Not usually ‘used’ so much as sometimes the result of a calculation. Analogous to “a+b=0”

0000

0000

0000

0000

The Transpose of a Matrix A' Taking the transpose is an

operation that creates a new matrix based on an existing one.

The rows of A = the columns of A' Hold upper left and lower right

corners and rotate 180 degrees.

Example of a transpose

654

321

63

52

41

', AA

The Transpose of a Matrix A' If A = A', then A is symmetric (i.e.

correlation matrix) If AA’ = A then A' is idempotent (and A' =

A) The transpose of a sum = sum of

transposes

The transpose of a product = the product of the transposes in reverse order

''')'( CBACBA

''')'( ABCABC

An example: Suppose that you wish to obtain

the sum of squared errors from the vector e. Simply pre-multiply e by its transpose e'.

which, in matrices looks like

222

21 neeeee ..'

An example - cont Since the matrix product is a scalar

found by summing the elements of the vector squared.

The Determinant of a Matrix The determinant of a matrix A is

denoted by |A|. Determinants exist only for square

matrices. They are a matrix characteristic,

and they are also difficult to compute

The Determinant for a 2x2 matrix

If A =

Then

That one is easy

21122211 aaaaA

2221

1211

aa

aa

The Determinant for a 3x3 matrix If A =

Then

333231

232221

131211

aaa

aaa

aaa

312213322113332112312312322311332211 aaaaaaaaaaaaaaaaaaA

Determinants For 4 x 4 and up don't try. For those

interested, expansion by minors and cofactors is the preferred method.

(However the spaghetti method works well! Simply duplicate all but the last column of the matrix next to the original and sum the products of the diagonals along the following pattern.)

Spaghetti Method of |A|

3231

2221

1211

333231

232221

131211

aa

aa

aa

aaa

aaa

aaa

Properties of Determinates Determinants have several

mathematical properties which are useful in matrix manipulations.

1 |A|=|A'|. 2. If a row of A = 0, then |A|= 0. 3. If every value in a row is multiplied by

k, then |A| = k|A|. 4. If two rows (or columns) are

interchanged the sign, but not value, of |A| changes.

5. If two rows are identical, |A| = 0.

Properties of Determinates 6. |A| remains unchanged if each

element of a row or each element multiplied by a constant, is added to any other row.

7. Det of product = product of Det's |A| = |A| |B|

8. Det of a diagonal matrix = product of the diagonal elements

The Inverse of a Matrix (A-

1) For an nxn matrix A, there may be a B

such that AB = I = BA. The inverse is analogous to a reciprocal) A matrix which has an inverse is

nonsingular. A matrix which does not have an

inverse is singular. An inverse exists only if

0A

Inverse by Row or column operations Set up a tableau matrix A tableau for inversions consists of

the matrix to be inverted post multiplied by a conformable identity matrix.

Matrix Inversion by Tableau Method

Rules: You may interchange rows. You may multiply a row by a scalar. You may replace a row with the sum of that

row and another row multiplied by a scalar. Every operation performed on A must

be performed on I When you are done; A = I & I = A-1

The Tableau Method of Matrix Inversion: An Example Step 1: Set up Tableau

100

010

001

231

452

341

Matrix Inversion – cont. Step 2: Add –2(Row 1) to Row 2

Step 3: Add –1(Row 1) to Row 3

100

012

001

231

230

341

101

012

001

570

230

341

Matrix Inversion – cont. Step 4: Multiply Row 2 by –1/3

Step 5: Add –4 (Row 2) to Row 1

101

03132

001

570

3210

341

///

101

03132

03435

570

3210

3101

//

//

/

/

Matrix Inversion – cont. Step 6: Add 7(Row 2) to Row 3

Step 7: Add Row 3 to Row 1

137311

03132

03435

3100

3210

3101

//

//

//

/

/

/

137311

03132

112

3100

3210

001

//

//

/

/

Matrix Inversion – cont. Step 9: Add 2(Row 3) to Row 2

Step 9: Multiply Row 3 by -3

137311

258

112

3100

010

001

///

3711

258

112

100

010

001

Checking the calculation Remember AA-1=I

Thus

100

010

001

3711

258

112

231

452

341

etc

0735411

11138421

***

***

The Matrix Model The multiple regression model may

be easily represented in matrix terms.

Where the Y, X, B and e are all matrices of data, coefficients, or residuals

Y XB e

The Matrix Model (cont.) The matrices in are

represented by

Note that we postmultiply X by B since this order makes them conformable.

Y

Y

Y

Yn

1

2

X

X X X

X X X

X X X

ik

k

n n nk

11 12

21 22 2

1 2

. . .

. . .

. . . . . . . . . . . .

. . .

B

B

B

B k

1

2 e

e

e

en

1

2

Y XB e

The Assumptions of the ModelScalar Version

1. The ei's are normally distributed. 2. E(ei) = 0 3. E(ei

2) = 2

4. E(eiej) = 0 (ij) 5. X's are nonstochastic with values fixed in repeated

samples and (Xik-Xbark)2/n is a finite nonzero number. 6. The number of observations is greater than the

number of coefficients estimated. 7. No exact linear relationship exists between any of

the explanatory variables.

The Assumptions of the Model: The Matrix Version These same assumptions

expressed in matrix format are:

1. e N(0,) 2. = 2I 3. The elements of X are fixed in

repeated samples and (1/ n)X'X is nonsingular and its elements are finite