Matrix Algebra Usefulin Statistics

Leif E. Peterson, Ph.D.

Dept. of MedicineBaylor College of Medicine

Scalars

• A scalar is a single data point

• A scalar is real-valued (negative, positive, zero)

• Continuous type of data point (integer with decimal component)

• Assumed infinitely precise

• Examples: 2.1384, -14.801,1E-12

Vectors• In statistics, a vector is an n × 1 matrix (column

vector)• In statistics, a vector commonly represents n

measurements for a single variable• Vector elements are assumed infinitely precise and are

continuously scaled

1

21 2 or [ , ,..., ]n

n

xx

x x x

x

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ ′= =⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

x x

Columnvector

Rowvector

1 2( , )P x x

O(0,0)

Length of a vector x between origin (0,0) and point P

2 2 21 2

2 2 21 2

2 2 21 2

2

1

1 1 2 2

( ,0) ( 0) ( 0) ( 0)

( 0) ( 0) ( 0)

+ + =

n

n

n

n

ii

n n

L d P x x x

x x x

x x x

x

x x x x x x=

= = = − + − + + −

= − + − + + −

= + + +

=

= +′

∑

x x

x x

x

y

(age)

(weight)

2 2( , )Q x y

1 1( , )P x y

O

2 21 2 1 2( , ) ( ) ( )d P Q x x y y= − + −

( , )d P Q

Geometric interpretation of two-dimensional vectors (one for each patient)

x

y

z

(age)

(weight)

(BP)

2 2 2( , , )P x y z1 1 1( , , )Q x y z

2 2 21 2 1 2 1 2( , ) ( ) ( ) ( )d P Q x x y y z z= − + − + −

O

Geometric interpretation of three-dimensional vectors (one for each patient)

( , )d P Q

(1,0)

Unit circle

(30 )6π

(45 )4π

(60 )3π(90 )

2π

2 (120 )3π

3 (135 )4π

5 (150 )6π

(180 )π

3 (270 )2π

2 (360 )π

2 2 1x y+ =

θ(radians) θ(degrees) cos(θ) 0 0 1

6π

30 0.866

4π

45 0.707

3π

60 0.5

2π

90 0

34π

135 -0.707

π 180 -1

Trigonometric functions of angles

cos( )x L θ= a

sin( )y L θ= aa

x

y

( , )P x y

θ

cos( )xL

θ =a

sin( )yL

θ =a

1

21 2(1 ) ( 1)

1 1

1 1 2 2

[ , ,..., ]

= + +

nn n

n

n n

yy

x x x

yx y x y x y

× ×

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥′ = ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦+

x y

Inner product between two vectors x and y

Inner product between two vectors x and y

1

21 2 1 1[ , ,..., ] gives n

n

y

yx x x x y

y

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ →⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

1

21 2 2 2[ , ,..., ] n

n

y

yx x x x y

y

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ →⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

1

2

1 2[ , ,..., ] n n n

n

yy

x x x x y

y

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ →⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

Step 1:

Step 2: Step n:

Outer product between two vectors x and y

1

21 2( 1) (1 )

1 1 1 2 1

2 1 2 2 2

1 2

[ , ,..., ]

=

nn n

n nn

n

n

n n n n

x

x y y y

x

x y x y x y

x y x y x y

x y x y x y

× ×

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥′ = ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

x y

θ

Trigonometric functions of angles

1θ 2θ

1x

2x2y

1y

x y

11cos( )

xL

θ =x

= 12cos( )

yL

θy

= 21sin( )

xL

θx

22sin( )

yL

θ =y

x-component

y-component

For x:

For y:x-component

y-component

Geometric Interpretation of inner product= − = +

⎛ ⎞ ⎛ ⎞⎛ ⎞ ⎛ ⎞⎟ ⎟⎜ ⎜⎟ ⎟⎜ ⎜= +⎟ ⎟⎜ ⎜⎟ ⎟⎜ ⎜⎟ ⎟⎟ ⎟⎜ ⎜⎟ ⎟⎜ ⎜⎝ ⎠ ⎝ ⎠⎝ ⎠ ⎝ ⎠+

=

1 2 2 1 2 1

1 1 2 2

1 1 2 2

cos( ) cos( ) cos( )cos( ) sin( )sin( )

y x y xL L L Lx y x yL L

θ θ θ θ θ θ θ

y x y x

x y

1 1 2 2x y x y′ = +x y

1 1 2 2L x x x x′= = +x x x

cos( )L L

θ′ ′

= =′ ′x y

x y x yx x y y

′ = cos( )L L θx yx y

Correlation as angle between deviation vectors

1

2

j j

jj

n j

x xx x

x x

⎡ ⎤−⎢ ⎥⎢ ⎥−⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥−⎢ ⎥⎣ ⎦

d

=

= =

− −′ ′= = = =

′ ′ − −

∑

∑ ∑1

2 2

1 1

( )( )cos( )

( ) ( )j k

n

ij j ik kj k j k jki

n njj kk

j j k k ij j ik ki i

x x x xs

L L s sx x x x

θd d

d d d d

d d d d

1

1 =1,2,...,n

j iji

x x j pn =

= ∑

== − −∑

1

1( )( )

n

jk ij j ik ki

s x x x xn

== = −∑2 2

1

1( )

n

j jj ij ji

s s x xn

== −∑ 2

1

1( )

n

j ij ji

s x xn

Correlation as angle between deviation vectors

41

3

⎡ ⎤⎢ ⎥⎢ ⎥= −⎢ ⎥⎢ ⎥⎣ ⎦

x13 5

⎡ ⎤⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦

y

4 2 21 2 3

3 2 1

⎡ ⎤ ⎡ ⎤−⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= − − = −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦

xd1 3 23 3 05 3 2

⎡ ⎤ ⎡ ⎤− −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= − =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦

yd

[ ]2

2 3 1 3 141

⎡ ⎤⎢ ⎥

′ ⎢ ⎥= − − =⎢ ⎥⎢ ⎥⎣ ⎦

x xd d [ ]2

2 0 2 0 82

⎡ ⎤−⎢ ⎥

′ ⎢ ⎥= − =⎢ ⎥⎢ ⎥⎣ ⎦

y yd d

Correlation as angle between deviation vectors

[ ]2

2 3 1 0 22

⎡ ⎤−⎢ ⎥

′ ⎢ ⎥= − =−⎢ ⎥⎢ ⎥⎣ ⎦

x yd d

′ −= = = = −

′ ′

2cos( ) 0.189

14 8xyr θ x y

x x y y

d d

d d d d

−

= = = −2 2

23 0.189

14 83 3

xyxy

x y

sr

s s

Matrix definition

11 12 13 1 1

21 22 23 2 2

( ) 1 2 3

1 2 3

j p

j p

n p i i i ij ip

n n n nj np

x x x x x

x x x x x

x x x x x

x x x x x

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

X

Matrix Operations: Transpose11 12 13 1 1

21 22 23 2 2

( ) 1 2 3

1 2 3

j p

j p

n p i i i ij ip

n n n nj np

x x x x x

x x x x x

x x x x x

x x x x x

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

X

11 12 13 1 1

21 22 23 2 2

( ) ( ) 1 2 3

1 2 3

i n

i n

T

p n p n j j j ji jn

p p p pj pn

x x x x x

x x x x x

x x x x x

x x x x x

× ×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥′= = ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

X X

Transpose of a Matrix

(5 4)

1 123 33 3.5

1 99 30 2.7

1 102 45 1.4

0 212 39 4.6

0 187 57 2.8

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

X

(4 5)

1 1 1 0 0

123 99 102 212 187

33 30 45 39 57

3.5 2.7 1.4 4.6 2.8

T

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

X

Matrix Addition

(2 3)

4 1 7

3 2 6×

⎡ ⎤⎢ ⎥=⎢ ⎥⎣ ⎦

X

(2 3)

2 3 6

5 4 1×

⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦

Y

× ×

+ + +⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥+ = =⎢ ⎥ ⎢ ⎥+ + +⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦(2 3) (2 3)

4 2 1 3 7 6 6 4 13

3 5 2 4 6 1 8 6 7X Y

Matrix-vector Multiplication

1

211 12 1

321 22 2

( ) ( 1)

1 2( )

( 1)

( 1)

n

nT

p n ni

p p pnp n

nn

p

y

yx x x

yx x x

yx x x

y

× ×

×

×

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

X y

Matrix-vector Multiplication

(2 3) (3 1)

32 1 4

25 3 7

5

T

× ×

⎡ ⎤⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎣ ⎦

X y

2 3 3 1

32 1 4

2 2(3) 1(2) 4(5) 6 2 20 285 3 7

5

28

T

× ×

⎡ ⎤⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + = + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥= ⎢ ⎥⎣ ⎦

X y

Step 1:

Matrix-vector Multiplication

Step 2:

(2 3) (3 1)

32 1 42 5(3) 3(2) 7(5) 15 6 35 56

5 3 75

28

56

T

× ×

⎡ ⎤⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + = + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦

X y

Matrix Multiplication

11 12 1 11 12 1

21 22 2 21 22 2

( ) ( )

1 2 1 2( ) ( )

( )

n p

n pT

p n n p

p p pn n n npp n n p

p p

x x x x x x

x x x x x x

x x x x x x

× ×

× ×

×

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

X X

Matrix Multiplication

Step 1:

(2 4) (4 2)

2 6

1 42 1 3 42(2) 1(1) 3(3) 4(4) 4 1 9 16 30

3 26 4 2 7

4 7

30

T

× ×

⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥= ⎢ ⎥⎣ ⎦

X X

Matrix MultiplicationStep 2:

Step 3:

(2 4) (4 2)

2 6

2 1 3 4 1 46(2) 4(1) 2(3) 7(4) 12 4 6 28 50

6 4 2 7 3 2

4 7

30

50

T

× ×

⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦

X X

(2 4) (4 2)

2 6

1 42 1 3 42(6) 1(4) 3(2) 4(7) 12 4 6 28 50

3 26 4 2 7

4 7

30 50

50

T

× ×

⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦

X X

Matrix Multiplication

Step 4:

(2 4) (4 2)

2 6

2 1 3 4 1 46(6) 4(4) 2(2) 7(7) 36 16 4 49 105

6 4 2 7 3 2

4 7

30 50

50 105

T

× ×

⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦

X X

Matrix Multiplication

Step 1:

(2 4) (4 2)

2 6

1 42 1 3 42(2) 1(1) 3(3) 4(4) 4 1 9 16 30

3 26 4 2 7

4 7

30

T

× ×

⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥= ⎢ ⎥⎣ ⎦

X X

Mean vector

1

11 12 11

2 21 22 22

1 2

1

11 11

1

n

nT

p p pnpp

n x x xxx x xx

nn nx x xx

n

′⎡ ⎤⎢ ⎥⎢ ⎥ ⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥′⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= = = =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦⎣ ⎦ ⎢ ⎥ ⎣ ⎦′⎢ ⎥⎢ ⎥⎣ ⎦

x 1

x 1x X 1

x 1

Mean matrix

1 2

1 2

( 1) (1 )

1 2( )

1

p

p

n p

pn p

x x x

x x x

x x x

× ×

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥′ = ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

x

Deviation matrix

11 1 12 2 1

21 1 22 2 2

( ) ( 1) (1 )

1 1 2 2( )

1

p p

p p

n p n p

n n np pn p

x x x x x x

x x x x x x

x x x x x x

× × ×

×

− − −⎡ ⎤⎢ ⎥

− − −⎢ ⎥⎢ ⎥′− = ⎢ ⎥⎢ ⎥⎢ ⎥− − −⎢ ⎥⎣ ⎦

X x

Variance-Covariance Matrix

( ) ( )11 1 21 1 1 1 11 1 12 2 1

12 2 22 2 2 2 21 1 22 2 2

1 2 1 1 2 2( ) ( )

( )

11 1

n p p

n p pT

p p p p np p n n np pp n n p

p p

x x x x x x x x x x x x

x x x x x x x x x x x x

n

x x x x x x x x x x x x× ×

×

− − − − − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥

− − − − − −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥′ ′= − − = ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− − − − − −⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

S X x X x

11 12 1

21 22 2

1 2

p

p

p p

p p pp

s s s

s s s

s s s

×

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

S

Diagonal matrices: sample standard deviation matrix (inverse)

11 1

22 21/2

0 0 0 0

0 0 0 0

0 0 0 0

p p

pp p

s s

s s

s s

×

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

D

11 1

1/2 22 2

1 10 0 0 0

1 10 0 0 0

1 10 0 0 0

p p

pp p

s s

s s

s s

−×

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

D

Obtaining correlation matrix by sandwiching covariance matrix between inverted standard deviation matrices

11 1111 12 1

21 22 21/2 1/2 22 22

1 2

111 12

11 11 11 22 11

1 10 0 0 0

1 10 0 0 0

1 10 0 0 0

p

p

p p pp

pp pp

p

p

s ss s s

s s ss s

s s s

s sss s

s s s s s s

− −

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥= = ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

=

R D SD

12 1221 22

21 211 22 22 22 22

1 21 2

11 22

1

1

1

p pp

ppp

p pp p pp

pp pp pp pp

r rss s

r rs s s s s s

r rs s ss s s s s s

⎡ ⎤⎢ ⎥⎢ ⎥ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ = ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎣ ⎦⎢ ⎥⎢ ⎥⎣ ⎦

Identity matrices1 0 0

0 1 0

0 0 1

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

I

× × × × ×= =

( ) ( ) ( ) ( ) ( )n p n p n p n p n pI X X I X

1 0 3 1 2 3 1 2

0 1 6 7 4 6 7 4

⎡ ⎤ − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦⎢ ⎥⎣ ⎦

Trace of a matrix

11 221

tr( )n

nn iii

x x x x=

= + + + = ∑X

2 9 7

tr 5 -1 3 2 1 7 8

3 4 7

⎡ ⎤⎢ ⎥⎢ ⎥ = − + =⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

tr( ) tr( )T =X X

1 5 6 1 4 -2

tr 4 6 7 tr 5 6 4 1 6 9 16

-2 4 9 6 7 9

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= = + + =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

Multiple Linear Regression

+y = Xb e

01 11 12 1 1

2 21 22 2 21

1 2( 1) ( ) ( 1)( 1)

( 1)

p

p

n n n np npn n p np

n

y x x x

y x x x

y x x x

β εεβ

εβ× × ××

×

⎡ ⎤⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥= +⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦⎢ ⎥⎣ ⎦

Multiple Linear Regression

1( )T T−=b X X X y

( )−

× × × × ×× ×

× ×

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢= = ⎢ ⎥ ⎢⎢ ⎥ ⎢⎢ ⎥ ⎢⎢ ⎥ ⎢⎣ ⎦ ⎣ ⎦

11 12 1 11 12 1

1 21 22 2 21 22 2

( 1) ( )( ) ( )( 1)( ) ( 1) 1 2 1 2

( 1) ( )

n p

n pT T

p p n n p p n np p p p p pn n n np

p p n

x x x x x x

x x x x x x

x x x x x x

b X X X y

× × ×

× ×

×

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

11 12 1 1

21 22 2 2

1 2( ) ( ) ( 1)

( ) ( 1)

( 1)

n

n

p p pn nn p p n n

p p p

p

x x x y

x x x y

x x x y

References

• Gentle, J.E. Numerical Linear Algebra for Applications in Statistics. New York(NY), Springer-Verlag, 1998.

• Morrison, D.F. Multivariate Statistical Methods. New York(NY), McGraw-Hill, 1990.