Notes on Matrix De�niteness

De�nition 1:An n� n matrix A is positive semi-de�nite (p.s.d.) if for any non-zero n� 1 column vector t; we have t0At � 0

De�nition 21 :An n� n matrix A is positive de�nite (p.d.) if for any non-zero n� 1 column vector t; we have t0At > 0

Result 1:For any n� k matrix X; X 0X is positive semi-de�nite.(Proof)Assume that

t =

26664t1t2...tk

37775 6= 0k�1Then,

t0X 0Xt = (Xt)0Xt

= s0s

=nPi=1

s2i � 0;

where I de�ned that

s|{z}n�1

�

26664s1s2...sn

37775 � X|{z}n�k

t|{z}k�1

:

Result 2:If A is a positive semi-de�nite k � k matrix, then A0 is positive semi-de�nite.(Proof)Assume that

t =

26664t1t2...tk

37775 6= 0k�1Since A is positive semi-de�nite, we have

0 � t0At = (t0A0t)0 :

However, since (t0A0t)0 is scalar, i.e. (t0A0t)0 = (t0A0t) : Then, we have

0 � (t0A0t)

and A0 is positive semi-de�nite.

1 In econometrics, p.s.d. matrices appear more frequently than p.d. matrices. So, I mainly discuss p.s.d. matrices on this note..

1

Result 3:Any k � k variance-covariance matrix � of column random vector X is positive semi-de�nite.(Proof)Since � is the variance of column random vector X; by the de�nition of variance,

V arX [X] = EX�(X � EX [X]) (X � EX [X])0

�:

I de�ned �X � EX [X] ; for simplicity of notationAlso, assume that

t =

26664t1t2...tk

37775 6= 0k�1:Then,

t0�t = t0V arX [X] t

= t0EX�(X � �X) (X � �X)

0�t

= EX�t0 (X � �X) (X � �X)

0t�

= EX

h�(X � �X)

0t0 �

(X � �X)0ti

= EX�s2�� 0;

where I de�ned that

s|{z}1�1

� (X � �X)0| {z }

1�k

t|{z}k�1

:

Result 4:Any n� n idempotent and symmetric matrices are positive semi-de�nite.(Proof)Assume that

t =

26664t1t2...tn

37775 6= 0n�1:Also, assume that X is arbitrary n� n idempotent and symmetric matrix. Then,

t0Xt = t0XXt (Since X is idempotent)

= t0X 0Xt (Since X is symmetric)

= (Xt)0Xt

= s0s =nPi=1

s2i � 0;

where I de�ned that

s|{z}n�1

�

26664s1s2...sn

37775 � X|{z}n�n

t|{z}n�1

:

As special cases of this result, the identity matrix In and the projection matrix PX = X (X 0X)X 0 and the annihilatormatrix MX = In � PX = In �X (X 0X)X 0 in the linear regression model are positive semi-de�nite.

Result 5:

2

If A is a k � k positive semi-de�nite and non-singular matrix. Then A�1 is positive semi-de�nite.(Proof)Assume that

t =

26664t1t2...tk

37775 6= 0k�1:Since A is k � k positive semi-de�nite, we know that A0 is also positive semi-de�nite by Result 2. Then,

0 � s0A0s = s0A0A�1A| {z }created

s

= (As)0A�1 (AS) :

Now, de�ne that

t � As

t|{z}k�1

� A|{z}k�k

s|{z}k�1

:

Then, we have

0 � t0A�1t:

Result 6:If A is k � k positive semi-de�nite matrix and B is any k � n matrix. Then B0AB is positive semi-de�nite.(Proof)Assume that

s =

26664s1s2...sn

37775 6= 0n�1Then, we have

s0B0ABs = (Bs)0ABs

= t0At � 0, (Since A is positive semi-de�nite)

where I de�ne that

t � As

t|{z}k�1

� B|{z}k�n

s|{z}n�1

:

Result 7:If A and B are non-singular, k�k, positive semi-de�nite, and symmetric matrices. Then A�B is positive semi-de�nite

if and only if B�1 �A�1 is positive semi-de�nite. (Equivalently, A�B is positive semi-de�nite if and only if A�1 �B�1is negative semi-de�nite.)(Proof)Incomplete.

3