Lecture 2 4 , § 4 . 9

Suppose you have a population distribution where 6 0 % of t h e population i s i n the

city a n d 4 0% i n the suburb. Th i s c a n be represented by t h e vector

i.: "key feature: The s u m of entries of above vector = 1 .

More generally ,

Probability : A vector i n 112" whose entries a r e non-negative and a d d-

up to 1 i s called a probability vector.-

Eg: (§.}})

i s a probability vector but not {§→ Negative entry

S t o c h a s l i r i x : Th is i s a square ma t r i x whose columns a r e probability vectors.

t o : : ? )Suppose

you have a n n x n stochastic ma t r i x P a n d a probality vector I 'of size n . Then PT ' i s a lso a probability vector of size n .

Eg: A--poi::?) 1%1=0.18%1+09/8:) =/!!:) and

O ' 33 t 0 . 6 7 = 1 to

M a r t a i n : A Maka i chain i s a n infinite collection of probability vectors-

X o , I i , I I , I I , . . . of size n a n d a n x n stochastic matrix PSuch that

P IO = I , , P I , =x I , . . . , P I ; = Fit,,...

This i s a n example of a r e c u r s i v e definition w e d i d a while back!

E-g. duppose a city loses 5 % of its population t o t h e suburbsand t h e suburbs lose 1 5 % of its population t o t h e city every

year.Suppose t h e current population of city i s 1 5 0 o o o and

that of the suburb i s 5 0 0 0 0 .

We get a Markov chain from th i s setup a s follows.

Population of city after year 1 : 0 . 9 5 (150000) t 0.15/50000)11 I , 1 1" suburb 2 : 0 . 0 5 (150ooo) t 0.85 (50000).

write i n matrix:(8¥ 8¥11'::::)After year 2 , population will be f:?÷ %.FI/::iE.:II('s::::)':"!...

T h e matrix (g. I f

8¥] i s stochastic, but (151%0) i s No t a probability

vector. B u t c a n force i t t o be , by scaling by 1 -, i e . ,

population of city+ suburb

set ÷ . f i : : : : -1=1:':L.

Then population distribution after n years will be obtained v i a t h e

Mar ron expressionqq.gs, oo.!:) (i:)

but scaled by 200000 , i e . , t h e actua l population will b e

s o o o o o (I:::::'t" 1%1).The advantage here i s that t h e numbers become m u c h smaller when youconvert (soooooo;

]t o (3,4;]. so it's less t ime and expense for a

computer t o do t h e computations!

Steady S t a t e s : I f P i s a stochastic mat r i x (all columns a r e-

probability vectors) , then a steady state vector forP i s a probability vector I (oo. I t o b e i ts entries add u pt o 1 )Such t h a t

P I = I.

I n other words, I i s a steady state vector if i t i s a n eigenvector of Pwith eigenvalue 1 a n d i s also a probability vector.

Eg: Let's try t o find a steady sta te vector for 1%5, oj.gs].

Goal : F i n d a probability vector i n E , .

I:::::'÷) - ' I .f i?::::.. I*...f:osoo'I§ Linear system

- 0 . 0 5 X , t 0 . 1 5 × 2 = 0 .

⇒ x , = O I x , = 3×2 .

0 . 0 5

% General solution = [3×-2]

. Can w e find × , s o that (3×1)X u

i s a probability vector?

Well w e know 3×2 t x , = L ⇒ 4×2 = I ⇒ × , , I, .

% (3,41,]

i s a steady state rector!

Upshot: A stochastic matrix always has a steady sta te vector!I n other words, o n e c a n always find a probability vector

i n E , .

Also if P i s a stochastic mat r i x and Tio i s a steady sta te vector,then t h e chain

XI , P i o ,patio, 133×1,...

will a l l have v a l u e t o !