27
1 3. Random Variables Let (, F, P) be a probability model for an experiment, and X a function that maps every to a unique point the set of real numbers. Since the outcome is not certain, so is the value Thus if B is some subset of R, we may want to determine the probability of ”. To determine this probability, we can look at the set that contains all that maps into B under the function X. , ξ , R x ξ . ) ( x X = ξ B X ) (ξ = ) ( 1 B X A ξ ξ R ) (ξ X x A B Fig. 3.1 PILLAI

lecture slides for probability and statistics

Embed Size (px)

DESCRIPTION

book is probability and statistics by papoulis

Citation preview

Page 1: lecture slides for probability and statistics

1

3. Random Variables

Let (Ω, F, P) be a probability model for an experiment, and X a function that maps every to a unique point the set of real numbers. Since the outcome is not certain, so is the value Thus if B is some subset of R, we may want to determine the probability of “ ”. To determine this probability, we can look at the set that contains all that maps into B under the function X.

,Ω∈ξ,Rx∈ ξ

.)( xX =ξ

BX ∈)(ξ

Ω∈= − )(1 BXA Ω∈ξΩ

ξ

R)(ξX

x

A

B

Fig. 3.1 PILLAI

Page 2: lecture slides for probability and statistics

2

Obviously, if the set also belongs to the associated field F, then it is an event and the probability of A is well defined; in that case we can say

However, may not always belong to F for all B, thus creating difficulties. The notion of random variable (r.v) makes sure that the inverse mapping always results in an event so that we are able to determine the probability for any

Random Variable (r.v): A finite single valued function that maps the set of all experimental outcomes into the set of real numbers R is said to be a r.v, if the set is an event for every x in R.

)(1 BXA −=

)).((" )("event theofy Probabilit 1 BXPBX −=∈ξ (3-1)

)(1 BX −

.RB ∈

) ( ⋅XΩ

)(| xX ≤ξξ)( F∈

PILLAI

Page 3: lecture slides for probability and statistics

3

Alternatively X is said to be a r.v, if where Brepresents semi-definite intervals of the form and all other sets that can be constructed from these sets by performing the set operations of union, intersection and negation any number of times. The Borel collection B of such subsets of R is the smallest σ-field of subsets of R that includes all semi-infinite intervals of the above form. Thus if X is a r.v, then

is an event for every x. What about Are they also events ? In fact with since and are events, is an event and hence is also an event.

ax ≤<−∞

ab > ? , aXbXa =≤<

bX ≤

bXabXaX ≤<=≤∩> aXaX c >=≤

)(| xXxX ≤=≤ξξ

FBX ∈− )(1

aX ≤

(3-2)

PILLAI

Page 4: lecture slides for probability and statistics

4

Thus, is an event for every n. Consequently

is also an event. All events have well defined probability. Thus the probability of the event must depend on x. Denote

The role of the subscript X in (3-4) is only to identify the actual r.v. is said to the Probability Distribution Function (PDF) associated with the r.v X.

≤<− 1 aX

na

∩∞

=

==

≤<−

1

1 n

aXaXn

a

)(| xX ≤ξξ

.0)( )(| ≥=≤ xFxXP Xξξ (3-4)

)(xFX

(3-3)

PILLAI

Page 5: lecture slides for probability and statistics

5

Distribution Function: Note that a distribution function g(x) is nondecreasing, right-continuous and satisfies

i.e., if g(x) is a distribution function, then

(i)

(ii) if then

and

(iii) for all x.

We need to show that defined in (3-4) satisfies all properties in (3-6). In fact, for any r.v X,

,0)( ,1)( =−∞=+∞ gg

,0)( ,1)( =−∞=+∞ gg

,21 xx < ),()( 21 xgxg ≤

),()( xgxg =+

(3-6)

)(xFX

(3-5)

PILLAI

Page 6: lecture slides for probability and statistics

6

1)( )(| )( =Ω=+∞≤=+∞ PXPFX ξξ

.0)( )(| )( ==−∞≤=−∞ φξξ PXPFX

(i)

and

(ii) If then the subset Consequently the event since implies As a result

implying that the probability distribution function is nonnegative and monotone nondecreasing.(iii) Let and consider the event

since

,21 xx < ).,(),( 21 xx −∞⊂−∞

, )(| )(| 21 xXxX ≤⊂≤ ξξξξ

1)( xX ≤ξ .)( 2xX ≤ξ

( ) ( ) ),()()()( 2211 xFxXPxXPxF XX =≤≤≤= ξξ (3-9)∆ ∆

(3-7)

(3-8)

,121 xxxxx nn <<<<< −

.)(| kk xXxA ≤<= ξξ (3-10)

, )( )( )( kk xXxXxXx ≤=≤∪≤< ξξξ (3-11)PILLAI

Page 7: lecture slides for probability and statistics

7

using mutually exclusive property of events we get

But and hence

Thus

But the right limit of x, and hence

i.e., is right-continuous, justifying all properties of a distribution function.

( ) ).()()()( xFxFxXxPAP XkXkk −=≤<= ξ (3-12)

,11 −+ ⊂⊂ kkk AAA

.0)(lim hence and lim1

===∞→

=∞→ kk

kkkk

APAA φ∩ (3-13)

.0)()(lim)(lim =−=∞→∞→

xFxFAP XkXkkk

),()( xFxF XX =+

)(xFX

(3-14)

,lim +

∞→= xxkk

PILLAI

Page 8: lecture slides for probability and statistics

8

Additional Properties of a PDF

(iv) If for some then

This follows, since implies is the null set, and for any will be a subset of the null set.

(v)

We have and since the two events are mutually exclusive, (16) follows.

(vi)

The events and are mutually exclusive and their union represents the event

0)( 0 =xFX ,0x . ,0)( 0xxxFX ≤= (3-15)

( ) 0)()( 00 =≤= xXPxFX ξ 0)( xX ≤ξ

)( ,0 xXxx ≤≤ ξ

).(1 )( xFxXP X−=>ξ (3-16)

, )( )( Ω=>∪≤ xXxX ξξ

. ),()( )( 121221 xxxFxFxXxP XX >−=≤< ξ (3-17)

)( 21 xXx ≤< ξ )( 1xX ≤ξ

. )( 2xX ≤ξ

PILLAI

Page 9: lecture slides for probability and statistics

9

(vii)

Let and From (3-17)

or

According to (3-14), the limit of as from the right always exists and equals However the left limit value need not equal Thus need not be continuous from the left. At a discontinuity point of the distribution, the left and right limits are different, and from (3-20)

( ) ).()()( −−== xFxFxXP XXξ (3-18)

,0 ,1 >−= εεxx .2 xx =

),(lim)( )( lim00

εξεεε

−−=≤<−→→

xFxFxXxP XX (3-19)

).()( )( −−== xFxFxXP XXξ (3-20)

),( 0+xFX )(xFX 0xx →

).( 0xFX

)( 0−xFX ).( 0xFX )(xFX

.0)()( )( 000 >−== −xFxFxXP XXξ (3-21)PILLAI

Page 10: lecture slides for probability and statistics

10

Thus the only discontinuities of a distribution function are of the jump type, and occur at points where (3-21) is satisfied. These points can always be enumerated as a sequence, and moreover they are at most countable in number. Example 3.1: X is a r.v such that Find Solution: For so that and for so that (Fig.3.2)

Example 3.2: Toss a coin. Suppose the r.v X is such that Find

)( xFX

0x

. ,)( Ω∈= ξξ cX ).(xFX

, )( , φξ =≤< xXcx ,0)( =xFX

.,TH=Ω.1)( ,0)( == HXTX

)(xFX

xc

1

Fig. 3.2

).(xFX

.1)( =xFX ,)( , Ω=≤> xXcx ξ

PILLAI

Page 11: lecture slides for probability and statistics

11

Solution: For so that

•X is said to be a continuous-type r.v if its distribution function is continuous. In that case for all x, and from (3-21) we get

•If is constant except for a finite number of jump discontinuities(piece-wise constant; step-type), then X is said to be a discrete-type r.v. If is such a discontinuity point, then from (3-21)

, )( ,0 φξ =≤< xXx .0)( =xFX

3.3) (Fig. .1)( that so , , )( ,1

,1 )( that so , )( ,10=Ω==≤≥

−===≤<≤xFTHxXx

pTPxFTxXx

X

X

ξξ

).()( −−=== iXiXii xFxFxXPp (3-22)

)(xFX)()( xFxF XX =−

.0== xXP

)(xFX

)(xFX

x

Fig.3.3

1q

1

ix

PILLAI

Page 12: lecture slides for probability and statistics

12

From Fig.3.2, at a point of discontinuity we get

and from Fig.3.3,

Example:3.3 A fair coin is tossed twice, and let the r.v Xrepresent the number of heads. Find Solution: In this case and

.101)()( =−=−== −cFcFcXP XX

.0)0()0( 0 qqFFXP XX =−=−== −

, ,,, TTTHHTHH=Ω).(xFX

.0)(,1)(,1)(,2)( ==== TTXTHXHTXHHX

3.4) (Fig. .1)()( ,2

,43,, )(,, )( ,21

,41)()( )( )( ,10

,0)()( ,0

=⇒Ω=≤≥

==⇒=≤<≤

===⇒=≤<≤

=⇒=≤<

xFxXx

THHTTTPxFTHHTTTxXx

TPTPTTPxFTTxXx

xFxXx

X

X

X

X

ξ

ξ

ξ

φξ

PILLAI

Page 13: lecture slides for probability and statistics

13

From Fig.3.4,

Probability density function (p.d.f)

The derivative of the distribution function is called the probability density function of the r.v X. Thus

Since

from the monotone-nondecreasing nature of

.2/14/14/3)1()1(1 =−=−== −XX FFXP

)(xFX)( xf X

.

)()(dx

xdFxf XX =

,0)()(lim

)(0

≥∆

−∆+=

→∆ xxFxxF

dxxdF XX

x

X

(3-23)

(3-24)

),( xFX

)(xFX

x

Fig. 3.4

1

4/11

4/3

2

PILLAI

Page 14: lecture slides for probability and statistics

14

it follows that for all x. will be a continuous function, if X is a continuous type r.v. However, if X is a discrete type r.v as in (3-22), then its p.d.f has the general form (Fig. 3.5)

where represent the jump-discontinuity points in As Fig. 3.5 shows represents a collection of positive discrete masses, and it is known as the probability mass function (p.m.f ) in the discrete case. From (3-23), we also obtain by integration

Since (3-26) yields

0)( ≥xf X )( xf X

,)()( ∑ −=i

iiX xxpxf δ

).(xFX

(3-25)

.)()( duufxFx

xX ∫ ∞−= (3-26)

,1)( =+∞XF

,1)( =∫+∞

∞−dxxf x

(3-27)

ix

)(xfX

xix

ip

Fig. 3.5

)( xf X

PILLAI

Page 15: lecture slides for probability and statistics

15

which justifies its name as the density function. Further, from (3-26), we also get (Fig. 3.6b)

Thus the area under in the interval represents the probability in (3-28).

Often, r.vs are referred by their specific density functions -both in the continuous and discrete cases - and in what follows we shall list a number of them in each category.

.)()()( )( 2

11221 dxxfxFxFxXxP

x

x XXX ∫=−=≤< ξ (3-28)

Fig. 3.6

)( xf X ),( 21 xx

)(xfX

(b)

x1x 2x

)(xFX

x

1

(a)1x 2x

PILLAI

Page 16: lecture slides for probability and statistics

16

Continuous-type random variables

1. Normal (Gaussian): X is said to be normal or Gaussian r.v, if

This is a bell shaped curve, symmetric around the parameter and its distribution function is given by

where is often tabulated. Since depends on two parameters and the notation ∼will be used to represent (3-29).

.2

1)(22 2/)(

2

σµ

πσ−−= x

X exf (3-29)

,2

1)(22 2/)(

2∫ ∞−

−−

==x y

XxGdyexFσµ

πσσµ (3-30)

dyexG yx 2/2

21)( −

∞−∫=π

,( 2σµNX)(xfX

)(xfX

µ ,2σ )

xµFig. 3.7 PILLAI

Page 17: lecture slides for probability and statistics

17

2. Uniform: ∼ if (Fig. 3.8), ),,( babaUX <

≤≤

−=otherwise. 0,

, ,1)( bxa

abxf X(3.31)

3. Exponential: ∼ if (Fig. 3.9))( λεX

≥=

otherwise. 0,

,0 ,1)(

/ xexfx

X

λ

λ (3-32)

)(xfX

x

Fig. 3.9

)(xfX

xa b

ab −1

Fig. 3.8

PILLAI

Page 18: lecture slides for probability and statistics

18

4. Gamma: ∼ if (Fig. 3.10)

If an integer

5. Beta: ∼ if (Fig. 3.11)

where the Beta function is defined as

),( βαGX )0 ,0( >> βα

≥Γ=

−−

otherwise. 0,

,0 ,)()(

/1

xexxf

x

X

βα

α

βα (3-33)

n=α )!.1()( −=Γ nn

),( baX β )0 ,0( >> ba

<<−

=−−

otherwise. 0,

,10 ,)1(),(

1)(

11 xxxbaxf

ba

X β (3-34)

),( baβ

∫ −− −=1

0

11 .)1(),( duuuba baβ (3-35)

x

)( xf X

x

Fig. 3.1110

)( xf X

Fig. 3.10

PILLAI

Page 19: lecture slides for probability and statistics

19

6. Chi-Square: ∼ if (Fig. 3.12)

Note that is the same as Gamma

7. Rayleigh: ∼ if (Fig. 3.13)

8. Nakagami – m distribution:

),( 2 nX χ

)(2 nχ ).2 ,2/(n

≥=

otherwise. 0,

,0 ,)(22 2/

2 xexxf

x

X

σ

σ

(3-36)

(3-37)

,)( 2σRX

x

)( xf X

Fig. 3.12

)( xf X

xFig. 3.13

PILLAI

Γ=−−

otherwise. 0,

,0 ,)2/(2

1)(

2/12/2/ xex

nxfxn

nX

22 1 /2 , 0( ) ( )

0 otherwiseX

mm mxm x e x

f x m− − Ω ≥ = Γ Ω

(3-38)

Page 20: lecture slides for probability and statistics

20

9. Cauchy: ∼ if (Fig. 3.14)

10. Laplace: (Fig. 3.15)

11. Student’s t-distribution with n degrees of freedom (Fig 3.16)

( ) . ,1)2/(

2/)1()(2/)1(2

+∞<<∞−

+

Γ+Γ

=+−

tnt

nnntf

n

T π

,),( µαCX

. ,)(

/)( 22 +∞<<∞−−+

= xx

xf X µαπα

. ,21)( /|| +∞<<∞−= − xexf x

λ

)( xf X

x

Fig. 3.14µ

(3-41)

(3-40)

(3-39)

x

)( xf X

Fig. 3.15t

( )Tf t

Fig. 3.16PILLAI

Page 21: lecture slides for probability and statistics

21

12. Fisher’s F-distribution/ 2 / 2 / 2 1

( ) / 2

( ) / 2 , 0( ) ( / 2) ( / 2) ( )

0 otherwise

m n m

m nz

m n m n z zf z m n n mz

+

Γ +≥= Γ Γ +

(3-42)

PILLAI

Page 22: lecture slides for probability and statistics

22

Discrete-type random variables

1. Bernoulli: X takes the values (0,1), and

2. Binomial: ∼ if (Fig. 3.17)

3. Poisson: ∼ if (Fig. 3.18)

.)1( ,)0( pXPqXP ==== (3-43)

),,( pnBX

.,,2,1,0 ,)( nkqpkn

kXP knk =

== − (3-44)

, )( λPX

.,,2,1,0 ,!

)( ∞=== − kk

ekXPkλλ (3-45)

k

)( kXP =

Fig. 3.171 2 n

)( kXP =

Fig. 3.18 PILLAI

Page 23: lecture slides for probability and statistics

23

4. Hypergeometric:

5. Geometric: ∼ if

6. Negative Binomial: ~ if

7. Discrete-Uniform:

We conclude this lecture with a general distribution duePILLAI

(3-49)

(3-48)

(3-47)

.,,2,1 ,1)( NkN

kXP ===

),,( prNBX1

( ) , , 1, .1

r k rkP X k p q k r r

r−−

= = = + −

.1 ,,,2,1,0 ,)( pqkpqkXP k −=∞===

)( pgX

, max(0, ) min( , )( )

m N mk n k

Nn

m n N k m nP X k

−−

+ − ≤ ≤= = (3-46)

Page 24: lecture slides for probability and statistics

24

PILLAI

to Polya that includes both binomial and hypergeometric asspecial cases.

Polya’s distribution: A box contains a white balls and b blackballs. A ball is drawn at random, and it is replaced along withc balls of the same color. If X represents the number of whiteballs drawn in n such draws, find the probability mass function of X. Solution: Consider the specific sequence of draws where kwhite balls are first drawn, followed by n – k black balls. The probability of drawing k successive white balls is given by

Similarly the probability of drawing k white balls

0, 1, 2, , ,X n=

2 ( 1)

2 ( 1)W

a a c a c a k cpa b a b c a b c a b k c

+ + + −=

+ + + + + + + − (3-50)

Page 25: lecture slides for probability and statistics

25

PILLAI

followed by n – k black balls is given by

Interestingly, pk in (3-51) also represents the probability of drawing k white balls and (n – k) black balls in any otherspecific order (i.e., The same set of numerator and denominator terms in (3-51) contribute to all other sequences as well.) But there are such distinct mutually exclusivesequences and summing over all of them, we obtain the Polyadistribution (probability of getting k white balls in n draws)to be

nk

1 1

0 0( )

( 1) ( 1) ( 1)

.

k w

k n k

i j

b jca ica b ic a b j k c

b b c b n k cp pa b kc a b k c a b n c

− − −

= =

+++ + + + +

+ + − −=

+ + + + + + + −

=∏ ∏ (3-51)

1 1

0 0( )( ) , 0,1,2, , .

k n k

ki j

n nk k

b jca ica b ic a b j k cP X k p k n

− − − = =

+++ + + + += = = =∏ ∏

(3-52)

Page 26: lecture slides for probability and statistics

26

PILLAI

Both binomial distribution as well as the hypergeometric distribution are special cases of (3-52).

For example if draws are done with replacement, then c = 0 and (3-52) simplifies to the binomial distribution

where

Similarly if the draws are conducted without replacement, Then c = – 1 in (3-52), and it gives

( ) , 0,1,2, ,k n knk

P X k p q k n −

= = = (3-53)

, 1 .a bp q pa b a b

= = = −+ +

( 1)( 2) ( 1) ( 1) ( 1)( )

( )( 1) ( 1) ( ) ( 1)nk

a a a a k b b b n kP X ka b a b a b k a b k a b n

− − − + − − + += =

+ + − + − + + − + − +

Page 27: lecture slides for probability and statistics

27

which represents the hypergeometric distribution. Finally c = +1 gives (replacements are doubled)

we shall refer to (3-55) as Polya’s +1 distribution. the general Polya distribution in (3-52) has been used to study the spreadof contagious diseases (epidemic modeling).

1 1

1

( 1)! ( 1)! ( 1)! ( 1)!( )( 1)! ( 1)! ( 1)! ( 1)!

= .

nk

a k b n kk n k

a b nn

a k a b b n k a b kP X ka a b k b a b n

+ − + − −−

+ + −

+ − + + + − − + + −= =

− + + − − + + −

(3-55)

! !( )! !( )!( )!( )! ( )!( )! ( )!( )!

a bk n k

a bn

n a a b k b a b nP X kk n k a k a b b n k a b k

−+

+ − + −= = =

− − + − + + −

(3-54)

PILLAI