Probability theory 2008 Conditional probability mass function Discrete case Continuous case

Probability theory 2008

Conditional probability mass function

Discrete case

Continuous case

)(

),(

)(

),(|)|( ),(

| xf

yxf

xXP

xXyYPxXyYPxyf

X

YXXY

)(

),()|( ),(

| xf

yxfxyf

X

YXXY


Conditional probability mass function- examples

Throwing two dice Let Z1 = the number on the first die

Let Z2 = the number on the second die

Set Y = Z1 and X = Z1+Z2

Radioactive decay Let X = the number of atoms decaying within 1 unit of time Let Y = the time of the first decay

?)5|(| yf XY

?)1|(| yf XY


Conditional expectation

Discrete case

Continuous case

Notation

y

XYy

xyfyxXyYPyxXYE )|(|)|( |

dyxyfyxXYE XY )|()|( |

)()|( xhxXYE )()|( XhXYE


Conditional expectation - rules

...)|( 21 xXYYE

cxXcE )|(

...)|( xXcYE

...)|),(( xXYXgE

...if)()|( YExXYE


Calculation of expected valuesthrough conditioning

Discrete case

Continuous case

General formula

x

Xx

xXYExfxXYExXPYE )|()()|()()(

dxxXYExfYE X )|()()(

)())|(( YEXYEE


Calculation of expected values through conditioning- example

Primary and secondary events

Let N denote the number of primary events Let X1, X2, … denote the number of secondary events for each primary

event Set Y = X1 + X2 + … + XN

Assume that X1, X2, … are i.i.d. and independent of N

?)( YE


Calculation of variances through conditioning

))|(())|(()( XYEVarXYVarEYVar

Variation in theexpected value of Y

induced byvariation in X

Average remainingvariation in Y

after X has been fixed


Variance decomposition in linear regression

0

1

2

3

4

5

6

7

8

0 1 2 3 4 5

x

y

y fitted y-value

j

jj

jjj

j yyyyyy 222 )ˆ()ˆ()(


Proof of the variance decomposition

We shall prove that

It can easily be seen that

))]|(([)())]|([))|(())|(( 2222 XYEEYEXYEXYEEXYVarE

))|(())|(()( XYEVarXYVarEYVar

2222 )]([))]|(([)]|(([))]|(([))|(( YEXYEEXYEEXYEEXYEVar


Regression and prediction

Regression function:

Theorem: The regression function is the best predictor of Y based on X

Proof:

)|()...,,|()...,,( 111 xX YExXxXYExxh nnn

22

22

))()|(())}()|())(|({(2))|((

))()|()|(())((

XdXYEEXdXYEXYEYEXYEYE

XdXYEXYEYEXdYE

Function of XFunction of X


Best linear predictor

Theorem: The best linear predictor of Y based on X is

Proof: …….

)()( xx

yy XXL

Ordinary linear regression


Expected quadratic prediction errorof the best linear predictor

Theorem:

Proof: …….

)1())(( 222 yXLYE

Ordinary linear regression


Martingales

The sequence X1, X2,… is called a martingale if

Example 1: Partial sums of independent variables with mean zero

Example 2: Gambler’s fortune if he doubles the stake as long as he loses and leaves as soon as he wins

1 allfor )...,,|( 11 nXXXXE nnn


Exercises: Chapter II

2.6, 2.9, 2.12, 2.16, 2.22, 2.26, 2.28

Use conditional distributions/probabilities to explain why the envelop-rejection method works


Transforms

YX TT and

YXT

YX ff and

YXf


The probability generating function

Let X be an integer-valued nonnegative random variable. The probability generating function of X is

Defined at least for | t | < 1 Determines the probability function of X uniquely Adding independent variables corresponds to multiplying their generating functions

Example 1: X Be(p)

Example 2: X Bin(n;p)

Example 3: X Po(λ)

Addition theorems for binomial and Poisson distributions

0

)()()(n

nXX nXPttEtg


The moment generating function

Let X be a random variable. The moment generating function of X is

provided that this expectation is finite for | t | < h, where h > 0

Determines the probability function of X uniquely Adding independent variables corresponds to multiplying their moment

generating functions

)()( tXX eEt


The moment generating functionand the Laplace transform

Let X be a non-negative random variable. Then

)()()()()(0

)(

0

tLdxxfedxxfeeEt XXxt

XtxtX

X


The moment generating function- examples

The moment generating function of X is

Example 1: X Be(p)

Example 2: X Exp(a)

Example 3: X (2;a)

)()( tXX eEt


The moment generating function- calculation of moments

)(!

...)()()(0

k

k

k

XtxtX

X XEk

tdxxfeeEt

)0(!

...)()( )(

0

kX

k

ktX

X k

teEt


The moment generating function- uniqueness

...,2,1,0,)()()()( kdxxfxdxxfxtt Yk

Xk

YX

)()()( where...,2,1,0,0)( xfxfxhkdxxhx YXk


Normal approximation of a binomial distribution

Let X1, X2, …. be independent and Be(p) and let

Then

.

n

npXXY n

n

...1

n

nntntp

nntntpY

non

ppt

epe

pepetn

)/1(2

)1(1

))1(1(

)1()(

2

//

/


Distributions for which the moment generating function does not exist

Let X = eY, where YN( ;)

Then

and

.

)2/exp()()()( 22rrreEXE YrYr

)2/logexp(!

1)2/exp(

!)(

!

!

)())(exp()(

2222 nntnn

nnn

tn

n

t

n

teEteEeE

n

Y

n

nYYtX


The characteristic function

Let X be a random variable. The characteristic function of X is

Exists for all random variables Determines the probability function of X uniquely Adding independent variables corresponds to multiplying their

characteristic functions

)(sin)(cos)()( tXiEtXEeEt itXX


Comparison of the characteristic function and the moment generating function

Example 1: Exp(λ)

Example 2: Po(λ)

Example 3: N( ; )

Is it always true that

.

)()( itt XX


The characteristic function- uniqueness

For discrete distributions we have

For continuous distributions with

we have

.

dttX |)(|

)()(2

1xfdtte XX

itx

)()(2

1xXPdtte

T X

T

T

itx


The characteristic function- calculation of moments

If the k:th moment exists we have

.

)()0()( kkkX XEi


Using a normal distribution to approximate a Poisson distribution

Let XPo(m) and set

Then

.

Xm

mm

mXY

1

...)( tY


Using a Poisson distribution to approximate a Binomial distribution

Let XBin(n ; p)

Then

If p = 1/n we get.

nitX pept )1()(

))exp(1()( itetX


Sums of a stochastic number of stochastic variables

Probability generating function:

Moment generating function:

Characteristic function:

NN XXS ...1


Branching processes

Suppose that each individual produces j new offspring with probability pj, j ≥ 0, independently of the number produced by any other individual.

Let Xn denote the size of the nth generation

Then

where Zi represents the number of offspring of the ith individual of the (n - 1)st generation.

1

1

nX

iin ZX

generation


Generating function of a branching processes

Let Xn denote the number of individuals in the n:th generation of a population, and assume that

where Yk, k = 1, 2, … are i.i.d. and independent of Xn

Then

Example:

nX

kkn YX

X

11

0 1

))((...)(1

tggtg YXX nn

tp

pppttg

k

kkY )1(1

)1()(0


Branching processes- mean and variance of generation size

Consider a branching process for which X0 = 1, and and respectively depict the expectation and standard deviation of the offspring distribution.

Then

.

nn

nnnn

XE

ZEXEXXEEXE

)(

)()(...)]|([)( 11

1if,

1if,1

1)(

)]|([)]|([)(

2

12

11

n

XVar

XXEVarXXVarEXVarn

n

n

nnnnn


Branching processes- extinction probability

Let 0 = P(population dies out) and assume that X0 = 1

Then

where g is the probability generating function of the offspring distribution

jj

jj

j

ppjXP

0

010

0 )|out dies population(

)1('

)( 00

g

g


Exercises: Chapter III

3.1, 3.2, 3.3, 3.7, 3.15, 3.25, 3.26, 3.27, 3.32

Documents

Probability theory 2008 Conditional probability mass function Discrete case Continuous case