Documents3

1. Fundamentals

1.1 Probability Theory

2.1 Theoretical Distributions

2.1.1 Discrete Distributions

The Binomial Distribution

The Poisson Distribution

2.1.2 Continuous Distributions

The Normal Distribution

Gamma Distribution

Gumbel Distribution

The x2 Distribution

The t Distribution

The F distribution

2.1.3 Multivariate Distributions


Bernoulli Distribution

A Bernoulli experiment is one in which there are just two outcomes of interests event A occurs or does not occur.

The indicator function of the event A is called a Bernoulli random variable:

1 ,0for ,)(or 1)()0( ,)()1(

occurs if 0,occurs if ,1

1 =======

=

xqpxfpqAPfpAPf

AA

X

xx

c

c

Examples: - tossing a coin (heads or tails)

- testing a product (good or defective)

- precipitation (rain or non rain)

The mean and the variance are

pqppEXEXXVarpqpEX

====+=

222 )()(01


Binomial Distribution B(n,p)

Consider the independent and identically distributed random variables X1,,Xn, which are the results of n Bernoulli trials. The number of successes among n trials, which is the sum of the 0s and 1s resulting from the individual trails

is described by a Binomial distribution and has the probability

nn XXS ++= L1

successesk exactly are herein which t patternsdistict ofnumber the:

n trials among successesk ofpattern particular a ofy probabilit the:

) trialsin successes ()(

===

kn

qp

qpkn

nkPkSP

knk

knkn

The mean and variance are

npqXVarXVarSVarnpEXEXES

nn

n

=++==++=

)()()( 121

LL

10

,!)!(

!

k) choose(n t coefficien Binomial

=

=

n

kknn

kn


Approximate Binomial Probabilities

According to the Central Limit Theorem, the distribution of the sum Sn and hence the binomial distribution, is asymptotically normal for large n. More precisely, for fixed p

)()1,0(lim zFznpqnpSP Nn

n=

=

Example:

Consider a binomial distribution with n=8 and p=1/2. The binomial distribution is

and the standard normal distribution is

which approximates the binomial distribution well

knk

xk kxXP

= 21218

)(

=24)( xFxXP


Example: Binomial Distribution and the Freezing of Cayuga Lake

Given the years, during which the Cayuga Lake in central New York State was observed to have frozen in the last 200 years,

1796 1816 1856 1875 1884 1904 1912 1934 1961 1979

what is the probability for the lake freezing at least once during some decade in the future?

Model: the number of years of lake freezing in 10 years is a binomial distributed random variable

The model is appropriate, since one can assume

- Lake freezing is a Bernoulli experiment

- Whether it freezes in a given winter is independent of whether it froze in recent years

- The probability that the lake will freeze in a given winter is constant

Estimating the model parameter

05.0200/10 ==pPrediction given by the model

32.0)95.0)(05.0(!9!1!10)05.01(05.0

110

)1( 91101 ==

== nSP


Poisson Distribution P(t)The random variable

X=number of events in an interval of width t

is described by the density

!)(e

!)(e ) in time events ()(

-mt-

xm

xttxPxf

xx

===

=number of events per unit time,

t=m= number of events in t

A Poisson variable has the mean and variancetXVartEX == )( ,


Relation to Binomial Distribution

-The Poisson distribution arises when concerning rare events (e.g. wind speed larger than Vc)

-Under the assumption of a constant , the base interval t is divided into n equal length sub-intervals with n being large enough so that the likelihood of two exceedances in any one sub-interval is negligible

- The occurrence of an exceedance in any one sub-interval can be approximated as a Bernoulli trial with probability t/n of success- Assume that events in adjacent sub-intervals are independent. The number of exceedance X in interval t is binomially distributed with B(n,t/n)- )()/,( , tPntnBn


Example: Poisson Distribution and Annual Tornado Counts

Given the annual tornado counts in New York State between 1959-1988, what are the probabilities for zero tornado per year or for greater than nine tornados per year?

Model: the annual number of tornados is a Poisson distributed random variable

Estimating the model parameter tThe rate of tornado occurrence:

6.430/138 ==m

Prediction of the model:

+==>

===

>

L!10 6.4)()9(01.0

!06.4)1()0(

106.4

9

06.4

0

exfxP

ePf

x

Histogram of number of tornados reported annually in New York State for 1959-1988 (dashed), and fitted Poisson distribution with m=4.6 tornados per year (solid)


The Normal Distribution N (,2)

=

=

x

N

N

dttxF

xxf

2

2

2

2

2)(exp

21)(

2)(exp

21)(

- described by two parameters and - symmetric

- for a N(0,1) random variable, probability for values larger than 1.96 (2.58) relative to the mean is smaller than 5% (1%). Nearly all values lie within [-3,3].- the c.d.f cannot be given explicitly

The Standard Normal Distribution N (0,1)

Any normal distribution can be transformed to N(0,1) using Z=(X-)/

= 221exp

21)( xxf


Normal Distribution

)(xfN )(xFN


Example: The normal distribution and the mean January temperature

Suppose that the mean January temperature at Ithaca (New York State) is a random variable with a Gaussian distribution with

=22.2F =4.4 FWhat is the probability that an arbitrarily selected January will have mean temperature as cold as or colder than 21.32 F (23.08F)?

- transforming the temperature into a standard normal variable

z=(21.32F-22.2F)/4.4F=-0.2

z=(23.08F-22.2F)/4.4F=+0.2

-looking up the table

F(z=-0.2)=0.421

F(z=+0.2)=0.579

Since Gaussian distribution is symmetric, one has

)(1)(1)(1)()( zFzZPzZPzZPzF ===


Example: The normal distribution and the mean January temperature

The average January temperature over the US are available for the period 1951-1980 and the year 1989. How can one assess the probabilities of the 1989 temperatures?

Model: the mean January temperatures at each stations are normally distributed random variables

Estimating the model parameters:

data available :

)(1

1

2

1

2

1

i

N

ii

N

ii

x

xN

xN

=

=

=

=


Probabilities of 1989 January mean temperature obtained from the model

- The probabilities are described in terms of percentiles of local Gaussian distributions

- Florida and most of the midwest were substantially warmer than usual, while only a small portion of the western United States was cooler than usual


Gamma Distribution: G (,)

0,, ,)(

)/exp()/()(1

>=

xxxxf

- defined for positive real numbers

- characterized by two parameters:

the shape parameter and the scale parameter - skewed

- mean and variance:

- approaches normal distribution for large - the c.d.f. is not explicit

- any gamma distribution can be transformed

to the standard Gamma distribution using

the transformation

22 , ==

x=

1(1) ),()1( ==+

0(0) :1on)distributi al(exponenti

/1)0(:10 as )(:1

=>

==