STATISTICS Univariate Distributions Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University

STATISTICSUnivariate Distributions

Professor Ke-Sheng ChengDepartment of Bioenvironmental Systems Engineering

National Taiwan University

Probability density functions of discrete random variables

• Discrete uniform distribution • Bernoulli distribution• Binomial distribution• Negative binomial distribution• Geometric distribution• Hypergeometric distribution• Poisson distribution

04/10/23 2Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Discrete uniform distribution

N ranges over the possible integers.

)(1

0

,,2,11

);( ,,2,1 xINotherwise

NxNNxf NX

2/)1(][ NXE

N

j

jtX N

etm

NXVar

1

2

1)(

12/)1(][


Bernoulli distribution

1-p is often denoted by q.

)()1(0

10)1();( 1,0

11

xIppotherwise

or xpppxf xx

xx

X

10 p

pXE ][

qpetm

pqXVart

X

)(

][


Binomial distribution

• Binomial distribution represents the probability of having exactly x success in n independent and identical Bernoulli trials.

)()1(

0

,,1,0)1(),;( ,,1,0 xIpp

x

n

otherwise

nxppx

npnxf n

xnxxnx

X

npXE ][nt

X peqtm

npqpnpXVar

)()(

)1(][


Negative binomial distribution• Negative binomial distribution represents the

probability of achieving the r-th success in x independent and identical Bernoulli trials.

• Unlike the binomial distribution for which the number of trials is fixed, the number of successes is fixed and the number of trials varies from experiment to experiment. The negative binomial random variable represents the number of trials needed to achieve the r-th success.


,1,;,2,1)1(1

1),;(

rrx rppr

xprxf rrx

X

prXE /][

rtrtX qepetm

prqXVar

)1/()()(

/][ 2


Geometric distribution

• Geometric distribution represents the probability of obtaining the first success in x independent and identical Bernoulli trials.

,3,2,1)1();( 1 x pppxf xX

pXE /1][

)1/()()(

/][ 2

ttX qepetm

pqXVar


Hypergeometric distribution

where M is a positive integer, K is a nonnegative integer that is at most M, and n is a positive integer that is at most M.

otherwise

nx for

n

Mxn

KM

x

K

nKMxfX

0

,,1,0),,;(


• Let X denote the number of defective products in a sample of size n when sampling without replacement from a box containing M products, K of which are defective.

MnKXE /][

1][

M

nM

M

KM

M

KnXVar


Poisson distribution

• The Poisson distribution provides a realistic model for many random phenomena for which the number of occurrences within a given scope (time, length, area, volume) is of interest. For example, the number of fatal traffic accidents per day in Taipei, the number of meteorites that collide with a satellite during a single orbit, the number of defects per unit of some material, the number of flaws per unit length of some wire, etc.


0

,2,1,0!

);(

x x

exf

x

X

)(!

xIx

e0,1,

x

][XE ][XVar

)1()( te

X etm


Assume that we are observing the occurrence of certain happening in time, space, region or length. Also assume that there exists a positive quantity which satisfies the following properties:

1.


2.

3.



The probability of success (occurrence) in each trial.




,2,1,0!

);(

x x

exf

x

X


0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35 40 45 50

alpha=0.05 alpha=0.1 alpha=0.2 alpha=0.5

Comparison of Poisson and Binomial distributions


• Example Suppose that the average number of telephone calls

arriving at the switchboard of a company is 30 calls per hour.

(1) What is the probability that no calls will arrive in a 3-minute period?

(2) What is the probability that more than five calls will arrive in a 5-minute interval?

Assume that the number of calls arriving during any time period has a Poisson distribution.


Assuming time is measured in minutes


Poisson distribution is NOT an appropriate choice.

Assuming time is measured in seconds


Poisson distribution is an appropriate choice.

• The first property provides the basis for transferring the mean rate of occurrence between different observation scales.

• The “small time interval of length h” can be measured in different observation scales.

• represents the time length measured in scale of .

• is the mean rate of occurrence when observation scale is used.

i

i

hhi

i

i


• If the first property holds for various observation scales, say , then it implies the probability of exactly one happening in a small time interval h can be approximated by

• The probability of more than one happenings in time interval h is negligible.

12

21

1

21 21

p

hhh

hhh

nn

n n

n ,,1


• probability that more than five calls will arrive in a 5-minute interval

• Occurrences of events which can be characterized by the Poisson distribution is known as the Poisson process.

.042021.0

)5()5()5()5()5()5(1 543210

PPPPPP


Probability density functions of continuous random variables

• Uniform or rectangular distribution• Normal distribution (also known as the Gaussian

distribution)• Exponential distribution (or negative exponential

distribution)• Gamma distribution (Pearson Type III)• Chi-squared distribution• Lognormal distribution


Uniform or rectangular distribution

)()(

1),;( ],[ xI

abbaxf baX

2/)(][ baXE

tab

eetm

abXVaratbt

X )()(

12/)(][ 2


PDF of U(a,b)


Normal distribution (Gaussian distribution)

2

2

2

1

2

1),;(

x

X exf

][XE

2/

2

22

)(

][tt

X etm

XVar



Z




Z~N(0,1)X~N(μ1, σ1) Y~N(μ2, σ2)

2

2

1

1

YX

Z

Commonly used values of normal distributions


Exponential distribution(negative exponential distribution)

.0)();( ),0[ ,xIexf x

X

/1][ XE

t for t

tm

XVar

X )(

/1][ 2

Mean rate of occurrence in a Poisson process.



Gamma distribution

.0,0,)()(

1),;( ),0[

/1

xIex

xf xX

][XE

./1for )1()(

][ 2

tttm

XVar

X

represents the mean rate of occurrence in a Poisson process.

is equivalent to in the exponential density.

/1

/1


• The exponential distribution is a special case of gamma distribution with

• The sum of n independent identically distributed exponential random variables with parameter has a gamma distribution with parameters .

.1

/1 and n


Pearson Type III distribution (PT3)

, and are the mean, standard deviation and skewness coefficient of X, respectively.

It reduces to Gamma distribution if = 0.

xex

xfx

X

,)(

1)(

1

22


• The Pearson type III distribution is widely applied in stochastic hydrology.

• Total rainfall depths of storm events can be characterized by the Pearson type III distribution.

• Annual maximum rainfall depths are also often characterized by the Pearson type III or log-Pearson type III distribution.


Chi-squared distribution

• The chi-squared distribution is a special case of the gamma distribution with

.1,2,k, )(2)2/(2

1);( ),0[

2/1)2/(

xIex

kkxf x

k

X

kXE ][

.2/1)21()(

2][2/

t for ttm

kXVark

X

.2 and 2/ k



Log-Normal DistributionLog-Pearson Type III Distribution (LPT3)

• A random variable X is said to have a log-normal distribution if Log(X) is distributed with a normal density.

• A random variable X is said to have a Log-Pearson type III distribution if Log(X) has a Pearson type III distribution.


Lognormal distribution

)(2

1),;( ),0(

ln

2

12

2

xIex

xf

x

X

)2/( 2

][ eXE22 222][ eeXVar


Approximations between random variables

• Approximation of binomial distribution by Poisson distribution

• Approximation of binomial distribution by normal distribution

• Approximation of Poisson distribution by normal distribution


Approximation of binomial distribution by Poisson distribution


Approximation of binomial distribution by normal distribution

• Let X have a binomial distribution with parameters n and p. If , then for fixed a<b,

is the cumulative distribution function of the standard normal distribution.

It is equivalent to say that as n approaches infinity X can be approximated by a normal distribution with mean np and variance npq.

n

)()( abnpqbnpXnpqanpPbnpq

npXaP

)(x


Approximation of Poisson distribution by normal distribution

• Let X have a Poisson distribution with parameter . If , then for fixed a<b

• It is equivalent to say that as approaches infinity X can be approximated by a normal distribution with mean and variance .

)()( abbXaPbX

aP



Example

• Suppose that two fair dice are tossed 600 times. Let X denote the number of times that a total of 7 dots occurs. What is the probability that ?


11090 X


Transformation of random variables

• [Theorem] Let X be a continuous RV with density fx. Let Y=g(X), where g is strictly

monotonic and differentiable. The density for Y, denoted by fY, is given by

.)(

))(()(1

1

dy

ydgygfyf XY


• Proof: Assume that Y=g(X) is a strictly monotonic increasing function of X.


Example• Let X be a gamma random variable with

04/10/23 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

55

1

1

1

11)(

1

)(

1)(

,,Let

Y

Y

Y

XXX

eY

eY

yf

dY

dXYX

XXY

.0,0,)()(

1),;( ),0[

/1

xIex

xf xX

Y is also a gamma random variable with scale parameter 1/ and shape parameter .

Definition of the location parameter


Example of location parameter


Definition of the scale parameter


Example of scale parameter


Simulation Simulation • Given a random variable X with CDF FX(x), there

are situations that we want to obtain a set of n random numbers (i.e., a random sample of size n) from FX(.) .

• The advances in computer technology have made it possible to generate such random numbers using computers. The work of this nature is termed “simulation”, or more precisely “stochastic simulation”.



Pseudo-random number generation

• Pseudorandom number generation (PRNG) is the technique of generating a sequence of numbers that appears to be a random sample of random variables uniformly distributed over (0,1).


• A commonly applied approach of PRNG starts with an initial seed and the following recursive algorithm (Ross, 2002)

modulo m where a and m are given positive integers, and the

above equation means that is divided by m and the remainder is taken as the value of .

1 nn axx

1naxnx


• The quantity is then taken as an approximation to the value of a uniform (0,1) random variable.

• Such algorithm will deterministically generate a sequence of values and repeat itself again and again. Consequently, the constants a and m should be chosen to satisfy the following criteria:– For any initial seed, the resultant sequence has the “appearance” of

being a sequence of independent uniform (0,1) random variables.– For any initial seed, the number of random variables that can be

generated before repetition begins is large.– The values can be computed efficiently on a digital computer.

mxn /


• A guideline for selection of a and m is that m be chosen to be a large prime number that can be fitted to the computer word size. For a 32-bit word computer, m = and a = result in desired properties (Ross, 2002).

1231 57


Simulating a continuous random variable

• probability integral transformation



67

The cumulative distribution function of a continuous random variable is a monotonic increasing function.

Example

04/10/23 Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU

68

• Generate a random sample of random variable V which has a uniform density over (0, 1).

• Convert to using the above V-to-X transformation.

)1,0(~,ln)1ln(

1)()(

)(

0

UiidVVU

X

UeduufxF

exf

xx

x

},,,{ 21 nvvv

},,,{ 21 nxxx },,,{ 21 nvvv

Random number generation in R• R commands for stochastic simulation (for

normal distribution – pnorm – cumulative probability– qnorm – quantile function– rnorm – generating a random sample of a specific

sample size– dnorm – probability density function

For other distributions, simply change the distribution names. For examples, (punif, qunif, runif, and dunif) for uniform distribution and (ppois, qpois, rpois, and dpois) for Poisson distribution.

04/10/23 69Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU

Generating random numbers of discrete distribution in R

• Discrete uniform distribution– R does not provide default functions for random

number generation for the discrete uniform distribution.

– However, the following functions can be used for discrete uniform distribution between 1 and k.• rdu<-function(n,k) sample(1:k,n,replace=T) # random number• ddu<-function(x,k) ifelse(x>=1 & x<=k & round(x)==x,1/k,0) # density• pdu<-function(x,k) ifelse(x<1,0,ifelse(x<=k,floor(x)/k,1)) # CDF• qdu <- function(p, k) ifelse(p <= 0 | p > 1, return("undefined"),

ceiling(p*k)) # quantile


70

– Similar, yet more flexible, functions are defined as follows• dunifdisc<-function(x, min=0, max=1) ifelse(x>=min & x<=max &

round(x)==x, 1/(max-min+1), 0)>dunifdisc(23,21,40)>dunifdisc(c(0,1))

• punifdisc<-function(q, min=0, max=1) ifelse(q<min, 0, ifelse(q>max, 1, floor(q-min+1)/(max-min+1)))>punifdisc(0.2)>punifdisc(5,2,19)

• qunifdisc<-function(p, min=0, max=1) floor(p*(max-min+1))+min>qunifdisc(0.2222222,2,19)>qunifdisc(0.2)

• runifdisc<-function(n, min=0, max=1) sample(min:max, n, replace=T)>runifdisc(30,2,19)>runifdisc(30)


71

• Binomial distribution


72

• Negative binomial distribution


73

• Geometric distribution


74

• Hypergeometric distribution


75

• Poisson distribution


76

An example of stochastic simulation

• The travel time from your home (or dormitory) to NTU campus may involve a few factors:– Walking to bus stop (stop for traffic lights,

crowdedness on the streets, etc.)– Transportation by bus– Stop by 7-11 or Starbucks for breakfast (long queue)– Walking to campus


• If you leave home at 8:00 a.m. for a class session of 9:10, what is the probability of being late for the class?

)36,15(~ 21 NX

~2X Gamma distribution with mean 30 minutes and standard deviation 10 minutes.

~3X Exponential distribution with a mean of 20 minutes.

)25,10(~ 24 NX All Xi’s are independently distributed.


4321 XXXXY

Documents

STATISTICS Univariate Distributions Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University