07 Random Variate Generation · 2012-09-03 · Purpose & Overview • Develop understanding of generating samples from a specified distribution as input to a simulation model. •

Chapter 7Chapter 7

Random-Variate Generation

7.1Prof. Dr. Mesut Güneş ▪ Ch. 7 Random-Variate Generation

Contents

• Inverse-transform Technique• Acceptance-Rejection Techniquep j q• Special Properties


Purpose & Overview

• Develop understanding of generating samples from a specified distribution as input to a simulation model.

• Illustrate some widely-used techniques for generating drandom variates:

• Inverse-transform technique•Acceptance-rejection technique•Acceptance-rejection technique•Special properties


Preparation

• It is assumed that a source of uniform [0,1] random numbers exists.• Linear Congruential Method (LCM)

f(x)• Random numbers R, R1, R2, … with

•PDF

⎧ ≤≤ 101

f(x)

⎩⎨⎧ ≤≤

=otherwise0

101)(

xxfR

0 1x

•CDF

⎪⎧ < 00 x

0 1F(x)

⎪⎩

⎪⎨

>≤≤=

1110)(

xxxxFR


0 1x

Inverse-transform TechniqueInverse transform Technique


Inverse-transform Technique

• The concept:• For CDF function: r = F(x)( )•Generate r from uniform (0,1), a.k.a U(0,1)• Find x, x = F-1(r)

F(x) F(x)

r = F(x)1

r = F(x)1

r1 r1

r2

x1

xx1

x

2

x2


Inverse-transform Technique

• The inverse-transform technique can be used in principle for any distribution.

• Most useful when the CDF F(x) has an inverse F -1(x)which is easy to compute.

• Required steps1 Compute the CDF of the desired random variable X1. Compute the CDF of the desired random variable X2. Set F(X) = R on the range of X3. Solve the equation F(X) = R for X in terms of Rq ( )4. Generate uniform random numbers R1, R2, R3, ... and compute

the desired random variate by Xi = F-1(Ri)


Inverse-transform Technique: Example

• Exponential Distribution• PDF

• To generate X1, X2, X3 …

1 Re X− =λ

• CDF

xexf λλ −=)(1

1Re

ReX− −=

=−λ

CDFxexF λ−−=1)(

)1ln()1ln(

RX

RX−−=−λ

)1ln(

)(

R

X

−−

=λ

• Simplification

)(

)1ln(

1 RFX

RX

−

−=λ

λ)ln(RX −=

)(1 RFX =• Since R and (1-R) are uniformly

distributed on [0,1]






Inverse-transform technique for exp(λ = 1)


• Example: • Generate 200 or 500 variates Xi with distribution exp(λ= 1)• Generate 200 or 500 Rs with U(0,1), the histogram of Xs becomes:

0 7

0,5

0,6

0,7

0,5

0,6

0,3

0,4

0 2

0,3

0,4

0

0,1

0,2

0

0,1

0,2

0

0,5 1 1,5 2 2,5 3 3,5 4 4,5 5 5,5 6 6,5 7

Empirical Histogram

0

0,57 1,15 1,72 2,30 2,87 3,45 4,02 4,60 5,17 5,75

Rel Prob. Theor. PDF


Inverse-transform Technique• Check: Does the random variable X1 have the desired distribution?

)())(()( 00101 xFxFRPxXP =≤=≤ )())(()( 00101


Inverse-transform Technique: Other Distributions

• Examples of other distributions for which inverse CDF works are:•Uniform distribution•Weibull distribution

T i l di t ib ti• Triangular distribution


Inverse-transform Technique: Uniform Distribution

• Random variable X uniformly distributed over [a, b]

)( RXF =

RabaX=

−−

)()(

abRaXabRaX

+−=−

)( abRaX −+=


Inverse-transform Technique: Weibull Distribution

)( RXF =• The Weibull Distribution is

described by• The variate is

( )βα1

)(

Re

RXFX

=−

=−

y

• PDF

( )ββ ( )

( )ββ

α

)1l (

1

R

ReX

X

−=−( )βαβ

βαβ x

exxf −−= 1)(

( )β

( )β

βα

)1ln(

)1ln(

RX

RX

−−=

−=−• CDF

( )βαx

eXF −−=1)(ββ

β

αα

)1ln(

)1ln(

RX

R

−⋅−=

=

β

β βα

)1l (

)1ln(

RX

RX −⋅−=

7.15

βα )1ln( RX −−⋅=

Prof. Dr. Mesut Güneş ▪ Ch. 7 Random-Variate Generation

Inverse-transform Technique: Triangular Distribution

• The CDF of a Triangular Distribution with endpoints (0 2) i i b(0, 2) is given by

⎧10

2X⎪⎧ ≤

10

002x

x

⎪⎪⎩

⎪⎪⎨

⎧

≤≤−

−

≤≤=

21)2(1

102)( 2

XX

XX

XR

⎪⎪

⎪⎪⎪

⎨≤<

−−

≤<=

212

)2(1

102)( 2

xx

xx

xF

• X is generated by

⎪⎩≤≤ 21

21 X

⎪⎪⎩ > 21

2x

X is generated by

⎪⎩

⎪⎨⎧

≤<−−≤≤

=1)1(22

02

21

21

RRRR

X


⎪⎩ )( 2

Inverse-transform Technique: Empirical Continuous Distributions

• When theoretical distributions are not applicable• To collect empirical data: p

•Resample the observed data• Interpolate between observed data points to fill in the gaps


Inverse-transform Technique: Empirical Continuous Distributions• For a small sample set (size n):

• Arrange the data from smallest to largest

• Set x(0)=0A i h b bili h i l

(n)(2)(1) x x x ≤…≤≤

i 21≤≤• Assign the probability 1/n to each interval • The slope of each line segment is defined as

ni ,,2,1 xxx (i)1)-(i K=≤≤

xxxx iiii )1()()1()( −−

nni

nia iiii

i 1)1()1()()1()( −− =

−−

=

• The inverse CDF is given by

⎟⎠⎞

⎜⎝⎛ −

−+== −−

niRaxRFX ii

)1()(ˆ)1(

1 iRi≤<

− )1(when

⎠⎝ n)( nn



i Interval PDF CDF Slope ai

1 0.0 < x ≤ 0.8 0.2 0.2 4.00

2 0.8 < x ≤ 1.24 0.2 0.4 2.20

3 1.24 < x ≤ 1.45 0.2 0.6 1.05

4 1.45 < x ≤ 1.83 0.2 0.8 1.90

5 1.83 < x ≤2.76 0.2 1.0 4.65

)/)14((71.0

14)14(1

1

−−+==

− nRaxXR

66.1)6.071.0(90.145.1

=−+=



• What happens for large samples of data•Several hundreds or tens of thousand

• First summarize the data into a frequency distribution with smaller number of intervals

• Afterwards, fit continuous empirical CDF to the frequency distribution

• Slight modifications• Slight modifications•Slope

− xx ci cumulative probability of th fi t i i t l

1

)1()(

−

−

−

−=

ii

iii cc

xxa the first i intervals

•The inverse CDF is given by

( ) iiiii cRccRaxRFX ≤<−+== −−−−

11)1(1 when )(ˆ


Inverse-transform Technique: Empirical Continuous Distributions• Example: Suppose the data collected for 100 broken-widget

repair times are:

Interval (Hours) Frequency

Relative Frequency

Cumulative Frequency, c i Slope, a i

0.25 ≤ x ≤ 0.5 31 0.31 0.31 0.810 5 ≤ x ≤ 1 0 10 0 10 0 41 5 000.5 ≤ x ≤ 1.0 10 0.10 0.41 5.001.0 ≤ x ≤ 1.5 25 0.25 0.66 2.001.5 ≤ x ≤ 2.0 34 0.34 1.00 1.47

Consider R1 = 0.83:Consider R1 0.83:

c3 = 0.66 < R1 < c4 = 1.00

X1 = x(4 1) + a4(R1 – c(4 1))X1 x(4-1) a4(R1 c(4-1))= 1.5 + 1.47(0.83-0.66)= 1.75


Inverse-transform Technique:Empirical Continuous Distributions

• Problems with empirical distributions•The data in the previous example is restricted in the rangep p g

0.25 ≤ X ≤ 2.0•The underlying distribution might have a wider range

Th s t to find a theo etical dist ib tion•Thus, try to find a theoretical distribution

• Hints for building empirical distributions based on Hints for building empirical distributions based on frequency tables• It is recommended to use relatively short intervals

• Number of bins increase

•This will result in a more accurate estimate


Inverse-transform Technique: Continuous Distributions

• A number of continuous distributions do not have a closed form expression for their CDF, e.g.

•NormalG

( )( )dtxFx

t exp)(2

21

21∫

∞

−−= σμ

πσ

•Gamma•Beta

• The presented method does not work for these

∞−

• The presented method does not work for these distributions

• Solution•Approximate the CDF or numerically integrate the CDF

• Problem•Computationally slow


Inverse-transform Technique: Discrete Distribution

• All discrete distributions can be generated via inverse-transform technique

• Method: numerically, table-lookup procedure, algebraically, or a formula

l f l• Examples of application:•Empirical•Discrete uniform•Discrete uniform•Geometric



• Example: Suppose the number of shipments, x, on the loading dock of a company is either 0, 1, or 2•Data - Probability distribution:

x P(x) F(x)0 0.50 0.501 0.30 0.80

• The inverse-transform technique as table-lookup

2 0.20 1.00

The inverse transform technique as table lookup procedure

)()( xFrRrxF =≤<=

•Set X = x

)()( 11 iiii xFrRrxF =≤<= −−

7.25

•Set X = xi



Method - Given R, the generation scheme

5.0,0 ≤⎪⎧ R

gbecomes: 0.8

0.18.08.05.0

,2,1

≤<≤<

⎪⎩

⎪⎨=

RRx

Table for generating the discrete variate X

Consider R1 = 0.73:F(x ) < R ≤ F(x )

i Input ri Output xi

1 0.5 0

F(xi-1) < R ≤ F(xi)F(x0) < 0.73 ≤ F(x1)Hence, X1 = 1

0.5 02 0.8 13 1.0 2


Acceptance-Rejection TechniqueAcceptance Rejection Technique


Acceptance-Rejection Technique

• Useful particularly when inverse CDF does not exist in closed form• Thinning

• Illustration: To generate random variates, X ~ U(1/4,1)

Generate R

Procedure:Step 1. Generate R ~ U(0,1)

Condition

nono

Step 2. If R ≥ ¼, accept X=R.Step 3. If R < ¼, reject R, return to Step 1 yesyes

• R does not have the desired distribution, but R conditioned (R’) on

Output R’

the event {R ≥ ¼} does.• Efficiency: Depends heavily on the ability to minimize the number

of rejections.

7.28

of rejections.


Acceptance-Rejection Technique: Poisson Distribution

• Probability mass function of a Poisson Distribution

αα −enNPn

)(

• Exactly n arrivals during one time unit

== en

nNP!

)(

• Since interarrival times are exponentially distributed we can set

12121 1 +++++<≤+++ nnn AAAAAAA LL

• Since interarrival times are exponentially distributed we can set

)ln( ii

RA −=

• Well known, we derived this generator in the beginning of the class

αiA



• Substitute the sum by

∑∑+ −

≤− 1 )ln(1)ln( n

in

i RR

• Simplify by

∑∑==

<≤11

)(1)(i

i

i

i

αα

•multiply by -α, which reverses the inequality sign• sum of logs is the log of a product

∑∑+

==

>−≥

1

1

11)ln()ln(

n

ii

n

ii RR α

∏∏+

==

>−≥1

11

lnlnn

ii

n

ii RR α

• Simplify by eln(x) = x

∏∏+

− >≥1n

i

n

i ReR α


∏∏== 11 i

ii

i


• Procedure of generating a Poisson random variate N is as follows1. Set n=0, P=12. Generate a random number Rn+1, and replace P by P x Rn+1

3 If P ( ) th t N3. If P < exp(-α), then accept N=n• Otherwise, reject the current n, increase n by one, and return

to step 2.


Acceptance-Rejection Technique: Poisson Distribution• Example: Generate three Poisson variates with mean α=0.2

• exp(-0.2) = 0.8187• Variate 1• Variate 1

• Step 1: Set n = 0, P = 1• Step 2: R1 = 0.4357, P = 1 x 0.4357• Step 3: Since P = 0 4357 < exp(- 0 2) accept N = 0• Step 3: Since P 0.4357 < exp( 0.2), accept N 0

• Variate 2• Step 1: Set n = 0, P = 1• Step 2: R1 = 0.4146, P = 1 x 0.4146Step 2: R1 0.4146, P 1 x 0.4146• Step 3: Since P = 0.4146 < exp(-0.2), accept N = 0

• Variate 3• Step 1: Set n = 0, P = 1p ,• Step 2: R1 = 0.8353, P = 1 x 0.8353• Step 3: Since P = 0.8353 > exp(-0.2), reject n = 0 and return to Step 2 with n = 1• Step 2: R2 = 0.9952, P = 0.8353 x 0.9952 = 0.8313• Step 3: Since P = 0.8313 > exp(-0.2), reject n = 1 and return to Step 2 with n = 2• Step 2: R3 = 0.8004, P = 0.8313 x 0.8004 = 0.6654• Step 3: Since P = 0.6654 < exp(-0.2), accept N = 2



• It took five random numbers to generate three Poisson variates

• In long run, the generation of Poisson variates requires some overhead!

N Rn+1 P Accept/Reject Result0 0.4357 0.4357 P < exp(- α) Accept N=0

0 0.4146 0.4146 P < exp(- α) Accept N=0

0 0.8353 0.8353 P ≥ exp(- α) Reject

1 0.9952 0.8313 P ≥ exp(- α) Reject1 0.9952 0.8313 P ≥ exp( α) Reject

2 0.8004 0.6654 P < exp(- α) Accept N=2


Special PropertiesSpecial Properties


Special Properties

• Based on features of particular family of probability distributions

• For example:•Direct Transformation for normal and lognormal distributions•Convolution


Direct Transformation

• Approach for N(0,1)•PDF

2

2

21)(

x

exf−

=π

•CDF, No closed form available

∫∞−

−=

x t

dtexF 2

2

21)(π



• Approach for N(0,1)• Consider two standard normal random variables, Z1 and Z2, plotted as 1 2 p

a point in the plane:• In polar coordinates:

• Z = B cos(α)• Z1 = B cos(α)• Z2 = B sin(α)

(Z1,Z2)ZZ2

B

α

Z1


Z1


• Chi-square distribution•Given k independent N(0, 1) random variables X1, X2, …, Xk, then p ( ) 1 2 k,

the sum is according to the Chi-square distribution

•PDF ∑=k

ik X 22χPDF ∑=i

ik1

χ

( )22

2

1

21),(

xk

k exkxfk

−−

Γ=

( ) 22 2kΓ



• The following relationships are known• B2 = Z2

1 + Z22 ~ χ2 distribution with 2 degrees of freedom = exp(λ = 1/2). 1 2 χ g p( )

• Hence:

RB ln2−=

• The radius B and angle α are mutually independent.

RB ln2

)2cos(ln2 RRZ π−=

)2sin(ln2

)2cos(ln2

212

211

RRZ

RRZ

π

π

−=

−=



• Approach for N(μ, σ 2):pp (μ, )•Generate Zi ~ N(0,1)

Xi = μ + σ Zi

• Approach for Lognormal(μ,σ2):pp g (μ, )•Generate X ~ N(μ,σ2)

Y = eXiYi = e i


Direct Transformation: Example

• Let R1 = 0.1758 and R2=0.1489• Two standard normal random variates are generated as g

follows:11.1)1489.02cos()1758.0ln(21 =−= πZ

50.1)1489.02sin()1758.0ln(22 =−= πZ

• To obtain normal variates Xi with mean μ=10 and variance σ 2 = 4

00.1350.121022.1211.1210

2

1

=⋅+==⋅+=

XX


Convolution

• Convolution•The sum of independent random variablesp

• Can be applied to obtain•Erlang variates•Binomial variates


Convolution

• Erlang Distribution•Erlang random variable X with parameters (k, θ) can be g p ( , )

depicted as the sum of k independent exponential random variables Xi, i = 1, …, k each having mean 1/(k θ)

=∑k

iiXX

1

−=∑=

k

ii

i

Rk1

1

)ln(1θ

⎟⎟⎠

⎞⎜⎜⎝

⎛−= ∏

=

k

i

i

Rk

k1

ln1θ

θ

⎠⎝ =ik 1θ


Summary

• Principles of random-variate generation via• Inverse-transform techniqueq•Acceptance-rejection technique•Special properties

• Important for generating continuous and discrete distributionsdistributions


Documents

07 Random Variate Generation · 2012-09-03 · Purpose & Overview • Develop understanding of generating samples from a specified distribution as input to a simulation model. •