Random Numbers and Their Applicationsrejohnso/bios524/Random Numbers and...Random Numbers and Their Applications Pseudo Uniform Random Numbers: Background! Multiplicative Congruential

Random Numbersand Their Applications

Bios 524: Biostatistical Computing

Random Numbers and Their Applications

Summary! Overview! Pseudo Uniform Random Numbers! Transformations Out of Uniform! Applications! Discussion! Examples


X/M = I + FX (mod M) = MF

Pseudo Uniform Random Numbers: Background

! Linear Congruential Generators

Z = {Z0, Zi = AZi−1 + C (mod M); i = 1, 2, …}

U = {Ui = Zi /M; i = 1, 2, …}

Or M−−−−1


1 2 34

56789

1011

12131415161718

1920

2122232425

2627

28 29 30

Zi = 24Zi−1 + 0 (mod 31)

full cycle


13 2 175

2728

2186

2015

1922124182914

264

3102325

1116

129 30 7



! Multiplicative Congruential Generators (C=0)! Good ones with modulus 231−1:

! Fishman and Moore (JASA, 1982): Tests! {Ui ; i=1(1)n} is a sequence of iid random variables! {Ui}, i=1(1)n, have a uniform distribution on (0,1)! {Ui−1, Ui}, i=2(2)n, have a uniform distribution on

the unit square! {Ui−2, Ui−1, Ui}, i=3(3)n, have a uniform distribution on the unit

cube



! Multiplicative Congruential Generators (C=0)! Good ones with modulus 231−1:

! Fishman and Moore (JASA, 1982): Values of A

! 397204094 (V)! 764261123 (VIII)! 507435369 (XII)! 1277850838 (XIV)

Used in current SAS versions



! SAS RANUNI! Function: U=Ranuni(seed);! Call routine: Call Ranuni(seed, U);! Seed

! ≤ 0, the system clock is sampled for the seed

! Integer between 1 and 231−2, inclusive

! Note: 231−1 is not allowed in V8.1+

but was inadvertently allowed in earlier versions.


Pseudo Uniform Random Numbers: Transformations! New random variables out of uniform

Some techniques:! Inverse Probability Transform (IT)! Acceptance-Rejection (AT)! Other Transformations (T)

! Composition Method! Box-Muller! Many others

Good reference:Rubinstein (1981)


Inverse Probability Transform! Define F to be the cdf from which we

wish to sample.! F is nondecreasing, thus F−1 may be

defined as F−1(u) = inf{x: F(x) ≥ u}, 0 ≤ u ≤ 1.

! X= F−1(U), where U is generated from U(0,1), is a pseudo random variable sampled from the desired distribution.


Inverse Probability TransformF: cdf of desired distribution

0

0.25

0.5

0.75

1

-3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3

U

X


Inverse Probability TransformF: cdf of desired distribution

0

0.25

0.5

0.75

1

0 1 2 3 4 5 6 7 8 9 10 11 12 13

U

XX


Other Transformations:Box-Muller Method

! Used to generate standard normal random values.

! Generate U1 and U2 from U(0,1), then compute! Z1= (−2 ln U1)1/2 cos 2πU2

! Z1= (−2 ln U1)1/2 sin 2πU2

! Z1 and Z2 are iid pseudo standard normal random variables.


Table 1: SAS Functionsof the form RANxxx

SAS Function

Technique Distribution

RANBIN IT, T Binomial (user: # trials and success probability)

RANCAU AR Cauchy (location=0, scale=1) RANEXP IT Exponential (λ=1) RANGAM AR Gamma (user: shape parameter, scale=1) RANNOR T Standard Normal RANPOI IT, T Poisson (user: mean) RANTBL IT Finite Discrete or Nominal

(user specifies all probabilities) RANTRI IT Triangular on (0,1)

(user: mode)


Table 2: SAS Quantile Functions(inverse CDF)

Inverse CDF Distribution BETAINV Beta (user: two shape parameters) CINV Chi-squared (user: df and ncp) FINV F (user: numerator & denominator df and ncp) GAMINV Gamma distribution (user: shape parameter, scale=1) PROBIT Standard normal TINV Student’s t (user: df and ncp)


Some Applications: Examples! Determine which of several kinds of traffic signals

is most effective at controlling flow at a busy intersection.

! Arrange data in a large computer database in order to minimize mean access time.

! Estimate the probability of a given event resulting from a complex process.

! Select a simple random sample, or other probability sample, from a database.


Simple Random Sampling (SRS)

! In a population of size N, each sample of size n has the same probability of selection: [NCn] −1

! Each population unit has the same probability of selection: n/N

! Each combination of m (≤ n) population units has the same probability of selection: [N−mCn−m] /[NCn]

50C10=10272278170

m=5: 45C5 /50C10≈≈≈≈0.00012


Simple Random Sampling (SRS)Random-Sort

! Random-Sort Method! Generate U from U(0,1) for each

population unit (save the unit number and U in a data set). The data set will have N observations.

! Sort the data set on U.! Select the first n units from the sorted data

set.


Random Sort Routine%Let k=10; * Sample size;%Let seed=325476; * Random # generator seed;

data RandSort (keep=pop_id ranorder);retain seed &seed;set Popltion (keep=pop_id);call ranuni(seed,ranorder);

proc sort data=RandSort;by ranorder;

proc sort data=RandSort(obs=&k);by pop_id;


Simple Random Sampling (SRS)Probability-Tree Method

! Probability-Tree Method

! Unit 1

! Unit 2

! Unit 3! Etc.

n/(N−1) (n−1) /(N−1)

n /(N−2) (n−1) /(N−2) (n−2) /(N−2)

(Conditional) Probability of Selection

n/N


Probability-Tree Algorithm

! Probability-Tree Method! Initialize k=n and m=N.! Generate U from U(0,1)

! If U ≤ k/m! Select the current unit.! k=k−1

! m=m−1

! Repeat until n units are selected.


Probability-Tree RoutineA One-Pass Algorithm%Let n=50; * Population size;%Let k=10; * Sample size;%Let seed=325476; * Random number generator seed;data SRSample;

keep n k select_k;n=&n;k=&k;seed=&seed;do select_k=n to 1 by -1 while (k>0);call ranuni(seed,selprob);if selprob < k/select_k then do;

k=k-1;output;end;

end;run;


Simple Random Sampling (SRS)Floyd’s Method

! Floyd’s Method (Bentley and Floyd, 1987)

! Initialize k=N−n+1! Generate U from U(0,1)

! Transform U to X ∈ {1, 2, … , k}: X = ceil(U*k)! If this value of X has not been previously

selected, then accept X.! Else, accept X=k.! k=k+1

! Repeat until k=N.


Floyd’s MethodExample: Sample three integers from {1, 2, 3, 4, 5, 6}

00.20.40.60.8

1

1 2 3 4 5 6

FirstSecond

Third

Select one from:1, 2, 3, 4

Result: 2

Previously selected:none

Add to selected:2



Select one from:1, 2, 3, 4, 5

Result: 4

Previously selected:2

Add to selected:40

0.20.40.60.8

1

1 2 3 4 5 6

FirstSecond

Third



Select one from:1, 2, 3, 4, 5, 6

Result: 4

Previously selected:2, 4

Add to selected:60

0.20.40.60.8

1

1 2 3 4 5 6

FirstSecond

Third


Simple Random Sampling (SRS)SURVEYSELECT

! SAS version 8: PROC SURVEYSELECT! This new procedure uses Floyd’s method.

proc surveyselectdata=Popltion out=SampleSampSize=&k Method=SRSSeed=&seed;


Discussion! Pseudo uniform numbers are out of uniform.

! Transformations do not produce variables with the exact desired distributions.

! Robustness of Generator/Transformation! Deng and Chhikara (1992), Statistica Neerlandica Vol 2-3,

pp. 195-207.

! Applications may be biased or inaccurate.! Inaccuracies in Transformations

! Approximation of transformation! Numerical inaccuracies (floating-point)


Discussion! Do we really get a Simple Random Sample

using PROC SURVEYSELECT or one of the other methods?! There are M−1 seeds one can select to initiate the

generator (recall M is the modulus).! Thus there are at most min(M−1, NCn) samples that

can be generated with any of the methods.! Some potential samples have no chance of being

sampled if N is large relative to n.


Discussion! Each method provides a systematic sample from

the population of integers 1(1)M-1.! The population of numbers in the generated sequence

has a very dense discrete uniform distribution on (0,1).

! Population is very close to U(0,1).! Robustness is not likely a problem

! To evaluate a random number generator, we need to ask how does it sample from this population?

! Generators that pass the standard tests nicely approximate true uniform random variables.


Examples! Selecting a stratified random sample.! Generating a sequence of non-standard

normal random values! Generating a sequence of binomial

random values! Estimating an event probability.! Simulating a regression problem.


Estimating an Event Probability! Estimate p=P(E), where E is some event that is

the result of a random process with probability p.! We wish to estimate p with a given margin of

error with 100δ% confidence.! Needed:

! δ (find z such that the P(-z≤Z≤z)=δ for Z a standard normal variable)

! Margin of error (E)! Feasible value for p. Choose the closest value to 0.5

that is feasible for p. If no knowledge of the value of p, choose 0.5 as the feasible value (p*).


Estimating an Event Probability

! Choose the sample size to be no smaller than

! If p*=0.5, then the above simplifies to

2

*)1(*

−

Ezpp

2

2

⋅ Ez

Documents

Random Numbers and Their Applicationsrejohnso/bios524/Random Numbers and...Random Numbers and Their Applications Pseudo Uniform Random Numbers: Background! Multiplicative Congruential