EE317: Computation and Simulation - ee317/Course_notes/handout7-1up.pdfآ  EE317: Computation and Simulation

  • View
    0

  • Download
    0

Embed Size (px)

Text of EE317: Computation and Simulation - ee317/Course_notes/handout7-1up.pdfآ  EE317: Computation and...

  • EE317: Computation and Simulation Handout Seven - Random Number Generation

    Dr. Conor Brennan School of Electronic Engineering

    January 16, 2014

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Randomness is a fact of life.... In many ways it enriches it, or at least makes it more interesting.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Probability and statistics and troublesome concepts as humans often have a poor intuition for them.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Let’s examine the so-called birthday paradox. N people are in a room. How large must N be before the chances that at least two people in the roomshare a birthday?

    Lets work out the complementary probability - that no-one shares a birthday. This is given by (assuming independence)

    PN = 365

    365

    364

    365

    363

    365 · · · 365− N + 1

    365

    and the probability of a shared birthday is thus

    P = 1− PN

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • For engineers randomness can be beneficial or problematic. For example radio engineers must contend with random fading of the power received by a mobile. However recently engineers have used this randomness for security communications.

    Ray tracing – Image theory 

    Ground 

    TransmiYer 

    Receiver 

    TransmiYer   image 

    ˆ n

    θr

    θi

    Ray path 

    Reflec/on point 

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Fading is caused by destructive interference of the EM waves and it causes large fluctuations in the power received. It is random in nature and radio engineers use many models to characterise it (Rayleigh, Ricean etc).

    (a) Fading along a straight line caused by two rays interfering

    (b) More general random fading in two-dimensional urban region

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • The Monte Carlo method is a numerical method for statistical simulation which utilises sequences of random numbers to perform the simulation.

    The name “Monte Carlo” was chosen by John Von Neumann. The name refers to the Monte Carlo casino in Monaco, where the uncle of a colleague used to gamble....

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • The Monte Carlo method has a wide range of engineering applications, from the simulation of bit error rates in telecommunications to tolerance analysis in VLSI designs.

    A “random number generator” is one of the primary compo- nents of any Monte Carlo sim- ulation.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • We would like to create a se- quence of independent random numbers with a specified distri- bution, e.g. uniform, Gaussian.

    This means, loosely, that each number is obtained purely by chance, having nothing to do with other numbers of the sequence, and that each number generated has a specified probability of falling in any given range of values.

    A uniform distribution on a finite interval is one in which each possible number within the interval is equally probable. In these notes a distribution is generally understood to be uniform unless some other distribution is specifically mentioned.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Each of the ten digits 0 through 9 will occur about 1/10 of the time in a uniform sequence of random digits.

    Each pair of digits (01,35,64 etc) should occur about 1/100 of the time, etc.

    Yet if we take a truly random sequence of a million digits, it will not always have exactly 100,000 zeros, 100,000 ones, etc. In fact, chances of this are quite slim; a sequence of such sequences will only have this character on the average.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Any specified sequence of a million digits is equally as probable as the seemingly special sequence consisting of a million zeros. Equally the choice 1, 2, 3, 4, 5, 6 is as likely as any else to win the Lotto (and you may have to share it with fewer people).

    Also - truly independent random processes are memoryless. If we are choosing a million digits at random and if the first 999,999 of them happen to come out to be zero, the chance that the final digital is zero is still exactly 1/10, in a truly random situation.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • “Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.” - John von Neumann (1951)

    All random number generators are based upon specific mathematical algorithms which are repeatable and sequential. As such, the numbers are just pseudo random, but they are often called random numbers for simplicity, with the understanding that they only appear to be random, but actually are deterministic.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Desirable properties include:

    • Uncorrelated Sequences - The sequences of random numbers should be serially uncorrelated. This means that any subsequence of random numbers should not be correlated with any other subsequence of random numbers. Most especially, n-tuples of random numbers should be independent of one another.

    • Long Period - The generator should be of long period. Ideally, the generator should not repeat; practically, the repetition should occur only after the generation of a very large set of random numbers.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Desirable properties continued:

    • Uniformity - The random numbers should be uniformly distributed within a specified region. For example, if random numbers on [0,1) are to be generated, it would be poor practice were more than half to fall into [0, 0.1), assuming the sample size is sufficiently large. The properties of uniformity and uncorrelated sequences are loosely related. Often, when there is a lack of uniformity, there are n-tuples of random numbers which are correlated. In this case, the space might be filled in a definite, easily observable pattern.

    • Efficiency - As literally millions or even billions of random numbers are required in a large-scale simulation, the generator should be efficient.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Almost all random number generators have as their basis a sequence of pseudorandom integers (there are exceptions).

    The integers or fixed point numbers can be manipulated arithmetically to yield floating point or real numbers.

    The random number cycle can be presented in terms of either integers or real numbers. Here, for clarity, we confine ourselves to integers.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Start  Finish  Numbers traversed 

    in sequence 

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • The above figure illustrates a random number cycle representing, in this case, a sequence of 12 integers. Each red dot represents a distinct integer.

    The nature of the cycle illustrates that:

    (1) the sequence has a finite number of integers,

    (2) the sequence gets traversed in a particular order, and

    (3) the sequence repeats if the period of the generator is exceeded

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • Linear Congruential Generators are by far the most commonly used generators. They generate uniform random numbers, based on the following formula.

    Xn+1 = (aXn + c) mod m

    where

    m is the modulus satisfying m > 0

    a is the multiplier satisfying 0 ≤ a ≤ m

    c is the increment satisfying 0 ≤ c ≤ m and

    X0 the initial seed satisfying 0 ≤ X0 ≤ m.

    mod m means take the remainder left over after integer division by m. Note that in practical situations we would choose a large value of m. In these notes however we shall use smaller values of m so that we can explicitly explore concepts such as maximum period, randomness etc.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • In the case of c = 0, it is called a multiplicative congruential random number generator.

    The case c 6= 0 is termed a mixed congruential random number generator.

    Multiplicative LCGs are faster than mixed LCGs because of one less addition operation.

    Multiplicative LCGs are adequate to good for many applications.

    Dr. Conor Brennan School of Electronic Engineering EE317: Computation and Simulation

  • The modulo operation provides the appearance of randomness.

    Note that Xn+1 depends only on the previous integer Xn. This characteristic of LCG minimises storage requirements.

    At the same it imposes restrictions on the period: the longest peri