Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
CS 521 Computational Science1
Random NumbersRandom Numbersandand
Monte Carlo MethodsMonte Carlo Methods
Charles Erwin and Jay Hatcher
CS 521 Computational Science2
Random NumbersRandom Numbers
CS 521 Computational Science3
IntroductionIntroduction
What are “Random Numbers?”
Where do Random Numbers Come from?
How to make Random numbers?
CS 521 Computational Science4
Random NumbersRandom Numbers
Not just looking for a random number.– Looking for a sequence of numbers that
doesn’t repeat.– Do not want a sequence that is deterministic!
Seed: Number that begins the sequence.– Generally the only number that is known
initially.
CS 521 Computational Science5
Why use Random Numbers?Why use Random Numbers?
Statistical Sampling Cryptography/Cryptanalysis Gaming Analysis of Algorithms Simulation Input
CS 521 Computational Science6
Where to find Random NumbersWhere to find Random Numbers
Tables of Random Numbers– Classic Example: RAND corp’s A Million Random
Digits with 100,000 Normal Deviates– Generated with Electronic simulation of roulette
wheel attached to a computer.– Early example of hardware Random Number
generator.
Random Number Generator
CS 521 Computational Science7
Ideas for Random Sequences?Ideas for Random Sequences?
von Neumann’s “Squaring” Method– Take a number, square it, and then use the middle
digits as the random number.– What about a seed of 0000?
Linear Congruential Generator– Xn = (aXn-1 + c) mod m– Xo = seed, modulus m, muliplier a, and increment c– Repeats due to the modular arithmetic that forces
wrapping of values into the desired range.
CS 521 Computational Science8
Linear Linear CongruentialCongruential Generator Generator Problems (cont)
CS 521 Computational Science9
Random Numbers from HardwareRandom Numbers from Hardware Linux Example: /dev/random
– Generated by many different factors observedby the operating systems
– Avoids tracking things such as network trafficthat can be manipulated by outsiders.
Other inputs for random information:– Readings from a Geiger counter– Detected noise from a radio receiver– Thermal or quantum-mechanical noise, amplified to provide a
random voltage source.
CS 521 Computational Science10
Random Numbers from HardwareRandom Numbers from Hardware(cont)(cont)
Internal CPU Random Number Generator– Modern Intel chips feature an internal RNG– Samples thermal noise by amplifying the voltage measured across
undriven resistors.– Analysis determines it cryptographically sound.
Unconventional Source: Lava Lamps– Random bits are extracted from images of the erupting blobs
inside six Lava Lite lamps. No longer available.– LavaRnd provided random data using similar methods.– http://www.lavarnd.org/what/index.html
CS 521 Computational Science11
Random Numbers from HardwareRandom Numbers from Hardware(cont)(cont)
Commercial RandomNumber Generators– Operate from USB or Serial connections– Standard mode is to deliver one byte of
data at a time.– Current models pass DIEHARD battery of
tests– Very fast generation.– Prices range anywhere from ~$100 to more
than $1,000 per unit.
CS 521 Computational Science12
Software Random GeneratorsSoftware Random Generators
Random Number Algorithm an oxymoron.– von Neumann: “Anyone who uses software to
produce random numbers is in a state of sin”– Deterministic Random number generators are
called “Pseudorandom Number generators”– Clever algorithms can create sequences that
are relatively random.
CS 521 Computational Science13
Modern Generator Example 1Modern Generator Example 1
Mersenne Twister by Makoto Matsumotoand Takuji Nishimura– Period size: 2^19937-1– Freely Available in C source code– Fast (used to be much faster than the C rand()
functionality but rand() has since beenimproved)
– Not cryptographically secure
CS 521 Computational Science14
Modern Generator Example 2Modern Generator Example 2
Blum Blum Shub (BBS) by Lenore Blum,Manuel Blum and Michael Shub
–Xn+1 = (Xn)2 mod M– M=pq is the product of two large primes p and q– Proven to be cryptographically secure– Very slow
CS 521 Computational Science15
Summary of AlgorithmsSummary of Algorithms
Different generators are more appropriatefor specific circumstances.– For Cryptanalysis or Cryptography, finding an
algorithm which is cryptographically secureis essential.
– For Simulations such as Monte-Carlo, a fastalgorithm is preferred.
CS 521 Computational Science16
Summary (cont.)Summary (cont.)
If your application is security based, yourchoices are limited to cryptographicallysecure random number generators.
Otherwise, use the normal algorithmanalysis questions of correctness andcomplexity.
CS 521 Computational Science17
Quick list of ReferencesQuick list of References Linear Congruential Generator:
http://eternallyconfuzzled.com/tuts/random.htmlhttp://www.taygeta.com/rwalks/node1.html
Randomness as a Resource:http://www.americanscientist.org/template/AssetDetail/assetid/20829/page/3
Hardware Random Number Generators: http://www.robertnz.net/hwrng.htm Mersenne Twister: http://www.math.sci.hiroshima-u.ac.jp/~m-
mat/MT/emt.html Blum Blum Shub: http://en.wikipedia.org/wiki/Blum-Blum-
Shub_pseudorandom_number_generator DIEHARD Random Tests: http://www.cs.hku.hk/~diehard/cdrom/ Security Requirements for Random Numbers:
http://www.ietf.org/rfc/rfc4086.txt Intel’s internal RNG:
http://www.cryptography.com/resources/whitepapers/IntelRNG.pdf
CS 521 Computational Science18
Monte Carlo MethodsMonte Carlo Methods
CS 521 Computational Science19
IntroductionIntroduction
“Monte Carlo” was coined by Metropolisduring the Manhattan Project of WWII
Named after a resort town in Monacofamous for its casinos
Monte Carlo methods use statisticalsimulation to approximate a solution to aproblem
CS 521 Computational Science20
Integration MethodsIntegration Methods
Direct Sampling
Random Walk and Markov Chains
CS 521 Computational Science21
Integration Methods (cont)Integration Methods (cont)
Direct Sampling– Importance Sampling
– Stratified Sampling
– Adaptive Monte Carlo– VEGAS algorithm
– Recursive Stratified Sampling
CS 521 Computational Science22
Integration Methods (cont)Integration Methods (cont)
Markov Chain Monte Carlo (MCMC)– Random Walks
– Metropolis-Hastings Algorithm
– Gibbs Sampling
CS 521 Computational Science23
Optimization MethodsOptimization Methods
Genetic Algorithms
Simulated Annealing
Parallel Tempering
CS 521 Computational Science24
Optimization Methods (cont)Optimization Methods (cont)
Genetic Algorithms– Explore solution space using subsequent
generations
– Generations that are better are combinedand/or mutated to try to produce an evenbetter solution
CS 521 Computational Science25
Optimization Methods (cont)Optimization Methods (cont)
Simulated Annealing– Inspired by annealing in metallurgy
– Nearby solutions are compared to the globaltemperature T
– T is gradually decreased
CS 521 Computational Science26
Optimization Methods (cont)Optimization Methods (cont)
Parallel Tempering– Run two or more simulations concurrently at
different temperatures
– Pay attention to overlap between the twosystems and use the overlap to pick morelikely solutions
CS 521 Computational Science27
Other MethodsOther Methods
Direct Simulation– Stochastic simulation of rarefied gas flows– DSMC Demo
Dynamic Monte Carlo– models the dynamic behaviors of molecules
Quantum Monte Carlo– Applies MC to the Schrödinger wave equation
in quantum mechanics
CS 521 Computational Science28
Example: Approximating Example: Approximating ππ
π is 4 times thenumber of dartsin the shaded partof the quadrantdivided by thetotal darts in thequadrant
Buffon’s Needle
CS 521 Computational Science29
ApplicationsApplicationsRadiation transport Operations researchNuclear criticality Design of nuclear reactorsDesign of nuclear weapons Statistical physicsPhase transitions Wetting and growth of thin filmsReinforcement Learning Genetic ProgrammingAtomic wave functions and Intranuclear cascade reactions eigenvalues Thermodynamic propertiesLong chain coiling polymers Reaction kineticsPartial differential equations Large sets of linear equationsNumerical integration Uncertainty analysisDevelopment of statistical tests Cell population studiesCombinatorial problem Search and optimizationSignal detection WarGames
CS 521 Computational Science30
References and Additional InfoReferences and Additional Info Introduction to Monte Carlo Methods
– http://csep1.phy.ornl.gov/mc/mc.html Adaptive and Recursive Monte Carlo Methods
– http://www.library.cornell.edu/nr/bookcpdf/c7-8.pdf The WWW Virtual Library: Monte Carlo Methods
– http://random.mat.sbg.ac.at/links/monte.html The Basics of Monte Carlo Simulations
– http://www.chem.unl.edu/zeng/joy/mclab/mcintro.html Advanced Monte Carlo Methods
– http://www.cs.pitt.edu/~milos/courses/cs3750/lectures/class10.pdf Monte Carlo Methods (Reinforcement Learning)
– http://www.cs.ualberta.ca/~sutton/book/5/node1.html Buffon’s Needle (applet)
– http://www.angelfire.com/wa/hurben/buff.html Direct Simulation MC
– http://www.simba.us/misc/dsmc/dsmca.html