Monte Carlo methods Agent-Based Modeling and Social ......Monte Carlo methods Agent-Based Modeling and Social System Simulation Lecturer: Dr. Nino Antulov-Fantulin Lecturer: Dr. Nino

Monte Carlo methodsAgent-Based Modeling and Social SystemSimulationLecturer: Dr. Nino Antulov-Fantulin

Lecturer: Dr. Nino Antulov-Fantulin 1

Outline

1. Estimation

2. Sampling

3. Markov Chain Monte Carlo (MCMC)

4. Simulation* of stochastic dynamical processes

5. Optimization*

*Will be covered in Monte Carlo methods II lecture (18.11).


History...

The modern version of the Monte Carlo method was invented in thelate 1940s by Stanislaw Ulam, while he was working on nuclearweapons projects at the Los Alamos National Laboratory.Immediately after Ulam’s breakthrough, John von Neumannunderstood its importance and programmed the ENIAC computer tocarry out Monte Carlo calculations. *https://en.wikipedia.org/wiki/Monte_Carlo_method#History


https://en.wikipedia.org/wiki/Stanislaw_Ulam

https://en.wikipedia.org/wiki/John_von_Neumann

https://en.wikipedia.org/wiki/Monte_Carlo_method#History

Monte Carlo methods

Monte Carlo methods are a broad class of computational algorithmsthat rely on repeated random sampling to obtain numerical results.Used for:

• Estimation - integrals in high dimensional spaces

• Sampling - probability distributions

• Optimization - maximization of function

• Simulation


Why sampling and estimations?

Monte Carlo techniques play a fundamental role in statisticalphysics, modeling stochastic processes, machine learning andothers. ∫ ∫

...

∫f (x1, x2, ..., xn)dx1dx2...dxn (1)

p(x |y ) =p(y |x)p(x)∫

p(y |x ′)p(x ′)dx ′ (2)

ddt~P = A~P, (3)

(1) Statistical Mechanics: Algorithms and Computations, Book by Werner Krauth(2) Andrieu et al., An Introduction to MCMC for Machine Learning,(3) Gillespie, D. T. A rigorous derivation of the chemical master equation. Physica A: Statistical Mechanics and itsApplications 188, 404–425 (1992)


1. Monte Carlo Estimation

• Geometric view on estimating integrals over a functionf (x) : [a, b]→ [c, d ]:∫ b

a f (x)dx(b − a)(d − c)

≈ MN

=⇒∫ b

af (x)dx ≈ M

N(b − a)(d − c) (4)

where M denotes the number of samples that fall below f (x)and N denotes total number of samples.

• How good is this estimate?


1. Monte Carlo EstimationExample: Estimate the value of constant π if we have access to thegenerator of random numbers from uniform distributions [-1,1].

• Draw circle with radius 1 and square of length 2 centered atorigin.

• X,Y ∼ U[-1,1]

Z =

1, if√

X 2 + Y 2 ≤ 1

0, otherwise(5)

• M =∑N

i=1 Zi ,

2∫ b

af (x)dx = π̂ =

MN

(b − a)(d − c) =MN

4

How good is this estimate?Lecturer: Dr. Nino Antulov-Fantulin 7


Example: Estimate the value of constant π if we have access to thegenerator of random numbers from uniform distributions [-1,1].

• E [π̂] = π

• Chernoff bound: Let X1, ..., XN be independent Poisson trialsP(Xi = 1) = pi . Let X =

∑Ni=1 Xi and µ = E [x ]. For 0 < ε < 1,

P(|X − µ| ≥ εµ) ≤ 2exp(−µε2/3).

• P(|π̂ − π| ≥ επ) ≤ 2exp(−Nπε2/12)The last bound can be generalized for any set of i.i.d. indicator variables to provide (ε, δ)-approximation (see Probabilityand Computing, book by Mitzenmacher and Upfal).



• Algebra view on estimating integrals:

I =∫ b

af (x)dx = (b−a)

∫ b

a

1(b − a)

f (x)dx = E [(b−a)f (X )] (6)

where X denotes the random variable distributed with Uniformdistribution i.e. density f (x) = 1/(b − a).



E [(b − a)f (X )] ≈ IN =(b − a)

N

N∑i=1

f (xi ) (7)

• This estimate IN is unbiased and by the strong law of largenumbers, it will almost surely (a.s.) converge to I

• If the standard deviation of f (x) is finite σ <∞, then thestandard deviation of estimation IN is σ/

√N

• Central Limit Theorem (Lindeberg-Levy) states that thedistribution of errors converge in distribution to standardNormal distribution.

√N(IN − I) =⇒

N=∞N (0, σ2) (8)


1. Monte Carlo Estimation example

Estimate the integral

f (x) =∫ b

a

e−x

1 + x2 dx , (9)

where a = 0, b = 10 with N = 1000. What happens if b = 100?


2. Sampling

Up to this point we were assuming that we have access to thegenerator of Uniform numbers.What happens when we need to get the sample from differentdensity function p(x)?


2. Sampling

Rejection method for sampling with easy to sample proposal q(x).You can also use Uniform distribution for q(x).


2. SamplingMethod for sampling with easy to sample proposal q(x). Functionq(x) multiplied by constant M is always above function p(x) i.e.Mq(x) ≥ p(x).

i = 1repeat

Sample x (i) ∼ q(x), u ∼ U(0, 1)if u ∗Mq(x (i)) ≤ p(x (i)) then

accept x (i)

i + +else

rejectend if

until i==NLecturer: Dr. Nino Antulov-Fantulin 14

2. SamplingRejection-free method for sampling - Inverse Smirnov Transform.Probability integral transform: if F is c.d.f of X , thenY = F (X ) ∼ U[0, 1].


2. Sampling

Rejection-free method for sampling - Inverse Smirnov Transform

1. sample from f (x) density function

2. calculate cumulative density function F (x) =∫ x−∞ f (x)dx

3. calculate the inverse function F−1(x)

4. generate u from Uniform distribution [0,1]

5. sample is F−1(u)

Corollary of probability integral transform X = F−1(Y ) has c.d.f. F ifY ∼ U[0, 1].


2. Monte Carlo Importance sampling

We choose density function g(x) which is easy to sample and whichis positive g(x) > 0 whenever f (x) > 0.

I =∫ b

af (x)dx =

∫ b

a

f (x)g(x)

g(x)dx = E [f (Y )/g(Y )], (10)

where Y is a random variable with g(y ) density function.

E [f (Y )/g(Y )] ≈ IN =1N

∑i

f (G−1(xi ))g(G−1(xi ))

, (11)

where xi is the sample from Uniform distribution [0,1] and G−1(x) isthe inverse function of cumulative density functionG(x) =

∫ x−∞ g(x)dx i.e. Inverse Transform Sampling.


2. Importance sampling example


f (x) =∫ b

a

e−x

1 + x2 dx , (12)

where a = 0, b = 100.Which proposal to take?


2. Importance sampling example


f (x) =∫ b

a

e−x

1 + x2 dx , (13)

where a = 0, b = 100.

g(x) = C ∗ e−x . (14)


3. MCMC methods

Metropolis-Hastings is one algorithm in the MCMC class, whichfrom current sample σ randomly proposes another state σ′ byg(σ → σ′) proposal distribution and accepts it with probability

min(

1,P(σ′)g(σ′ → σ)P(σ)g(σ → σ′)

)


3. MCMC methods

The existence of stationary distribution of the Markov Chain indiscrete time and discrete states is guaranteed by detailed balanceproperty and the uniqueness by ergodicity.

Prove it!


3. MCMC methods

min(

1,P(σ′)g(σ′ → σ)P(σ)g(σ → σ′)

)This acceptance rule is direct consequence of imposing a detailedbalance condition P(σj )Wσj ,σi = P(σi )Wσi ,σj , which assures theconvergence to equilibrium.


3. MCMC methods

Andrieu et. al., An Introduction to MCMC for Machine LearningLecturer: Dr. Nino Antulov-Fantulin 23

Readings

• Christophe Andrieu, Nando de Freitas, Arnaud Doucet, MichaelI. Jordan, An Introduction to MCMC for Machine Learning,doi:10.1023/A:1020281327116

• Statistical Mechanics: Algorithms and Computations, Book byWerner Krauth

• Probability and Statistics, Book by DeGroot and Schervish

• Probability and Computing, Book by Mitzenmacher and Upfal

• Information, Physics and Computation, Book by Mezard andMontanari


Documents

Monte Carlo methods Agent-Based Modeling and Social ......Monte Carlo methods Agent-Based Modeling and Social System Simulation Lecturer: Dr. Nino Antulov-Fantulin Lecturer: Dr. Nino