Upload
fred-j-hickernell
View
34
Download
0
Embed Size (px)
Citation preview
Simulating the Mean Efficientlyand to a Given Tolerance
Fred J. HickernellDepartment of Applied Mathematics, Illinois Institute of Technology
[email protected] mypages.iit.edu/~hickernell
Thanks to Lan Jiang, Tony Jiménez Rugama, Jagadees Rathinavel,and the rest of the the Guaranteed Automatic Integration Library (GAIL) team
Supported by NSF-DMS-1522687
Thanks for your kind invitation
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Estimating/Simulating/Computing an Integral
Gaussian probability =
ż
[a,b]
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2 dx
option price =
ż
Rdpayoff(x)
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2
looooooomooooooon
PDF of Brownian motion at d times
dx
Bayesian β̂j =
ż
Rdβj prob(β|data)dβ =
ş
Rd βj prob(data|β) probprior(β)dβş
Rd prob(data|β) probprior(β)dβ
Sobol’ indexj =
ş
[0,1]2d
[output(x)´ output(xj, x
1´j)]output(x 1)dxdx 1
ş
[0,1]d output(x)2 dx´[ş
[0,1]d output(x)dx]2
µ =
ż
Rdg(x)dx = E[f (X)] =
ż
Rdf (x)ν(dx) =?, µ̂n =
nÿ
i=1wif (xi)
How to choose ν, txiuni=1, and twiu
ni=1 to make |µ´ µ̂n| small? (trio identity)
Given εa, how big must n be to guarantee |µ´ µ̂n| ď εa? (adaptive cubature)
2/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Estimating/Simulating/Computing an Integral
Gaussian probability =
ż
[a,b]
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2 dx
option price =
ż
Rdpayoff(x)
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2
looooooomooooooon
PDF of Brownian motion at d times
dx
Bayesian β̂j =
ż
Rdβj prob(β|data)dβ =
ş
Rd βj prob(data|β) probprior(β)dβş
Rd prob(data|β) probprior(β)dβ
Sobol’ indexj =
ş
[0,1]2d
[output(x)´ output(xj, x
1´j)]output(x 1)dxdx 1
ş
[0,1]d output(x)2 dx´[ş
[0,1]d output(x)dx]2
µ =
ż
Rdg(x)dx = E[f (X)] =
ż
Rdf (x)ν(dx) =?, µ̂n =
nÿ
i=1wif (xi)
How to choose ν, txiuni=1, and twiu
ni=1 to make |µ´ µ̂n| small? (trio identity)
Given εa, how big must n be to guarantee |µ´ µ̂n| ď εa? (adaptive cubature)
2/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Estimating/Simulating/Computing an Integral
Gaussian probability =
ż
[a,b]
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2 dx
option price =
ż
Rdpayoff(x)
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2
looooooomooooooon
PDF of Brownian motion at d times
dx
Bayesian β̂j =
ż
Rdβj prob(β|data)dβ =
ş
Rd βj prob(data|β) probprior(β)dβş
Rd prob(data|β) probprior(β)dβ
Sobol’ indexj =
ş
[0,1]2d
[output(x)´ output(xj, x
1´j)]output(x 1)dxdx 1
ş
[0,1]d output(x)2 dx´[ş
[0,1]d output(x)dx]2
µ =
ż
Rdg(x)dx = E[f (X)] =
ż
Rdf (x)ν(dx) =?, µ̂n =
nÿ
i=1wif (xi)
How to choose ν, txiuni=1, and twiu
ni=1 to make |µ´ µ̂n| small? (trio identity)
Given εa, how big must n be to guarantee |µ´ µ̂n| ď εa? (adaptive cubature)
2/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Estimating/Simulating/Computing an Integral
Gaussian probability =
ż
[a,b]
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2 dx
option price =
ż
Rdpayoff(x)
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2
looooooomooooooon
PDF of Brownian motion at d times
dx
Bayesian β̂j =
ż
Rdβj prob(β|data)dβ =
ş
Rd βj prob(data|β) probprior(β)dβş
Rd prob(data|β) probprior(β)dβ
Sobol’ indexj =
ş
[0,1]2d
[output(x)´ output(xj, x
1´j)]output(x 1)dxdx 1
ş
[0,1]d output(x)2 dx´[ş
[0,1]d output(x)dx]2
µ =
ż
Rdg(x)dx = E[f (X)] =
ż
Rdf (x)ν(dx) =?, µ̂n =
nÿ
i=1wif (xi)
How to choose ν, txiuni=1, and twiu
ni=1 to make |µ´ µ̂n| small? (trio identity)
Given εa, how big must n be to guarantee |µ´ µ̂n| ď εa? (adaptive cubature)
2/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Estimating/Simulating/Computing the MeanGaussian probability =
ż
[a,b]
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2 dx
option price =
ż
Rdpayoff(x)
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2
looooooomooooooon
PDF of Brownian motion at d times
dx
Bayesian β̂j =
ż
Rdβj prob(β|data)dβ =
ş
Rd βj prob(data|β) probprior(β)dβş
Rd prob(data|β) probprior(β)dβ
Sobol’ indexj =
ş
[0,1]2d
[output(x)´ output(xj, x
1´j)]output(x 1)dxdx 1
ş
[0,1]d output(x)2 dx´[ş
[0,1]d output(x)dx]2
µ =
ż
Rdg(x)dx = E[f (X)] =
ż
Rdf (x)ν(dx) =?, µ̂n =
nÿ
i=1wif (xi)
How to choose ν, txiuni=1, and twiu
ni=1 to make |µ´ µ̂n| small? (trio identity)
Given εa, how big must n be to guarantee |µ´ µ̂n| ď εa? (adaptive cubature) 2/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Product Rules Using Rectangular Gridsµ =
ż
Rdf (x)ν(dx) « µ̂n =
nÿ
i=1wif (xi)
If
∣∣∣∣∣ż 1
0f (x)dx´
mÿ
i=1wif (ti)
∣∣∣∣∣ = O(m´r), then∣∣∣∣ż[0,1]d
f (x)dx
´
mÿ
i1=1¨ ¨ ¨
mÿ
id=1wi1 ¨ ¨ ¨wid f (ti1 , . . . , tid)
∣∣∣∣∣= O(m´r) = O(n´r/d)
assuming rth derivatives in each direction exist.
But the computational costbecomes prohibitive for large dimensions, d:
d 1 2 5 10 100n = 8d 8 64 3.3E4 1.0E9 2.0E90
Product rules are typically a bad idea unless d is small.
3/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Product Rules Using Rectangular Gridsµ =
ż
Rdf (x)ν(dx) « µ̂n =
nÿ
i=1wif (xi)
If
∣∣∣∣∣ż 1
0f (x)dx´
mÿ
i=1wif (ti)
∣∣∣∣∣ = O(m´r), then∣∣∣∣ż[0,1]d
f (x)dx
´
mÿ
i1=1¨ ¨ ¨
mÿ
id=1wi1 ¨ ¨ ¨wid f (ti1 , . . . , tid)
∣∣∣∣∣= O(m´r) = O(n´r/d)
assuming rth derivatives in each direction exist.
But the computational costbecomes prohibitive for large dimensions, d:
d 1 2 5 10 100n = 8d 8 64 3.3E4 1.0E9 2.0E90
Product rules are typically a bad idea unless d is small.
3/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Product Rules Using Rectangular Gridsµ =
ż
Rdf (x)ν(dx) « µ̂n =
nÿ
i=1wif (xi)
If
∣∣∣∣∣ż 1
0f (x)dx´
mÿ
i=1wif (ti)
∣∣∣∣∣ = O(m´r), then∣∣∣∣ż[0,1]d
f (x)dx
´
mÿ
i1=1¨ ¨ ¨
mÿ
id=1wi1 ¨ ¨ ¨wid f (ti1 , . . . , tid)
∣∣∣∣∣= O(m´r) = O(n´r/d)
assuming rth derivatives in each direction exist. But the computational costbecomes prohibitive for large dimensions, d:
d 1 2 5 10 100n = 8d 8 64 3.3E4 1.0E9 2.0E90
Product rules are typically a bad idea unless d is small. 3/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Monte Carlo Simulation in the News
Sampling with a computer can be fastHow big is our error?
4/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
IID Monte Carloµ =
ż
Rdf (x)ν(dx) « µ̂n =
1n
nÿ
i=1f (xi), xi
IID„ ν
µ´ µ̂n =µ´ µ̂n
std(f (X))/?
nlooooooomooooooon
CNF„(0,1)
ˆ1?
nloomoon
DSC(txiu)
ˆ std(f (X))loooomoooon
VAR(f)
trio identity (Meng, 2017+)
µ´ µ̂n =
ż
Rdf (x) (ν´ ν̂n)(dx)
ν̂n =1n
nÿ
i=1δxi
P[|µ´ µ̂n| ď errn]99%for
by the ,
where is the sample variance
5/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Central Limit Theorem Stopping Rule for IID Monte Carlo
µ =
ż
Rdf (x)ν(dx) « µ̂n =
1n
nÿ
i=1f (xi), xi
IID„ ν
µ´ µ̂n =µ´ µ̂n
std(f (X))/?
nlooooooomooooooon
CNF„(0,1)
ˆ1?
nloomoon
DSC(txiu)
ˆ std(f (X))loooomoooon
VAR(f)
P[|µ´ µ̂n| ď errn] « 99%
for errn =2.58ˆ 1.2σ̂
?n
by the Central Limit Theorem (CLT),
where σ̂2 is the sample variance.
5/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Central Limit Theorem Stopping Rule for IID Monte Carlo
µ =
ż
Rdf (x)ν(dx) « µ̂n =
1n
nÿ
i=1f (xi), xi
IID„ ν
µ´ µ̂n =µ´ µ̂n
std(f (X))/?
nlooooooomooooooon
CNF„(0,1)
ˆ1?
nloomoon
DSC(txiu)
ˆ std(f (X))loooomoooon
VAR(f)
P[|µ´ µ̂n| ď errn] « 99%
for errn =2.58ˆ 1.2σ̂
?n
by the Central Limit Theorem (CLT),
where σ̂2 is the sample variance. But the CLT is only an asymptotic result, and1.2σ̂ may be an overly optimistic upper bound on σ.
5/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Berry-Esseen Stopping Rule for IID Monte Carlo
µ =
ż
Rdf (x)ν(dx) « µ̂n =
1n
nÿ
i=1f (xi), xi
IID„ ν
µ´ µ̂n =µ´ µ̂n
std(f (X))/?
nlooooooomooooooon
CNF„(0,1)
ˆ1?
nloomoon
DSC(txiu)
ˆ std(f (X))loooomoooon
VAR(f)
P[|µ´ µ̂n| ď errn] ě 99%for Φ
(´?
n errn /(1.2σ̂nσ))
+ ∆n(´?
n errn /(1.2σ̂nσ), κmax) = 0.0025
by the Berry-Esseen Inequality,
where σ̂2nσ
is the sample variance using an independent sample from that used tosimulate the mean, and provided that kurt(f (X)) ď κmax(nσ) (H. et al., 2013;Jiang, 2016)
5/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Normally n should be a power of 2
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f . Let tω(k)uk be someweights. Then
µ´ µ̂n =
´ÿ
0‰kPdual
pf (k)∥∥∥! pf(k)ω(k)
)
k
∥∥∥2‖tω(k)u0‰kPdual‖2
loooooooooooooooooooomoooooooooooooooooooon
CNFP[´1,1]
ˆ ‖tω(k)u0‰kPdual‖2loooooooooomoooooooooon
DSC(txiuni=1)=O(n´1+ε)
ˆ
∥∥∥∥∥#
pf (k)ω(k)
+
k
∥∥∥∥∥2
looooooomooooooon
VAR(f)
6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Adaptive Low Discrepancy Sampling Cubature
µ =
ż
[0,1]df (x)dx
µ̂n =1n
nÿ
i=1f (xi), xi Sobol’ or lattice
Let tpf (k)uk denote the coefficients of theFourier Walsh or complex exponentialexpansion of f .
Let tω(k)uk be someweights. Then
Assuming that the pf (k) do not decay erratically as kÑ∞, the discretetransform,
rfn(k)(
k, may be used to bound the error reliably (H. and Jiménez
Rugama, 2016; Jiménez Rugama and H., 2016; H. et al., 2017+):
|µ´ µ̂n| ď errn := C(n)ÿ
certaink
∣∣∣rfn(k)∣∣∣6/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Bayesian Cubature—f Is Randomµ =
ż
Rdf (x)ν(dx) « µ̂n =
nÿ
i=1wi f (xi)
Assume f „ GP(0, s2Cθ) (Diaconis, 1988;O’Hagan, 1991; Ritter, 2000; Rasmussen andGhahramani, 2003)
c0 =
ż
RdˆRdCθ(x, t)ν(dx)ν(dt)
c =
(ż
RdCθ(xi, t)ν(dt)
)n
i=1, C =
(Cθ(xi, xj)
)ni,j=1
Choosingw =(wi)n
i=1 = C´1c is optimal
µ´ µ̂n =µ´ µ̂n
b(c0 ´ cTC´1c
)yTC´1y
nlooooooooooooooomooooooooooooooon
CNF„N(0,1)
ˆa
c0 ´ cTC´1cloooooooomoooooooon
DSC
ˆ
c
yTC´1y
nlooooomooooon
VAR(f)
where y =(f (xi)
)ni=1.
But, θ needs to be inferred (by MLE), and C´1 typicallyrequires O(n3) operations
7/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Bayesian Cubature—f Is Random
µ =
ż
Rdf (x)ν(dx) « µ̂n =
nÿ
i=1wi f (xi)
Assume f „ GP(0, s2Cθ) (Diaconis, 1988;O’Hagan, 1991; Ritter, 2000; Rasmussen andGhahramani, 2003)
c0 =
ż
RdˆRdCθ(x, t)ν(dx)ν(dt)
c =
(ż
RdCθ(xi, t)ν(dt)
)n
i=1, C =
(Cθ(xi, xj)
)ni,j=1
Choosingw =(wi)n
i=1 = C´1c is optimal
P[|µ´ µ̂n| ď errn] = 99% for errn = 2.58c(
c0 ´ cTC´1c) yTC´1y
n
where y =(f (xi)
)ni=1. But, θ needs to be inferred (by MLE), and C´1 typically
requires O(n3) operations7/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Gaussian Probability
µ =
ż
[a,b]
exp(´ 1
2tTΣ´1t
)a
(2π)d det(Σ)dt Genz (1993)
=
ż
[0,1]d´1f (x)dx
For some typical choice of a, b, Σ, d = 3, εa = 0; µ « 0.6763Worst 10% Worst 10%
εr Method % Accuracy n Time (s)IID Monte Carlo 100% 8.1E4 1.8E´2
1E´2 Sobol’ Sampling 100% 1.0E3 5.1E´3Bayesian Lattice 100% 1.0E3 2.8E´3
IID Monte Carlo 100% 2.0E6 3.8E´11E´3 Sobol’ Sampling 100% 2.0E3 7.7E´3
Bayesian Lattice 100% 1.0E3 2.8E´3
1E´4 Sobol’ Sampling 100% 1.6E4 1.8E´2Bayesian Lattice 100% 8.2E3 1.4E´2
Bayesian lattice cubature uses covariance kernel C for which C is circulant,and operations on C require only O(n log(n)) operations 8/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Asian Option Pricing
fair price =
ż
Rde´rT max
1d
dÿ
j=1Sj ´ K, 0
e´xTΣ´1x/2
(2π)d/2 |Σ|1/2 dx « $13.12
Sj = S0e(r´σ2/2)jT/d+σxj = stock price at time jT/d,
Σ =(
min(i, j)T/d)d
i,j=1
Worst 10% Worst 10%εa = 1E´4 Method % Accuracy n Time (s)
Sobol’ Sampling 100% 2.1E6 4.3Sobol’ Sampling w/ control variates 97% 1.0E6 2.1
The coefficient of the control variate for low discrepancy sampling is different thanfor IID Monte Carlo (H. et al., 2005; H. et al., 2017+)
9/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Sobol’ IndicesY = output(X), where X „ U[0, 1]d; Sobol’ Indexj(µ) describes how muchcoordinate j of input X influences output Y (Sobol’, 1990; 2001):
Sobol’ Indexj(µ) :=µ1
µ2 ´ µ23, j = 1, . . . , d
µ1 :=
ż
[0,1)2d[output(x)´ output(xj, x
1´j)]output(x 1)dxdx 1
µ2 :=
ż
[0,1)doutput(x)2 dx, µ3 :=
ż
[0,1)doutput(x)dx
output(x) = ´x1 + x1x2 ´ x1x2x3 + ¨ ¨ ¨+ x1x2x3x4x5x6 (Bratley et al., 1992)
εa = 1E´3, εr = 0 j 1 2 3 4 5 6n 65 536 32 768 16 384 16 384 2 048 2 048
Sobol’ Indexj 0.6529 0.1791 0.0370 0.0133 0.0015 0.0015{Sobol’ Indexj 0.6528 0.1792 0.0363 0.0126 0.0010 0.0012
Sobol’ Indexj(pµn) 0.6492 0.1758 0.0308 0.0083 0.0018 0.0039
10/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Summary
The error in simulating the mean can be decomposed as a trio identity(Meng, 2017+; H., 2017+)Knowing when to stop a simulation of the mean is not trivial (H. et al., 2017+)The Berry-Esseen inequality can tell us when to stop an IID simulationFourier analysis can tell us when to stop a low discrepancy simulationBayesian cubature can tell us when to stop a simulation if you can afford thecomputational costAll methods can be fooled by nasty functions, fRelative error tolerances and problems involving functions of integrals canbe handled (H. et al., 2017+)Our algorithms are implemented in the Guaranteed Automatic IntegrationLibrary (GAIL) (Choi et al., 2013–2015), which is under continuousdevelopment
11/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Upcoming SAMSI Quasi-Monte Carlo Program
12/16
Thank you
Slides available at www.slideshare.net/fjhickernell/tulane-march-2017-talk
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
References I
Bratley, P., B. L. Fox, and H. Niederreiter. 1992. Implementation and tests of low-discrepancysequences, ACM Trans. Model. Comput. Simul. 2, 195–213.
Choi, S.-C. T., Y. Ding, F. J. H., L. Jiang, Ll. A. Jiménez Rugama, X. Tong, Y. Zhang, and X. Zhou.2013–2015. GAIL: Guaranteed Automatic Integration Library (versions 1.0–2.1).
Cools, R. and D. Nuyens (eds.) 2016. Monte Carlo and quasi-Monte Carlo methods: MCQMC,Leuven, Belgium, April 2014, Springer Proceedings in Mathematics and Statistics, vol. 163,Springer-Verlag, Berlin.
Diaconis, P. 1988. Bayesian numerical analysis, Statistical decision theory and related topics IV,Papers from the 4th Purdue symp., West Lafayette, Indiana 1986, pp. 163–175.
Genz, A. 1993. Comparison of methods for the computation of multivariate normal probabilities,Computing Science and Statistics 25, 400–405.
H., F. J. 2017+. Error analysis of quasi-Monte Carlo methods. submitted for publication,arXiv:1702.01487.
H., F. J., L. Jiang, Y. Liu, and A. B. Owen. 2013. Guaranteed conservative fixed width confidenceintervals via Monte Carlo sampling, Monte Carlo and quasi-Monte Carlo methods 2012, pp. 105–128.
H., F. J. and Ll. A. Jiménez Rugama. 2016. Reliable adaptive cubature using digital sequences,Monte Carlo and quasi-Monte Carlo methods: MCQMC, Leuven, Belgium, April 2014, pp. 367–383.arXiv:1410.8615 [math.NA].
14/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
References IIH., F. J., Ll. A. Jiménez Rugama, and D. Li. 2017+. Adaptive quasi-Monte Carlo methods. submittedfor publication, arXiv:1702.01491 [math.NA].
H., F. J., C. Lemieux, and A. B. Owen. 2005. Control variates for quasi-Monte Carlo, Statist. Sci. 20,1–31.
Jiang, L. 2016. Guaranteed adaptive Monte Carlo methods for estimating means of randomvariables, Ph.D. Thesis.
Jiménez Rugama, Ll. A. and F. J. H. 2016. Adaptive multidimensional integration based on rank-1lattices, Monte Carlo and quasi-Monte Carlo methods: MCQMC, Leuven, Belgium, April 2014,pp. 407–422. arXiv:1411.1966.
Meng, X. 2017+. Statistical paradises and paradoxes in big data. in preparation.
O’Hagan, A. 1991. Bayes-Hermite quadrature, J. Statist. Plann. Inference 29, 245–260.
Rasmussen, C. E. and Z. Ghahramani. 2003. Bayesian Monte Carlo, Advances in Neural InformationProcessing Systems, pp. 489–496.
Ritter, K. 2000. Average-case analysis of numerical problems, Lecture Notes in Mathematics,vol. 1733, Springer-Verlag, Berlin.
Sobol’, I. M. 1990. On sensitivity estimation for nonlinear mathematical models, Matem. Mod. 2,no. 1, 112–118.
. 2001. Global sensitivity indices for nonlinear mathematical models and their monte carloestimates, Math. Comput. Simul. 55, no. 1-3, 271–280.
15/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Maximum Likelihood Estimation of the Covariance Kernelf „ GP(0, s2Cθ), Cθ =
(Cθ(xi, xj)
)ni,j=1
y =(f (xi)
)ni=1, µ̂n = cT
θ̂C´1θ̂y
θ̂ = argminθ
yTC´1θ y
[det(C´1θ )]1/n
P[|µ´ µ̂n| ď errn] = 99% for errn =2.58?
n
b(c0,θ̂ ´ c
Tθ̂C´1θ̂cθ̂)yTC´1
θ̂y
There is a de-randomized interpretation of Bayesian cubature (H., 2017+)
f P Hilbert space w/ reproducing kernel Cθ and with best interpolant rfy
θ̂ = argminθ
yTC´1θ y
[det(C´1θ )]1/n
= argminθ
vol(
z P Rn :∥∥rfz‖θ ď ‖rfy‖θ
(
)|µ´ µ̂n| ď
2.58?
n
b
c0,θ̂ ´ cTθ̂C´1θ̂cθ̂
loooooooooomoooooooooon
‖error representer‖θ̂
b
yTC´1θ̂y
looooomooooon
‖rfy‖θ̂
if∥∥f ´rfy
∥∥θ̂ď
2.58∥∥rf∥∥
θ̂?n
16/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Maximum Likelihood Estimation of the Covariance Kernelf „ GP(0, s2Cθ), Cθ =
(Cθ(xi, xj)
)ni,j=1
y =(f (xi)
)ni=1, µ̂n = cT
θ̂C´1θ̂y
θ̂ = argminθ
yTC´1θ y
[det(C´1θ )]1/n
P[|µ´ µ̂n| ď errn] = 99% for errn =2.58?
n
b(c0,θ̂ ´ c
Tθ̂C´1θ̂cθ̂)yTC´1
θ̂y
There is a de-randomized interpretation of Bayesian cubature (H., 2017+)
f P Hilbert space w/ reproducing kernel Cθ and with best interpolant rfy
θ̂ = argminθ
yTC´1θ y
[det(C´1θ )]1/n
= argminθ
vol(
z P Rn :∥∥rfz‖θ ď ‖rfy‖θ
(
)|µ´ µ̂n| ď
2.58?
n
b
c0,θ̂ ´ cTθ̂C´1θ̂cθ̂
loooooooooomoooooooooon
‖error representer‖θ̂
b
yTC´1θ̂y
looooomooooon
‖rfy‖θ̂
if∥∥f ´rfy
∥∥θ̂ď
2.58∥∥rf∥∥
θ̂?n
16/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Maximum Likelihood Estimation of the Covariance Kernelf „ GP(0, s2Cθ), Cθ =
(Cθ(xi, xj)
)ni,j=1
y =(f (xi)
)ni=1, µ̂n = cT
θ̂C´1θ̂y
θ̂ = argminθ
yTC´1θ y
[det(C´1θ )]1/n
P[|µ´ µ̂n| ď errn] = 99% for errn =2.58?
n
b(c0,θ̂ ´ c
Tθ̂C´1θ̂cθ̂)yTC´1
θ̂y
There is a de-randomized interpretation of Bayesian cubature (H., 2017+)
f P Hilbert space w/ reproducing kernel Cθ and with best interpolant rfy
θ̂ = argminθ
yTC´1θ y
[det(C´1θ )]1/n
= argminθ
vol(
z P Rn :∥∥rfz‖θ ď ‖rfy‖θ
(
)
|µ´ µ̂n| ď2.58?
n
b
c0,θ̂ ´ cTθ̂C´1θ̂cθ̂
loooooooooomoooooooooon
‖error representer‖θ̂
b
yTC´1θ̂y
looooomooooon
‖rfy‖θ̂
if∥∥f ´rfy
∥∥θ̂ď
2.58∥∥rf∥∥
θ̂?n
16/16
Introduction IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Examples References
Maximum Likelihood Estimation of the Covariance Kernelf „ GP(0, s2Cθ), Cθ =
(Cθ(xi, xj)
)ni,j=1
y =(f (xi)
)ni=1, µ̂n = cT
θ̂C´1θ̂y
θ̂ = argminθ
yTC´1θ y
[det(C´1θ )]1/n
P[|µ´ µ̂n| ď errn] = 99% for errn =2.58?
n
b(c0,θ̂ ´ c
Tθ̂C´1θ̂cθ̂)yTC´1
θ̂y
There is a de-randomized interpretation of Bayesian cubature (H., 2017+)
f P Hilbert space w/ reproducing kernel Cθ and with best interpolant rfy
θ̂ = argminθ
yTC´1θ y
[det(C´1θ )]1/n
= argminθ
vol(
z P Rn :∥∥rfz‖θ ď ‖rfy‖θ
(
)|µ´ µ̂n| ď
2.58?
n
b
c0,θ̂ ´ cTθ̂C´1θ̂cθ̂
loooooooooomoooooooooon
‖error representer‖θ̂
b
yTC´1θ̂y
looooomooooon
‖rfy‖θ̂
if∥∥f ´rfy
∥∥θ̂ď
2.58∥∥rf∥∥
θ̂?n
16/16