Vanilla Rao–Blackwellisation of
Metropolis–Hastings algorithms
Christian P. Robert
Universite Paris-Dauphine, IuF, and CRESTJoint works with Randal Douc, Pierre Jacob and Murray Smith
LGM2012, Trondheim, May 30, 2012
1 / 32
Main themes
1 Rao–Blackwellisation on MCMC
2 Can be performed in any Hastings Metropolis algorithm
3 Asymptotically more efficient than usual MCMC with acontrolled additional computing
4 Takes advantage of parallel capacities at a very basic level(GPUs)
2 / 32
Main themes
1 Rao–Blackwellisation on MCMC
2 Can be performed in any Hastings Metropolis algorithm
3 Asymptotically more efficient than usual MCMC with acontrolled additional computing
4 Takes advantage of parallel capacities at a very basic level(GPUs)
2 / 32
Main themes
1 Rao–Blackwellisation on MCMC
2 Can be performed in any Hastings Metropolis algorithm
3 Asymptotically more efficient than usual MCMC with acontrolled additional computing
4 Takes advantage of parallel capacities at a very basic level(GPUs)
2 / 32
Main themes
1 Rao–Blackwellisation on MCMC
2 Can be performed in any Hastings Metropolis algorithm
3 Asymptotically more efficient than usual MCMC with acontrolled additional computing
4 Takes advantage of parallel capacities at a very basic level(GPUs)
2 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Metropolis Hastings algorithm
1 We wish to approximate
I =
∫h(x)π(x)dx∫π(x)dx
=
∫
h(x)π(x)dx
2 π(x) is known but not∫π(x)dx.
3 Approximate I with δ = 1n
∑nt=1 h(x
(t)) where (x(t)) is a Markov
chain with limiting distribution π.
4 Convergence obtained from Law of Large Numbers or CLT forMarkov chains.
3 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Metropolis Hastings algorithm
1 We wish to approximate
I =
∫h(x)π(x)dx∫π(x)dx
=
∫
h(x)π(x)dx
2 π(x) is known but not∫π(x)dx.
3 Approximate I with δ = 1n
∑nt=1 h(x
(t)) where (x(t)) is a Markov
chain with limiting distribution π.
4 Convergence obtained from Law of Large Numbers or CLT forMarkov chains.
3 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Metropolis Hastings algorithm
1 We wish to approximate
I =
∫h(x)π(x)dx∫π(x)dx
=
∫
h(x)π(x)dx
2 π(x) is known but not∫π(x)dx.
3 Approximate I with δ = 1n
∑nt=1 h(x
(t)) where (x(t)) is a Markov
chain with limiting distribution π.
4 Convergence obtained from Law of Large Numbers or CLT forMarkov chains.
3 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Metropolis Hastings algorithm
1 We wish to approximate
I =
∫h(x)π(x)dx∫π(x)dx
=
∫
h(x)π(x)dx
2 π(x) is known but not∫π(x)dx.
3 Approximate I with δ = 1n
∑nt=1 h(x
(t)) where (x(t)) is a Markov
chain with limiting distribution π.
4 Convergence obtained from Law of Large Numbers or CLT forMarkov chains.
3 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Metropolis Hasting Algorithm
Suppose that x(t) is drawn.
1 Simulate yt ∼ q(·|x(t)).
2 Set x(t+1) = yt with probability
α(x(t), yt) = min
{
1,π(yt)
π(x(t))
q(x(t)|yt)q(yt|x(t))
}
Otherwise, set x(t+1) = x(t) .
3 α is such that the detailed balance equation is satisfied: ⊲ π is thestationary distribution of (x(t)).
◮ The accepted candidates are simulated with the rejection algorithm.
4 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Metropolis Hasting Algorithm
Suppose that x(t) is drawn.
1 Simulate yt ∼ q(·|x(t)).
2 Set x(t+1) = yt with probability
α(x(t), yt) = min
{
1,π(yt)
π(x(t))
q(x(t)|yt)q(yt|x(t))
}
Otherwise, set x(t+1) = x(t) .
3 α is such that the detailed balance equation is satisfied: ⊲ π is thestationary distribution of (x(t)).
◮ The accepted candidates are simulated with the rejection algorithm.
4 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Metropolis Hasting Algorithm
Suppose that x(t) is drawn.
1 Simulate yt ∼ q(·|x(t)).
2 Set x(t+1) = yt with probability
α(x(t), yt) = min
{
1,π(yt)
π(x(t))
q(x(t)|yt)q(yt|x(t))
}
Otherwise, set x(t+1) = x(t) .
3 α is such that the detailed balance equation is satisfied:
π(x)q(y|x)α(x, y) = π(y)q(x|y)α(y, x).
⊲ π is the stationary distribution of (x(t)).
◮ The accepted candidates are simulated with the rejection algorithm.
4 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Metropolis Hasting Algorithm
Suppose that x(t) is drawn.
1 Simulate yt ∼ q(·|x(t)).
2 Set x(t+1) = yt with probability
α(x(t), yt) = min
{
1,π(yt)
π(x(t))
q(x(t)|yt)q(yt|x(t))
}
Otherwise, set x(t+1) = x(t) .
3 α is such that the detailed balance equation is satisfied:
π(x)q(y|x)α(x, y) = π(y)q(x|y)α(y, x).
⊲ π is the stationary distribution of (x(t)).
◮ The accepted candidates are simulated with the rejection algorithm.
4 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Some properties of the HM algorithm
1 Alternative representation of the estimator δ is
δ =1
n
n∑
t=1
h(x(t)) =1
n
Mn∑
i=1
nih(zi) ,
where
zi’s are the accepted yj ’s,Mn is the number of accepted yj ’s till time n,ni is the number of times zi appears in the sequence (x(t))t.
5 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
q(·|zi) =α(zi, ·) q(·|zi)
p(zi)≤ q(·|zi)
p(zi),
where p(zi) =∫α(zi, y) q(y|zi)dy. To simulate from q(·|zi):
1 Propose a candidate y ∼ q(·|zi)2 Accept with probability
q(y|zi)/(
q(y|zi)p(zi)
)
= α(zi, y)
Otherwise, reject it and starts again.
◮ this is the transition of the HM algorithm.The transition kernel qadmits π as a stationary distribution:
π(x)q(y|x) =
6 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
q(·|zi) =α(zi, ·) q(·|zi)
p(zi)≤ q(·|zi)
p(zi),
where p(zi) =∫α(zi, y) q(y|zi)dy. To simulate from q(·|zi):
1 Propose a candidate y ∼ q(·|zi)2 Accept with probability
q(y|zi)/(
q(y|zi)p(zi)
)
= α(zi, y)
Otherwise, reject it and starts again.
◮ this is the transition of the HM algorithm.The transition kernel qadmits π as a stationary distribution:
π(x)q(y|x) =
6 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
q(·|zi) =α(zi, ·) q(·|zi)
p(zi)≤ q(·|zi)
p(zi),
where p(zi) =∫α(zi, y) q(y|zi)dy. To simulate from q(·|zi):
1 Propose a candidate y ∼ q(·|zi)2 Accept with probability
q(y|zi)/(
q(y|zi)p(zi)
)
= α(zi, y)
Otherwise, reject it and starts again.
◮ this is the transition of the HM algorithm.The transition kernel qadmits π as a stationary distribution:
π(x)q(y|x) =
6 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
q(·|zi) =α(zi, ·) q(·|zi)
p(zi)≤ q(·|zi)
p(zi),
where p(zi) =∫α(zi, y) q(y|zi)dy. To simulate from q(·|zi):
1 Propose a candidate y ∼ q(·|zi)2 Accept with probability
q(y|zi)/(
q(y|zi)p(zi)
)
= α(zi, y)
Otherwise, reject it and starts again.
◮ this is the transition of the HM algorithm.The transition kernel qadmits π as a stationary distribution:
π(x)q(y|x) = π(x)p(x)∫π(u)p(u)du
︸ ︷︷ ︸
π(x)
α(x, y)q(y|x)p(x)
︸ ︷︷ ︸
q(y|x)
6 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
q(·|zi) =α(zi, ·) q(·|zi)
p(zi)≤ q(·|zi)
p(zi),
where p(zi) =∫α(zi, y) q(y|zi)dy. To simulate from q(·|zi):
1 Propose a candidate y ∼ q(·|zi)2 Accept with probability
q(y|zi)/(
q(y|zi)p(zi)
)
= α(zi, y)
Otherwise, reject it and starts again.
◮ this is the transition of the HM algorithm.The transition kernel qadmits π as a stationary distribution:
π(x)q(y|x) = π(x)α(x, y)q(y|x)∫π(u)p(u)du
6 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
q(·|zi) =α(zi, ·) q(·|zi)
p(zi)≤ q(·|zi)
p(zi),
where p(zi) =∫α(zi, y) q(y|zi)dy. To simulate from q(·|zi):
1 Propose a candidate y ∼ q(·|zi)2 Accept with probability
q(y|zi)/(
q(y|zi)p(zi)
)
= α(zi, y)
Otherwise, reject it and starts again.
◮ this is the transition of the HM algorithm.The transition kernel qadmits π as a stationary distribution:
π(x)q(y|x) = π(y)α(y, x)q(x|y)∫π(u)p(u)du
6 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
q(·|zi) =α(zi, ·) q(·|zi)
p(zi)≤ q(·|zi)
p(zi),
where p(zi) =∫α(zi, y) q(y|zi)dy. To simulate from q(·|zi):
1 Propose a candidate y ∼ q(·|zi)2 Accept with probability
q(y|zi)/(
q(y|zi)p(zi)
)
= α(zi, y)
Otherwise, reject it and starts again.
◮ this is the transition of the HM algorithm.The transition kernel qadmits π as a stationary distribution:
π(x)q(y|x) = π(y)q(x|y) ,
6 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Lemma (Douc & X., AoS, 2011)
The sequence (zi, ni) satisfies
1 (zi, ni)i is a Markov chain;
2 zi+1 and ni are independent given zi;
3 ni is distributed as a geometric random variable with probabilityparameter
p(zi) :=
∫
α(zi, y) q(y|zi) dy ; (1)
4 (zi)i is a Markov chain with transition kernel Q(z, dy) = q(y|z)dyand stationary distribution π such that
q(·|z) ∝ α(z, ·) q(·|z) and π(·) ∝ π(·)p(·) .
7 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Lemma (Douc & X., AoS, 2011)
The sequence (zi, ni) satisfies
1 (zi, ni)i is a Markov chain;
2 zi+1 and ni are independent given zi;
3 ni is distributed as a geometric random variable with probabilityparameter
p(zi) :=
∫
α(zi, y) q(y|zi) dy ; (1)
4 (zi)i is a Markov chain with transition kernel Q(z, dy) = q(y|z)dyand stationary distribution π such that
q(·|z) ∝ α(z, ·) q(·|z) and π(·) ∝ π(·)p(·) .
7 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Lemma (Douc & X., AoS, 2011)
The sequence (zi, ni) satisfies
1 (zi, ni)i is a Markov chain;
2 zi+1 and ni are independent given zi;
3 ni is distributed as a geometric random variable with probabilityparameter
p(zi) :=
∫
α(zi, y) q(y|zi) dy ; (1)
4 (zi)i is a Markov chain with transition kernel Q(z, dy) = q(y|z)dyand stationary distribution π such that
q(·|z) ∝ α(z, ·) q(·|z) and π(·) ∝ π(·)p(·) .
7 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Lemma (Douc & X., AoS, 2011)
The sequence (zi, ni) satisfies
1 (zi, ni)i is a Markov chain;
2 zi+1 and ni are independent given zi;
3 ni is distributed as a geometric random variable with probabilityparameter
p(zi) :=
∫
α(zi, y) q(y|zi) dy ; (1)
4 (zi)i is a Markov chain with transition kernel Q(z, dy) = q(y|z)dyand stationary distribution π such that
q(·|z) ∝ α(z, ·) q(·|z) and π(·) ∝ π(·)p(·) .
7 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Old bottle, new wine [or vice-versa]
zi−1
8 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Old bottle, new wine [or vice-versa]
zi−1 zi
ni−1
indep
indep
8 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Old bottle, new wine [or vice-versa]
zi−1 zi zi+1
ni−1 ni
indep
indep
indep
indep
8 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Old bottle, new wine [or vice-versa]
zi−1 zi zi+1
ni−1 ni
indep
indep
indep
indep
δ =1
n
n∑
t=1
h(x(t)) =1
n
Mn∑
i=1
nih(zi) .
8 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Old bottle, new wine [or vice-versa]
zi−1 zi zi+1
ni−1 ni
indep
indep
indep
indep
δ =1
n
n∑
t=1
h(x(t)) =1
n
Mn∑
i=1
nih(zi) .
8 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Importance sampling perspective
1 A natural idea:
δ∗ =1
n
Mn∑
i=1
h(zi)
p(zi),
9 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Importance sampling perspective
1 A natural idea:
δ∗ ≃
∑Mn
i=1
h(zi)
p(zi)∑Mn
i=1
1
p(zi)
=
∑Mn
i=1
π(zi)
π(zi)h(zi)
∑Mn
i=1
π(zi)
π(zi)
.
9 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Importance sampling perspective
1 A natural idea:
δ∗ ≃
∑Mn
i=1
h(zi)
p(zi)∑Mn
i=1
1
p(zi)
=
∑Mn
i=1
π(zi)
π(zi)h(zi)
∑Mn
i=1
π(zi)
π(zi)
.
2 But p not available in closed form.
9 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Importance sampling perspective
1 A natural idea:
δ∗ ≃
∑Mn
i=1
h(zi)
p(zi)∑Mn
i=1
1
p(zi)
=
∑Mn
i=1
π(zi)
π(zi)h(zi)
∑Mn
i=1
π(zi)
π(zi)
.
2 But p not available in closed form.
3 The geometric ni is the replacement, an obvious solution that isused in the original Metropolis–Hastings estimate sinceE[ni] = 1/p(zi).
9 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
The Bernoulli factory
The crude estimate of 1/p(zi),
ni = 1 +∞∑
j=1
∏
ℓ≤j
I {uℓ ≥ α(zi, yℓ)} ,
can be improved:
Lemma (Douc & X., AoS, 2011)
If (yj)j is an iid sequence with distribution q(y|zi), the quantity
ξi = 1 +
∞∑
j=1
∏
ℓ≤j
{1− α(zi, yℓ)}
is an unbiased estimator of 1/p(zi) which variance, conditional on zi, is
lower than the conditional variance of ni, {1− p(zi)}/p2(zi).
10 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Rao-Blackwellised, for sure?
ξi = 1 +∞∑
j=1
∏
ℓ≤j
{1− α(zi, yℓ)}
1 Infinite sum but finite with at least positive probability:
α(x(t), yt) = min
{
1,π(yt)
π(x(t))
q(x(t)|yt)q(yt|x(t))
}
For example: take a symmetric random walk as a proposal.
2 What if we wish to be sure that the sum is finite?
Finite horizon k version:
ξki = 1 +
∞∑
j=1
∏
1≤ℓ≤k∧j
{1− α(zi, yj)}∏
k+1≤ℓ≤j
I {uℓ ≥ α(zi, yℓ)}
11 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Rao-Blackwellised, for sure?
ξi = 1 +∞∑
j=1
∏
ℓ≤j
{1− α(zi, yℓ)}
1 Infinite sum but finite with at least positive probability:
α(x(t), yt) = min
{
1,π(yt)
π(x(t))
q(x(t)|yt)q(yt|x(t))
}
For example: take a symmetric random walk as a proposal.
2 What if we wish to be sure that the sum is finite?
Finite horizon k version:
ξki = 1 +
∞∑
j=1
∏
1≤ℓ≤k∧j
{1− α(zi, yj)}∏
k+1≤ℓ≤j
I {uℓ ≥ α(zi, yℓ)}
11 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Variance improvement
Proposition (Douc & X., AoS, 2011)
If (yj)j is an iid sequence with distribution q(y|zi) and (uj)j is an iiduniform sequence, for any k ≥ 0, the quantity
ξki = 1 +
∞∑
j=1
∏
1≤ℓ≤k∧j
{1− α(zi, yj)}∏
k+1≤ℓ≤j
I {uℓ ≥ α(zi, yℓ)}
is an unbiased estimator of 1/p(zi) with an almost sure finite number ofterms.
12 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Variance improvement
Proposition (Douc & X., AoS, 2011)
If (yj)j is an iid sequence with distribution q(y|zi) and (uj)j is an iiduniform sequence, for any k ≥ 0, the quantity
ξki = 1 +
∞∑
j=1
∏
1≤ℓ≤k∧j
{1− α(zi, yj)}∏
k+1≤ℓ≤j
I {uℓ ≥ α(zi, yℓ)}
is an unbiased estimator of 1/p(zi) with an almost sure finite number of
terms. Moreover, for k ≥ 1,
V
[
ξki
∣
∣
∣zi
]
=1− p(zi)
p2(zi)− 1− (1− 2p(zi) + r(zi))
k
2p(zi)− r(zi)
(
2− p(zi)
p2(zi)
)
(p(zi)− r(zi)) ,
where p(zi) :=∫
α(zi, y) q(y|zi) dy. and r(zi) :=∫
α2(zi, y) q(y|zi) dy.
12 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Variance improvement
Proposition (Douc & X., AoS, 2011)
If (yj)j is an iid sequence with distribution q(y|zi) and (uj)j is an iiduniform sequence, for any k ≥ 0, the quantity
ξki = 1 +
∞∑
j=1
∏
1≤ℓ≤k∧j
{1− α(zi, yj)}∏
k+1≤ℓ≤j
I {uℓ ≥ α(zi, yℓ)}
is an unbiased estimator of 1/p(zi) with an almost sure finite number of
terms. Therefore, we have
V
[
ξi
∣∣∣ zi
]
≤ V
[
ξki
∣∣∣ zi
]
≤ V
[
ξ0i
∣∣∣ zi
]
= V [ni| zi] .
12 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
zi−1
ξki = 1 +∞∑
j=1
∏
1≤ℓ≤k∧j
{1− α(zi, yj)}∏
k+1≤ℓ≤j
I {uℓ ≥ α(zi, yℓ)}
13 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
zi−1 zi
ξki−1
not indep
not indep
ξki = 1 +∞∑
j=1
∏
1≤ℓ≤k∧j
{1− α(zi, yj)}∏
k+1≤ℓ≤j
I {uℓ ≥ α(zi, yℓ)}
13 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
zi−1 zi zi+1
ξki−1 ξki
not indep
not indep
not indep
not indep
ξki = 1 +∞∑
j=1
∏
1≤ℓ≤k∧j
{1− α(zi, yj)}∏
k+1≤ℓ≤j
I {uℓ ≥ α(zi, yℓ)}
13 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
zi−1 zi zi+1
ξki−1 ξki
not indep
not indep
not indep
not indep
δkM =
∑Mi=1 ξ
ki h(zi)
∑Mi=1 ξ
ki
.
13 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
zi−1 zi zi+1
ξki−1 ξki
not indep
not indep
not indep
not indep
δkM =
∑Mi=1 ξ
ki h(zi)
∑Mi=1 ξ
ki
.
13 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Let
δkM =
∑Mi=1 ξ
ki h(zi)
∑Mi=1 ξ
ki
.
For any positive function ϕ, we denote Cϕ = {h; |h/ϕ|∞ <∞}.
14 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Let
δkM =
∑Mi=1 ξ
ki h(zi)
∑Mi=1 ξ
ki
.
For any positive function ϕ, we denote Cϕ = {h; |h/ϕ|∞ <∞}. Assumethat there exists a positive function ϕ ≥ 1 such that
∀h ∈ Cϕ,∑Mi=1
h(zi)/p(zi)∑Mi=1
1/p(zi)
P−→ π(h)
Theorem (Douc & X., AoS, 2011)
Under the assumption that π(p) > 0, the following convergence property holds:
i) If h is in Cϕ, then
δkMP−→M→∞ π(h) (◮Consistency)
14 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Let
δkM =
∑Mi=1 ξ
ki h(zi)
∑Mi=1 ξ
ki
.
For any positive function ϕ, we denote Cϕ = {h; |h/ϕ|∞ <∞}.Assume that there exists a positive function ψ such that
∀h ∈ Cψ ,√M
(
∑Mi=1
h(zi)/p(zi)∑Mi=1
1/p(zi)− π(h)
)
L−→ N (0,Γ(h))
Theorem (Douc & X., AoS, 2011)
Under the assumption that π(p) > 0, the following convergence property holds:
ii) If, in addition, h2/p ∈ Cϕ and h ∈ Cψ , then
√M(δkM − π(h))
L−→M→∞ N (0, Vk[h− π(h)]) , (◮Clt)
where Vk(h) := π(p)∫
π(dz)V[
ξki
∣
∣
∣z
]
h2(z)p(z) + Γ(h) .
14 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
We will need some additional assumptions. Assume a maximal inequalityfor the Markov chain (zi)i: there exists a measurable function ζ such thatfor any starting point x,
∀h ∈ Cζ , Px
∣∣∣∣∣∣
sup0≤i≤n
i∑
j=0
[h(zi)− π(h)]
∣∣∣∣∣∣
> ǫ
≤ nCh(x)
ǫ2
Theorem (Douc & X., AoS, 2011)
Assume that h is such that h/p ∈ Cζ and {Ch/p, h2/p2} ⊂ Cφ. Assume
moreover that
√M
(δ0M − π(h)
) L−→ N (0, V0[h− π(h)]) .
Then, for any starting point x,
√
Mn
(∑nt=1 h(x
(t))
n− π(h)
)
n→+∞−→ N (0, V0[h− π(h)]) ,
where Mn is defined by15 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
We will need some additional assumptions. Assume a maximal inequalityfor the Markov chain (zi)i: there exists a measurable function ζ such thatfor any starting point x,
∀h ∈ Cζ , Px
∣∣∣∣∣∣
sup0≤i≤n
i∑
j=0
[h(zi)− π(h)]
∣∣∣∣∣∣
> ǫ
≤ nCh(x)
ǫ2
Moreover, assume that ∃φ ≥ 1 such that for any starting point x,
∀h ∈ Cφ, Qn(x, h)P−→ π(h) = π(ph)/π(p) ,
Theorem (Douc & X., AoS, 2011)
Assume that h is such that h/p ∈ Cζ and {Ch/p, h2/p2} ⊂ Cφ. Assume
moreover that
√M
(δ0M − π(h)
) L−→ N (0, V0[h− π(h)]) .
Then, for any starting point x,
√(∑n
h(x(t)))
n→+∞15 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
We will need some additional assumptions. Assume a maximal inequalityfor the Markov chain (zi)i: there exists a measurable function ζ such thatfor any starting point x,
∀h ∈ Cζ , Px
∣∣∣∣∣∣
sup0≤i≤n
i∑
j=0
[h(zi)− π(h)]
∣∣∣∣∣∣
> ǫ
≤ nCh(x)
ǫ2
Moreover, assume that ∃φ ≥ 1 such that for any starting point x,
∀h ∈ Cφ, Qn(x, h)P−→ π(h) = π(ph)/π(p) ,
Theorem (Douc & X., AoS, 2011)
Assume that h is such that h/p ∈ Cζ and {Ch/p, h2/p2} ⊂ Cφ. Assume
moreover that
√M
(δ0M − π(h)
) L−→ N (0, V0[h− π(h)]) .
Then, for any starting point x,
√(∑n
h(x(t)))
n→+∞15 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
∀h ∈ Cζ , Px
∣∣∣∣∣∣
sup0≤i≤n
i∑
j=0
[h(zi)− π(h)]
∣∣∣∣∣∣
> ǫ
≤ nCh(x)
ǫ2
∀h ∈ Cφ, Qn(x, h)P−→ π(h) = π(ph)/π(p) ,
Theorem (Douc & X., AoS, 2011)
Assume that h is such that h/p ∈ Cζ and {Ch/p, h2/p2} ⊂ Cφ. Assume
moreover that
√M
(δ0M − π(h)
) L−→ N (0, V0[h− π(h)]) .
Then, for any starting point x,
√
Mn
(∑nt=1 h(x
(t))
n− π(h)
)
n→+∞−→ N (0, V0[h− π(h)]) ,
where Mn is defined by15 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Theorem (Douc & X., AoS, 2011)
Assume that h is such that h/p ∈ Cζ and {Ch/p, h2/p2} ⊂ Cφ. Assume
moreover that
√M
(δ0M − π(h)
) L−→ N (0, V0[h− π(h)]) .
Then, for any starting point x,
√
Mn
(∑nt=1 h(x
(t))
n− π(h)
)
n→+∞−→ N (0, V0[h− π(h)]) ,
where Mn is defined by
Mn∑
i=1
ξ0i ≤ n <
Mn+1∑
i=1
ξ0i .
15 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Variance gain (1)
h(x) x x2 IX>0 p(x)τ = .1 0.971 0.953 0.957 0.207τ = 2 0.965 0.942 0.875 0.861τ = 5 0.913 0.982 0.785 0.826τ = 7 0.899 0.982 0.768 0.820
Ratios of the empirical variances of δ∞ and δ estimating E[h(X)]:100 MCMC iterations over 103 replications of a random walk Gaussianproposal with scale τ .
16 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Illustration (1)
Figure: Overlay of the variations of 250 iid realisations of the estimatesδ (gold) and δ∞ (grey) of E[X] = 0 for 1000 iterations, along with the90% interquantile range for the estimates δ (brown) and δ∞ (pink), inthe setting of a random walk Gaussian proposal with scale τ = 10.
17 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Extra computational effort
median mean q.8 q.9 timeτ = .25 0.0 8.85 4.9 13 4.2τ = .50 0.0 6.76 4 11 2.25τ = 1.0 0.25 6.15 4 10 2.5τ = 2.0 0.20 5.90 3.5 8.5 4.5
Additional computing effort due: median and mean numbers of additionaliterations, 80% and 90% quantiles for the additional iterations, and ratioof the average R computing times obtained over 105 simulations
18 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Formal importance samplingVariance reductionAsymptotic resultsIllustrations
Illustration (2)
Figure: Overlay of the variations of 500 iid realisations of the estimatesδ (deep grey), δ∞ (medium grey) and of the importance sampling version(light grey) of E[X] = 10 when X ∼ Exp(.1) for 100 iterations, alongwith the 90% interquantile ranges (same colour code), in the setting ofan independent exponential proposal with scale µ = 0.02. 19 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Integrating out white noise [C+X, 96]
In Casella+X. (1996), averaging over possible past and future histories(by integrating out uniforms) to improve weights of accepted values
20 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Integrating out white noise [C+X, 96]
In Casella+X. (1996), averaging over possible past and future histories(by integrating out uniforms) to improve weights of accepted valuesGiven the whole sequence of proposed values yt ∼ µ(yt), averaging overuniforms is possible: starting with y1, we can compute
1
T
T∑
t=1
E[h(Xt)|y1, . . . , yT ] =1
T
T∑
t=1
ϕth(yt)
through a recurrence relation:
20 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Integrating out white noise [C+X, 96]
In Casella+X. (1996), averaging over possible past and future histories(by integrating out uniforms) to improve weights of accepted values
ϕ(i)t = δt
p∑
j=t
ξtj
with δ0 = 1 , δt =
t−1∑
j=0
δjξj(t−1)ρjt
and ξtt = 1 , ξtj =
j∏
u=t+1
(1− ρtu)
occurence survivals of the yt’s, associated with Metropolis–Hastings ratio
ωt = π(yt)/µ(yt) , ρtu = ωu/ωt ∧ 1 .
20 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Integrating out white noise (2)
Extension to generic M-H feasible (C+X, 96)Potentialy large variance improvement but cost of O(T 2)...
Possible recovery of efficiency thanks to parallelisation:Moving from (ǫ1, . . . , ǫp) towards...
(ǫ(1), . . . , ǫ(p))
by averaging over ”all” possible orders
21 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Integrating out white noise (2)
Extension to generic M-H feasible (C+X, 96)Potentialy large variance improvement but cost of O(T 2)...
Possible recovery of efficiency thanks to parallelisation:Moving from (ǫ1, . . . , ǫp) towards...
(ǫ(1), . . . , ǫ(p))
by averaging over ”all” possible orders
21 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Integrating out white noise (2)
Extension to generic M-H feasible (C+X, 96)Potentialy large variance improvement but cost of O(T 2)...
Possible recovery of efficiency thanks to parallelisation:Moving from (ǫ1, . . . , ǫp) towards...
(ǫ(1), . . . , ǫ(p))
by averaging over ”all” possible orders
21 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Case of the independent Metropolis–Hastings algorithm
Starting at time t with p processors and a pool of p proposed values,
(y1, . . . , yp)
use processors to examine in parallel p different “histories”
22 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Case of the independent Metropolis–Hastings algorithm
Starting at time t with p processors and a pool of p proposed values,
(y1, . . . , yp)
use processors to examine in parallel p different “histories”
22 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Improvement
The standard estimator τ1 of Eπ [h(X)]
τ1(xt, y1:p) =1
p
p∑
k=1
h(xt+k)
is necessarily dominated by the average
τ2(xt, y1:p) =1
p2
p∑
k=0
nkh(yk)
where y0 = xt and n0 is the number of times xt is repeated.
23 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Further Rao-Blackwellisation
E.g., use of the Metropolis–Hastings weights wj : j being the index suchthat xt+i−1 = yj , update of the weights at each time t+ i:
wj = wj + 1− ρ(xt+i−1, yi)
wi = wi + ρ(xt+i−1, yi)
resulting into a more stable estimator
τ3(xt, y1:p) =1
p2
p∑
k=0
wkh(yk)
E.g., Casella+X. (1996)
τ4(xt, y1:p) =1
p2
p∑
k=0
ϕkh(yk)
24 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Further Rao-Blackwellisation
E.g., use of the Metropolis–Hastings weights wj : j being the index suchthat xt+i−1 = yj , update of the weights at each time t+ i:
wj = wj + 1− ρ(xt+i−1, yi)
wi = wi + ρ(xt+i−1, yi)
resulting into a more stable estimator
τ3(xt, y1:p) =1
p2
p∑
k=0
wkh(yk)
E.g., Casella+X. (1996)
τ4(xt, y1:p) =1
p2
p∑
k=0
ϕkh(yk)
24 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Markovian continuity
The Markov validity of the chain is not jeopardised! The chain continues
by picking one sequence at random and taking the corresponding x(j)t+p as
starting point of the next parallel block.
25 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Markovian continuity
The Markov validity of the chain is not jeopardised! The chain continues
by picking one sequence at random and taking the corresponding x(j)t+p as
starting point of the next parallel block.
25 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Impact of Rao-Blackwellisations
Comparison of
τ1 basic IMH estimator of Eπ [h(X)],
τ2 improving by averaging over permutations of proposed values andusing p times more uniforms
τ3 improving upon τ2 by basic Rao-Blackwell argument,
τ4 improving upon τ2 by integrating out ancillary uniforms, at a costof O(p2).
26 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Illustration
Variations of estimates based on RB and standard versions of parallelchains and on a standard MCMC chain for the mean and variance of thetarget N (0, 1) distribution (based on 10, 000 independent replicas).
27 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Illustration
Variations of estimates based on RB and standard versions of parallelchains and on a standard MCMC chain for the mean and variance of thetarget N (0, 1) distribution (based on 10, 000 independent replicas).
27 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Illustration
Variations of estimates based on RB and standard versions of parallelchains and on a standard MCMC chain for the mean and variance of thetarget N (0, 1) distribution (based on 10, 000 independent replicas).
27 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Illustration
Variations of estimates based on RB and standard versions of parallelchains and on a standard MCMC chain for the mean and variance of thetarget N (0, 1) distribution (based on 10, 000 independent replicas).
27 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Impact of the order
Parallelisation allows for the partial integration of the uniforms
What about the permutation order?Comparison of
τ2N with no permutation,
τ2C with circular permutations,
τ2R with random permutations,
τ2H with half-random permutations,
τ2S with stratified permutations,
28 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Impact of the order
Parallelisation allows for the partial integration of the uniforms
What about the permutation order?
28 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Impact of the order
Parallelisation allows for the partial integration of the uniforms
What about the permutation order?
28 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Impact of the order
Parallelisation allows for the partial integration of the uniforms
What about the permutation order?
28 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Impact of the order
Parallelisation allows for the partial integration of the uniforms
What about the permutation order?
28 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Importance target
Comparison with the ultimate importance sampling
29 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Importance target
Comparison with the ultimate importance sampling
29 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Importance target
Comparison with the ultimate importance sampling
29 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Importance target
Comparison with the ultimate importance sampling
29 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Extension to the general case
Same principle can be applied to any Markov update: if
xt+1 = Ψ(xt, ǫt)
then generate(ǫ1, . . . , ǫp)
in advance and distribute to the p processors in different permutationordersPlus use of Douc & X’s (2011) Rao–Blackwellisation ξki
30 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Extension to the general case
Same principle can be applied to any Markov update: if
xt+1 = Ψ(xt, ǫt)
then generate(ǫ1, . . . , ǫp)
in advance and distribute to the p processors in different permutationordersPlus use of Douc & X’s (2011) Rao–Blackwellisation ξki
30 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Implementation
Similar run of p parallel chains (x(j)t+i), use of averages
τ2(x(1:p)1:p ) =
1
p2
p∑
k=1
p∑
j=1
nkh(x(j)t+k)
and selection of new starting value at random at time t+ p:
31 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Implementation
Similar run of p parallel chains (x(j)t+i), use of averages
τ2(x(1:p)1:p ) =
1
p2
p∑
k=1
p∑
j=1
nkh(x(j)t+k)
and selection of new starting value at random at time t+ p:
31 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Illustration
Variations of estimates based on RB and standard versions of parallelchains and on a standard MCMC chain for the mean and variance of thetarget distribution (based on p = 64 parallel processors, 50 blocs of pMCMC steps and 500 independent replicas).
RB par org
−0.
10−
0.05
0.00
0.05
0.10
RB par org
0.9
1.0
1.1
1.2
1.3
32 / 32
Metropolis Hastings revisitedRao–Blackwellisation
Rao-Blackwellisation (2)
Independent caseGeneral MH algorithms
Illustration
Variations of estimates based on RB and standard versions of parallelchains and on a standard MCMC chain for the mean and variance of thetarget distribution (based on p = 64 parallel processors, 50 blocs of pMCMC steps and 500 independent replicas).
RB par org
−0.
10−
0.05
0.00
0.05
0.10
RB par org
0.9
1.0
1.1
1.2
1.3
32 / 32