Sequential Monte Carlo Methods (for DSGE Models) · Bayesian Estimation of DSGE Models, with E. Herbst, 2015, Princeton University Press \Sequential Monte Carlo Sampling for DSGE

$Page 1: Sequential Monte Carlo Methods (for DSGE Models) · Bayesian Estimation of DSGE Models, with E. Herbst, 2015, Princeton University Press \Sequential Monte Carlo Sampling for DSGE$
Sequential Monte Carlo Methods (for DSGEModels)

Frank Schorfheide∗

∗University of Pennsylvania, PIER, CEPR, and NBER

October 23, 2017

F. Schorfheide Econ 556a (Part 2, Guest Lecture): SMC for DSGE Models

Some References

• “Solution and Estimation of DSGE Models,” with J.Fernandez-Villaverde and Juan Rubio-Ramirez, 2016, in: Handbookof Macroeconomics, vol. 2A

• Bayesian Estimation of DSGE Models, with E. Herbst, 2015,Princeton University Press

• “Sequential Monte Carlo Sampling for DSGE Models,” with E.Herbst, 2014, Journal of Econometrics

• See syllabus for additional references.


Some Background

• DSGE model: dynamic model of the macroeconomy, indexed by θ –vector of preference and technology parameters. Used forforecasting, policy experiments, interpreting past events.

• Bayesian analysis of DSGE models:

p(θ|Y ) =p(Y |θ)p(θ)

p(Y )∝ p(Y |θ)p(θ).

• Computational hurdles: numerical solution of model leads tostate-space representation =⇒ likelihood approximation =⇒posterior sampler.

• “Standard” approach for (linearized) models (Schorfheide, 2000;Otrok, 2001):

• Model solution: log-linearize and use linear rational expectationssystem solver.

• Evaluation of p(Y |θ): Kalman filter• Posterior draws θi : MCMC


Sequential Monte Carlo (SMC) Methods

SMC can help to

Lecture 1

• approximate the posterior of θ: Chopin (2002) ... Durham andGeweke (2013) ... Creal (2007), Herbst and Schorfheide (2014)

Lecture 2

• approximate the likelihood function (particle filtering): Gordon,Salmond, and Smith (1993) ... Fernandez-Villaverde andRubio-Ramirez (2007)

• or both: SMC 2: Chopin, Jacob, and Papaspiliopoulos (2012) ...Herbst and Schorfheide (2015)


Lecture 1


A Loglinearized New Keynesian DSGE Model

• Consumption Euler equation:

yt = Et [yt+1]− 1

τ

(Rt − Et [πt+1]− Et [zt+1]

)+ gt − Et [gt+1]

• New Keynesian Phillips curve:

πt = βEt [πt+1] + κ(yt − gt),

where

κ = τ1− ννπ2φ

• Monetary policy rule:

Rt = ρR Rt−1 + (1− ρR )ψ1πt + (1− ρR )ψ2 (yt − gt) + εR,t


Exogenous Processes

• Technology:

lnAt = ln γ + lnAt−1 + ln zt , ln zt = ρz ln zt−1 + εz,t .

• Government spending / aggregate demand: define gt = 1/(1− ζt);assume

ln gt = (1− ρg ) ln g + ρg ln gt−1 + εg ,t .

• Monetary policy shock εR,t is assumed to be serially uncorrelated.


Solving A Linear Rational Expectations System – A SimpleExample

• Consider

yt =1

θEt [yt+1] + εt , (1)

where εt ∼ iid(0, 1) and θ ∈ Θ = [0, 2].

• Introduce conditional expectation ξt = Et [yt+1] and forecast errorηt = yt − ξt−1.

• Thus,

ξt = θξt−1 − θεt + θηt . (2)


A Simple Example

• Determinacy: θ > 1. Then only stable solution:

ξt = 0, ηt = εt , yt = εt (3)

• Indeterminacy: θ ≤ 1 the stability requirement imposes norestrictions on forecast error:

ηt = Mεt + ζt . (4)

• For simplicity assume now ζt = 0. Then

yt − θyt−1 = Mεt − θεt−1. (5)


At the End of the Day...

• We obtain a transition equation for the vector st :

st = Φ1(θ)st−1 + Φε(θ)εt .

• The coefficient matrices Φ1(θ) and Φε(θ) are functions of theparameters of the DSGE model.


Measurement Equation

• Connect model variables st with observables yt .

• In NK model:

YGRt = γ(Q) + 100(yt − yt−1 + zt)

INFLt = π(A) + 400πt

INTt = π(A) + r (A) + 4γ(Q) + 400Rt .

where

γ = 1 +γ(Q)

100, β =

1

1 + r (A)/400, π = 1 +

π(A)

400.

• More generically:

yt = Ψ0(θ) + Ψ1(θ)t + Ψ2(θ)st +ut︸︷︷︸optional

.


State-Space Representation and Likelihood

• Measurement:

yt = Ψ0(θ) + Ψ1(θ)t + Ψ2(θ)st +ut︸︷︷︸optional

• State transition:

st = Φ1(θ)st−1 + Φε(θ)εt

• Joint density for the observations and latent states:

p(Y1:T ,S1:T |θ) =T∏

t=1

p(yt , st |Y1:t−1,S1:t−1, θ)

=T∏

t=1

p(yt |st , θ)p(st |st−1, θ).

• Compute the marginal p(Y1:T |θ). We will discuss how in Lecture 2.


Where Are We Headed? ... Bayesian Inference

• Ingredients of Bayesian Analysis:

• Likelihood function p(Y |θ)

• Prior density p(θ)

• Marginal data density p(Y ) =∫p(Y |θ)p(θ)dφ

• Bayes Theorem:

p(θ|Y ) =p(Y |θ)p(θ)

p(Y )∝ p(Y |θ)p(θ)

• Implementation: usually by generating a sequence of draws (notnecessarily iid) from posterior

θi ∼ p(θ|Y ), i = 1, . . . ,N

• Algorithms: direct sampling, accept/reject sampling, importancesampling, Markov chain Monte Carlo sampling, sequential MonteCarlo sampling...


Bayesian Inference – Posterior

• Focus on SMC algorithm that generates draws θiNi=1 from

posterior distributions of parameters.

• Draws can then be transformed into objects of interest, h(θi ), andunder suitable conditions a Monte Carlo average of the form

hN =1

N

N∑i=1

h(θi ) ≈ Eπ[h]

∫h(θ)p(θ|Y )dθ.

• Strong law of large numbers (SLLN), central limit theorem (CLT)...


Bayesian Inference – Decision Making

• The posterior expected loss of decision δ(·):

ρ(δ(·)|Y

)=

∫Θ

L(θ, δ(Y )

)p(θ|Y )dθ.

• Bayes decision minimizes the posterior expected loss:

δ∗(Y ) = argmind ρ(δ(·)|Y

).

• Approximate ρ(δ(·)|Y

)by a Monte Carlo average

ρN

(δ(·)|Y

)=

1

N

N∑i=1

L(θi , δ(·)

).

• Then compute

δ∗N (Y ) = argmind ρN

(δ(·)|Y

).


Bayesian Inference

• Point estimation:

• Quadratic loss: posterior mean

• Absolute error loss: posterior median

• Interval/Set estimation Pπθ ∈ C (Y ) = 1− α:

• highest posterior density sets

• equal-tail-probability intervals


Implementation: Sampling from Posterior

• DSGE model posteriors are often non-elliptical, e.g., multimodalposteriors may arise because it is difficult to

• disentangle internaland externalpropagationmechanisms;

• disentangle therelative importance ofshocks.

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0θ2

θ1

• Economic Example: is wage growth persistent because

1 wage setters find it very costly to adjust wages?

2 exogenous shocks affect the substitutability of labor inputs andhence markups?


Sampling from Posterior

• If posterior distributions are irregular, standard MCMC methods canbe inaccurate (examples will follow).

• SMC samplers often generate more precise approximations ofposteriors in the same amount of time.

• SMC can be parallelized.

• SMC = importance sampling on steroids =⇒ We will first reviewimportance sampling.


Importance Sampling

• Approximate π(·) by using a different, tractable density g(θ) that iseasy to sample from.

• For more general problems, posterior density may be unnormalized.So we write

π(θ) =p(Y |θ)p(θ)

p(Y )=

f (θ)∫f (θ)dθ

.

• Importance sampling is based on the identity

Eπ[h(θ)] =

∫h(θ)π(θ)dθ =

∫Θh(θ) f (θ)

g(θ)g(θ)dθ∫Θ

f (θ)g(θ)g(θ)dθ

.

• (Unnormalized) importance weight:

w(θ) =f (θ)

g(θ).


Importance Sampling

1 For i = 1 to N, draw θi iid∼ g(θ) and compute the unnormalizedimportance weights

w i = w(θi ) =f (θi )

g(θi ).

2 Compute the normalized importance weights

W i =w i

1N

∑Ni=1 w

i.

An approximation of Eπ[h(θ)] is given by

hN =1

N

N∑i=1

W ih(θi ).


Illustration

If θi ’s are draws from g(·) then

Eπ[h] ≈1N

∑Ni=1 h(θi )w(θi )

1N

∑Ni=1 w(θi )

, w(θ) =f (θ)

g(θ).

6 4 2 0 2 4 60.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

f

g1

g2

6 4 2 0 2 4 6

weights

f/g1

f/g2


Accuracy

• Since we are generating iid draws from g(θ), it’s fairlystraightforward to derive a CLT:

√N(hN−Eπ[h]) =⇒ N

(0,Ω(h)

), where Ω(h) = Vg [(π/g)(h−Eπ[h])].

• Using a crude approximation (see, e.g., Liu (2008)), we can factorizeΩ(h) as follows:

Ω(h) ≈ Vπ[h](Vg [π/g ] + 1

).

The approximation highlights that the larger the variance of theimportance weights, the less accurate the Monte Carloapproximation relative to the accuracy that could be achieved withan iid sample from the posterior.

• Users often monitor

ESS = NVπ[h]

Ω(h)≈ N

1 + Vg [π/g ].


From Importance Sampling to Sequential ImportanceSampling

• In general, it’s hard to construct a good proposal density g(θ),

• especially if the posterior has several peaks and valleys.

• Idea - Part 1: it might be easier to find a proposal density for

πn(θ) =[p(Y |θ)]φnp(θ)∫[p(Y |θ)]φnp(θ)dθ

=fn(θ)

Zn.

at least if φn is close to zero.

• Idea - Part 2: We can try to turn a proposal density for πn into aproposal density for πn+1 and iterate, letting φn −→ φN = 1.


Illustration: Tempered Posteriors of θ1

θ1

0.00.2

0.40.6

0.81.0

n10

2030

4050

0

1

2

3

4

5

πn(θ) =[p(Y |θ)]φnp(θ)∫[p(Y |θ)]φnp(θ)dθ

=fn(θ)

Zn, φn =

(n

Nφ

)λ


SMC Algorithm: A Graphical Illustration

C S M C S M C S M

−10

−5

0

5

10

φ0 φ1 φ2 φ3

• πn(θ) is represented by a swarm of particles θin,W

inN

i=1:

hn,N =1

N

N∑i=1

W inh(θi

n)a.s.−→ Eπn [h(θn)].

• C is Correction; S is Selection; and M is Mutation.


SMC Algorithm

1 Initialization. (φ0 = 0). Draw the initial particles from the prior:

θi1

iid∼ p(θ) and W i1 = 1, i = 1, . . . ,N.

2 Recursion. For n = 1, . . . ,Nφ,

1 Correction. Reweight the particles from stage n − 1 by defining theincremental weights

w in = [p(Y |θi

n−1)]φn−φn−1 (6)

and the normalized weights

W in =

w inW

in−1

1N

∑Ni=1 w

inW i

n−1

, i = 1, . . . ,N. (7)

An approximation of Eπn [h(θ)] is given by

hn,N =1

N

N∑i=1

W inh(θi

n−1). (8)

2 Selection.


SMC Algorithm

1 Initialization.

2 Recursion. For n = 1, . . . ,Nφ,

1 Correction.2 Selection. (Optional Resampling) Let θN

i=1 denote N iid drawsfrom a multinomial distribution characterized by support points andweights θi

n−1, WinN

i=1 and set W in = 1.

An approximation of Eπn [h(θ)] is given by

hn,N =1

N

N∑i=1

W inh(θi

n). (9)

3 Mutation. Propagate the particles θi ,Win via NMH steps of a MH

algorithm with transition density θin ∼ Kn(θn|θi

n; ζn) and stationarydistribution πn(θ). An approximation of Eπn [h(θ)] is given by

hn,N =1

N

N∑i=1

h(θin)W i

n . (10)


Remarks

• Correction Step:• reweight particles from iteration n− 1 to create importance sampling

approximation of Eπn [h(θ)]

• Selection Step: the resampling of the particles• (good) equalizes the particle weights and thereby increases accuracy

of subsequent importance sampling approximations;• (not good) adds a bit of noise to the MC approximation.

• Mutation Step: changes particle values• adapts particles to posterior πn(θ);• imagine we don’t do it: then we would be using draws from prior

p(θ) to approximate posterior π(θ), which can’t be good!

θ1

0.00.2

0.40.6

0.81.0

n10

2030

4050

0

1

2

3

4

5


More on Transition Kernel in Mutation Step

• Transition kernel Kn(θ|θn−1; ζn): generated by running M steps of aMetropolis-Hastings algorithm.

• Lessons from DSGE model MCMC:• blocking of parameters can reduces persistence of Markov chain;• mixture proposal density avoids “getting stuck.”

• Blocking: Partition the parameter vector θn into Nblocks equally sizedblocks, denoted by θn,b, b = 1, . . . ,Nblocks . (We generate the blocksfor n = 1, . . . ,Nφ randomly prior to running the SMC algorithm.)

• Example: random walk proposal density:

ϑb|(θin,b,m−1, θ

in,−b,m,Σ

∗n,b) ∼ N

(θi

n,b,m−1, c2n Σ∗

n,b

).


Adaptive Choice of ζn = (Σ∗n, cn)

• Infeasible adaption:• Let Σ∗

n = Vπn [θ].• Adjust scaling factor according to

cn = cn−1f(1− Rn−1(ζn−1)

),

where Rn−1(·) is population rejection rate from iteration n − 1 and

f (x) = 0.95 + 0.10e16(x−0.25)

1 + e16(x−0.25).

• Feasible adaption – use output from stage n− 1 to replace ζn by ζn:

• Use particle approximations of Eπn [θ] and Vπn [θ] based onθi

n−1, WinN

i=1.

• Use actual rejection rate from stage n − 1 to calculatecn = cn−1f

(Rn−1(ζn−1)

).


More on Resampling

• So far, we have used multinomial resampling. It’s fairly intuitive andit is straightforward to obtain a CLT.

• But: multinominal resampling is not particularly efficient.

• The Herbst-Schorfheide book contains a section on alternativeresampling schemes (stratified resampling, residual resampling...)

• These alternative techniques are designed to achieve a variancereduction.

• Most resampling algorithms are not parallelizable because they relyon the normalized particle weights.


Application 1: Small Scale New Keynesian Model

• We will take a look at the effect of various tuning choices onaccuracy:

• Tempering schedule λ: λ = 1 is linear, λ > 1 is convex.

• Number of stages Nφ versus number of particles N.


Effect of λ on Inefficiency Factors InEffN [θ]

1 2 3 4 5 6 7 8100

101

102

103

104

λ

Notes: The figure depicts hairs of InEffN [θ] as function of λ. Theinefficiency factors are computed based on Nrun = 50 runs of the SMCalgorithm. Each hair corresponds to a DSGE model parameter.


Number of Stages Nφ vs Number of Particles N

ρg σr ρr ρz σg κ σz π(A) ψ1 γ(Q) ψ2 r (A) τ10−3

10−2

10−1

100

101

Nφ = 400, N = 250Nφ = 200, N = 500Nφ = 100, N = 1000

Nφ = 50, N = 2000Nφ = 25, N = 4000

Notes: Plot of V[θ]/Vπ[θ] for a specific configuration of the SMCalgorithm. The inefficiency factors are computed based on Nrun = 50runs of the SMC algorithm. Nblocks = 1, λ = 2, NMH = 1.


A Few Words on Posterior Model Probabilities

• Posterior model probabilities

πi,T =πi,0p(Y1:T |Mi )∑M

j=1 πj,0p(Y1:T |Mj )

where

p(Y1:T |Mi ) =

∫p(Y1:T |θ(i),Mi )p(θ(i)|Mi )dθ(i)

• For any model:

ln p(Y1:T |Mi ) =T∑

t=1

ln

∫p(yt |θ(i),Y1:t−1,Mi )p(θ(i)|Y1:t−1,Mi )dθ(i)

• Marginal data density p(Y1:T |Mi ) arises as a by-product of SMC.


Marginal Likelihood Approximation

• Recall w in = [p(Y |θi

n−1)]φn−φn−1 .

• Then

1

N

N∑i=1

w inW

in−1 ≈

∫[p(Y |θ)]φn−φn−1

pφn−1 (Y |θ)p(θ)∫pφn−1 (Y |θ)p(θ)dθ

dθ

=

∫p(Y |θ)φnp(θ)dθ∫p(Y |θ)φn−1p(θ)dθ

• Thus,

Nφ∏n=1

(1

N

N∑i=1

w inW

in−1

)≈∫

p(Y |θ)p(θ)dθ.


SMC Marginal Data Density Estimates

Nφ = 100 Nφ = 400N Mean(ln p(Y )) SD(ln p(Y )) Mean(ln p(Y )) SD(ln p(Y ))500 -352.19 (3.18) -346.12 (0.20)1,000 -349.19 (1.98) -346.17 (0.14)2,000 -348.57 (1.65) -346.16 (0.12)4,000 -347.74 (0.92) -346.16 (0.07)

Notes: Table shows mean and standard deviation of log marginal datadensity estimates as a function of the number of particles N computedover Nrun = 50 runs of the SMC sampler with Nblocks = 4, λ = 2, andNMH = 1.


Application 2: Estimation of Smets and Wouters (2007)Model

• Benchmark macro model, has been estimated many (many) times.

• “Core” of many larger-scale models.

• 36 estimated parameters.

• RWMH: 10 million draws (5 million discarded); SMC: 500 stageswith 12,000 particles.

• We run the RWM (using a particular version of a parallelizedMCMC) and the SMC algorithm on 24 processors for the sameamount of time.

• We estimate the SW model twenty times using RWM and SMC andget essentially identical results.


Application 2: Estimation of Smets and Wouters (2007)Model

• More interesting question: how does quality of posterior simulatorschange as one makes the priors more diffuse?

• Replace Beta by Uniform distributions; increase variances ofparameters with Gamma and Normal prior by factor of 3.


SW Model with DIFFUSE Prior: Estimation stability RWH(black) versus SMC (red)

l ιw µp µw ρw ξw rπ−4

−3

−2

−1

0

1

2

3

4


A Measure of Effective Number of Draws

• Suppose we could generate iid Neff draws from posterior, then

Eπ[θ]approx∼ N

(Eπ[θ],

1

NeffVπ[θ]

).

• We can measure the variance of Eπ[θ] by running SMC and RWMalgorithm repeatedly.

• Then,

Neff ≈Vπ[θ]

V[Eπ[θ]

]


Effective Number of Draws

SMC RWMHParameter Mean STD(Mean) Neff Mean STD(Mean) Neff

σl 3.06 0.04 1058 3.04 0.15 60l -0.06 0.07 732 -0.01 0.16 177ιp 0.11 0.00 637 0.12 0.02 19h 0.70 0.00 522 0.69 0.03 5Φ 1.71 0.01 514 1.69 0.04 10rπ 2.78 0.02 507 2.76 0.03 159ρb 0.19 0.01 440 0.21 0.08 3ϕ 8.12 0.16 266 7.98 1.03 6σp 0.14 0.00 126 0.15 0.04 1ξp 0.72 0.01 91 0.73 0.03 5ιw 0.73 0.02 87 0.72 0.03 36µp 0.77 0.02 77 0.80 0.10 3ρw 0.69 0.04 49 0.69 0.09 11µw 0.63 0.05 49 0.63 0.09 11ξw 0.93 0.01 43 0.93 0.02 8


A Closer Look at the Posterior: Two Modes

Parameter Mode 1 Mode 2ξw 0.844 0.962ιw 0.812 0.918ρw 0.997 0.394µw 0.978 0.267Log Posterior -804.14 -803.51

• Mode 1 implies that wage persistence is driven by extremelyexogenous persistent wage markup shocks.

• Mode 2 implies that wage persistence is driven by endogenousamplification of shocks through the wage Calvo and indexationparameter.

• SMC is able to capture the two modes.


A Closer Look at the Posterior: Internal ξw versus Externalρw Propagation

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ξw

ρw


Stability of Posterior Computations: RWH (black) versusSMC (red)

P (ξw > ρw) P (ρw > µw) P (ξw > µw) P (ξp > ρp) P (ρp > µp) P (ξp > µp)0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Documents

Sequential Monte Carlo Methods (for DSGE Models) · Bayesian Estimation of DSGE Models, with E. Herbst, 2015, Princeton University Press \Sequential Monte Carlo Sampling for DSGE