Markov Regime-Switching (and some State Space) Models in ...past.rinfinance.com/agenda/2015/talk/MatthewBrigda.pdf · Markov Regime-Switching (and some State Space) Models in Energy

Markov Regime-Switching (and some State Space)Models in Energy Markets

Matthew Brigida, Ph.D.

Department of FinanceCollege of Business AdministrationClarion University of Pennsylvania

Clarion, PA [email protected]

June 2015

Introduction

Time series often exhibit distinct changes in regime. Thus we mustallow for switches in model parameters and standard errors. Forexample events which may cause a distinct shift are:

recession or economic expansion (any economic variable orrelationship).central bank currency intervention (currency volatility).the introduction of hydrofracking (the relationship betweennatural gas and crude oil prices).a weather event or plant outage (power price volatility, averagereturns, autocorrelation).

However, we usually don’t know exactly when the event occurred, sowe cannot use introduce indicator variables. Moreover, it is preferableto have a probability law over the entire data generating process(because a priori the timing of the regime switch is uncertain).

Dr. Matt Brigida, [email protected] Markov Regime-Switching in Energy Markets

mailto:[email protected]

Introduction

Regime-switching may also explain deviations from normality oftenseen in time series.

Do stock returns really have fat tails (motivating a Cauchy typedistribution)? Or rather are returns normal, but generated bymultiple regimes?Skewness may be explained similarly.



An Example: Electricity Prices

rt = µSt + eSt , eSt ∼ N (0, σ2St )Low volatility regime: µL = -0.34%, σL = 13.5%High volatility regime: µH = 0.2%, σH = 19.8%Empirical: µ = -0.12%, σ = 16.3%, skewness = 0.354, kurtosis = 4.8The transition matrix for the Markov process is:

Low HighLow 63% 34%High 58% 41%



Introduction

Ultimately, we will allow the time series to follow n distinct densityfunctions, and switch between these density functions according to anunobserved Markov process. Thus, we need to make inference aboutthe parameters of the density functions, as well as the Markov process(transition probabilities and filtered (or smoothed) regime probabilitiesP(St = i|ϕt−1))1.

1where ϕt−1 is the information available through time t − 1.Dr. Matt Brigida, [email protected] Markov Regime-Switching in Energy Markets


Gregory and Hansen (1996) Test for Regime-Shifts in aCointegrating Relationship

Prior to using a Markov-switching model it is useful to test for regimeswitching.

Codeoil

Background

Say we have yt = xtβ + et , where et ∼ i.i.d.N (0, σ2), and xt is a kvector of exogenous variables. Let θ denote the vector of parameters(here θ = (β σ2)′).

In the non-switching case we may simply maximize thelog-likelihood L(θ) =

∑Tt=1 ln(f (yt)) where f (•) denotes the

normal distribution.If there is switching, and the regime at each time (St) is knowna priori then we have L(θ) =

∑Tt=1 ln(f (yt |St)) where the St are

indicator variables in βSt = β1(1− St) + β2St andσ2St = σ

21(1−St) +σ22St where St ∈ {0, 1} (in the case of 2 regimes).



Background: Unobserved Regimes

If the regime states are unobserved, however, then we’ll need the jointdensity f (yt ,St |ϕt−1) = f (yt |St , yt)f (St |ϕt−1) where ϕt−1 denotes theinformation set though time t − 1.

We may then obtain the marginal f (yt |ϕt−1) by summing over allpossible regimes. In the case of two regimes:f (yt |ϕt−1) =

∑2St=1 f (yt |St , ϕt)f (St |ϕt−1)

We then have L(θ) =∑T

t=1 ln(f (yt |ϕt−1)).We then maximize this wrt θ = (βS1 βS2 σ2S1 σ

2S2)′ and whatever

parameters govern the regime switching process.



The Regime Switching Process

Now we must consider the process governing regime-switching (i.e.f (St |ϕt−1)). We may consider:

Switching which is independent of prior regimes (can be dependenton exogenous variables).Markov-switching with constant transition probabilities(dependent on the prior or lagged regime).Markov-switching with time-varying transition probabilities (theregime is a function of other variables2).

2the variables must be conditionally uncorrelated with the regime of the Markovprocess (Filardo (1998))



Independent Switching

In the case of independent switching f (St |ϕt−1) may have the law:P[St = 1] = e

p

1+ep

P[St = 2] = 1− ep

1+ep

where L is maximized (unconstrained) wrt θ = (βS1 βS2 σ2S1 σ2S2 p)

′.That is the parameters governing f (St |ϕt−1) which is p, and theparameters governing f (yt |St , ϕt) which are βS1 ,βS2 , σ2S1 , σ

2S2

You could also replace p with a function of exogenous variables (zt−1):say, replace p with p0 + p1z1,t−1 + p2z2,t−1 and maximize wrt p0,p1, p2.

This formulation is often implausible from an economic standpoint.Usually the present regime is dependent on the prior regime (if we arein a recession today, there is a greater probability of a recessiontomorrow).



Markov Regime-Switching: Constant TransitionProbabilities

Continuing with 2 regimes, we’ll allow the regimes to evolve accordingto a first-order Markov process with constant transition probabilities:

P =[

p11 1− p221− p11 p22

].

We’ll need to make an optimal forecast, and optimal inference, ofthe regime at each t (this is where the Hamilton filter comes in).

That is, we’ll need P[St = j|ϕt−1,θ] and P[St = j|ϕt ,θ] wherej ∈ {1, 2}.

For the inference note:

P(St = j|ϕt ; θ) = P(St = j|yt ,xt , ϕt−1; θ) =P(yt ,St = j|xt , ϕt−1; θ)

f (yt |xt , ϕt−1; θ)

= P(St = j|xt , ϕt−1; θ)f (yt |St = j,xt , ϕt−1; θ)∑2j=1 P(St = j|xt , ϕt−1; θ)f (yt |St = j,xt , ϕt−1; θ)



Continued...Collect P(St = j|ϕt ; θ) for j ∈ {1, 2} into a [2x1] vector ξ̂t|t . Similarlylet ηt = f (yt |St = j,xt , ϕt−1; θ) be a [2x1] vector. Then we may rewritethe optimal inference in vector notation as (using Hamilton’s notation):ξ̂t|t =

ξ̂t|t−1�ηt(1 1)(ξ̂t|t−1�ηt)

where � denotes element-by-elementmultiplication.

Then

ξ̂t+1|t =[

P(St+1 = 1|ϕt ; θ)P(St+1 = 2|ϕt ; θ)

]=

=[

p11P(St = 1|ϕt ; θ) + p21P(St = 2|ϕt ; θ)p12P(St = 1|ϕt ; θ) + p22P(St = 2|ϕt ; θ)

]=

=[

p11 p21p12 p22

] [P(St = 1|ϕt ; θ)P(St = 2|ϕt ; θ)

]= Pξ̂t|t

and so ξ̂t+1|t = Pξ̂t|t is the optimal forecast.From here we use ξ̂t+1|t to get ξ̂t+1|t+1 and the filter proceeds.



The Log Likelihood

The above filter provides the log likelihood by:

L(θ) =T∑

t=1log((1 1)(ξ̂t|t−1 � ηt)) =

T∑t=1

log(f (yt |xt , ϕt−1; θ))

which can be maximized wrt θ = (βS1 βS2 σ2S1 σ2S2 p11 p22)

′.



Estimating Regime Probability

Once you have the estimated parameters (θ) you may use the data andparameters to estimate the probability of each regime at each t. Youuse the same filtering algorithm.



Flowchart of the Markov Regime Switching Model

Run throughHamilton Filter

From this we getthe Likelihood

maxθL (θ)

θ̂ and Stan-dard Errors

Run throughHamilton Fil-

ter again with θ̂

Regime Prob.P[St = j|ϕt , θ̂]

0 50 100 150 200

0.0

0.4

0.8

Filtered Probability of Being in State 1

Month

Pro

babi

lity

0 50 100 150 200

0.2

0.6


Month

Pro

babi

lity

0 50 100 150 200

0.0

0.4

0.8


Month

Pro

babi

lity



Markov-Switching with Constant TransitionProbabilities: An Example

There is are theoretical justification for cointegration between naturalgas and crude oil prices.

They are produced together and often sold in contracts whichprice NG as a fraction of crude oil.They are possible substitutes.

However, prior analyses have found conflicting results.Some find evidence for cointegration, while others find structuralbreaks in the relative pricing relationship (prompting claims of‘decoupling’).

My paper3 claims there are events (technological such as fracking andCCCT, and political such as NG deregulation) which cause distinctstructural breaks in the cointegrating relationship.

These breaks may be modeled as switches between regimes.3revise and resubmit at Energy Economics



Markov-Switching with Constant TransitionProbabilities: An Example

NG and crude oil cointegrating equation: First-Order, M-Regime,Markov-Switching

PHH = β0,St + β1,St PWTI + et , et ∼ N(0, σ2St

)

P(St = j|St−1 = i) = pij , ∀j ∈ 1, 2, ...,M , andM∑

j=1pij = 1

β0,St = β0,1S1t + β0,2S2t + ...+ β0,M SMtβ1,St = β1,1S1t + β1,2S2t + ...+ β1,M SMtσ0,St = σ0,1S1t + σ0,2S2t + ...+ σ0,M SMt

where for m ∈ 1, 2, ...M , if St = m ⇒ Smt = 1, and Smt = 0 else



Estimation

Construction of the likelihood function for the above MarkovSwitching cointegrating equation was done using the Hamiltonfilter4.Minimization of the negative log-likelihood was done using theoptim function in the R.Alternativelly, maximization can be done using the EM algorithm.

Closed-form solutions for parameters means no optimizationrequired in M-step.E-step is simply the smoothed probabilities of the unobservedregime, P(St = j|ϕT).This method is robust with respect to the initial starting values ofthe model parameters.

4see Hamilton (1998).Dr. Matt Brigida, [email protected] Markov Regime-Switching in Energy Markets


R CodeLog-Likelihood Functionlik

R Code

Log-Likelihood Function: Continuedxi.a[1,]

Getting regime probabilities

Once we have found maximum likelihood estimates of the parameters,we rerun the filter with these estimates to get filtered regimeprobabilities.

Estimating Regime Probabilitiesalpha.hat

Online app and code

Shinyapp: https://mattbrigida.shinyapps.io/MSCP

Code for app (GitHub): ‘Matt-Brigida/R-Finance-2015-MSCP’


https://mattbrigida.shinyapps.io/MSCPmailto:[email protected]

Regime-Weighted Residuals

Given estimated model parameters and transition probabilities, Icalculate the filtered5 regime probability P (St = i|ϕt−1) whereϕt−1 denotes the information set at time t − 1.

Note this also affords the expected duration of each regime:

E(Dm) =∑∞

j=1 j(P[Dm = j]) = 11−pmm

5as opposed to smoothed probabilities P (St = i|ϕT )Dr. Matt Brigida, [email protected] Markov Regime-Switching in Energy Markets


Results: 2 regimes & monthly prices

Table 1. Results of Estimating Cointegrating Equation Regressions (equations 1-5) with Monthly

Prices

There are 225 monthly observations for natural gas and oil prices. The one-state model has 222 degrees-

of-freedom, and the two-state model has 217. The alternative hypothesis in the augmented Dickey-Fuller

and Phillips-Perron tests is to conclude the residual series is stationary. Residuals in the two-state model

are weighted by the filtered state probability. p-values are below the coefficient in parentheses. *, **, and

*** denote significance at the 10%, 5%, and 1% levels respectively.

One-State Model Two-State Model

State 1 1 2

𝛽0 -0.6910 (5e-6)***

-0.0287

(0.7518)

0.3351

(0.2068)

𝛽1 0.5608 (

Results: 2 regimes & weekly prices

Table 2. Results of Estimating Cointegrating Equation Regressions (equations 1-5) with Weekly

Prices

There are 978 weekly observations for natural gas and oil prices. The one-state model has 975 degrees-of-

freedom, and the two-state model has 970. The alternative hypothesis in the augmented Dickey-Fuller

and Phillips-Perron tests is to conclude the residual series is stationary. Residuals in the two-state model

are weighted by the filtered state probability. p-values are below the coefficient in parentheses. *, **, and

*** denote significance at the 10%, 5%, and 1% levels respectively.

One-State Model Two-State Model

State 1 1 2

𝛽0 -0.6867 (

Monthly Natural Gas and Crude Oil Prices with the Probability of State 2

Time

1995 2000 2005 2010

01

Natural GasOilP(State=2)



Weekly Natural Gas and Crude Oil Prices with the Probability of State 2

Time

1995 2000 2005 2010

01

Natural GasOilP(State=2)



Regime Probabilities: Allowing 3 Regimes, monthlyprices

0 50 100 150 200

0.0

0.4

0.8


Month

Pro

babi

lity

0 50 100 150 200

0.2

0.6


Month

Pro

babi

lity

0 50 100 150 200

0.0

0.4

0.8


Month

Pro

babi

lity



Regime Probabilities: Allowing 3 Regimes, weekly prices


Time

Pro

babi

lity

1995 2000 2005 2010

0.0

0.4

0.8


Time

Pro

babi

lity

1995 2000 2005 2010

0.0

0.4

0.8


Time

Pro

babi

lity

1995 2000 2005 2010

0.0

0.4

0.8



Time-Varying Transition Probabilities (TVTP)

We may also wish to allow the transition probabilities to varyaccording to a set of explanatory variables.

The probability of transitioning to an electricity price spike regimeis likely dependent on reserve capacity and the weather.I’ll show weather and natural gas storage quantities affect therelative pricing of crude oil and natural gas.



TVTP: NG and Oil Cointegrating EquationLet Xt−1 be a vector of variables which affect the likelihood of switchesbetween regimes. The cointegrating equation with first-order,two-regime, endogenous Markov switching parameters withtime-varying transition probabilities may be written as:

PHH = β0,St + β1,St PWTI + et , et ∼ N (0, σ2St )

pij,t = P(St = j|St−1 = i,Xt−1) =exp

(φij,0 + X′t−1φij,1

)1 + exp

(φij,0 + X′t−1φij,1

) ,∀i, j ∈ 1, 2 and

∑2j=1 pij = 1

β0,St = β0,1S1t + β0,2S2t

β1,St = β1,1S1t + β1,2S2t

σ0,St = σ0,1S1t + σ0,2S2twhere for m ∈ 1, 2, if St = m, then Smt = 1, and Smt = 0 otherwise.



TVTP Example

The variables governing the transition probabilities are:the deviation of the number of heating degree days form the longterm norm.the logged difference of natural gas working gas in storage from its5-year average.An Indicator for Enron’s collpase.



TVTP: Estimation

Hamilton Filter: TVTPxi.a[1,]

TVTP Example: Filtered Regime ProbabilitiesFiltered Probability of Being in State 2

Time

Pro

babi

lity

2000 2005 2010

0.0

0.2

0.4

0.6

0.8

1.0



TVTP Example: Parameter Estimates

Table: Parameter estimates of PHH = β0,St + β1,St PWTI + eSt where St ∈ {1, 2} and the transitionprobabilities are time-varying functions of the deviation of the amount of working gas in storage from its5-year average (STOR) and the deviation of the cumulative number of monthly HDD from its long-termnorm (HDD DEV). P-values are below the coefficients in parentheses.

Cointegrating EquationS1 S2

β0 -0.06572 0.07525(6.969e-01) (7.752e-01)

β1 0.34383 0.46665(8.322e-13)*** (1.529e-11)***

σ 0.31294 0.18495(0.000e+00)*** (0.000e+00)***

Transition Probabilitiesintercept -0.90138 -0.45712

(3.758e-04)*** (7.656e-02)*HDD DEV 0.02312

(2.452e-10)***NG STOR 1.53150

(8.320e-03)***Neg Log Likelihood: -40.37859



TVTP Example: Parameter Estimates with ENRNIndicatorTable: Parameter estimates of PHH = β0,St + β1,St PWTI + eSt where St ∈ {1, 2} and the transitionprobabilities are time-varying functions of the deviation of the amount of working gas in storage from its5-year average (STOR), the deviation of the cumulative number of monthly HDD from its long-term norm(HDD DEV), and an indicator for the 6 months after the collapse of Enron (ENRN). P-values are belowthe coefficients in parentheses.

Cointegrating EquationS1 S2

β0 -0.2929 0.3881(3.498e-08)*** (8.513e-05)***

β1 0.3716 0.3736(0.0000)*** (0.0000)***

σ 0.1813 0.1945(0.0000)*** (0.0000)***

Transition Probabilitiesintercept -1.6204 -1.5313

(0.0000)*** (0.0000)***HDD DEV 0.0317

(0.0000)***NG STOR 0.7485

(0.1027)ENRN 1.4220

(0.0000)***Neg Log Likelihood: -12.6087



TVTP: Electricity Price Spikes

This methodology has recently been found useful for modeling powerprices and predicting price spikes. Some relevant research is:

Mount, Ning, Cai (2006)Janczura, Weron (2010)among others



Incorporating Parameter Uncertainty

We may wish to allow for parameter uncertainty within the model:

...a person’s uncertainty about the future arises not simplybecause of future random terms but also because of uncertaintyabout current parameter values and of the model’s ability tolink the present to the future. [Harrison and Stevens (1976),pg. 208]

This can be done by writing the model in state-space form6 andapplying the Kalman filter.

6For an introduction to state-space models see Harvey (1989).Dr. Matt Brigida, [email protected] Markov Regime-Switching in Energy Markets


Regime Uncertainty versus Parameter Uncertainty

In the Markov regime-switching models the regime is theunobservable variable.In dealing with parameter uncertainty we first build a structuralmodel of the time series. The model is formulated in terms ofunobservable components, which nonetheless have a directinterpretation.

We then write the model in state space form (measurement andstate transition equations), where the unobservable component isthe state7 vector which evolves according to the state transitionequation.We make inference about the state using the Kalman filter.The varying state vector is the set of time-varying parameters.

7The term ‘state’ here is different from ‘state’ when used in a regime-switchingmodel.



Testing for Parameter Stability

The Brown, Durbin, and Evans (1975) ‘homogeneity test’.Null hypothesis: coefficients are equal at all points in time.Sample period is divided into n non-overlapping intervals.Ratio of ’between groups over within groups’ mean sum of squaresis the test statistic.Test statistic follows an F distribution.



Code## Brown Durban Evans (1975) ’Homogeneity test’ ----n

Kalman Filter Example

The Kalman Filter is often used in finance to estimatetime-varying β coefficient in a CAPM type equation8.There are many implementation of the Kalman filter in R andother languages (though you must take care to note whatparameters a particular implementation will estimate).Through the Kalman filter, the model parameters may beestimated using prediction error decomposition.

8The Kalman filter was created by engineers working on control systems. Earlyapplications of the filter were to such things as tracking a missile with a time-seriesof noisy observations of its location. Because of this, in the first applications of theKalman filter the parameters were assumed to be known (often by the physics of thesituation). Later, creating a likelihood via prediction error decomposition wasintroduced.



CAPM: Time-Varying β

Below is an implementation of the CAPM with a time-varying βcoefficient.

rs,t = α+ βtrm,t + et , et ∼ i.i.d.N (0,R)

βt+1 = µ+ Fβt + νt , νt ∼ i.i.d.N (0,Q)

You can easily allow α to vary with time also. Below however, I haveset α = 0.




Kalman Filtered Beta Codelik


Kalman Filtered Beta Code Continuedtheta.start

Time-varying and Unconditional β: BPBP Time−varying and Unconditional Betas (market model)

Time

Bet

a

2000 2005 2010 2015

01

23

4

Kalman Filtered BetaUnconditional Beta



Time-varying β in NG and Oil Cointegrating EquationBeta from Natural Gas and Crude Oil

Cointegrating Equation

Time

Bet

a

2000 2005 2010

0.2

0.3

0.4

0.5

0.6



Shiny App and Code

I have put a shiny app where you can input a stock’s ticker and timeinterval, and see a time series of the Kalman filtered beta coefficient,here: https://mattbrigida.shinyapps.io/kalman_filtered_beta

Note the app is too slow at this point—I hope to speed it up withRcpp.

The code is on GitHub here:‘Matt-Brigida/R-Finance-2015-Kalman_beta’


https://mattbrigida.shinyapps.io/kalman_filtered_beta`Matt-Brigida/R-Finance-2015-Kalman_beta'mailto:[email protected]

Regime-Switching in State-Space Models

It is straight-forward to include time-varying parameters in aregime-switching model by putting it in state-space form. At this pointyou can combine the Hamilton filter with the Kalman Filter (Kim(1994)).

This recognizes the uncertainty in current and future parametervalues.For example: Does the sensitivity of electricity prices to outageschange in an uncertain fashion?

∆Pt = Xt−1βt + etβt = βt−1 + ηtη ∼ N (0,R)et ∼ N (0, σ2St )



State Space Model with Markov Switching: NG & OilTo incorporate parameter uncertainty into our two-state Markovregime-switching model of the natural gas and crude oil cointegratingequation, we can write the equation as:

PHH = β0,St + β1,t,St PWTI + et , et ∼ N (0, σ2St )

β1,t,St = µSt + FStβ1,t−1,St−1 + GSt vST

P(St = j|St−1 = i) = pij , ∀j ∈ 1, 2, and2∑

j=1pij = 1

β0,St = β0,1S1t + β0,2S2t

β1,t,St = β1,t,1S1t + β1,t,2S2t

σ0,St = σ0,1S1t + σ0,2S2t

where for m ∈ 1, 2, if St = m, then Smt = 1, and Smt = 0 otherwise.Dr. Matt Brigida, [email protected] Markov Regime-Switching in Energy Markets


Overview of Kim (1994) Algorithm

Initial Parameters

Kalman Filter

Hamilton Filter

L (θ) =∑Tt=1 ln (f (yt |ϕt−1))

Approximations

t − 1 = tSmoothed RegimeProbabilities

Smoothed TimeVarying Coefficients



Overview of Kim (1994) Algorithm

We then take the estimated parameters θ̂ and use them in thesmoothing algorithm. The smoothing algorithm will give us:

Smoothed regime probabilities: ∀t, j, P[St = j|ϕT ]Smoothed estimates of the time-varying coefficients: βt|T



State Space Model with Markov Switching: NG & Oil

Time

Sm

ooth

ed B

eta

1995 2000 2005 2010

0.30

0.45

0.60

Time

Pro

b S

tate

2

1995 2000 2005 2010

0.0

0.4

0.8



Contact Info

work: [email protected]: http://complete-markets.com

Github: github.com/Matt-BrigidaTwitter: @mattbrigida



References

Brown, R. L., J. Durbin, and J. M. Evans (1975). Techniques fortesting the constancy of regression relationships over time. Journalof the Royal Statistical Society B37, 149–192.Diebold, F.X., J.H. Lee, and G.C. Weinbach (1994). Regimeswitching with time varying transition probabilities. In‘Nonstationary Time Series Analysis and Cointegration’. OxfordUniversity press.Filardo, A.J. (1998). Choosing information variables for transitionprobabilities in a time-varying transition probability Markovswitching model. Federal Reserve Bank of Kansas City researchpaper.Gregory, A.W. and B.E. Hansen (1996). Residual-based tests forcointegration in models with regime shifts. Journal ofEconometrics 70, 99–126.



References

Hamilton, J.D. (1989). A new approach to the economic analysisof nonstationary time series and the business cycle. Econometrica57, 357-384.Harrison, P.J. and C.F. Stevens (1976). Bayesian forecasting.Journal of the Royal Statistical Society, Series B, 38, 205–247.Harvey, A.C. (1989). Forecasting structural time series models andthe Kalman filter. Cambridge: Cambridge University Press.Kim, C.J. (1994). Dynamic linear models with Markov-switching.Journal of Econometrics 60, 1-22.


Documents

Markov Regime-Switching (and some State Space) Models in ...past.rinfinance.com/agenda/2015/talk/MatthewBrigda.pdf · Markov Regime-Switching (and some State Space) Models in Energy