Forecasting electricity spot prices using time-series ...€¦ · Forecasting electricity spot prices using time-series models with a double temporal segmentation ... Markov-switching

Forecasting electricity spot prices usingtime-series models with a double temporal

segmentation

Marie Bessec∗, Julien Fouquau∗∗, Sophie Meritet∗

∗LEDa-CGEMP, University Paris Dauphine∗∗NEOMA Business School

IAEE 37th International Conference - New York 2014

M. Bessec, J. Fouquau, S. Meritet Forecasting electricity spot prices

Brief summaryFeatures of electricity prices

Modeling and forecasting proceduresDay-ahead forecasting

Aim of the paper: short-run forecast of day-aheadelectricity spot prices in France

Features of electricity prices⊲ Seasonality⊲ Sudden and fast-reverting price spikes

Modeling and forecasting procedure⊲ Double temporal segmentation⊲ Non-linear models: MS and STR models⊲ Forecast combinations

Extensive evaluation: 2880 models, 1728 combinations

Results of the out-of-sample evaluation on French data⊲ Considering each season separately improves the results⊲ Non-linear models leads to better forecasts⊲ Pooling results provide more reliable forecasts




Related literature

Table: Related papers on electricity price forecasting

Authors Market

Combinations Bordignon et al. (2013) UKPXNowotarski et al. (2013) EEX, NP, PJM

Non-linear models Karakatsani and Bunn (2008) UKPXMisiorek et al. (2006) CaliforniaWeron and Misiorek (2008) California and NPKosater and Mosler (2006) EEX

Seasonality Nowotarski et al. (2013) NSW, EEX, ISONP, NYISO, PJM




Spikes and drops




Seasonality

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour

Pric

e (�

/MW

h)

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4Season

Pric

e (�

/MW

h)

intraday cycle intrayear cycle




Seasonality

High thermal sensitivity in France: 35% of the total stock of housingheated with electricity (RTE, 2012)




DataTemporal segmentationLinear versus non-linear modelsIndividual versus pooled forecasts

Dependent variable: Hourly day-ahead prices (e/MWh) in EPEX

Explanatory variables (available before the day-ahead auction)

Demand forecast (in MW): available for each trading hour at 0:00in t-1,

Capacity margin (in MW): available at 8 p.m. in t-1,

Volatility of spot prices (in e/MWh): the coefficient of variation ofthe hourly prices over the last 5 days,

Past values of spot prices (in e/MWh),

Gas price (in e/MWh): the Dutch day-ahead price (TTF); toavoid endogeneity problem, we use lag-1 price,

Forecasted balance of exchange programs with Germany (inMW): available for each hour at the end of t-1.





Temporal segmentation

Significant variation of coefficients of the regressors acrosstrading hours and seasons

Figure: Coefficients of the drivers in linear modelspt = α + φ1pt−1 + φ2pt−2 + β1Capat−1 + β2Demandt + β3Volatt−1 + β4Gast−1 + β5exchgt−1 + εt

2 4 6 8 10 12 14 16 18 20 22 24−2

−1

0

1

2

3intercept

2 4 6 8 10 12 14 16 18 20 22 240

0.5

1

1.5price(−1)

2 4 6 8 10 12 14 16 18 20 22 24−0.5

0

0.5

1price(−2)

2 4 6 8 10 12 14 16 18 20 22 24−0.2

−0.1

0

0.1

0.2margin(−1)

2 4 6 8 10 12 14 16 18 20 22 240

0.1

0.2

0.3

0.4first−differenced demand

2 4 6 8 10 12 14 16 18 20 22 24−10

−5

0

5price volatility

2 4 6 8 10 12 14 16 18 20 22 24−3

−2

−1

0

1first−differenced gas price(−1)

2 4 6 8 10 12 14 16 18 20 22 24−0.04

−0.03

−0.02

−0.01

0

0.01exchange with DE(−1)

WinterSpringSummerFallNon seasonal

⇒ Double temporal segmentation: each hour of the day and eachseason of the year is considered separately.





Non linear models

Markov-switching models

pt = α(St ) + β′(St )xt + εt(St )

where εt (St ) → NID(0, σ2(St )), St = 1, 2, . . . ,M a first-order Markovchain.

Smooth Threshold Regressive models

pt = α0 + β′

0xt + (α1 + β′

1 xt ) G(qt ; γ, c) + εt

with εt i.i.d .(

0, σ2ε

)

and:

G(qt ; γ, c) = [1 + exp(−γ(qt − c))]−1 , γ > 0

Linear model pt = α+ β′xt + εt , t = 1, . . . ,T





Individual Models Code

Autoregressive model ARExponential model EXPOAR model augmented with forecasted demand AR-XLinear regression with regressors selected with a stepwise procedure LSTEPMS model, 2 regimes, FTP, variable intercept and variance MS 1MS model, 2 regimes, FTP, variable intercept, AR coefficients and var. MS 2MS model, 2 regimes, FTP, variation of coefficients MS 3MS model, 2 regimes, TVTP, variable intercept and variance MSV 1MS model, 2 regimes, TVTP, variable intercept, AR coefficients and var. MSV 2MS model, 2 regimes, TVTP, variation of all coefficients MSV 3MS model, 3 regimes, FTP, variation of intercept and variance MS3 1MS model, 3 regimes, FTP, variable intercept, AR coefficients and var. MS3 2STR model, variable intercept STR 1STR model, variable intercept and AR coefficients STR 2STR model, variation of all coefficients STR 3





Forecast combinations

Combination of the individual forecasts pct =

∑Kk=1 ωt,k p(k)

twith alternative weights for k = 1, . . . ,K :

ωt,k = 1K (simple average) [C1]

ωt,k =(∑t−1

τ=t−l e2τ,k )

−1

∑Kj=1(

∑t−1τ=t−l e2

τ,j)−1

[C2]

ωt,k =v−1/2

t−1,k exp[−e2

t−1,k2vt−1,k

]ωt−1,k

∑Ki=1 v−1/2

t−1,i exp[−e2

t−1,i2vt−1,i

]ωt−1,i

[C3]

where eτ,k = pτ − p(k)τ , ω1,k = 1/K and vt−1,k = 1

t−1

∑t−1τ=1 e2

τ,k .




Empirical designOut-of-sample results

Real time evaluation: rolling forecasts of the last 35observations of each season in 2012

Evaluation of the forecast accuracy:

⊲ RMSE =√

1N

∑Nt=1 et (1)2

⊲ MAE = 1N

∑Nt=1 |et (1)|

⊲ MAPE = 1N

∑Nt=1 |

et (1)pt+1

|

Tests for comparing predictive accuracy⊲ Diebold-Mariano test, Giacomini-White test⊲ Encompassing test

Extensive evaluation: 15 individual models, 9 combinationsover each trading hour h = 1, . . . ,24, 4 seasons

2880 individual models and 1728 combinations





Figure: Estimation and forecasting windows





Temporal segmentation

Comparison of seasonal and non-seasonal models (in %)Winter Spring Summer Fall

% of times the seasonal model yields a lower MSE or MAE than the non-seasonal model

Seasonal beats non seasonal 69.1 77.6 30.4 54.5Seasonal beats non seasonal (AR & EXPO) 95.8 77.1 52.1 70.8Seasonal beats non seasonal (linear models) 89.6 80.9 36.1 64.6

Significance of difference with the Diebold-Mariano test

Seasonal beats non seasonal 32.4 42.7 13.4 24.5Seasonal beats non seasonal (AR & EXPO) 50.0 51.0 20.8 41.7Seasonal beats non seasonal (Linear models) 46.9 50.4 16.7 33.0

Encompassing test (Harvey, Leybourne and Newbold, 1997)

H0: NS encompasses SA 34.1 66.5 31.3 60.7H0: NS encompasses SA (AR & EXPO) 52.1 56.3 33.3 50.0H0: NS encompasses SA (Linear models) 45.8 66.0 27.8 54.9





Linear versus non-linear models

Comparison of linear and non-linear models (in %)Winter Spring Summer Fall

SA NS SA NS SA NS SA NS

% of times the non-linear model yields a lower MSE or MAE than the linear model

Non-linear beats linear 31.8 36.1 72.6 67.7 42.0 58.5 55.4 57.3MS3 1 beats linear 41.7 54.9 77.8 77.8 55.6 77.8 69.4 79.9STR beats linear 22.0 35.4 66.9 65.5 42.1 42.8 44.0 33.1


Non-linear beats linear 9.2 11.7 41.2 29.2 15.6 21.2 20.6 29.7MS3 1 beats linear 14.6 13.9 52.1 47.2 27.1 27.1 20.8 52.8STR beats linear 4.2 13.0 37.3 23.4 14.4 12.3 19.0 14.6


H0: Linear encompasses NL 34.7 40.0 71.6 68.2 61.2 64.4 57.2 61.9H0: Linear encompasses MS3 1 38.9 47.2 73.6 76.4 76.4 75.0 65.3 79.2H0: Linear encompasses STR 31.9 39.4 68.1 69.0 59.3 60.7 46.3 47.7





Table: Best models - RMSE

Winter Spring Summer FallModel SA NS SA NS SA NS SA NS Total

Linear 13 7 2 2 12 7 5 4 52AR 9 6 1 1 6 4 2 1 30expo 4 0 1 1 6 1 3 3 19

AR-X 0 1 0 0 0 2 0 0 3MS 7 13 18 16 9 16 12 17 108MS 1 4 3 0 2 2 0 1 2 14MS 2 0 1 3 0 1 0 1 2 8MS 3 1 0 2 1 1 0 2 0 7MSV 1 0 1 1 3 2 2 2 0 11MSV 2 1 0 2 1 0 3 2 3 12MSV 3 0 3 2 0 1 0 1 0 7MS3 1 0 5 7 7 1 8 3 9 40MS3 2 1 0 1 2 1 3 0 1 9STR 4 4 4 6 3 1 7 3 32STR 1 3 0 3 0 3 1 4 3 17STR 2 1 2 1 1 0 0 1 0 6STR 3 0 2 0 5 0 0 2 0 9Total 24 24 24 24 24 24 24 24 192

(•) = number of times each model gives the best RMSE.





Table: Rank of individual and pooled forecasts in term of RMSE

Winter Spring Summer Fall AllModel SA NS SA NS SA NS SA NS all

AR 6 8 23 23 12 20 23 20 22EXPO 3 6 22 22 10 23 22 21 19AR-X 15 19 10 19 21 14 7 14 14MS 1 10 12 19 17 14 13 13 6 12MS 2 21 20 10 16 20 16 16 9 16MS 3 20 23 6 18 16 15 9 18 15

MSV 1 11 14 16 7 11 8 17 10 11MSV 2 16 18 6 14 23 12 15 4 13MSV 3 19 16 8 20 22 18 14 16 17MS3 1 12 9 9 6 9 10 12 2 8MS3 2 17 22 14 21 18 17 20 12 18STR 1 18 21 12 13 17 21 19 19 20STR 2 22 15 15 12 15 19 18 22 21STR 3 23 17 21 11 18 22 21 23 23

C1 7 5 3 1 7 3 3 7 3C1 L 9 7 2 3 5 1 1 3 2

C1 NL 13 11 4 2 6 5 6 1 6C2 5 3 5 5 4 2 2 5 1

C2 L 2 2 17 9 1 6 5 15 5C2 NL 4 4 13 8 2 4 4 13 4

C3 8 13 18 10 8 11 8 8 10C3 L 1 1 20 15 3 9 10 17 7

C3 NL 14 10 1 4 12 7 11 11 9





Table: Rank of individual and pooled forecasts in term of RMSE

Winter Spring Summer Fall AllModel SA NS SA NS SA NS SA NS all

AR 6 8 23 23 12 20 23 20 22EXPO 3 6 22 22 10 23 22 21 19AR-X 15 19 10 19 21 14 7 14 14MS 1 10 12 19 17 14 13 13 6 12MS 2 21 20 10 16 20 16 16 9 16MS 3 20 23 6 18 16 15 9 18 15

MSV 1 11 14 16 7 11 8 17 10 11MSV 2 16 18 6 14 23 12 15 4 13MSV 3 19 16 8 20 22 18 14 16 17MS3 1 12 9 9 6 9 10 12 2 8MS3 2 17 22 14 21 18 17 20 12 18STR 1 18 21 12 13 17 21 19 19 20STR 2 22 15 15 12 15 19 18 22 21STR 3 23 17 21 11 18 22 21 23 23

C1 7 5 3 1 7 3 3 7 3C1 L 9 7 2 3 5 1 1 3 2

C1 NL 13 11 4 2 6 5 6 1 6C2 5 3 5 5 4 2 2 5 1

C2 L 2 2 17 9 1 6 5 15 5C2 NL 4 4 13 8 2 4 4 13 4

C3 8 13 18 10 8 11 8 8 10C3 L 1 1 20 15 3 9 10 17 7

C3 NL 14 10 1 4 12 7 11 11 9





Forecast combinations

Table: Comparison of individual and combined forecasts (in %)

Winter Spring Summer FallSA NS SA NS SA NS SA NS

% of times the pooled models yield a lower MSE or MAE than the individual model

Combined beats individual 67.3 70.4 61.9 70.9 74.6 73.7 71.9 66.6


Combined beats individual 25.2 26.9 27.1 36.4 43.9 38.9 35.2 37.5


H0: indv encompasses comb 29.3 39.7 43.3 56.3 66.8 59.4 54.4 52.5




Conclusion

Main resultsThe double temporal segmentation improves theforecasting ability of the models.Non-linear models designed to capture the sudden andfast-reverting spikes in the price dynamics improve theforecast accuracy.Pooling forecasts gives more reliable results.

Possible extensionsEnlarge the comparison of the specifications with a largerset of regressors.Explore how revised weather forecasts available in themorning before the auction could add extra value beyondthe midnight release of demand forecasts.


Documents

Forecasting electricity spot prices using time-series ...€¦ · Forecasting electricity spot prices using time-series models with a double temporal segmentation ... Markov-switching