Embed Size (px)
Citation preview
1
Appendix B: A Primer of Time Series Forecasting Models
B.1 A Primer of Time Series Forecasting ModelsB.1 A Primer of Time Series Forecasting Models
2
The Universal Time Series Model
( ) ( , , )t t t tg Y f T S I
TREND
SEASONAL
ERROR(Irregular)
TRANSFORMATION
3
Additive Decomposition of the Airline Data
T: LinearTrend
S: SeasonalAverage
I: IrregularComponent
4
Types of Models
)()( tt IfYg
),()( ttt ITfYg
),()( ttt ISfYg
),,()( tttt ISTfYg
Stationary Only
Trend and Stationary
Seasonal and Stationary
Trend, Seasonal, and Stationary
5
Exponential Smoothing Models (ESM) Stationary Only
– Simple Exponential Smoothing (one parameter) Trend and Stationary
– Simple Exponential Smoothing (one parameter)– Linear (Holt) Exponential Smoothing (two
parameters)– Damped-Trend Exponential Smoothing (three
parameters)
continued...
6
Exponential Smoothing Models (ESM) Seasonal and Stationary
– Seasonal Exponential Smoothing (two parameters) (Both additive and multiplicative types are supported.)
Trend, Seasonal, and Stationary– Holt-Winters Additive (three parameters)– Holt-Winters Multiplicative (three parameters)
7
Exponential Smoothing Premise Weighted averages of past values can produce good
forecasts of the future. The weights should emphasize the most recent data. Forecasting should require only a few parameters. Forecast equations should be simple and easy to
implement.
8
ESM as Weighted AveragesW
eig
hts
Y1 Y2 Y3 Y4 Y5 Y6 Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8
Weights applied to past values to predict Y9
Y7 Y8
Sample Mean Random Walk
9
ESM as Weighted Averages
nw
YYn
Yn
YwYwYwYwY
t
n
ttt
n
t
nn
n
tttn
1
11
ˆ
11
22111
1
We
igh
ts
Sample Mean
The mean is a weightedaverage where all weightsare the same.
8
19 8
1ˆt
tYY
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8
10
ESM as Weighted Averages
1,,2,1for 0,1
ˆ1
1
ntww
YYwY
tn
n
n
tttn
Random Walk
89̂ YY
A random walk forecastis a weighted average where all weights are 0except the most recent,which is 1.
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8
11
The Exponential Smoothing CoefficientForecast Equation
it
T
i
i
tttt
tttt
ttt
ttt
ttt
Y
YYYY
YYYY
YYY
YYY
YYY
0
23
22
1
222
1
12
1
11
1
)1(
ˆ)1()1()1(
]ˆ)1([)1()1(
ˆ)1()1(
]ˆ)1()[1(
ˆ)1(ˆ
12
Simple Exponential Smoothing
As the parameter grows larger, the most recent values are emphasized more.
5.0W
eig
hts
25.0
Y3 Y4 Y5 Y6 Y7 Y8 Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8
Weights applied to past values to predict Y9
13
ESM for Seasonal Data
Weights decay with respect to the seasonal factor.
We
igh
ts
Jan00Jan01 Jan02 Jan03 Jan04 Feb00
Feb01 Feb02 Feb03 Feb04
…
14
ESM Seasonal Factors
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s
First seasonal factor s1 is always “natural.”First season: January, Monday, Q1
Additive model: factors average to 0
Multiplicative model: factors average to 1
Monthly Seasonal Factors
15
Smoothing Weights
The choice of a Greek letter is arbitrary. The software uses names rather than Greek symbols.
Level smoothing weight
Trend smoothing weight
Seasonal smoothing weight
Trend damping weight
16
ESM Parameters and Keywords
ESM Parameters Name in Repository
Simple simple
Double double
Linear (Holt) , linear
Damped-Trend , , damptrend
Seasonal , seasonal
Additive Winters , , addwinters
Multiplicative Winters , , winters
17
Box-Jenkins ARIMAX Models ARIMAX: AutoRegressive Integrated Moving Average
with eXogenous variables. AR: Autoregressive Time series is a function of its
own past. MA: Moving Average Time series is a function of
past shocks (deviations, innovations, errors, and so on).
I: Integrated Differencing provides stochastic trend and seasonal components, so forecasting requires integration (undifferencing).
X: Exogenous Time series is influenced by external factors. (These input variables can actually be endogenous or exogenous.)
18
Box-Jenkins ARMA Models Theory: Given a stationary time series, there exists
an ARMA model that approximates the true model arbitrarily closely universal approximator.
Reality: Given a stationary time series, there is no guarantee that you can find the best ARMA approximator.
Theory: Apply differencing operators until what remains is a stationary time series.
Reality: Differencing might not be the best way to model trend and seasonality. After differencing, the time series could still be nonstationary.
19
Box-Jenkins Forecasting Myths Myth: Box and Jenkins invented ARIMA models. Fact: Box and Jenkins brought together existing
theory and added some of their results, and thus popularized the use of ARIMA models.
Myth: Box-Jenkins forecasting only works for stationary time series.
Fact: Box-Jenkins forecasting provides a general methodology for forecasting any time series. ARIMA models are nonstationary models that can be decomposed into the usual trend, seasonal, and stationary components.
20
Historical Impediments to Box-Jenkins ModelingHistory Models are sophisticated and require training and
experience to use them successfully. Modelers are prone to overfitting the data, which leads
to poor forecasts. Software is unavailable, unreliable, or too slow for
forecasting many time series.
Today Techniques exist for automatic model selection. Honest assessment techniques prevent overfitting. Modern software is reliable and fast.
21
ARIMA Model SpecificationARIMA(p, d, q)(P, D, Q)
p indicates a simple autoregressive order.
P indicates a seasonal autoregressive order.
.
12
1
3
1
t1212
332211
1
tt
stst
ttttt
ttt
yy
sdatamonthlyFor
yyP
yyyyp
yyp
22
AR(1): The Toothpaste Series
ttt tpsttpst 18.
23
ARIMA Model SpecificationARIMA(p, d, q)(P, D, Q)
d indicates a simple difference of the series.
D indicates a seasonal difference.
.)(
12s
)(1
)(1
1212
1
ttt
sttts
ttt
yyy
datamonthlyFor
yyyD
yyyd
24
ARIMA (1, 1, 0)(0, 0, 0): The Crocs Series )()(8. 11 tttttt CrocCrocCrocandCrocCroc
25
ARIMA Model SpecificationARIMA(p, d, q)(P, D, Q)
q indicates a simple moving average order.
Q indicates a seasonal moving average order.
.
12s
1
3
1
1212t
332211
1
tt
ststt
ttttt
ttt
y
datamonthlyFor
yQ
yq
yq
26
ARIMA (0, 0, 1)(0, 1, 0): The Pork Bellies Series
Summer Peaks; “BLT effect”
)(4. 121 ttts
ttts PBPBPBandPB
27
Types of ARIMA Models
)()( tt IfYg
),()( ttt ITfYg
),()( ttt ISfYg
),,()( tttt ISTfYg
Stationary Only
Trend and Stationary
Seasonal and Stationary
Trend, Seasonal, and Stationary
ARIMAX models accommodate exogenous variables.
28
Intermittent Demand Models (IDM)Intermittent time series have a large number of values that are zero. These types of series commonly occur in Internet, inventory, sales, and other data where the demand for a particular item is intermittent. Typically, when the value of the series associated with a particular time period is nonzero, demand occurs. When the value is zero (or missing), no demand occurs.
Source: SAS®9 Online Help and Documentation
29
Intermittent Demand Data
Time
Demand
Mostly Zeros
30
Intermittent Demand Models (IDM)
Demand
Time
Size
Interval
31
Intermittent Demand Models (IDM)Demand
SizeDemandInterval
Index Index
Average Demand=Demand Size divided by Demand Interval
32
Two IDM Choices Croston’s Method = Two smoothing models
– The Interval component is fit with an ESM. – The Size component is fit with an ESM.– The forecast of Average Demand is
Forecast Size/Forecast Interval.
Average Demand Method = One smoothing model– Average demand is calculated directly from the
data and forecast with an ESM.
33
Unobserved Components Models (UCMs)Unobserved components models are also called structural models in the time series literature. A UCM decomposes the response series into components such as trend, seasonals, cycles, and the regression effects due to predictor series. The components in the model are supposed to capture the salient features of the series that are useful in explaining and predicting its behavior.
Source: SAS®9 Online Help and Documentation
34
Unobserved Components Models (UCMs) also known as structural time series models decomposed time series into four components:
– trend– season– cycle– Irregular
General form:
Yt = Trend + Season + Cycle + Regressors
35
UCMs Each component captures some important feature of
the series dynamics. Components in the model have their own models. Each component has its own source of error. The coefficients for trend, season, and cycle are
dynamic. The coefficients are testable. Each component has its own forecasts.
36
Types of UCM Models
)()( tt IfYg
),()( ttt ITfYg
),()( ttt ISfYg
),,()( tttt ISTfYg
Stationary Only
Trend and Stationary
Seasonal and Stationary
Trend, Seasonal, and Stationary
UCM models accommodate exogenous variables.
37
Types of Models
UCM Statement Model Types
irregular Stationary Only (White Noise)
level, slope, irregular Trend and Stationary
season (or cycle), irregular
Seasonal and Stationary
all statements Trend, Seasonal, and Stationary
38
Specifying UCMsUnobserved components models are available through the HPFDIAGNOSE and HPFUCMSPEC procedures. The syntax used by these procedures is similar to that used by the UCM procedure in SAS/ETS software.
39
Which Model Type? Performance: time required to derive coefficients
and create forecasts Accuracy Usability: ease of going from data to forecasts and
interpreting results
40
PerformanceBest to worst:
1. ESM
2. IDM
3. ARIMAX
4. UCM
41
AccuracyBest to Worst:
1. ARIMAX, UCM
2. ESM
Intermittent Demand - Best to Worst:
1. IDM (when appropriate).
2. Others can be used, but they generally provide unacceptable accuracy.
42
UsabilityBest to worst:
1. ESM
2. UCM
3. ARIMAX
43
Mean Absolute Percent Error (MAPE)Absolute percent error for one time point: 100% |Actual-Forecast|/Actual
MAPE is one of the most common accuracy measures in business forecasting. As a selection criterion, choose the model with the smallest value of MAPE.
Interpretation the size of forecast error relative to the magnitude of the actual value
Mean absolute percent error
the average of all of the individual absolute percent errors
44
Mean Absolute Error (MAE)Absolute error for one time point: |Actual-Forecast|
MAE is not commonly used as an accuracy measure in business forecasting. As a selection criterion, choose the model with the smallest value of MAE.
Interpretation the size of the forecast error
Mean absolute error
the average of all of the individual absolute errors
45
MAPE versus MAEHoliday Sales
DayLow Sales Day
Actual 1,000 300
Forecast 900 400
APE 10% 33.3%
AE 100 100
Mean
21.65%
100
An error of 100 on a large sales day is usually not as serious as an error of 100 on a low sales day,
but MAE weights both equally.
MAPE
MAE
46
Root Mean Square Error (RMSE)Squared error for one time point: (Actual-Forecast)2
RMSE is commonly used as an accuracy measure in industrial, economic, and scientific forecasting. As a selection criterion, choose the model with the smallest value of RMSE.
Interpretation The squared size of the forecast error
Mean squared error (MSE)
The average of all of the individual squared errors, adjusted for the number of estimated model parameters
Root mean square error
The square root of MSE
47
Classes of Models Exponential smoothing models ARIMAX models UCM models Simple regression models
– are predefined trend components: linear, quadratic, cubic, log-linear, exponential, and so on
– are predefined seasonal dummies– include a combination of one or more simple
predefined components Simple models
– the mean– a random walk– a random walk with drift
48
Performance Simple models have no performance issues. Exponential smoothing models can be constructed
quickly and easily, so they always have good performance.
ARIMAX models require many more computer cycles than simple or exponential smoothing models, but are based on algorithms that were refined over the past 30 years. Thus, creating a custom fit ARIMAX model is feasible even for large numbers of series.
UCM models are very computer intensive and should be tried only on small data sets or individual time series.
49
Forecasting with SAS Forecast StudioFunctionality: Only automatically generated and custom ARIMAX or
UCM models accommodate event, input, and outlier (exogenous) variables.
Pre-existing ESM models and ARIMA models (for example, those shipped in the default catalog) do not accommodate exogenous variables.
Automatically generated ARIMAX models can select best combinations of exogenous variables for each series diagnosed (identified).
Custom, user-defined ARIMAX models must be specified to explicitly accommodate exogenous variables.
50
Static Linear Regression with Two Variables Y = 0 + 1X1 + 2X2 +
Y is the target (response/dependent) variable.
X1 and X2 are input (predictor/independent) variables.
is the error term.
0, 1, and 2 are parameters.
0 is the intercept or constant term.
1 and 2 are partial regression coefficients.
51
Time Series RegressionStatic Regression
Time Series Regression with Ordinary Regressors
Time Series Regression with Dynamic Regressors
kk XXY ...110
tktktt XXY ...110
tmtkkmtkktkk
mtmttt
mtmttt
kkXXX
XXX
XXXY
,1,1,0
,221,2,2,220
,111,111,1100
22
11
52
Common Transfer FunctionsContemporaneous Regression
Model
0)( B
tt
ttt
BZB
ZXY
)()(00
53
Common Transfer FunctionsDynamic Regression: One Lag Term
Model
BB 10)(
tt
tttt
BZB
ZXXY
)()(1100
54
Common Transfer FunctionsDynamic Regression: One Shifted Term
Model
kkBB )(
tt
tktkt
BZB
ZXY
)()(0
55
Common Transfer FunctionsDynamic Regression: One Shifted and One Lag Term
Model
221)( BBB
tt
tttt
BZB
ZXXY
)()(22110
