View
7
Download
0
Category
Preview:
Citation preview
Exponential Smoothing
INSR 260, Spring 2009Bob Stine
1
Overview Smoothing
Exponential smoothing
Model behind exponential smoothingForecasts and estimatesHidden state model
Diagnostic: residual plots
Examples! ! ! ! (from Bowerman, Ch 8,9)Cod catchPaper sales
2
SmoothingHeuristic! ! Data = Pattern + Noise
Pattern is slowly changing, predictableNoise may have short-term dependence, but by-and-large is irregular and unpredictable
IdeaIsolate the pattern from the noise by averaging data that are nearby in time.
Noise should mostly cancel out, revealing the patternExample: moving averages
Example: JMP’s spline smoothing uses different weights3
st = yt!w+···+yt!1+yt+yt+1+···+yt+w
2w+1
Simple Exponential SmoothMoving averages have a problem
Not useful for prediction:Smooth st depends upon observations in the future.Cannot compute near the ends of the data series
Exponential smoothing is one-sidedAverage of current and prior valuesRecent values are more heavily weighted than Tuning parameter α = (1-w) controls weights (0≤w<1)
Two expressions for the smoothed value
4
!t = yt+wyt!1+w2yt!2+···1+w+w2+···
Weighted average Predictor/Corrector
!t =yt
1 + w + w2 + · · · +w(yt!1 + wyt!2 + · · ·
1 + w + w2 + · · ·= (1! w)yt + w!t!1
= "yt + (1! ")!t!1
= !t!1 + "(yt ! !t!1)
Smoothing Constantα controls the amount of smoothingα ≈ 0 very smoothα ≈ 1 little smoothing
Example (Bowerman): monthly tons, cod
5
250
300
350
400
Cod (
tons)
0 5 10 15 20 25
Rows
Overlay Plot
Your impression
of the smooth?
!t = !t!1 + "(yt ! !t!1)
Table 6.1
Example: SplinesInterpolating polynomial
always possible to find a polynomial for which p(x)=y when there is one y for each xJMP interactive tool
Cod catch
6
275
300
325
350
375
400
425
Cod (
tons)
0 5 10 15 20 25
Time
Smoothing Spline Fit, lambda=0.095749
R-Square
Sum of Squares Error
0.859312
3702.189
Smoothing Spline Fit, lambda=0.095749
Bivariate Fit of Cod (tons) By Time
275
300
325
350
375
400
425
Cod (
tons)
0 5 10 15 20 25
Time
Smoothing Spline Fit, lambda=29.62771
R-Square
Sum of Squares Error
0.133897
22791.47
Smoothing Spline Fit, lambda=29.62771
Bivariate Fit of Cod (tons) By Time
Example: Exponential SmoothJMP formula similar to Excel
7
275
300
325
350
375
400
Cod C
atc
h
0 5 10 15 20 25
Rows
Overlay Plot
Y
Cod (tons) ExpSmth .05 ExpSmth 0.2
ExpSmth 0.5 ExpSmth 0.8
Which is best?
!t = !t!1 + "(yt ! !t!1)
What value should we use for α?
ModelNeed statistical model to
Express source of randomness, uncertaintyChoose an optimal estimate for αDefine predictor and quantify probable accuracy
Want to have prediction intervals for exponential smoothing
Latent variable model (“state-space models”)Assume each observation has mean Lt-1 ! ! ! ! ! ! ! yt = Lt-1 + εtMean values fluctuate over time! ! ! ! ! ! ! Lt = Lt-1 + α εtDiscussion
Lt is the state and is not observedIf α = 0, Lt is constantIf α = 1, Lt is just as variable as the data
8
εt ~ N(0,σ2)
PredictionsModel implies a predictor and method for finding prediction intervals
Observations have mean Lt-1! ! ! ! ! yt = Lt-1 + εtMeans fluctuate over time!! ! ! ! ! Lt = Lt-1 + α εtErrors are normally distributed! ! ! ! εt ~ N(0,σ2)
Predictor is constantE yn+1 = Ln! ! ! ! ! ! ! ! ! ! ! ! ! ŷn+1 = Ln E yn+2 = Ln+1 = Ln + αεn+1!! ! ! ! ! ! ! ŷn+2 = Ln E yn+3 = Ln+2 = Ln+1+αεn+2 = Ln+α(εn+2+εn+2)!! ŷn+2 = Ln In general, set ŷn+f = Ln
Variance of prediction errors growsE(yn+1-ŷn+1)2 = E(εn+1)2 = σ2
E(yn+f-ŷn+f)2 = E(εn+f + α(εn+f-1 +…+ εn+1))2 = σ2(1+(f-1)α2)
9
Estimating α Model
Observations have mean Lt-1! ! ! ! ! yt = Lt-1 + εtMean values fluctuate over time!! ! ! Lt = Lt-1 + α εt
Correspondencelt is our estimate of Lt
â is our estimate of α (text uses , see page 392)
EstimationLike doing least squares but you don’t get to see how well your model captures the underlying state since it is not observed!Choose â based on forecasting
If Lt-1 were observed, we’d use it to predict yt: it’s the mean of yt
Pick â to minimize the sum of squared errors, Σ(yt - lt-1)2Estimation is not linear in the data
10
εt ~ N(0,σ2)
!̂
JMP ResultsTechniques for estimating â
Text illustrates using the Excel solverWe’ll use JMP’s time series methods
Analyze > Modeling > Time Series
Simple exponential smoothingLots of outputResults confirm little smoothpattern; near constant
11
DF
Sum of Squared Errors
Variance Estimate
Standard Deviation
Akaike's 'A' Information Criterion
Schwarz's Bayesian Criterion
RSquare
RSquare Adj
MAPE
MAE
-2LogLikelihood
22
26315.4779
1196.15809
34.5855184
232.424394
233.559888
-0.174024
-0.174024
9.14722694
31.0025784
230.424394
Stable
Invertible
Yes
Yes
Hessian is not positive definite.
Model Summary
Level Smoothing Weight
Term
0.00001974
Estimate
0
Std Error
.
t Ratio
0.0000*Prob>|t|
Parameter Estimates
260
280
300
320
340
360
380
400
420
Predic
ted V
alu
e
0 10 20 30 40 50
Row
Forecast
â≈0
Book “cheats” a little by setting y0 to mean of first 12 rather than smoothing after y1. Finds â≈ 0.03.
DiagnosticsSequence plot of residuals
One-step ahead prediction errors, yt - ŷtNormal quantile plot
No visual pattern remainsBut we will in a week or so more elaborate diagnostic routines associated with ARIMA modelsText discusses tracking methods
12
-100
-80
-60
-40
-20
0
20
40
60
Resid
ual V
alu
e
0 5 10 15 20 25
Row
-100
-50
0
50
1 2 3 4 5 6
Count
.01 .05.10 .25 .50 .75 .90.95 .99
-3 -2 -1 0 1 2 3
Normal Quantile Plot
ExampleData are weekly sales of paper towels
Goal is to forecast future salesUnits of data are 10,000 rolls
Level appears to change over time, trending down then up.
13
Table 9.1
0
5
10
15
20
Rolls (
x10000)
0 20 40 60 80 100 120
Row
Exponential SmoothApply simple exponential smoothing
Model results not very satisfyingValue for smoothing parameter, â = 1 (max allowed)Forecasts are constant
Motivates alternative smoothing methods
14
0
5
10
15
20
25
Predic
ted V
alu
e
0 20 40 60 80 100 120 140 160
Row
Level Smoothing Weight
Term
1.0000000
Estimate
0.1110102
Std Error
9.01
t Ratio
<.0001*Prob>|t|
Parameter Estimates
DF
Sum of Squared Errors
Variance Estimate
Standard Deviation
Akaike's 'A' Information Criterion
Schwarz's Bayesian Criterion
RSquare
RSquare Adj
MAPE
MAE
-2LogLikelihood
118
143.840613
1.21898825
1.10407801
362.267669
365.046793
0.93712456
0.93712456
18.7016851
0.86251008
360.267669
Stable
Invertible
Yes
Yes
Model Summary
SummarySmoothing!! ! ! ! ! ! ! ! ! ! locate patterns
Exponential smoothing!! ! ! ! ! ! ! uses past
Model for exponential smoothing!! ! latent state
Diagnostic: residual plots! ! ! ! ! patternless
DiscussionDesire predictions that are more dynamicExtrapolate trends
Linear patternsSeasonal patterns
15
Recommended