7/30/2019 Forecasting Models.pdf
1/45
Advanced Techniques for Engineering
Analysis
Forecasting Models
GC University Lahore
7/30/2019 Forecasting Models.pdf
2/45
2
What is Forecasting?
Forecasting is attempting to predict the future
i.e. expected state of an event, condition or
variable at some future time index
Decision makers want to reduce the risk by
predicting future events or values of related
variables
7/30/2019 Forecasting Models.pdf
3/45
3
Forecasting Methods
Forecasting methods
Qualitative
intuitive, educated guesses that may or may not depend on
past data
Quantitative
based on mathematical or statistical models
7/30/2019 Forecasting Models.pdf
4/45
4
Forecasting Methods contd
We will consider two types of forecasts based on
mathematical models:
Regression forecasting
Time Series forecasting
7/30/2019 Forecasting Models.pdf
5/45
5
Regression Forecasting
We predict by modeling the relationship
between dependent variable and independent
variables
We use some latest observations to fit a least
square regression line to data and then use it for
forecasting purpose.
7/30/2019 Forecasting Models.pdf
6/45
6
Regression Forecasting contd
Suppose that Yis a variable of interest, andX1,,Xp are explanatory or predictor variables
such that
Y=f(X1, ,Xp; )
fis the mathematical modelthat determines
the relationship between the variable ofinterest and the explanatory variables
= (0, , m) are the modelparameters.
7/30/2019 Forecasting Models.pdf
7/45
7
Regression Forecasting contd
1. We choose to minimize
2. We use least squares estimation to estimate
3. Then we forecasty as
.);,,(1
p
xxfy
n
i
n
iiipiiexxfy
1 1
22
1.);,,(
7/30/2019 Forecasting Models.pdf
8/45
8
Regression Forecasting contd
Examples of Regression Models:
)st variableindependen
with twomodellinear(.5
)modelquadratic(.4
)modelgrowthlexponentia()exp(.3
)modelregressionlinearsimple(.2
)modelmeanconstant(.1
22110
2
210
10
10
0
iiii
iiii
iii
iii
ii
xxy
xxy
xy
xy
y
7/30/2019 Forecasting Models.pdf
9/45
9
Constant Mean Regression
Suppose that theyis are a constant value plus noise:
yi = 0 + i,
i.e., = 0.We want to determine the value of0 that minimizes
.)()(1
2
00
n
i
iyS
),0(~ 2 Ni
7/30/2019 Forecasting Models.pdf
10/45
10
Constant Mean Regression contd
Taking the derivative ofS(0) gives
Finally setting this equal to zero leads to
Hence the sample mean is the least squaresestimator for0.
.)(2)(1
00
n
i
iyS
.1
1
0 yyn
n
i
i
y
7/30/2019 Forecasting Models.pdf
11/45
11
Constant Mean Regression contd
Example:yi = 0 + i,
y
98.30963
99.18569
101.2684
97.52997
103.4013
98.84521
111.1842
98.70812
93.08922 5.1020 y
80
85
90
95
100
105
110
115
120
1 2 3 4 5 6 7 8 9
y
7/30/2019 Forecasting Models.pdf
12/45
12
Simple Linear Regression
Consider the model
yi = 0 + 1xi+ i,
i.e., = (0, 1).We want to determine the values of0 and 1 that
minimize.)(),(
1
2
1010
n
i
ii xyS
7/30/2019 Forecasting Models.pdf
13/45
13
Simple Linear Regression contd
Setting the first partial derivatives equal to zero gives
n
i
iii
n
iii
xxyS
xyS
1
1010
1
11010
0
.0)(2),(
0)(2),(
7/30/2019 Forecasting Models.pdf
14/45
14
Simple Linear Regression contd
Solving for0 and 1 leads to the least squares
estimates
.
10
221
xy
xxn
yxyxn
ii
iiii
7/30/2019 Forecasting Models.pdf
15/45
15
Simple Linear Regression contd
Example:
x y
1 2.6
2.3 2.8
3.1 3.1
4.8 4.7
5.6 5.1
6.3 5.30
1
2
3
4
5
6
0 1 2 3 4 5 6 7
X
Y
7/30/2019 Forecasting Models.pdf
16/45
16
Simple Linear Regression contd
Example continued:
x y
1 2.62.3 2.8
3.1 3.1
4.8 4.7
5.6 5.16.3 5.3 93.3 85.3
99.109
16.103
6.23
1.236
2
yx
x
yx
y
xn
i
ii
i
i
68.158.0
0
1
7/30/2019 Forecasting Models.pdf
17/45
17
Simple Linear Regression contd
Example (continued):
Regression equation:
xy 58.068.1
0
1
2
3
4
5
6
0 1 2 3 4 5 6 7
Y
Yhat
7/30/2019 Forecasting Models.pdf
18/45
18
General Linear Regression
Consider the linear regression model
or
where xi = (1,xi1, ,xip) and = (0, , p).
iippiii xxxy 22110
iiiy
x
7/30/2019 Forecasting Models.pdf
19/45
19
General Linear Regression contd
Suppose that we have n observations of y i.e.yi ,i=1,2,..n. We introduce matrix notation and define
y = (y1, ,yn), = (1, , n),
Note that y is n 1, is n 1, and X is n (p + 1).
.
1
1
1
1
221
111
npn
p
p
xx
xx
xx
X
7/30/2019 Forecasting Models.pdf
20/45
20
General Linear Regression contd
Then we can write the regression model as
y has a mean vector and covariance matrixgiven by
where I is the n n identity matrix.
Xy
,)var(][ 2
IyXy
E
7/30/2019 Forecasting Models.pdf
21/45
21
General Linear Regression contd
Note that by var(y), we mean the matrix
This is a symmetric matrix.
)var(),cov(),cov(
),cov()var(),cov(),cov(),cov()var(
21
2212
1211
nnn
n
n
yyyyy
yyyyyyyyyy
7/30/2019 Forecasting Models.pdf
22/45
22
General Linear Regression contd
In matrix notation, the least squares criterion can beexpressed as minimizing
Setting , the least squares estimator is givenby
.)( 1 yXXX
.)()()()(1
2
n
i
iiyS XyXyx
0)(
S
Prediction forYis then given by
.)( 1 yXXXXXy
7/30/2019 Forecasting Models.pdf
23/45
23
General Linear Regression contd
Example: Simple Linear Regression
.)( 1 yXXX
x y
1 2.6
2.3 2.83.1 3.1
4.8 4.7
5.6 5.1
6.3 5.3
0
1
2
3
4
5
6
0 1 2 3 4 5 6 7
X
Y
7/30/2019 Forecasting Models.pdf
24/45
24
General Linear Regression contd
Example: Simple Linear Regression (p = 1)
x y
1 2.6
2.3 2.83.1 3.1
4.8 4.7
5.6 5.1
6.3 5.3
3.616.518.411.31
3.2111
X
3.51.57.41.3
8.26.2
y
7/30/2019 Forecasting Models.pdf
25/45
25
General Linear Regression contd
Example: Simple Linear Regression (p = 1)
99.1091.23
1.236XX
047.018.018.087.01
XX
58.068.11
yXXX
7/30/2019 Forecasting Models.pdf
26/45
26
Homework-1
1. Suppose that the following data represent the total costs and
the number of units produced by a company.
a. Graph the relationship betweenXand Y.
b. Determine the simple linear regression line relating YtoX.
c. Predict the costs for producing 10 units.
d. Compute the SST, SSR, SSE,R andR2. Interpret the value
ofR2.
Total Cost (Y) 25 11 34 23 32
Units Produced (X) 5 2 8 4 6
7/30/2019 Forecasting Models.pdf
27/45
27
Homework-1 contd
2. Consider the fuel consumption data on the next slide, and the
following model which relates fuel consumption (Y) to the
average hourly temperature (X1) and the chill index (X2):
a. Plot YversusX1 and YversusX2.
b. Determine the least squares estimates for the model parameters.
c. Predict the fuel consumption when the temperature is 35 and the chill
index is 10.
d. Compute the SST, SSR, SSE andR2. Interpret the value ofR2.
.22110 XXY
7/30/2019 Forecasting Models.pdf
28/45
28
Data for Problem 2
Average Hourly
Temperature,xi1
Chill Index,xi2 Weekly Fuel
Consumption,yi
28 18 12.4
32.5 24 12.3
28 14 11.7
39 22 11.2
57.8 16 9.5
45.9 8 9.4
58.1 1 8.0
62.5 0 7.5
7/30/2019 Forecasting Models.pdf
29/45
29
Time Series Forecasting
Time series
Sequence of observations of response variable at
regular time intervals
Stochastic or dynamic, it does change over time.
For forecasting
We use past history of response variable to predictthe future.
Predictions exploit correlations between past
history and the future.
7/30/2019 Forecasting Models.pdf
30/45
30
Time Series Forecasting contd
1
1.5
2
2.5
3
3.5
0 12 24 36 48 60 72 84
Month
Sales
1
1.5
2
2.5
3
0 12 24 36 48 60 72 84
Month
Trend
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0 12 24 36 48 60 72 84
Month
Seasona
lity
7/30/2019 Forecasting Models.pdf
31/45
31
Time Series Forecasting contd
Classical Decomposition Model:
where
mt= trend component, the gradual upward or downward movement
of the data over time.
st= seasonal component, pattern of the demand fluctuation aboveor below the trend line that repeats at regular intervals
d= seasonal period
Zt= random noise component
tttt ZsmY
.0and,,)var(,0][1
2
d
j
jtdttt sssZZE
7/30/2019 Forecasting Models.pdf
32/45
32
Simple Average
Suppose that the time series {Yt} is generated by a constant
process subject to random noise,
Lety1, ,ynbe observations from the time series. We know
that average of the observations is a least squares estimator for
the mean m. Hence we can use thesimple average
as a forecast for time period t+ 1, t= 1, , n.
.tt ZmY
.11
1 t
i
it yt
y
7/30/2019 Forecasting Models.pdf
33/45
33
Moving Average
Now suppose that the mean mtof {Yt} changes slowlyover time. Hence it may be desirable to reduce theinfluence of past data on the forecast. A movingaverage forecast of order kis given by
The moving average of orderkdeals only with the latest
kperiods of data. It does not handle trend or seasonality very well,
although it does better than the simple mean.
.1,,,,1
1
1
nktyk
yt
kti
it
7/30/2019 Forecasting Models.pdf
34/45
34
Exponential Smoothing
An extension to the moving average is forecasting
using a weighted moving average, which gives
more weight to the most recent observations.
Exponential smoothingis a class of methods thatapply exponentially decreasing weights as the
observations get older.
7/30/2019 Forecasting Models.pdf
35/45
35
Exponential Smoothing contd
Lety1, ,ynbe observations of a time series {Yt} with
mean mtand no seasonality, and [0, 1]. Define
weighted averages
Application of these equations is calledsimple
exponential smoothing.
.,,2,)1(,
11
11
ntyyy
yy
ttt
7/30/2019 Forecasting Models.pdf
36/45
36
Exponential Smoothing contd
Note that
Hence the forecast is a weighted average of pastobservations, with exponentially decreasing weights.
1
1
2
2
2
2
1
222
1
11
1
)1()1()1()1(
])1([)1()1(
])1()[1(
)1(
yyyyy
yyyy
yyy
yyy
tt
ttt
tttt
ttt
ttt
7/30/2019 Forecasting Models.pdf
37/45
Seasonal Adjustment
Compute a seasonal index for each data pointin time series
By dividing the value of each data point by total
average of all data
Divide the time series by seasonal indices
This minimizes the effect of seasonal variations
Fit a suitable model in adjusted data of timeseries
Obtain the forecast and then multiply it with
corresponding seasonal index37
7/30/2019 Forecasting Models.pdf
38/45
Regression with Seasonal and Trend
Components
38
Multiple regression can be used to forecast both trend and
seasonal components in a time series
One independent variable is time
Dummy independent variables are used to represent the
seasons
The model is an additive decomposition model
44332211XbXbXbXbaY
whereX1 = time period
X2 = 1 if season 2, 0 otherwise
X3 = 1 if season 3, 0 otherwise
X4 = 1 if season 4, 0 otherwise
7/30/2019 Forecasting Models.pdf
39/45
39
Measuring Forecast Accuracy
Mean Squared Error (MSE):
Mean Absolute Error (MAE):
Mean Absolute Percentage Error (MAPE):
n
t
tt yyn 1
2)(1
n
t
tt yyn 1
||1
n
t t
tt
y
yy
n 1
100
7/30/2019 Forecasting Models.pdf
40/45
40
Homework-2
1. The Paris Chamber of Commerce and Industry has been
asked to prepare a forecast of the French index of industrial
production (see data on next slide).
a. Compute a forecast using moving averages with 12 observations in
each average.b. Now compute a series of moving average forecasts using six
observations in each average.
c. Compute a series of exponential smoothing forecasts with = 0.7.
d. Graph the data and each of your forecasts.
e. Compare your forecasts using the three metrics of forecast accuracy
discussed in class. How accurate would you say the forecasts are?
7/30/2019 Forecasting Models.pdf
41/45
41
Homework-2 contd
Period French indexof industrial
prod.
Period French indexof industrial
prod.
Period French indexof industrial
prod.
1 108 10 95 19 101
2 108 11 95 20 104
3 110 12 92 21 101
4 106 13 95 22 99
5 108 14 95 23 95
6 108 15 98 24 95
7 105 16 97 25 96
8 100 17 101 26 98
9 97 18 104 27 94
7/30/2019 Forecasting Models.pdf
42/45
Auto Regressive Models
42
Observation or dependent variable is a function of
itself at the previous moments of period or time.
),...,,(,21 tptttt
yyyfy
p
i
titit eybby1
0 Linear Model
where:
ytthe dependent variable values at the moment t,
yt-i
(i = 1, 2, ..., p)the dependent variable values at the
moment t-i,
bo, bi (i=1,..., p)regression coefficient,
pauto-regression rank,
etdisturbance term.
7/30/2019 Forecasting Models.pdf
43/45
Auto Regressive Models contd
43
A first-order autoregressive model is concerned withonly the correlation between consecutive values in a
series.
A second-order autoregressive model considers theeffect of relationship between consecutive values in a
series as well as the correlation between values two
periods apart.
ttt eybby 110
tttt
eybybby 22110
ipipiiieYbYbYbbY
22110
Auto Regressive Model Order P
7/30/2019 Forecasting Models.pdf
44/45
44
t yt
1 1.89
2 2.46
3 3.23
4 3.95
5 4.56
6 5.07
7 5.62
8 6.16
9 6.26
10 6.56
11 6.98
12 7.36
13 7.53
14 7.84
15 8.09
Example: For following data, find coefficients of AR-1 and
AR-2 models and compare both for prediction accuracy.
Homework-3
7/30/2019 Forecasting Models.pdf
45/45
Hints Homework-3
45
tyt
yt-1
yt-2
1 1.89 - -
2 2.46 1.89 -
3 3.23 2.46 1.89
4 3.95 3.23 2.46
5 4.56 3.95 3.23
6 5.07 4.56 3.957 5.62 5.07 4.56
8 6.16 5.62 5.07
9 6.26 6.16 5.62
10 6.56 6.26 6.16
11 6.98 6.56 6.26
12 7.36 6.98 6.5613 7.53 7.36 6.98
14 7.84 7.53 7.36
15 8.09 7.84 7.53
tttteyyy 21 08.08.01.1
yXXXb TT 1)(
ttt eybby 110
tttteybybby
22110
Compare two forecasts usingthe three metrics of forecast
accuracy
Recommended