Forecasting Models.pdf

7/30/2019 Forecasting Models.pdf

1/45

Advanced Techniques for Engineering

Analysis

Forecasting Models

GC University Lahore


2/45

2

What is Forecasting?

Forecasting is attempting to predict the future

i.e. expected state of an event, condition or

variable at some future time index

Decision makers want to reduce the risk by

predicting future events or values of related

variables


3/45

3

Forecasting Methods

Forecasting methods

Qualitative

intuitive, educated guesses that may or may not depend on

past data

Quantitative

based on mathematical or statistical models


4/45

4

Forecasting Methods contd

We will consider two types of forecasts based on

mathematical models:

Regression forecasting

Time Series forecasting


5/45

5

Regression Forecasting

We predict by modeling the relationship

between dependent variable and independent

variables

We use some latest observations to fit a least

square regression line to data and then use it for

forecasting purpose.


6/45

6

Regression Forecasting contd

Suppose that Yis a variable of interest, andX1,,Xp are explanatory or predictor variables

such that

Y=f(X1, ,Xp; )

fis the mathematical modelthat determines

the relationship between the variable ofinterest and the explanatory variables

= (0, , m) are the modelparameters.


7/45

7


1. We choose to minimize

2. We use least squares estimation to estimate

3. Then we forecasty as

.);,,(1

p

xxfy

n

i

n

iiipiiexxfy

1 1

22

1.);,,(


8/45

8


Examples of Regression Models:

)st variableindependen

with twomodellinear(.5

)modelquadratic(.4

)modelgrowthlexponentia()exp(.3

)modelregressionlinearsimple(.2

)modelmeanconstant(.1

22110

2

210

10

10

0

iiii

iiii

iii

iii

ii

xxy

xxy

xy

xy

y


9/45

9

Constant Mean Regression

Suppose that theyis are a constant value plus noise:

yi = 0 + i,

i.e., = 0.We want to determine the value of0 that minimizes

.)()(1

2

00

n

i

iyS

),0(~ 2 Ni


10/45

10

Constant Mean Regression contd

Taking the derivative ofS(0) gives

Finally setting this equal to zero leads to

Hence the sample mean is the least squaresestimator for0.

.)(2)(1

00

n

i

iyS

.1

1

0 yyn

n

i

i

y


11/45

11

Constant Mean Regression contd

Example:yi = 0 + i,

y

98.30963

99.18569

101.2684

97.52997

103.4013

98.84521

111.1842

98.70812

93.08922 5.1020 y

80

85

90

95

100

105

110

115

120

1 2 3 4 5 6 7 8 9

y


12/45

12

Simple Linear Regression

Consider the model

yi = 0 + 1xi+ i,

i.e., = (0, 1).We want to determine the values of0 and 1 that

minimize.)(),(

1

2

1010

n

i

ii xyS


13/45

13

Simple Linear Regression contd

Setting the first partial derivatives equal to zero gives

n

i

iii

n

iii

xxyS

xyS

1

1010

1

11010

0

.0)(2),(

0)(2),(


14/45

14


Solving for0 and 1 leads to the least squares

estimates

.

10

221

xy

xxn

yxyxn

ii

iiii


15/45

15


Example:

x y

1 2.6

2.3 2.8

3.1 3.1

4.8 4.7

5.6 5.1

6.3 5.30

1

2

3

4

5

6

0 1 2 3 4 5 6 7

X

Y


16/45

16


Example continued:

x y

1 2.62.3 2.8

3.1 3.1

4.8 4.7

5.6 5.16.3 5.3 93.3 85.3

99.109

16.103

6.23

1.236

2

yx

x

yx

y

xn

i

ii

i

i

68.158.0

0

1


17/45

17


Example (continued):

Regression equation:

xy 58.068.1

0

1

2

3

4

5

6

0 1 2 3 4 5 6 7

Y

Yhat


18/45

18

General Linear Regression

Consider the linear regression model

or

where xi = (1,xi1, ,xip) and = (0, , p).

iippiii xxxy 22110

iiiy

x


19/45

19

General Linear Regression contd

Suppose that we have n observations of y i.e.yi ,i=1,2,..n. We introduce matrix notation and define

y = (y1, ,yn), = (1, , n),

Note that y is n 1, is n 1, and X is n (p + 1).

.

1

1

1

1

221

111

npn

p

p

xx

xx

xx

X


20/45

20


Then we can write the regression model as

y has a mean vector and covariance matrixgiven by

where I is the n n identity matrix.

Xy

,)var(][ 2

IyXy

E


21/45

21


Note that by var(y), we mean the matrix

This is a symmetric matrix.

)var(),cov(),cov(

),cov()var(),cov(),cov(),cov()var(

21

2212

1211

nnn

n

n

yyyyy

yyyyyyyyyy


22/45

22


In matrix notation, the least squares criterion can beexpressed as minimizing

Setting , the least squares estimator is givenby

.)( 1 yXXX

.)()()()(1

2

n

i

iiyS XyXyx

0)(

S

Prediction forYis then given by

.)( 1 yXXXXXy


23/45

23


Example: Simple Linear Regression

.)( 1 yXXX

x y

1 2.6

2.3 2.83.1 3.1

4.8 4.7

5.6 5.1

6.3 5.3

0

1

2

3

4

5

6

0 1 2 3 4 5 6 7

X

Y


24/45

24


Example: Simple Linear Regression (p = 1)

x y

1 2.6

2.3 2.83.1 3.1

4.8 4.7

5.6 5.1

6.3 5.3

3.616.518.411.31

3.2111

X

3.51.57.41.3

8.26.2

y


25/45

25


Example: Simple Linear Regression (p = 1)

99.1091.23

1.236XX

047.018.018.087.01

XX

58.068.11

yXXX


26/45

26

Homework-1

1. Suppose that the following data represent the total costs and

the number of units produced by a company.

a. Graph the relationship betweenXand Y.

b. Determine the simple linear regression line relating YtoX.

c. Predict the costs for producing 10 units.

d. Compute the SST, SSR, SSE,R andR2. Interpret the value

ofR2.

Total Cost (Y) 25 11 34 23 32

Units Produced (X) 5 2 8 4 6


27/45

27

Homework-1 contd

2. Consider the fuel consumption data on the next slide, and the

following model which relates fuel consumption (Y) to the

average hourly temperature (X1) and the chill index (X2):

a. Plot YversusX1 and YversusX2.

b. Determine the least squares estimates for the model parameters.

c. Predict the fuel consumption when the temperature is 35 and the chill

index is 10.

d. Compute the SST, SSR, SSE andR2. Interpret the value ofR2.

.22110 XXY


28/45

28

Data for Problem 2

Average Hourly

Temperature,xi1

Chill Index,xi2 Weekly Fuel

Consumption,yi

28 18 12.4

32.5 24 12.3

28 14 11.7

39 22 11.2

57.8 16 9.5

45.9 8 9.4

58.1 1 8.0

62.5 0 7.5


29/45

29

Time Series Forecasting

Time series

Sequence of observations of response variable at

regular time intervals

Stochastic or dynamic, it does change over time.

For forecasting

We use past history of response variable to predictthe future.

Predictions exploit correlations between past

history and the future.


30/45

30

Time Series Forecasting contd

1

1.5

2

2.5

3

3.5

0 12 24 36 48 60 72 84

Month

Sales

1

1.5

2

2.5

3

0 12 24 36 48 60 72 84

Month

Trend

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0 12 24 36 48 60 72 84

Month

Seasona

lity


31/45

31

Time Series Forecasting contd

Classical Decomposition Model:

where

mt= trend component, the gradual upward or downward movement

of the data over time.

st= seasonal component, pattern of the demand fluctuation aboveor below the trend line that repeats at regular intervals

d= seasonal period

Zt= random noise component

tttt ZsmY

.0and,,)var(,0][1

2

d

j

jtdttt sssZZE


32/45

32

Simple Average

Suppose that the time series {Yt} is generated by a constant

process subject to random noise,

Lety1, ,ynbe observations from the time series. We know

that average of the observations is a least squares estimator for

the mean m. Hence we can use thesimple average

as a forecast for time period t+ 1, t= 1, , n.

.tt ZmY

.11

1 t

i

it yt

y


33/45

33

Moving Average

Now suppose that the mean mtof {Yt} changes slowlyover time. Hence it may be desirable to reduce theinfluence of past data on the forecast. A movingaverage forecast of order kis given by

The moving average of orderkdeals only with the latest

kperiods of data. It does not handle trend or seasonality very well,

although it does better than the simple mean.

.1,,,,1

1

1

nktyk

yt

kti

it


34/45

34

Exponential Smoothing

An extension to the moving average is forecasting

using a weighted moving average, which gives

more weight to the most recent observations.

Exponential smoothingis a class of methods thatapply exponentially decreasing weights as the

observations get older.


35/45

35

Exponential Smoothing contd

Lety1, ,ynbe observations of a time series {Yt} with

mean mtand no seasonality, and [0, 1]. Define

weighted averages

Application of these equations is calledsimple

exponential smoothing.

.,,2,)1(,

11

11

ntyyy

yy

ttt


36/45

36

Exponential Smoothing contd

Note that

Hence the forecast is a weighted average of pastobservations, with exponentially decreasing weights.

1

1

2

2

2

2

1

222

1

11

1

)1()1()1()1(

])1([)1()1(

])1()[1(

)1(

yyyyy

yyyy

yyy

yyy

tt

ttt

tttt

ttt

ttt


37/45

Seasonal Adjustment

Compute a seasonal index for each data pointin time series

By dividing the value of each data point by total

average of all data

Divide the time series by seasonal indices

This minimizes the effect of seasonal variations

Fit a suitable model in adjusted data of timeseries

Obtain the forecast and then multiply it with

corresponding seasonal index37


38/45

Regression with Seasonal and Trend

Components

38

Multiple regression can be used to forecast both trend and

seasonal components in a time series

One independent variable is time

Dummy independent variables are used to represent the

seasons

The model is an additive decomposition model

44332211XbXbXbXbaY

whereX1 = time period

X2 = 1 if season 2, 0 otherwise




39/45

39

Measuring Forecast Accuracy

Mean Squared Error (MSE):

Mean Absolute Error (MAE):

Mean Absolute Percentage Error (MAPE):

n

t

tt yyn 1

2)(1

n

t

tt yyn 1

||1

n

t t

tt

y

yy

n 1

100


40/45

40

Homework-2

1. The Paris Chamber of Commerce and Industry has been

asked to prepare a forecast of the French index of industrial

production (see data on next slide).

a. Compute a forecast using moving averages with 12 observations in

each average.b. Now compute a series of moving average forecasts using six

observations in each average.

c. Compute a series of exponential smoothing forecasts with = 0.7.

d. Graph the data and each of your forecasts.

e. Compare your forecasts using the three metrics of forecast accuracy

discussed in class. How accurate would you say the forecasts are?


41/45

41

Homework-2 contd

Period French indexof industrial

prod.


prod.


prod.

1 108 10 95 19 101

2 108 11 95 20 104

3 110 12 92 21 101

4 106 13 95 22 99

5 108 14 95 23 95

6 108 15 98 24 95

7 105 16 97 25 96

8 100 17 101 26 98

9 97 18 104 27 94


42/45

Auto Regressive Models

42

Observation or dependent variable is a function of

itself at the previous moments of period or time.

),...,,(,21 tptttt

yyyfy

p

i

titit eybby1

0 Linear Model

where:

ytthe dependent variable values at the moment t,

yt-i

(i = 1, 2, ..., p)the dependent variable values at the

moment t-i,

bo, bi (i=1,..., p)regression coefficient,

pauto-regression rank,

etdisturbance term.


43/45

Auto Regressive Models contd

43

A first-order autoregressive model is concerned withonly the correlation between consecutive values in a

series.

A second-order autoregressive model considers theeffect of relationship between consecutive values in a

series as well as the correlation between values two

periods apart.

ttt eybby 110

tttt

eybybby 22110

ipipiiieYbYbYbbY

22110

Auto Regressive Model Order P


44/45

44

t yt

1 1.89

2 2.46

3 3.23

4 3.95

5 4.56

6 5.07

7 5.62

8 6.16

9 6.26

10 6.56

11 6.98

12 7.36

13 7.53

14 7.84

15 8.09

Example: For following data, find coefficients of AR-1 and

AR-2 models and compare both for prediction accuracy.

Homework-3


45/45

Hints Homework-3

45

tyt

yt-1

yt-2

1 1.89 - -

2 2.46 1.89 -

3 3.23 2.46 1.89

4 3.95 3.23 2.46

5 4.56 3.95 3.23

6 5.07 4.56 3.957 5.62 5.07 4.56

8 6.16 5.62 5.07

9 6.26 6.16 5.62

10 6.56 6.26 6.16

11 6.98 6.56 6.26

12 7.36 6.98 6.5613 7.53 7.36 6.98

14 7.84 7.53 7.36

15 8.09 7.84 7.53

tttteyyy 21 08.08.01.1

yXXXb TT 1)(

ttt eybby 110

tttteybybby

22110

Compare two forecasts usingthe three metrics of forecast

accuracy

Documents

Forecasting Models.pdf