Introduction to Time Series Analysishalweb.uc3m.es/.../personas/kaiser/esp/doctorado/docto1b.pdf · 2008-04-09 · Introduction to time series (2008 ) 1 Introduction to Time Series

Introduction to time series (2008 ) 1

Introduction to Time Series Analysis

Regina KaiserOffice. 10.1.17E-mail:[email protected]ágina web de la asignatura


Outline

Chapter 1. Univariate ARIMA models.Chapter 2. Model fitting and checking.Chapter 3. Prediction and model selection.Chapter 4. Outliers and influential observations.Chapter 5. Heterocedastic models.Chapter 6. The TRAMO software.


Textbooks Abraham, B. and Ledolter, J. (1983) . Statistical Methods for

Forecasting. Wiley, New York. Anderson, T. W. (1971). The Statistical Analysis of Time Series.

Springer Verlag. New York. Box, G.E.P., Jenkins, G.M. and Reinsel, G. (1994). Time Series

Analysis: Forecasting and Control, 3rd. Ed. Prentice-Hall, Englewood Cliffs, NJ.

Brockwell, P.J. and Davis, R.A. (1996). Time Series: Theory and Methods. Springer-Verlag. New York.


Grading

• Final exam (60%).• Empirical project (40%).


Web resources• Time series data library.(http://www-personal.buseco.monash.edu.au/~hyndman/TSDL/)• Macroeconomic time serieshttp://www.fgn.unisg.ch/eurmacro/macrodata/index.html• Time series applets 1.http://www.aranya.com/resources/java/• Time series applets 2http://www.ubmail.ubalt.edu/~harsham/zero/scientificCal.htm• Time series applets 3.https://stat.tamu.edu/~jhardin/applets/index.html


Motivation

• Time series: sequence of observations takenat regular intervals of time

• Data in bussines, engineering, enviroment, medicine, etc.


Motivation.


Motivation.


Motivation.


Motivation.


Motivation.


Motivation.World,Oecd crude oil price(Q)

0

10

20

30

40

50

60

1 16 31 46 61 76 91 106 121 136 151 166 181


Motivation.

World, food price (Q)

0

0,5

1

1,5

2

1 10 19 28 37 46 55 64 73 82 91 100 109


Motivation.World, minerals price (Q)

0

0,5

1

1,5

2

1 10 19 28 37 46 55 64 73 82 91 100 109


Motivation.

World, agricultural raw price (Q)

00,20,40,60,8

11,21,41,6

1 10 19 28 37 46 55 64 73 82 91 100 109


Motivation.World, tropical drinks price (q)

0

0,5

1

1,5

2

2,5

3

1 10 19 28 37 46 55 64 73 82 91 100 109


Motivation.

0

0,5

1

1,5

2

2,5

3

1 15 29 43 57 71 85 99

Tropical drinks

Agricultural rawmat

Minerals, oreand metals

Food


Motivation.


Motivation.


Motivation.

• Knowledge of the dynamic structure willhelp to produce accurate forecasts of futureobservations and design optimal control schemes.


Motivation

Main objectives

• Understanding the dynamic structure of theobservations in a single series.

• Ascertain the leading, lagging and feedback relationships among several variables.


Chapter 1

Univariate ARIMA Models

Recommended readings: chapters 1 to 7 of Peña (2005)


Chapter 1. Contents.

1.1. Introduction. Definitions, examples,

1.2. Properties of univariate ARMA.ARMA weights.Stationarity and covariance structure.Autocorrelation function.Partial autocorrelación function.

1.3. Model especification.1.4. Examples.


Introduction

• Stochastic Process. Real-valued randomvariable that follows a distribution.

• The T-dimensional variable will have a joint distribution that dependson

tz )( tt zf

),...,,( 21 tTtt zzz

),...,,( 21 Tttt


Introduction

• Time Series will denote a particular realization of the stochasticprocess .

• To simplify notation

],...,,[ 21 tTtt zzz

),...,,( 21 Tzzz


IntroductionAssumptions

1. The process is stationary

2. The joint distribution of isa multivariate normal distribution

),...,,( 21 Tzzz


IntroductionImplications of stationarity• The joint distribution remains constant

over time

• It is then true that

),...,,(),...,,( 2121 kTkkT zzzfzzzf

;ztEz zt VVz


Introduction

• Stationarity implies a constant mean andbounded deviations from it.

• Strong requirement, few actual economicseries will satisfy it.


Introduction


Introduction


Introduction


Introduction

Transformations to achieve stationarity

– Constant variance: log/level plus outliercorrection.

– Stationary in mean: differencing.


Introduction

Differencing• Let B be the backward operator

• We shall use the operators:

– Regular difference– Seasonal difference

jttj zzB

B 1s

s B 1


Introduction

If is a deterministic trend

Where

• In general, will reduce a polynomial of degreed to a constant.

tz )( tt baz

;0

;2

t

t

z

bz

).(2tt zz

d


Introduction

• Example: will cancel a constant, but it will also cancel otherdeterministic periodic functions.

44 ttt zzz


Introduction

• Homogeneous difference equation

• Characteristic equation:• Solution is given by (four roots of

the unit circle)

– Two real roots and two complex conjugateswith modulus 1and frequency

0)1( 44

4 tttt zzzBz

irirrr 4321 ,,1,1

014 r4 1r

2/


Introduction

Therefore,

1. One in the zero frequency (trend)2. One in the twice-a-year seasonality3. Associated with once-a-year seasonality

)1)(1)(1( 24 BBB

2/


Univariate ARMA models

“Building block” is the white noise process

Implications:

),0( 2Niidat

0,...),,/()( 321 ttttt aaaaEaE

0),(),( jttjtt aaCovaaE2

21 ,...),/(),( atttjtt aaaVaraaVar


Introduction

General ARMA(p,q) processes:

• is the observable time series• is sequence of white noise• is the constant term

tt aBczB )()(

}{ ta}{ tz

c


Introduction

• Autoregressive polynomial

• Moving-average polynomial

• The AR and MA polynomials are assumedto have no common factor

ppBBBB ...1)( 2

21

qqBBBB ...1)( 2

21


Introduction

• Stationarity implies that all zeros ofare restricted to lie outside the unit circleand in this case:

Where is the mean of the series.

• Overall behavior of the series remains thesame over time.

)(B

)....1( 1 pc


Introduction

• In real world, however, time series data often exhibits a drifting behavior. Nonsta-stionarity can be modeled allowing some ofthe zeroes in to be equal to one. Thus,

• This is known as the ARIMA (p,d,q) model.

)(B

ttd aBczBB )()1)((


Introduction

• Some special cases of ARIMA(p,d,q):

– AR(1)– MA(1)– ARMA(1,1)– IMA(1,1)– ARIMA(0,1,1)(0,1,1)


Introduction

• AR(1) to MA( ) by recursive substitution.

ttt azz 1

ttttttt aazaazz 122

12 )(

tttktk

ktk

t aaaazz

122

11 ...


Introduction

• If , then

Which is a MA( )

1

0jjt

jt az


Introduction

• Some considerations:

• ARMA models are not unique (variety ofpossible representations.

• Parsimony is always a desirable property.• AR representations are easiest to estimate

(OLS) and to be interpretated.


Properties

• For simplicity, we will assume zero mean and an starting point m. The ARIMA (p,d,q)

• Can be rewritten as:

• Where,

tt aBzB )()(

aDzD

)',....,( tm zzz )',...,( tm aaa


Properties

1..................

...............

...1

1

1

p

pD

1..................

...............

...1

1

1

q

qD

)1)(1( mtmt


Properties

• ARMA weights

Where,

aDDz 1

1......

1

1

11

mt

DD


Properties

• The -weights can be obtained by equatingcoefficients of powers of B from therelations:

• Where

)()()( BBB

...1)( 221 BBB


Properties

• The ARIMA (p,d,q) model can then be rewritten in the MA form as:

mt

hhthtt aaz

1


Properties

• In the same way,

• Where,

azDD

1

1......

1

1

11

mt

DD


Properties

• With

• The AR form of the model is then,

).()()( BBB

t

mt

hhtht azz

1


Properties

• These two expressions are of fundamental importance in understanding the nature ofthe model.

• The MA form with the weights, showshow the observation is affected by current and past shocks or innovations.

• The AR form with the weigths, indicateshow the observation is related to its ownpast values.

tz


Properties

Examples : AR(1) model

• MA representation:

• AR representation:

....22

11 tttt aaaz

ttt azz 1


Properties

Examples : MA(1) model



tjtj

tt azzz .......1

1 ttt aaz


Properties

Examples : IMA(1,1) model



ttttt azzzz ...)1()1()1( 32

21

....))(1( 21 tttt aaaz


Stationarity condition

• Consider,

• Since the innovations are assumed normallydistributed, it follows that the observationsare also normally distributed.

• It is seen that, if in the characteristicequation:

mt

hhthtt aaz

1

kpp

kk rArA 0011 ...



• (Where the A’s denote polinomials in k andthe r’s are the distinct zeroes of )

• Then as we have that

)(B

,1jr poj ,...,1

,mt

,0)( tzE

0

2),cov(h

khhaktt zz



• So that will be stationary in thisasymptotic sense.

• This is the stationarity condition of anARMA model, and is equivalent to requirethat the zeroes of are lying outside theunit circle.

tz

)(B


Lag k autocovariance

• Let us denote,

• For an alternative expressions in terms ofthe ARMA parameters,

)(),cov(),cov()( kzzzzk kttktt

)...( 11 ptpttkt zzzz )...( 11 qtqttkt aaaz


Lag k autocovariance

• By taking expectations on both sides andusing the MA form, we obtain, for

• Where,

k

p

hh ghkk

1)()(

qk

kgkq

hkhha

k0

,...2,1,00

2

,0k


Autocorrelation function

• The autocorrelation function is defined as

• By substitution of , we obtain that

,...2,1,0,)0()()( kkk

)(k

0)0(

)()(1

kghkkp

h

kh


Autocorrelation function

• When , in a MA(q) model, then,

• That is, the autocorrelation function cuts offafter lag q.

0)( B

qkqkk qq

,0,)...1()(

1221


Partial autocorrelation function

• Intuition

• AR(1):

• AR(2):

tttt zzzz 123

tttt zzzz 123



• The autocorrelation function only takes intoaccount that and are related in bothcases.

• But if we want to measure the directrelationship (without the intermediate ), we found that it is zero for the AR(1) anddifferent from zero for the AR(2) model

tz 2tz

1tz



• The partial autocorrelation function is, therefore, a measure of the linear relationamong observations k-periods apart, independently of the intermediate values.



• Consider first an stationary AR(p) model. The autoregressive coefficients are relatedto the autocorrelations by the Yule-Walkerequations,

• Where, is the matrix,

pppG

pG )( pxp



1)1(...)2()1()1(1)2(

)2(1)1()1()2(...)1(1

ppp

ppp

Gp



• Regarding this as system of p equations andp unknowns (the coefficients), thesolution is, for p>1, the ratio of twodeterminants

ppp GH /



)()1(...)2()1()1(1)2(

)2(1)1()1()2(...)1(1

ppppp

p

Hp

where



•A pxp matrix, and for p=1. This leads to define, for any stationary model

•Which is known as the Partial Autocorrelation Function.

)1( p

1/1)(

kGHkk

kkk



It has the property that, for a stationary AR(p) model,

In other words, vanishes for k>p when the model is AR(p).

pkpkk

k 0

k


properties

Slow decay towards zero

Slow decay towards zero

ARMA(p,q)

slow decay towards zero

q different from zero

MA(q)

p different from zero

slow decay towards zero

AR(P)

PAFSAF


Examples

• AR(1) model

• Model:

• Variance:

ttt azz 1

2222azz

2

22

1

az


Examples

• Autocovariance function:

1)1(

11

0

)( 2

2

2

kk

k

k

k a

z


Examples

• Autocorrelation function:

• Partial autocorrelation:

1101

)(kkk

kk

)1()1(


Examples

• AR(2) model

• Model:

• Variance:

tttt azzz 2211

21

22

2

2

22

)1(11

az


Examples


2

1

11

)(

2

21

2

2

1

k

kk

22211 kkkk


Examples

• MA(1) model

• Model:

• Variance:

1 ttt aaz

2222aaz

)1( 222 az


Examples

• Autocovariance function:

1010

)( 2

2

kkk

k a

z


Examples


10

11

01)( 2

k

kk

k


Specification

• The class of ARMA(p,q) models isextensive.

• Guidelines are needed in selecting a member of the class to represent the time series data at hand


Specification

• Box and Jenkins (1976) proposed aniterative model building strategy.

– Tentative specification or identification of a model.

– Efficient estimation of model parameters.– Diagnostic checking of fitted model for further

improvement.


Specification

• Tentative specification. The aim is toemploy statistics that:

– 1. Can be readily calculated from thedata.

– 2. Allow the user to tentatively select a model .


Specification

• Three methods:

– The Sample Autocorrelation Function(SAF)

– The Sample Partial AutocorrelationFunction (SPAF)

– Use of diagnostic tools (AIC,BIC) to findthe best model (TRAMO-automatic).


Specification

• Sample Autocorrelation Function. TheSACF of are defined as

With

and the sample mean of the n availableobservations

tz

,...2,1/ˆ 0 kCCkk

))((1

zzzzC jt

jn

ttj

z


Specification

• Properties:

• 1.For stationary models,

• 2. When there exists a unit root in the AR polynomial

nkk ̂

1ˆ p

k


Specification

• If the SACF of the original series ispersistently close to 1 as k increases, oneforms the first difference , , and studiesits SACF to determine whether furtherdifferencing is called for.

tz


Specification

• Once stationary is achieved, a cutting offpattern after, say a lag “q”, in the SACF willthen lead to tentative specification of a MA(q) model.


Specification

• The Sample Partial AutocorrelationFunction:

• Are obtained by replacing the in

• by their sample estimates

,...2,1ˆ kkk

1/1)(

kGHkk

kkk

k̂


Specification

Properties:• 1.For stationary models,

• 2. The are asymptotically normallydistributed

• 3. For a stationary AR(p) model

nkk ̂

k̂

pknVar k 1)ˆ(


Specification

• Properties 1, 2 and 3 make SPACF a convenient tool for specifying the order or a stationary AR model (cutting off patternafter lag p)

• Not valid for non-stationary models.


Specification

Weakness of the SACF and SPACF.

1. Subjective judgment is often required todecide on the order of differencing.

2. For stationary ARMA models, both SACF and SPACF tend to exhibit a gradual “tapering off” behavior, making thespecification very difficult.


Specification

Diagnostic Tools

• The program (TRAMO), in an automaticway, specifies a set of possible models, estimate them and select the best one basedon AIC and BIC criteria.

• IMPOSSIBLE 10-15 years before

Documents

Introduction to Time Series Analysishalweb.uc3m.es/.../personas/kaiser/esp/doctorado/docto1b.pdf · 2008-04-09 · Introduction to time series (2008 ) 1 Introduction to Time Series