Linear Regression Basics IIIViolating Assumptions

Fin250f: Lecture 7.2

Spring 2010

Brooks, chapter 4(skim)

4.1-2, 4.4, 4.5, 4.7, 4.9-13

Outline

Violating assumptionsParameter stabilityModel building

OLS Assumptions

Error variancesError correlationsError normalityFunctional forms and linearityOmitting variablesAdding irrelevant variables

Error Variance

var(ut )=σ2

var(ut) =σ t2

OLS unbiased, consistent, but inefficient

Weighting observations by noise (ARCH/GARCH)

Error VarianceWhich is a bigger error?

yt

yt

*

*

*

***

Y

X

Error Correlations

E(utut+ j ) ≠0

Patterns in residualsPlot residuals/residual diagnosticsFurther modeling necessary

If you can forecast u(t+1), need to work harder

Error Normality

Skewness and kurtosis in residualsTesting

Plots Bera-Jarque test

How can this impact results?

Bera-Jarque Test for Normality

b1 =E(u3)σ 3 ,b2 =

E(u4 )σ 4

W =Tb12

6+(b2 −3)2

24⎛

⎝⎜⎞

⎠⎟

W : χ 2 (2)

Nonnormal Errors: Impact

For some theory: No In practice can be big problem Many extreme data points Forecasting models work hard to fit these

extreme outliers Some solutions:

Drop data points Robust forecast objectives (absolute errors)

Functional Forms

Y=a+bXActual function is nonlinearSeveral types of diagnostics

Higher order (squared) terms (RESET) Think about specific nonlinear models

Neural networks Threshold models

Tricky: More later

yt =a+β1xt,1 +β2xt,2 +ut

Omitting Variables

Leave out x(2)

If it is correlated with x(1) this is a problem.

Beta(1) will be biased and inconsistent.

Forecast will not be optimal

Irrelevant Variables

Overfitting/data snooping Model fits to noise

Impacts standard errors for coefficientsCoefficients still consistent and

unbiased

Parameter Stability

Known break point Chow test Predictive failure test

Unknown break Quant likelihood ratio test Recursive least squares

Chow Test

yt =a+ βxt + ut

yt1 =a1 +β1xt

1 + ut1

yt2 =a2 +β 2xt

2 + ut2

RSS= u∑ t

2

RSS1 = (∑ ut1)2

RSS2 = (∑ ut2 )2

Chow Test

RSS−(RSS1 + RSS2 )RSS1 + RSS2

(T −2k)k

k=number of regressors

2k = number of regressors unrestricted

Test statistic F(k,T-2k)

Predictive Failureyt =a+ βxt + ut

yt1 =a1 +β1xt

1 + ut1, Large subsample 1,T1

RSS= ut2

t=1

T

∑

RSS1 = (t=1

T1

∑ ut1)2

T2 =T −T1

Predictive Failure

(RSS−RSS1) /T2

RSS1 / (T1 −k)=

Expected squared error at endExpected squared error before end

F(T2 ,T1 −k)

Unknown Breaks

Search for breakLook for maximum Chow levelDistribution is tricky

Monte-carlo/bootstrap

Recursive/rolling estimation

Recursive Estimate (1,T1) move T1 to full sample T See if parameters converge

Rolling Roll bands (t-T,t) through data Watch parameters move through time

We’ll use some of these

Pure Out of Sample Tests

Estimate parameters over (1,T1)Get errors over (T1+1,T)

Model Construction

General -> specific Less financial theory More statistics Problems: large unwieldy models

Simple -> general More theory at the start Problems: can leave out important stuff