# Linear Regression and Testing - uni- Linear Regression and Testing Pag. 5 Under these assumptions, the

• View
0

0

Embed Size (px)

Transcript

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 1

Linear Regression and Testing

1. Assumptions of the Classical Linear Regression Model.

2. CLRM assumption and the time series analysis.

3. Usual estimation procedure.

4. Example: estimating the Euro-area Phillips curve.

5. Summary Statistics: Coefficient Results, S. E., R-sq., Adj. R-

sq., Sum-of-Sq. Residuals, Mean and S.D. of the Dep. Variable.

6. Verifying the basic assumptions: overview

7. Verifying the basic assumptions: OLS tests: t-Statistics, p-

value, F-Statistic, DW Statistic, RESET Test.

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 2

1. Assumptions of the Classical Linear Regression Model

Using the matrix notation, the standard regression model may be

written as:

(1)

where is a T-dimensional vector containing observations on the

dependent variable, X is a T x k matrix of independent variables,

is a k-vector of coefficients, and is a T-vector of disturbances.

Alternatively:

t=1…T (2)

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 3

The following assumptions permit to consider the Ordinary Least

Squares estimates (OLS) b for the vector :

A1. Linearity: The model specifies a linear relationship between y and the columns of X. A2. Full rank: There is no exact linear relationship among any of the independent variables in the model. This is necessary for estimating the parameters of the model. A3. Exogeneity of the independent variables: E[ |X] = 0. This states that the expected value of the disturbance at each observation in the sample is not a function of the independent variables observed at any observation.

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 4

A4. Homoscedasticity and non-autocorrelation: Each disturbance, has the same finite variance,

and is uncorrelated with every other disturbance, .

A5. Exogenously generated data: The data in X may be any mixture of a constant and random variables. The process generating the data is independent of the process that generates . Analysis is done conditionally on the observed X. A6. Normal distribution: The disturbances are normally distributed . This assumption is made for convenience.

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 5

Under these assumptions, the Gauss–Markov Theorem holds: In the classical linear regression model, the least squares estimator b is the minimum variance linear unbiased estimator of β whether X is stochastic or non-stochastic, so long as the other assumptions of the model continue to hold. Where b is defined as:

or

(3)

And: (4)

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 6

The following finite sample properties hold:

 ; (5)

 . (6)  Gauss−Markov theorem: MVLUE.  (7)

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 7

Results that follow from Assumption A6, normally distributed disturbances:  b and are statistically independent. It follows that b and are

uncorrelated and statistically independent.  The exact distribution of b|X, is .  The ratio is chi-squared distributed with T-k

degrees of freedom, .

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 8

2. CLRM assumption and the time series analysis Consider the estimation of the parameters of a pth-order

autoregression, AR(P), by OLS:

(8)

with roots of

outside the unit

circle and with an i.i.d sequence with mean zero, variance

and finite fourth moment.

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 9

An autoregression has the form of the standard regression model

with

).

 However, an autoregression cannot satisfy usual condition that

is independent of for all t and s. See A3.

 In fact, although and are independent, this is not the case for

and .

 Without this independence, none of the small-sample results are

valid for the classical linear regression model applies.

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 10

Even if is Gaussian, the OLS coefficient b gives biased estimate of

for an autoregression and the standard t and F statistics can only

be justified asymptotically. However, one may rely on consistency:

B1. Stationarity: given a stochastic process generating t=1,..,T if

neither its mean nor its autocovariances depend on the date

t, then the process for is said to be autocovariance-stationary or

weakly stationary.

B2. Ergodicity: A covariance-stationary process is said to be ergodic

for the mean if converges in probability to

as T goes to infinity.

B1 and B2 imply for the OLS estimator that :

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 11

In order to find the distribution of , suppose the sample consists

of T+p observations on : ,…, , ,…, ),

OLS estimation will thus use observations 1 through T. Then

(9)

One may assume that:

(10)

with Q a non singular and non stochastic matrix.

 is assumed to be a martingale difference sequence,

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 12

thus one can show:

(11)

Substituting (10) and (11) into (9),

(12)

from which the asymptotical application of the t and F statistics

follows.

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 13

3. Usual estimation procedure

 As a first step of the estimation procedure one should find b,

the estimate of and other basic descriptive statistics.

 As a second step, one should proceed in verifying the above

assumptions A1-A6 plus B1-B2 if any.

 If all of these assumptions are verified one could treat the point

and interval estimates as reliable, and test potential restriction

suggested by the theory.

 Otherwise one should find some remedy provided in the

literature.

• Andrea Beccarini (CQE) Empirical Methods Summer 2013

Linear Regression and Testing Pag. 14

4. Example: estimating the Euro-area Phillips curve

Dependent Variable: HICPEA Sample(adjusted): 1996:3 2008:1 Included observations: 47 after adjusting endpoints

Variable Coefficient Std. Error t-Statistic Prob.

C 0.602236 0.088140 6.832720 0.0000 HICPEA(-1) -0.195737 0.151978 -1.287924 0.2045

OGEAP 0.101738 0.059188 1.718913 0.0927

R-squared 0.079167 Mean dependent var 0.503383 Adjusted R-squared 0.037311