Linear Regression and Testing - uni- Linear Regression and Testing Pag. 5 Under these assumptions, the

  • View
    0

  • Download
    0

Embed Size (px)

Transcript

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 1

    Linear Regression and Testing

    1. Assumptions of the Classical Linear Regression Model.

    2. CLRM assumption and the time series analysis.

    3. Usual estimation procedure.

    4. Example: estimating the Euro-area Phillips curve.

    5. Summary Statistics: Coefficient Results, S. E., R-sq., Adj. R-

    sq., Sum-of-Sq. Residuals, Mean and S.D. of the Dep. Variable.

    6. Verifying the basic assumptions: overview

    7. Verifying the basic assumptions: OLS tests: t-Statistics, p-

    value, F-Statistic, DW Statistic, RESET Test.

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 2

    1. Assumptions of the Classical Linear Regression Model

    Using the matrix notation, the standard regression model may be

    written as:

    (1)

    where is a T-dimensional vector containing observations on the

    dependent variable, X is a T x k matrix of independent variables,

    is a k-vector of coefficients, and is a T-vector of disturbances.

    Alternatively:

    t=1…T (2)

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 3

    The following assumptions permit to consider the Ordinary Least

    Squares estimates (OLS) b for the vector :

    A1. Linearity: The model specifies a linear relationship between y and the columns of X. A2. Full rank: There is no exact linear relationship among any of the independent variables in the model. This is necessary for estimating the parameters of the model. A3. Exogeneity of the independent variables: E[ |X] = 0. This states that the expected value of the disturbance at each observation in the sample is not a function of the independent variables observed at any observation.

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 4

    A4. Homoscedasticity and non-autocorrelation: Each disturbance, has the same finite variance,

    and is uncorrelated with every other disturbance, .

    A5. Exogenously generated data: The data in X may be any mixture of a constant and random variables. The process generating the data is independent of the process that generates . Analysis is done conditionally on the observed X. A6. Normal distribution: The disturbances are normally distributed . This assumption is made for convenience.

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 5

    Under these assumptions, the Gauss–Markov Theorem holds: In the classical linear regression model, the least squares estimator b is the minimum variance linear unbiased estimator of β whether X is stochastic or non-stochastic, so long as the other assumptions of the model continue to hold. Where b is defined as:

    or

    (3)

    And: (4)

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 6

    The following finite sample properties hold:

     ; (5)

     . (6)  Gauss−Markov theorem: MVLUE.  (7)

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 7

    Results that follow from Assumption A6, normally distributed disturbances:  b and are statistically independent. It follows that b and are

    uncorrelated and statistically independent.  The exact distribution of b|X, is .  The ratio is chi-squared distributed with T-k

    degrees of freedom, .

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 8

    2. CLRM assumption and the time series analysis Consider the estimation of the parameters of a pth-order

    autoregression, AR(P), by OLS:

    (8)

    with roots of

    outside the unit

    circle and with an i.i.d sequence with mean zero, variance

    and finite fourth moment.

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 9

    An autoregression has the form of the standard regression model

    with

    ).

     However, an autoregression cannot satisfy usual condition that

    is independent of for all t and s. See A3.

     In fact, although and are independent, this is not the case for

    and .

     Without this independence, none of the small-sample results are

    valid for the classical linear regression model applies.

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 10

    Even if is Gaussian, the OLS coefficient b gives biased estimate of

    for an autoregression and the standard t and F statistics can only

    be justified asymptotically. However, one may rely on consistency:

    B1. Stationarity: given a stochastic process generating t=1,..,T if

    neither its mean nor its autocovariances depend on the date

    t, then the process for is said to be autocovariance-stationary or

    weakly stationary.

    B2. Ergodicity: A covariance-stationary process is said to be ergodic

    for the mean if converges in probability to

    as T goes to infinity.

    B1 and B2 imply for the OLS estimator that :

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 11

    In order to find the distribution of , suppose the sample consists

    of T+p observations on : ,…, , ,…, ),

    OLS estimation will thus use observations 1 through T. Then

    (9)

    One may assume that:

    (10)

    with Q a non singular and non stochastic matrix.

     is assumed to be a martingale difference sequence,

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 12

    thus one can show:

    (11)

    Substituting (10) and (11) into (9),

    (12)

    from which the asymptotical application of the t and F statistics

    follows.

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 13

    3. Usual estimation procedure

     As a first step of the estimation procedure one should find b,

    the estimate of and other basic descriptive statistics.

     As a second step, one should proceed in verifying the above

    assumptions A1-A6 plus B1-B2 if any.

     If all of these assumptions are verified one could treat the point

    and interval estimates as reliable, and test potential restriction

    suggested by the theory.

     Otherwise one should find some remedy provided in the

    literature.

  • Andrea Beccarini (CQE) Empirical Methods Summer 2013

    Linear Regression and Testing Pag. 14

    4. Example: estimating the Euro-area Phillips curve

    Dependent Variable: HICPEA Sample(adjusted): 1996:3 2008:1 Included observations: 47 after adjusting endpoints

    Variable Coefficient Std. Error t-Statistic Prob.

    C 0.602236 0.088140 6.832720 0.0000 HICPEA(-1) -0.195737 0.151978 -1.287924 0.2045

    OGEAP 0.101738 0.059188 1.718913 0.0927

    R-squared 0.079167 Mean dependent var 0.503383 Adjusted R-squared 0.037311