May 2004 Prof. Himayatullah 1 Basic Econometrics Chapter 5: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis Testing

May 2004Prof. Himayatullah1

Basic EconometricsBasic Econometrics

Chapter 5: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis Testing


Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION:Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing

5-1. Statistical Prerequisites See Appendix A with key concepts

such as probability, probability distributions, Type I Error, Type II Error,level of significance, power of a statistic test, and confidence interval


Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing

5-2. Interval estimation: Some basic Ideas How “close” is, say, ^2 to 2 ?

Pr (^2 - 2 ^2 + ) = 1 - (5.2.1)

Random interval ^2 - 2 ^2 +

if exits, it known as confidence interval

^2 - is lower confidence limit

^2 + is upper confidence limit



5-2. Interval estimation: Some basic Ideas

(1 - ) is confidence coefficient,

0 < < 1 is significance level

Equation (5.2.1) does not mean that the Pr of

2 lying between the given limits is (1 - ), but

the Pr of constructing an interval that contains

2 is (1 - )

(^2 - , ^2 + ) is random interval



5-2. Interval estimation: Some basic Ideas In repeated sampling, the intervals will

enclose, in (1 - )*100 of the cases, the true value of the parameters

For a specific sample, can not say that the probability is (1 - ) that a given fixed interval includes the true 2

If the sampling or probability distributions of the estimators are known, one can make confidence interval statement like (5.2.1)



5-3. Confidence Intervals for Regression Coefficients

Z= (^2 - 2)/se(^2) = (^2 - 2) x2

i / ~N(0,1)(5.3.1)

We did not know and have to use ^ instead, so:

t= (^2 - 2)/se(^2) = (^2 - 2) x2i /^ ~ t(n-2)

(5.3.2)=> Interval for 2

Pr [ -t /2 t t /2] = 1- (5.3.3)



5-3. Confidence Intervals for Regression Coefficients

Or confidence interval for 2 is

Pr [^2-t /2se(^2) 2 ^2+t /2se(^2)] = 1- (5.3.5)

Confidence Interval for 1

Pr [^1-t /2se(^1) 1 ^1+t /2se(^1)] = 1- (5.3.7)



5-4. Confidence Intervals for 2

Pr [(n-2)^2/ 2/2 2

(n-2)^2/ 21- /2] = 1-

(5.4.3)The interpretation of this interval is: If we

establish (1- ) confidence limits on 2 and if we maintain a priori that these limits will include true 2, we shall be right in the long run (1- ) percent of the time



5-5. Hypothesis Testing: General Comments The stated hypothesis is known as the null hypothesis: Ho

The Ho is tested against and alternative hypothesis: H1

5-6. Hypothesis Testing: The confidence interval approach

One-sided or one-tail Test H0: 2 * versus H1: 2 > *



Two-sided or two-tail Test H0: 2 = * versus H1: 2 # *

^2 - t /2se(^2) 2 ^2 + t /2se(^2) values

of 2 lying in this interval are plausible under Ho

with 100*(1- )% confidence. If 2 lies in this region we do not reject Ho (the

finding is statistically insignificant) If 2 falls outside this interval, we reject Ho (the

finding is statistically significant)



5-7. Hypothesis Testing: Hypothesis Testing: The test of significance approachThe test of significance approach

A test of significance is a procedure by which A test of significance is a procedure by which sample results are used to verify the truth or sample results are used to verify the truth or falsity of a null hypothesisfalsity of a null hypothesis

Testing the significance of regression Testing the significance of regression coefficient: The t-testcoefficient: The t-test

Pr [^2-t /2se(^2) 2 ^2+t /2se(^2)]= 1- (5.7.2)



5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approach

Table 5-1: Decision Rule for t-test of significanceTable 5-1: Decision Rule for t-test of significance

Type of Hypothesis

H0 H1 Reject H0 if

Two-tail 2 = 2* 2 # 2* |t| > t/2,df

Right-tail 2 2* 2 > 2* t > t,df

Left-tail 2 2* 2 < 2* t < - t,df



5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approachTesting the significance of Testing the significance of 2 2 : The : The 22 Test Test

Under the Normality assumption we have:Under the Normality assumption we have:

^2

22 = = (n-2) ------- ~ 22 (n-2) (5.4.1)

2 From (5.4.2) and (5.4.3) on page 520 =>



5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approach

Table 5-2: A summary of theTable 5-2: A summary of the 22 Test Test

H0 H1 Reject H0 if

2 = 20 2 > 2

0 Df.(^2)/ 20 > 22 ,df

2 = 20 2 < 2

0 Df.(^2)/ 20 < 22

((1-),df

2 = 20 2 # 2

0 Df.(^2)/ 20 > 22

/2,df

or < 2 2 ((1-/2), df



5-8. Hypothesis Testing: Hypothesis Testing: Some practical aspectsSome practical aspects1) The meaning of “Accepting” or 1) The meaning of “Accepting” or

“Rejecting” a Hypothesis“Rejecting” a Hypothesis2) The Null Hypothesis and the Rule of 2) The Null Hypothesis and the Rule of ThumbThumb3) Forming the Null and Alternative 3) Forming the Null and Alternative HypothesesHypotheses4) Choosing 4) Choosing , the Level of Significance, the Level of Significance



5-8. Hypothesis Testing: Hypothesis Testing: Some practical aspectsSome practical aspects5) The Exact Level of Significance: 5) The Exact Level of Significance: The p-Value [The p-Value [See page 132See page 132]6) Statistical Significance versus 6) Statistical Significance versus Practical Significance Practical Significance 7) The Choice between Confidence-7) The Choice between Confidence- Interval and Test-of-Significance Interval and Test-of-Significance Approaches to Hypothesis TestingApproaches to Hypothesis Testing [Warning: Read carefully pages 117-134 ]



5-9. Regression Analysis and Analysis of Variance TSS = ESS + RSSTSS = ESS + RSS F=[MSS F=[MSS of ESS]of ESS]/[MSS /[MSS of RSS] of RSS] = =

= = 22^̂22 xxii

22/ / ^̂2 2 (5.9.1)(5.9.1) If uIf uii are normally distributed; H are normally distributed; H00:: 2 2 = 0= 0

then F follows the F distribution with 1 then F follows the F distribution with 1 and n-2 degree of freedomand n-2 degree of freedom



5-9. Regression Analysis and Analysis of Variance

F provides a test statistic to test the F provides a test statistic to test the null hypothesis that true null hypothesis that true 22 is zero by is zero by compare this F ratio with the F-critical compare this F ratio with the F-critical obtained from F tables at the chosen obtained from F tables at the chosen level of significance, or obtain the p-level of significance, or obtain the p-value of the computed F statistic to value of the computed F statistic to make decisionmake decision



5-9. Regression Analysis and Analysis of Variance

Table 5-3. ANOVA for two-variable regression modelTable 5-3. ANOVA for two-variable regression model

Source of Variation

Sum of square ( SS) Degree of Freedom -(Df)

Mean sum of square ( MSS)

ESS (due to regression)

y^yîi2 2 = = 22^̂22

xxii22 1 22^̂22

xxii22

RSS (due to residuals)

uûîi22 n-2 uûîi

2 2 /(n-2)=/(n-2)=^̂22

TSS y y ii

22 n-1



5-10. Application of Regression Analysis: Problem of Prediction

By the data of Table 3-2, we obtained the sample regression (3.6.2) : Yî = 24.4545 + 0.5091Xi , where Yî is the estimator of true E(Yi)

There are two kinds of prediction as follows:



5-10. Application of Regression Analysis: Problem of Prediction Mean prediction: Prediction of the

conditional mean value of Y corresponding to a chosen X, say X0, that is the point on the population regression line itself (see pages 137-138 for details)

Individual prediction: Prediction of an individual Y value corresponding to X0

(see pages 138-139 for details)



5-11. Reporting the results of regression analysis

An illustration:

YÎ= 24.4545 + 0.5091Xi (5.1.1)

Se = (6.4138) (0.0357) r2= 0.9621t = (3.8128) (14.2405) df= 8P = (0.002517) (0.000000289) F1,2=2202.87



5-12. Evaluating the results of regression analysis:

Normality Test: The Chi-Square (2) Goodness of fit Test

2N-1-k = (Oi – Ei)2/Ei

(5.12.1)Oi is observed residuals (uî) in interval iEi is expected residuals in interval iN is number of classes or groups; k is number ofparameters to be estimated. If p-value of

obtaining 2N-1-k is high (or 2

N-1-k is small) =>The Normality Hypothesis can not be rejected




Normality Test: The Chi-Square (2) Goodness of fit Test

H0: ui is normally distributed H1: ui is un-normally distributedCalculated-2

N-1-k = (Oi – Ei)2/Ei (5.12.1)Decision rule: Calculated-2

N-1-k > Critical-2N-1-k then H0 can

be rejected




The Jarque-Bera (JB) test of normalityThis test first computes the Skewness (S)and Kurtosis (K) and uses the followingstatistic:JB = n [S2/6 + (K-3)2/24] (5.12.2)

Mean= xbar = xi/n ; SD2 = (xi-xbar)2/(n-1)S=m3/m2

3/2 ; K=m4/m22 ; mk= (xi-xbar)k/n



5-12. (Continued)

Under the null hypothesis H0 that the residuals are normally distributed Jarque and Bera show that in large sample (asymptotically) the JB statistic given in (5.12.12) follows the Chi-Square distribution with 2 df. If the p-value of the computed Chi-Square statistic in an application is sufficiently low, one can reject the hypothesis that the residuals are normally distributed. But if p-value is reasonable high, one does not reject the normality assumption.



5-13. Summary and Conclusions1. Estimation and Hypothesis testing

constitute the two main branches of classical statistics

2. Hypothesis testing answers this question: Is a given finding compatible with a stated hypothesis or not?

3. There are two mutually complementary approaches to answering the preceding question: Confidence interval and test of significance.



5-13. Summary and Conclusions

4. Confidence-interval approach has a specified probability of including within its limits the true value of the unknown parameter. If the null-hypothesized value lies in the confidence interval, H0 is not rejected, whereas if it lies outside this interval, H0 can be rejected



5-13. Summary and Conclusions5. Significance test procedure develops a

test statistic which follows a well-defined probability distribution (like normal, t, F, or Chi-square). Once a test statistic is computed, its p-value can be easily obtained.

The p-value The p-value of a test is the lowest significance level, at which we would reject H0. It gives exact probability of obtaining the estimated test statistic under H0. If p-value is small, one can reject H0, but if it is large one may not reject H0.



5-13. Summary and Conclusions 6. Type I error is the error of rejecting

a true hypothesis. Type II error is the error of accepting a false hypothesis. In practice, one should be careful in fixing the level of significance , the probability of committing a type I error (at arbitrary values such as 1%, 5%, 10%). It is better to quote the p-value of the test statistic.



5-13. Summary and Conclusions

7. This chapter introduced the normality test to find out whether ui follows the normal distribution. Since in small samples, the t, F,and Chi-square tests require the normality assumption, it is important that this assumption be checked formally

May 2004Prof. Himayatullah 32


5-13. Summary and Conclusions (ended)

8. If the model is deemed practically adequate, it may be used for forecasting purposes. But should not go too far out of the sample range of the regressor values. Otherwise, forecasting errors can increase dramatically.

Documents

May 2004 Prof. Himayatullah 1 Basic Econometrics Chapter 5: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis Testing