May 2004Prof. Himayatullah1
Basic EconometricsBasic Econometrics
Chapter 5: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis Testing
May 2004Prof. Himayatullah2
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION:Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-1. Statistical Prerequisites See Appendix A with key concepts
such as probability, probability distributions, Type I Error, Type II Error,level of significance, power of a statistic test, and confidence interval
May 2004Prof. Himayatullah3
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-2. Interval estimation: Some basic Ideas How “close” is, say, ^2 to 2 ?
Pr (^2 - 2 ^2 + ) = 1 - (5.2.1)
Random interval ^2 - 2 ^2 +
if exits, it known as confidence interval
^2 - is lower confidence limit
^2 + is upper confidence limit
May 2004Prof. Himayatullah4
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-2. Interval estimation: Some basic Ideas
(1 - ) is confidence coefficient,
0 < < 1 is significance level
Equation (5.2.1) does not mean that the Pr of
2 lying between the given limits is (1 - ), but
the Pr of constructing an interval that contains
2 is (1 - )
(^2 - , ^2 + ) is random interval
May 2004Prof. Himayatullah5
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-2. Interval estimation: Some basic Ideas In repeated sampling, the intervals will
enclose, in (1 - )*100 of the cases, the true value of the parameters
For a specific sample, can not say that the probability is (1 - ) that a given fixed interval includes the true 2
If the sampling or probability distributions of the estimators are known, one can make confidence interval statement like (5.2.1)
May 2004Prof. Himayatullah6
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-3. Confidence Intervals for Regression Coefficients
Z= (^2 - 2)/se(^2) = (^2 - 2) x2
i / ~N(0,1)(5.3.1)
We did not know and have to use ^ instead, so:
t= (^2 - 2)/se(^2) = (^2 - 2) x2i /^ ~ t(n-2)
(5.3.2)=> Interval for 2
Pr [ -t /2 t t /2] = 1- (5.3.3)
May 2004Prof. Himayatullah7
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-3. Confidence Intervals for Regression Coefficients
Or confidence interval for 2 is
Pr [^2-t /2se(^2) 2 ^2+t /2se(^2)] = 1- (5.3.5)
Confidence Interval for 1
Pr [^1-t /2se(^1) 1 ^1+t /2se(^1)] = 1- (5.3.7)
May 2004Prof. Himayatullah8
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-4. Confidence Intervals for 2
Pr [(n-2)^2/ 2/2 2
(n-2)^2/ 21- /2] = 1-
(5.4.3)The interpretation of this interval is: If we
establish (1- ) confidence limits on 2 and if we maintain a priori that these limits will include true 2, we shall be right in the long run (1- ) percent of the time
May 2004Prof. Himayatullah9
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-5. Hypothesis Testing: General Comments The stated hypothesis is known as the null hypothesis: Ho
The Ho is tested against and alternative hypothesis: H1
5-6. Hypothesis Testing: The confidence interval approach
One-sided or one-tail Test H0: 2 * versus H1: 2 > *
May 2004Prof. Himayatullah10
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
Two-sided or two-tail Test H0: 2 = * versus H1: 2 # *
^2 - t /2se(^2) 2 ^2 + t /2se(^2) values
of 2 lying in this interval are plausible under Ho
with 100*(1- )% confidence. If 2 lies in this region we do not reject Ho (the
finding is statistically insignificant) If 2 falls outside this interval, we reject Ho (the
finding is statistically significant)
May 2004Prof. Himayatullah11
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-7. Hypothesis Testing: Hypothesis Testing: The test of significance approachThe test of significance approach
A test of significance is a procedure by which A test of significance is a procedure by which sample results are used to verify the truth or sample results are used to verify the truth or falsity of a null hypothesisfalsity of a null hypothesis
Testing the significance of regression Testing the significance of regression coefficient: The t-testcoefficient: The t-test
Pr [^2-t /2se(^2) 2 ^2+t /2se(^2)]= 1- (5.7.2)
May 2004Prof. Himayatullah12
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approach
Table 5-1: Decision Rule for t-test of significanceTable 5-1: Decision Rule for t-test of significance
Type of Hypothesis
H0 H1 Reject H0 if
Two-tail 2 = 2* 2 # 2* |t| > t/2,df
Right-tail 2 2* 2 > 2* t > t,df
Left-tail 2 2* 2 < 2* t < - t,df
May 2004Prof. Himayatullah13
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approachTesting the significance of Testing the significance of 2 2 : The : The 22 Test Test
Under the Normality assumption we have:Under the Normality assumption we have:
^2
22 = = (n-2) ------- ~ 22 (n-2) (5.4.1)
2 From (5.4.2) and (5.4.3) on page 520 =>
May 2004Prof. Himayatullah14
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approach
Table 5-2: A summary of theTable 5-2: A summary of the 22 Test Test
H0 H1 Reject H0 if
2 = 20 2 > 2
0 Df.(^2)/ 20 > 22 ,df
2 = 20 2 < 2
0 Df.(^2)/ 20 < 22
((1-),df
2 = 20 2 # 2
0 Df.(^2)/ 20 > 22
/2,df
or < 2 2 ((1-/2), df
May 2004Prof. Himayatullah15
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-8. Hypothesis Testing: Hypothesis Testing: Some practical aspectsSome practical aspects1) The meaning of “Accepting” or 1) The meaning of “Accepting” or
“Rejecting” a Hypothesis“Rejecting” a Hypothesis2) The Null Hypothesis and the Rule of 2) The Null Hypothesis and the Rule of ThumbThumb3) Forming the Null and Alternative 3) Forming the Null and Alternative HypothesesHypotheses4) Choosing 4) Choosing , the Level of Significance, the Level of Significance
May 2004Prof. Himayatullah16
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-8. Hypothesis Testing: Hypothesis Testing: Some practical aspectsSome practical aspects5) The Exact Level of Significance: 5) The Exact Level of Significance: The p-Value [The p-Value [See page 132See page 132]6) Statistical Significance versus 6) Statistical Significance versus Practical Significance Practical Significance 7) The Choice between Confidence-7) The Choice between Confidence- Interval and Test-of-Significance Interval and Test-of-Significance Approaches to Hypothesis TestingApproaches to Hypothesis Testing [Warning: Read carefully pages 117-134 ]
May 2004Prof. Himayatullah17
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-9. Regression Analysis and Analysis of Variance TSS = ESS + RSSTSS = ESS + RSS F=[MSS F=[MSS of ESS]of ESS]/[MSS /[MSS of RSS] of RSS] = =
= = 22^̂22 xxii
22/ / ^̂2 2 (5.9.1)(5.9.1) If uIf uii are normally distributed; H are normally distributed; H00:: 2 2 = 0= 0
then F follows the F distribution with 1 then F follows the F distribution with 1 and n-2 degree of freedomand n-2 degree of freedom
May 2004Prof. Himayatullah18
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-9. Regression Analysis and Analysis of Variance
F provides a test statistic to test the F provides a test statistic to test the null hypothesis that true null hypothesis that true 22 is zero by is zero by compare this F ratio with the F-critical compare this F ratio with the F-critical obtained from F tables at the chosen obtained from F tables at the chosen level of significance, or obtain the p-level of significance, or obtain the p-value of the computed F statistic to value of the computed F statistic to make decisionmake decision
May 2004Prof. Himayatullah19
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-9. Regression Analysis and Analysis of Variance
Table 5-3. ANOVA for two-variable regression modelTable 5-3. ANOVA for two-variable regression model
Source of Variation
Sum of square ( SS) Degree of Freedom -(Df)
Mean sum of square ( MSS)
ESS (due to regression)
y^y^ii2 2 = = 22^̂22
xxii22 1 22^̂22
xxii22
RSS (due to residuals)
u^u^ii22 n-2 u^u^ii
2 2 /(n-2)=/(n-2)=^̂22
TSS y y ii
22 n-1
May 2004Prof. Himayatullah20
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-10. Application of Regression Analysis: Problem of Prediction
By the data of Table 3-2, we obtained the sample regression (3.6.2) : Y^i = 24.4545 + 0.5091Xi , where Y^i is the estimator of true E(Yi)
There are two kinds of prediction as follows:
May 2004Prof. Himayatullah21
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-10. Application of Regression Analysis: Problem of Prediction Mean prediction: Prediction of the
conditional mean value of Y corresponding to a chosen X, say X0, that is the point on the population regression line itself (see pages 137-138 for details)
Individual prediction: Prediction of an individual Y value corresponding to X0
(see pages 138-139 for details)
May 2004Prof. Himayatullah22
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-11. Reporting the results of regression analysis
An illustration:
Y^I= 24.4545 + 0.5091Xi (5.1.1)
Se = (6.4138) (0.0357) r2= 0.9621t = (3.8128) (14.2405) df= 8P = (0.002517) (0.000000289) F1,2=2202.87
May 2004Prof. Himayatullah23
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-12. Evaluating the results of regression analysis:
Normality Test: The Chi-Square (2) Goodness of fit Test
2N-1-k = (Oi – Ei)2/Ei
(5.12.1)Oi is observed residuals (u^i) in interval iEi is expected residuals in interval iN is number of classes or groups; k is number ofparameters to be estimated. If p-value of
obtaining 2N-1-k is high (or 2
N-1-k is small) =>The Normality Hypothesis can not be rejected
May 2004Prof. Himayatullah24
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-12. Evaluating the results of regression analysis:
Normality Test: The Chi-Square (2) Goodness of fit Test
H0: ui is normally distributed H1: ui is un-normally distributedCalculated-2
N-1-k = (Oi – Ei)2/Ei (5.12.1)Decision rule: Calculated-2
N-1-k > Critical-2N-1-k then H0 can
be rejected
May 2004Prof. Himayatullah25
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-12. Evaluating the results of regression analysis:
The Jarque-Bera (JB) test of normalityThis test first computes the Skewness (S)and Kurtosis (K) and uses the followingstatistic:JB = n [S2/6 + (K-3)2/24] (5.12.2)
Mean= xbar = xi/n ; SD2 = (xi-xbar)2/(n-1)S=m3/m2
3/2 ; K=m4/m22 ; mk= (xi-xbar)k/n
May 2004Prof. Himayatullah26
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-12. (Continued)
Under the null hypothesis H0 that the residuals are normally distributed Jarque and Bera show that in large sample (asymptotically) the JB statistic given in (5.12.12) follows the Chi-Square distribution with 2 df. If the p-value of the computed Chi-Square statistic in an application is sufficiently low, one can reject the hypothesis that the residuals are normally distributed. But if p-value is reasonable high, one does not reject the normality assumption.
May 2004Prof. Himayatullah27
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-13. Summary and Conclusions1. Estimation and Hypothesis testing
constitute the two main branches of classical statistics
2. Hypothesis testing answers this question: Is a given finding compatible with a stated hypothesis or not?
3. There are two mutually complementary approaches to answering the preceding question: Confidence interval and test of significance.
May 2004Prof. Himayatullah28
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-13. Summary and Conclusions
4. Confidence-interval approach has a specified probability of including within its limits the true value of the unknown parameter. If the null-hypothesized value lies in the confidence interval, H0 is not rejected, whereas if it lies outside this interval, H0 can be rejected
May 2004Prof. Himayatullah29
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-13. Summary and Conclusions5. Significance test procedure develops a
test statistic which follows a well-defined probability distribution (like normal, t, F, or Chi-square). Once a test statistic is computed, its p-value can be easily obtained.
The p-value The p-value of a test is the lowest significance level, at which we would reject H0. It gives exact probability of obtaining the estimated test statistic under H0. If p-value is small, one can reject H0, but if it is large one may not reject H0.
May 2004Prof. Himayatullah30
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-13. Summary and Conclusions 6. Type I error is the error of rejecting
a true hypothesis. Type II error is the error of accepting a false hypothesis. In practice, one should be careful in fixing the level of significance , the probability of committing a type I error (at arbitrary values such as 1%, 5%, 10%). It is better to quote the p-value of the test statistic.
May 2004Prof. Himayatullah31
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-13. Summary and Conclusions
7. This chapter introduced the normality test to find out whether ui follows the normal distribution. Since in small samples, the t, F,and Chi-square tests require the normality assumption, it is important that this assumption be checked formally
May 2004Prof. Himayatullah 32
Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing
5-13. Summary and Conclusions (ended)
8. If the model is deemed practically adequate, it may be used for forecasting purposes. But should not go too far out of the sample range of the regressor values. Otherwise, forecasting errors can increase dramatically.