View
223
Download
3
Tags:
Embed Size (px)
Citation preview
1
Pertemuan 13Uji Koefisien Korelasi dan Regresi
Matakuliah : A0392 – Statistik Ekonomi
Tahun : 2006
3
Inference About The Slope and Coefficient Of Correlation
Inference about the slope with Analysis of Variance
4
Measures of Variation: The Sum of Squares
SST = SSR + SSE
Total Sample
Variability
= Explained Variability
+ Unexplained Variability
5
Measures of Variation: The Sum of Squares
• SST = Total Sum of Squares – Measures the variation of the Yi values
around their mean,
• SSR = Regression Sum of Squares – Explained variation attributable to the
relationship between X and Y
• SSE = Error Sum of Squares – Variation attributable to factors other than the
relationship between X and Y
(continued)
Y
6
Measures of Variation: The Sum of Squares
(continued)
Xi
Y
X
Y
SST = (Yi - Y)2
SSE =(Yi - Yi )2
SSR = (Yi - Y)2
_
_
_
7
Venn Diagrams and Explanatory Power of Regression
Sales
Sizes
Variations in Sales explained by Sizes or variations in Sizes used in explaining variation in Sales
Variations in Sales explained by the error term or unexplained by Sizes
Variations in store Sizes not used in explaining variation in Sales
SSE
SSR
8
The ANOVA Table in Excel
ANOVA
df SS MS FSignificance
F
Regression
kSSR
MSR
=SSR/kMSR/MSE
P-value of
the F Test
Residualsn-k-1
SSE
MSE
=SSE/(n-k-1)
Total n-1SST
9
Measures of VariationThe Sum of Squares: Example
ANOVA
df SS MS F Significance F
Regression 1 30380456.12 30380456 81.17909 0.000281201
Residual 5 1871199.595 374239.92
Total 6 32251655.71
Excel Output for Produce Stores
SSR
SSERegression (explained) df
Degrees of freedom
Error (residual) df
Total df
SST
11
Standard Error of Estimate
•
• Measures the standard deviation (variation) of the Y values around the regression equation
2
1
ˆ
2 2
n
ii
YX
Y YSSE
Sn n
12
Measures of Variation: Produce Store Example
Regression StatisticsMultiple R 0.9705572R Square 0.94198129Adjusted R Square 0.93037754Standard Error 611.751517Observations 7
Excel Output for Produce Stores
r2 = .94
94% of the variation in annual sales can be explained by the variability in the size of the store as measured by square footage.
Syxn
13
Linear Regression Assumptions
• Normality– Y values are normally distributed for each X– Probability distribution of error is normal
• Homoscedasticity (Constant Variance)
• Independence of Errors
14
Consequences of Violationof the Assumptions
• Violation of the Assumptions– Non-normality (error not normally distributed)– Heteroscedasticity (variance not constant)
• Usually happens in cross-sectional data– Autocorrelation (errors are not independent)
• Usually happens in time-series data• Consequences of Any Violation of the Assumptions
– Predictions and estimations obtained from the sample regression line will not be accurate
– Hypothesis testing results will not be reliable• It is Important to Verify the Assumptions
15
• Y values are normally distributed around the regression line.
• For each X value, the “spread” or variance around the regression line is the same.
Variation of Errors Aroundthe Regression Line
X1
X2
X
Y
f(e)
Sample Regression Line
16
Inference about the Slope: t Test
• t Test for a Population Slope– Is there a linear dependency of Y on X ?
• Null and Alternative Hypotheses– H0: 1 = 0 (no linear dependency)
– H1: 1 0 (linear dependency)
• Test Statistic–
–
1
1
1 1
2
1
where
( )
YXb n
bi
i
b St S
SX X
. . 2d f n
17
Example: Produce Store
Data for 7 Stores:Estimated Regression Equation:Annual
Store Square Sales Feet ($000)
1 1,726 3,681
2 1,542 3,395
3 2,816 6,653
4 5,555 9,543
5 1,292 3,318
6 2,208 5,563
7 1,313 3,760
ˆ 1636.415 1.487i iY X
The slope of this model is 1.487.
Does square footage affect annual sales?
18
Inferences about the Slope: t Test Example
H0: 1 = 0
H1: 1 0
.05
df 7 - 2 = 5
Critical Value(s):
Test Statistic:
Decision:
Conclusion:There is evidence that square footage affects annual sales.
t0 2.5706-2.5706
.025
Reject Reject
.025
From Excel Printout
Reject H0.
Coefficients Standard Error t Stat P-valueIntercept 1636.4147 451.4953 3.6244 0.01515Footage 1.4866 0.1650 9.0099 0.00028
1b 1bS t
p-value
19
Inferences about the Slope: Confidence Interval Example
Confidence Interval Estimate of the Slope:
11 2n bb t S Excel Printout for Produce Stores
At 95% level of confidence, the confidence interval for the slope is (1.062, 1.911). Does not include 0.
Conclusion: There is a significant linear dependency of annual sales on the size of the store.
Lower 95% Upper 95%Intercept 475.810926 2797.01853Footage 1.06249037 1.91077694
20
Inferences about the Slope: F Test
• F Test for a Population Slope– Is there a linear dependency of Y on X ?
• Null and Alternative Hypotheses– H0: 1 = 0 (no linear dependency)
– H1: 1 0 (linear dependency)
• Test Statistic
–
– Numerator d.f.=1, denominator d.f.=n-2
1
2
SSR
FSSEn
21
Relationship between a t Test and an F Test
• Null and Alternative Hypotheses– H0: 1 = 0 (no linear dependency)– H1: 1 0 (linear dependency)
•
• The p –value of a t Test and the p –value of an F Test are Exactly the Same
• The Rejection Region of an F Test is Always in the Upper Tail
2
2 1, 2n nt F
22
ANOVAdf SS MS F Significance F
Regression 1 30380456.12 30380456.12 81.179 0.000281Residual 5 1871199.595 374239.919Total 6 32251655.71
Inferences about the Slope: F Test Example
Test Statistic:
Decision:Conclusion:
H0: 1 = 0H1: 1 0 .05numerator df = 1denominator df 7 - 2 = 5
There is evidence that square footage affects annual sales.
From Excel Printout
Reject H0.
0 6.61
Reject
= .05
1, 2nF
p-value
23
Purpose of Correlation Analysis
• Correlation Analysis is Used to Measure Strength of Association (Linear Relationship) Between 2 Numerical Variables– Only strength of the relationship is concerned– No causal effect is implied
24
Purpose of Correlation Analysis
• Population Correlation Coefficient (Rho) is Used to Measure the Strength between the Variables
(continued)
XY
X Y
25
• Sample Correlation Coefficient r is an Estimate of and is Used to Measure the Strength of the Linear Relationship in the Sample Observations
Purpose of Correlation Analysis
(continued)
1
2 2
1 1
n
i ii
n n
i ii i
X X Y Yr
X X Y Y
27
Features of and r
• Unit Free
• Range between -1 and 1
• The Closer to -1, the Stronger the Negative Linear Relationship
• The Closer to 1, the Stronger the Positive Linear Relationship
• The Closer to 0, the Weaker the Linear Relationship
28
• Hypotheses – H0: = 0 (no correlation)
– H1: 0 (correlation)
• Test Statistic
–
2
2 1
2 2
1 1
where
2n
i ii
n n
i ii i
rt
rn
X X Y Yr r
X X Y Y
t Test for Correlation
29
Example: Produce Stores
Regression StatisticsMultiple R 0.9705572R Square 0.94198129Adjusted R Square 0.93037754Standard Error 611.751517Observations 7
From Excel Printout r
Is there any evidence of linear relationship between annual sales of a store and its square footage at .05 level of significance?
H0: = 0 (no association)
H1: 0 (association)
.05
df 7 - 2 = 5
30
Example: Produce Stores Solution
0 2.5706-2.5706
.025
Reject Reject
.025
Critical Value(s):
Conclusion:There is evidence of a linear relationship at 5% level of significance.
Decision:Reject H0.
2
.97069.0099
1 .942052
rt
rn
The value of the t statistic is exactly the same as the t statistic value for test on the slope coefficient.
31
Estimation of Mean Values
Confidence Interval Estimate for :
The Mean of Y Given a Particular Xi
2
22
1
( )1ˆ
( )
ii n YX n
ii
X XY t S
n X X
t value from table with df=n-2
Standard error of the estimate
Size of interval varies according to distance away from mean, X
| iY X X
32
Prediction of Individual Values
Prediction Interval for Individual Response Yi at a Particular Xi
Addition of 1 increases width of interval from that for the mean of Y
2
22
1
( )1ˆ 1( )
ii n YX n
ii
X XY t S
n X X
33
Interval Estimates for Different Values of X
Y
X
Prediction Interval for a Individual Yi
a given X
Confidence Interval for the Mean of Y
Y i = b0 + b1X i
X
34
Example: Produce Stores
Yi = 1636.415 +1.487Xi
Data for 7 Stores:
Regression Model Obtained:
Annual Store Square Sales
Feet ($000)
1 1,726 3,681
2 1,542 3,395
3 2,816 6,653
4 5,555 9,543
5 1,292 3,318
6 2,208 5,563
7 1,313 3,760
Consider a store with 2000 square feet.
35
Estimation of Mean Values: Example
Find the 95% confidence interval for the average annual sales for stores of 2,000 square feet.
2
22
1
( )1ˆ 4610.45 612.66( )
ii n YX n
ii
X XY t S
n X X
Predicted Sales Yi = 1636.415 +1.487Xi = 4610.45 ($000)
X = 2350.29 SYX = 611.75 tn-2 = t5 = 2.5706
Confidence Interval Estimate for| iY X X
|3997.02 5222.34iY X X
36
Prediction Interval for Y : Example
Find the 95% prediction interval for annual sales of one particular store of 2,000 square feet.
Predicted Sales Yi = 1636.415 +1.487Xi = 4610.45 ($000)
X = 2350.29 SYX = 611.75 tn-2 = t5 = 2.5706
2
22
1
( )1ˆ 1 4610.45 1687.68( )
ii n YX n
ii
X XY t S
n X X
Prediction Interval for Individual
2922.00 6297.37iX XY
iX XY
37
Estimation of Mean Values and Prediction of Individual Values in PHStat
• In Excel, use PHStat | Regression | Simple Linear Regression …– Check the “Confidence and Prediction Interval
for X=” box
• Excel Spreadsheet of Regression Sales on Footage
Microsoft Excel Worksheet
38
Pitfalls of Regression Analysis
• Lacking an Awareness of the Assumptions Underlining Least-Squares Regression
• Not Knowing How to Evaluate the Assumptions
• Not Knowing What the Alternatives to Least-Squares Regression are if a Particular Assumption is Violated
• Using a Regression Model Without Knowledge of the Subject Matter
39
Strategy for Avoiding the Pitfalls of Regression
• Start with a scatter plot of X on Y to observe possible relationship
• Perform residual analysis to check the assumptions
• Use a histogram, stem-and-leaf display, box-and-whisker plot, or normal probability plot of the residuals to uncover possible non-normality
40
Strategy for Avoiding the Pitfalls of Regression
• If there is violation of any assumption, use alternative methods (e.g., least absolute deviation regression or least median of squares regression) to least-squares regression or alternative least-squares models (e.g., curvilinear or multiple regression)
• If there is no evidence of assumption violation, then test for the significance of the regression coefficients and construct confidence intervals and prediction intervals
(continued)
41
Chapter Summary
• Introduced Types of Regression Models
• Discussed Determining the Simple Linear Regression Equation
• Described Measures of Variation
• Addressed Assumptions of Regression and Correlation
• Discussed Residual Analysis
• Addressed Measuring Autocorrelation