View
220
Download
2
Embed Size (px)
Citation preview
Reading – Linear Regression
• Le (Chapter 8 through 8.1.6)• C &S (Chapter 5:F,G,H)
Issues with hypothesis testing
• Significance does not imply causality– Need a proper prospective experiment
• Significance does not imply practical importance– Trivial but significant differences
• Run lots of tests, will find significant difference by chance– With α = 0.05, expect 1 in 20 results to be sig. by chance
Issues with hypothesis testing
• Large p-values because sample size is small– Effect could exist but we may not have a large enough
sample size
• Outliers may cause problems especially in small samples.
Issues With Hypothesis Testing
What is the population of inference?
Example: A statistics class of n=15 women and n=5 men yield the following exam scores:
Women: mean = 90% SD = 10%Men: mean = 85% SD = 11%
Test the hypothesis that women did better on the exam then men.
Hypothesis tests and Confidence Intervals
bapba
nnstxx
11)( *
Two sampletest statistic:
bap
ba
nns
xxt
11
CI for differencein means:
If 95% CI excludes 0 then the p-value will be <0.05.
Linear Regression
• Investigate the relationship between two variables– Does blood pressure relate to age?– Does weight loss relate to blood pressure loss– Does income relate to education?– Do sales relate to years of experience?
• Dependent variable – The variable that is being predicted or explained
• Independent variable – The variable that is doing the predicting or explaining
• Think of data in pairs (xi, yi)
Linear Regression - Purpose
• Is there an association between the two variables– Does weight change relate to BP change?
• Estimation of impact– How much BP change occurs per pound of weight change
• Prediction – If a person loses 10 pounds how much of a drop in blood
pressure can be expected
Regression History
• Sir Francis Galton (1822-1911) studied the relationship between a father’s height and the son’s height.
• He found that although there was a relationship between father and son’s height the relationship was not perfect.
• If the father was above average in height so was the son (typically) but not as much above average. This was called regression to the mean
Example of Regression Equation
We know systolic BP increases with age. How much does it increase per year and is the increase constant over time?
SBP = 90 + 0.8*AGE
Interpretation: For each year of age SBP increases by 0.8 mmHg.
At age 50: SBP = 90 + 0.8*50 = 130 mmHg
At age 60: SBP = 90 + 0.8*60 = 138 mmHg
Y or Dependent Variable
X or Dependent Variable
Simple Linear Regression EquationSimple Linear Regression Equation
The The simple linear regression equationsimple linear regression equation is: is:
yy = = 00 + + 11xx
• Graph of the regression equation is a Graph of the regression equation is a straight line.straight line.
• 00 is the is the yy intercept of the regression line. intercept of the regression line.
• 11 is the slope of the regression line. is the slope of the regression line.
• yy is the mean value of is the mean value of yy for a given for a given xx value.value.
Simple Linear Regression ModelSimple Linear Regression Model
The equation that describes how y is related to x and an The equation that describes how y is related to x and an error term is called the error term is called the regression modelregression model..
The The simple linear regression modelsimple linear regression model is: is:
yy = = 00 + + 11xx + +
• 00 and and 11 are called are called parameters of the modelparameters of the model..
• is a random variable called theis a random variable called the error term error term..
Simple Linear Regression EquationSimple Linear Regression Equation
Positive Linear RelationshipPositive Linear Relationship
EE((yy))
xx
Slope Slope 11
is positiveis positive
Regression lineRegression line
InterceptIntercept00
Simple Linear Regression EquationSimple Linear Regression Equation
Negative Linear RelationshipNegative Linear Relationship
EE((yy))
xx
Slope Slope 11
is negativeis negative
Regression lineRegression lineInterceptIntercept00
Simple Linear Regression EquationSimple Linear Regression Equation
No RelationshipNo Relationship
EE((yy))
xx
Slope Slope 11
is 0is 0
Regression lineRegression lineInterceptIntercept
00
Estimated Simple Linear Regression Estimated Simple Linear Regression EquationEquation
The The estimated simple linear regression estimated simple linear regression equationequation is: is:
• The graph is called the estimated The graph is called the estimated regression line.regression line.
• bb00 is the is the yy intercept of the line. intercept of the line.
• bb11 is the slope of the line. is the slope of the line.
• is the estimated value of is the estimated value of yy for a given for a given xx value.value.
0 1y b b x 0 1y b b x
yy
Estimation ProcessEstimation Process
Regression ModelRegression Modelyy = = 00 + + 11xx + +
Regression EquationRegression Equationyy = = 00 + + 11xx
Unknown ParametersUnknown Parameters00, , 11
Sample Data:Sample Data:x yx y
xx11 y y11
. .. . . .. . xxnn yynn
EstimatedEstimatedRegression EquationRegression Equation
Sample StatisticsSample Statistics
bb00, , bb11
bb00 and and bb11
provide estimates ofprovide estimates of00 and and 11
0 1y b b x 0 1y b b x
Least Squares MethodLeast Squares Method
Least Squares Criterion: Choose Least Squares Criterion: Choose and and to minimizeto minimize
where:where:
yyii = = observedobserved value of the dependent variable value of the dependent variable
for the for the iith observationth observation
S = Yi – 01
Estimation
Slope: Slope:
The Least Squares EstimatesThe Least Squares Estimates
21 )(
))((
xx
yyxxb
i
ii
21 )(
))((
xx
yyxxb
i
ii
0 1b y b x 0 1b y b x Intercept:Intercept:
Example Restaurant Student Population
(Thousands)Quarterly Sales
1 2 58
2 6 105
3 8 88
4 8 118
5 12 117
6 16 137
7 20 157
8 20 169
9 22 149
10 26 202
X-Y PLOT OF DATA
CalculationsObs Xi Yi Xi-XBAR Yi-YBAR (Xi – XBAR)*
(Yi – YBAR)
(Xi – XBAR)2
1 2 58 -12 -72 864 144
2 6 105 -8 -25 200 64
3 8 88 -6 -42 252 36
4 8 118 -6 -12 72 36
5 12 117 -2 -13 26 4
6 16 137 2 7 14 4
7 20 157 6 27 162 36
8 20 169 6 39 234 36
9 22 149 8 19 152 64
10 26 202 12 72 864 144
Tot 140 1300 2840 568
Estimates for Dataset
b1 = 2840/568 = 5 b0 = 130 – 5*14 = 60
Y = Sales; X = # thousands of students
Equation:
Y = 60 + 5* X
21 )(
))((
xx
yyxxb
i
ii
21 )(
))((
xx
yyxxb
i
ii0 1b y b x 0 1b y b x
DATA sales;INFILE DATALINES;INPUT restaurant studentpop quarsales;DATALINES;1 2 582 6 1053 8 884 8 1185 12 1176 16 1377 20 1578 20 1699 22 14910 26 202;
PROC PRINT DATA=sales;PROC MEANS DATA=sales;
PROC REG DATA=sales SIMPLE; MODEL quarsales = studentpop; PLOT quarsales * studentpop ;RUN;
OUTPUT FROM PROC REG
The REG Procedure
Descriptive Statistics
Uncorrected StandardVariable Sum Mean SS Variance Deviation
Intercept 10.00000 1.00000 10.00000 0 0studentpop 140.00000 14.00000 2528.00000 63.11111 7.94425quarsales 1300.00000 130.00000 184730 1747.77778 41.80643
Parameter Estimates
Parameter StandardVariable DF Estimate Error t Value Pr > |t|
Intercept 1 60.00000 9.22603 6.50 0.0002
studentpop 1 5.00000 0.58027 8.62 <.0001
REGRESSION EQUATION:
Y = 60.0 + 5.0*X
QUARSALES = 60 + 5*STUDENTPOP
The Coefficient of DeterminationThe Coefficient of Determination
Relationship Among SST, SSR, SSERelationship Among SST, SSR, SSE
SST = SSR + SSESST = SSR + SSE
where:where: SST = total sum of squaresSST = total sum of squares SSR = sum of squares due to regressionSSR = sum of squares due to regression SSE = sum of squares due to errorSSE = sum of squares due to error
( ) ( ) ( )y y y y y yi i i i 2 2 2( ) ( ) ( )y y y y y yi i i i 2 2 2^^
The The coefficient of determinationcoefficient of determination is: is:
rr22 = SSR/SST = SSR/SST
where:where:
SST = total sum of squaresSST = total sum of squares
SSR = sum of squares due to SSR = sum of squares due to regressionregression
The Coefficient of DeterminationThe Coefficient of Determination
OUTPUT FROM PROC REG
Dependent Variable: quarsales
Analysis of Variance
Sum of MeanSource DF Squares Square F Value Pr > F
Model 1 SSR 14200 14200 74.25 <.0001
Error 8 SSE 1530 191.25000
Corrected Total 9 SST 15730
Root MSE 13.82932 R-Square 0.9027Dependent Mean 130.00000 Coeff Var 10.63794
Coefficient of Determination
42 130
46 115
42 148
71 100
80 156
74 162
70 151
80 156
85 162
72 158
64 155
81 160
41 125
61 150
75 165
First value is age
Second value is SBP
Find the regression equation
SBP = b0 + b1*age
Your TURN