Upload
phungkhue
View
222
Download
0
Embed Size (px)
Citation preview
REGRESSION Statistics for Language Research Fall 2016
12/6/2016
1
陈芳
华东师大英语系
OVERVIEW Regression
The regression line Intercept and slope estimates in a regression line
Unstandardized Standardized
Accuracy of prediction Handling outliers Hypothesis testing Prediction as a purpose SPSS Extension
12/6/2016
2
陈芳
华东师大英语系
REGRESSION Regression is the process of finding the best
fitting line using the data we have. This line is most often used to predict or
estimate a score of one variable (Y) from another (X). Predict for future occurrences Estimate a mean value Examples
12/6/2016
3
陈芳
华东师大英语系
12/6/2016 陈芳
华东师大英语系
4
PREDICTING A SCORE If we know nothing else but
that the mean psychological symptom score of students in ECNU is 90 (based on measures such as Hopkins Symptom Checklist).
We know a student comes
from ECNU
12/6/2016
5
陈芳
华东师大英语系
PREDICTING A SCORE If we have some extra information
such as the level of stress of ECNU students and we know that level of stress is related to the number of symptoms. We could give a more precise estimate of a student’s symptom score given that we know his stress level.
E.g. I can tell you the student’s stress score is 42, now your estimated symptom score maybe 105.
E.g. I can tell you that the student’s stress score is 18, now your estimated symptom score is 92.
12/6/2016
6
18 42
陈芳
华东师大英语系
FIND THE LINE Find the line is the same as finding a and b that
defines the line.
12/6/2016
7
陈芳
华东师大英语系
THE REGRESSION LINE
Think back to your high school math class…the equation for a straight line is:
12/6/2016
8
Predicted value of Y
Slope of regression line
Value of X
Intercept
bXaY +=ˆ
陈芳
华东师大英语系
These are our regression coefficients
THE BEST-FITTING LINE ---VISUAL HELP the best-fitting line minimizes the errors of
prediction. the difference between observed Y and predicted
Ŷ is called the residual, it is represented as (Y- Ŷ ).
Ordinal least squared estimation: sum the square of the residuals.
Draw on the board. No matter how the line tilts, it will always pass one
point, that is
12/6/2016
9
陈芳
华东师大英语系
),( YX
THE REGRESSION LINE The slope of the best fitting line we just described can
be estimated as below, this can be proven, but we will not do that here.
==
X
Y
X
XY
ssr
sb 2
cov
12/6/2016
10
陈芳
华东师大英语系
NXbY
XbYa ∑ ∑−=−=
STANDARDIZED DATA So what do we do if our data are standardized—that is,
both X and Y are z-scores? The mean of X in this case is ____. The mean of Y in this case is also ____. The standard deviation of X and Y are both _____. Centering
Think now about the equations we used to estimate our regression coefficient—and plug in the values above
12/6/2016
11 XXbXbaY β+=+=+= 00ˆ '''
SPSS call this beta, standardized coefficient.
陈芳
华东师大英语系
0)0(0 ''' =−=−= bXbYa rrssrb
X
Y =
=
=
11'
INTERPRETATION OF STANDARDIZED SLOPE COEFFICIENT (BETA)
This changes our interpretation a bit… Now, an one unit increase in X or Y corresponds to
an increase of one standard deviation If β = 0.75, this means that an increase of one
standard deviation in X relates to a ¾ standard deviation increase in Y.
12/6/2016
12
XY β+= 0ˆ
陈芳
华东师大英语系
HYPOTHESIS TESTING
Regression has two coefficients (slope & intercept) We still use t-test to test this. In SPSS, the t-test has a p-value attached to it. As usual,
if p<alpha(usually 0.05), we reject the null hypothesis and conclude the coefficient is significantly different from 0.
12/6/2016
13
陈芳
华东师大英语系
JUST FOR YOUR INFORMATION t-test for correlation r. You can use look up the critical t value. But we will
leave this to SPSS. t-test for slope coefficient b. It follows a t distribution
with n-2 degrees of freedom.
You can use either equation to double-check the t statistic in the following example
12/6/2016
14
[ ]21
22
−=−
−== ndf
rNr
srtr
21)1(
1)(1)(
12ˆˆ
−−
−
−=
−=
−
=−−
NNrs
Nsbs
Nsb
Nss
bt
Y
X
YY
X
X
YY
陈芳
华东师大英语系
SPSS DEMO
12/6/2016
15
Stress and Syptom data online. Predicting symptom scores from level of stress. Can you write the equation for this best fitting line?
Intercept
Slope
Standardized slope coefficient. =r (simple regression only.)
Does not include 0. Yes, significant.
Smaller than .05. Significant. Reject null hypothesis and conclude the slope is different from 0.
We don’t care about this.
Unstandardized slope coefficient
陈芳
华东师大英语系
PREDICTED VS. ACTUAL SCORE
For a score of X=25 we can predict a score of about 93.5 on Y. (How do we find the value?)
We can compare how the real values of Y that occurred with a score of X = 25 compare to our prediction.
12/6/2016
16
陈芳
华东师大英语系
STANDARD ERROR OF ESTIMATE (Y – Ŷ) is the error of prediction, or the residual. Remember sum of squared error or SSerror? The standard error of estimate (SEE) is defined as This equation, like standard deviation, describes on
average, how our predicted scores are from the actual scores.
SEE can be more easily estimated with:
12/6/2016
17
( )2ˆ
112YY Y
Ns s rN−
− = − −
∑ − 2)ˆ( YY
22)ˆ( 2
.ˆ −=
−−
== ∑− N
SSN
YYss error
XYYY
陈芳
华东师大英语系
SEE AS A MEASURE OF THE ACCURACY OF PREDICTION SEE is one way we can assess how well our
regression equation is working. predicting with a dependent variable is better that
predicting without one, but it is not without error. If the regression equation works well, it’s safe to assume
the predicted values of Y should be very close to the real values.
The smaller the SEE, the better the equation predicts.
12/6/2016
18
陈芳
华东师大英语系
R2 AS A MEASURE OF ACCURACY OF PREDICTION
In this equation, SSY is the total variance in Y that are composed of two parts a. The part of the variability in Y that is associated
with X, , learn to use the following phrases. that can be explained by X that can be predicted by X that is accounted by X that is attributable to X
b. The part of the variability in Y that is independent
of X, which is SSerror.
12/6/2016
19
Y
errorY
Y
Y
SSSSSS
SSSS
r −== ˆ2
YSS ˆ
陈芳
华东师大英语系
12/6/2016 陈芳
华东师大英语系
20
CALCULATE THE SSY-HAT 12/6/2016
21
陈芳
华东师大英语系
256.702.410173.105
=SSY SSY-hat
CALCULATE THE SSERROR 12/6/2016
22
陈芳
华东师大英语系
173.105702.410529.305 −=
PREDICTABLE VARIABILITY & R2
The higher the r2, the better the predictors are working (the more variance in Y that are explained by the predictors).
If a correlation (r) is found to be 0.8, we calculate r2 = 0.82 = 0.64
How do we interpret this? This means that 64% of the variance of Y can be
explained by the variability of X. You can use the phrases on the previous slide
interchangeably. Remember, this does NOT mean that 64% of Y is
caused by X.
12/6/2016
23
陈芳
华东师大英语系
SPSS DEMO Predicting course quality from grades. Interpret R2. Based on the regression equation, what overall rate will be
if the student’s expected grade is 3.1?
12/6/2016
24
陈芳
华东师大英语系
WHAT AFFECTS REGRESSION ANALYSIS? REMEMBER THIS?
4 major factors affect the calculation of correlations: 1. Nonlinearity of relationship 2. Restriction of range (or variance) of X and/or Y 3. Use of heterogeneous sub-samples 4. Outliers
12/6/2016
25
陈芳
华东师大英语系
12/6/2016
26
陈芳
华东师大英语系
USE SPSS Sleep and mood: This is fictional data Create scatter plot to check the linear
relationship assumption. If not, transformation of data might be necessary.
Check for outliers. Delete outliers.
Read the output. Make your conclusion. Interpret the results. Evaluate model fit.
12/6/2016
27
陈芳
华东师大英语系
EXTENSION: COMPARING T-TEST AND REGRESSION
12/6/2016
28
陈芳
华东师大英语系
EXTENSION: COMPARING T-TEST AND REGRESSION
12/6/2016
29
陈芳
华东师大英语系
SOME EXTENSIONS OF REGRESSION Multiple Regression
This method involves the prediction of Y from several independent variables. There is no regression line in this case We calculate a single a (intercept) for this multiple
regression, but each IV has a separate b (slope) estimated.
These slopes are not the exactly the slope of the line, but more like a weighting function .
We can perform a hypothesis test to see if each of these weights help predict Y using t-test. Again, SPSS gives help.
Our r2 should increase as we add more predictors—We can explain more and more of the variance in Y by adding different predictors
12/6/2016
30
陈芳
华东师大英语系
SOME EXTENSIONS OF REGRESSION Chapter 15 contains a introduction to multiple
regression.
12/6/2016
31
陈芳
华东师大英语系
EXTRA Testing the difference
between independent and nonindependent rs.
PIP (proportional improvement in prediction)
Homogeneity of variance in arrays
Errors of prediction as a function of r: r=.2, SEE reduced by 2%,
SEM is 98% of the prediction without x; r=.95, SEE reduced by two-thirds.
12/6/2016 陈芳
华东师大英语系
32