Regression and Correlation
Jake BlanchardFall 2010
IntroductionWe can use regression to find
relationships between random variables
This does not necessarily imply causation
Correlation can be used to measure predictability
Regression with Constant VarianceLinear Regression: E(Y|
X=x)=+xIn general, variance is function of
xIf we assume the variance is a
constant, then the analysis is simplified
Define total absolute error as the sum of the squares of the errors
Linear Regression
n
ii
n
iii
n
iiii
n
iii
n
iii
n
iii
xx
xxyy
xysolve
xyx
xy
xyxy
1
2
1
1
2
1
2
1
2
1
22
02
02
Variance in Regression AnalysisRelevant variance is conditional:
Var(Y|X=x)
2
2|2
22|
1
22
1
22|
1
22|
1
2
2121
Y
XY
XY
n
ii
n
iiXY
n
iiiXY
ss
r
ns
xxyyn
s
xyn
s
Confidence IntervalsRegression coefficients are t-
distributed with n-2 dofStatistic below is thus t-
distributed with n-2 dof
And the confidence interval is
n
ii
ixY
xYi
xx
xxn
s
Yi
1
2
2
|
|
1
n
ii
iXY
nixY
xx
xxn
styi
1
2
2
|2,
211|
1
ExampleExample 8.1Data for compressive strength (q)
of stiff clay as a function of “blow counts” (N)
038.08305.0
2
029.0
112.0
22.191
12.9591123.27.18
22|
22
222
222
ns
Nq
NnNqNnqN
qnqs
NnNs
qN
Nq
i
ii
iq
iN
744.0,21.07.18*104353
7.184101038.*306.2477.
477.04*112.0029.04
306.2
1
95.0|
2
2
95.0|
8,975.0
1
2
2
|2,
211|
Nq
Nq
i
n
ii
iXY
nixY
yNat
t
xx
xxn
styi
Plot
Correlation Estimate
22
2|2
,
,
1,
1,
121
11
11
rss
nn
ss
ss
yxnyx
n
ss
yyxx
n
Y
xYyx
Y
Xyx
YX
n
iii
yx
YX
n
iii
yx
Regression with Non-Constant VarianceNow relax
assumption of constant variance
Assume regions with large conditional variance weighted less
)(2
)(1
)(1
|1
)|()(|
|
1
2
2
22
2
11
2
1
1111
1
11
1
22
22
22
xsgsn
yyws
xgww
xwxww
ywxwyxww
w
xwyw
xyw
xgxXYVarw
weightsxxXYExgxXYVar
xY
n
iii
iii
n
iii
n
iii
n
ii
n
iii
n
iii
n
iiii
n
ii
n
ii
n
iii
n
iii
n
iiii
iii
Example (8.2)Data for maximum settlement (x)
of storage tanks and maximum differential settlement (y)
From looking at data, assume g(x)=x (that is, standard deviation of y increases linearly with x
2
22
1|
ii xw
xxXYVar
Example (8.2) continued
96.0
243.00589.0
65.0045.0
627.0923.011.165.1
|
2
xss
ssyx
xy
y
x
Multiple Regression
ikkiii xxxy ...22110
“Nonlinear” Regression
)()|( xgxYE
Use LINEST in Excel
Recommended
