Regression & Correlation

Review:Types of Variables

&Steps in Analysis

Types of Variables

 Nominal / Categorical: each value is distinct category [gender, blood type, city]

 

Scale / Interval: linear measure, same intervalbetween each value [age, weight, IQ, GPA, SAT, income]

 

Ordinal: ranking, un-equal intervalsbetween values

[Likert scale, preference ranking]

Variables & Statistical TestsVariable Type Example Common Stat

MethodNominal by nominal

Blood type by gender

Chi-square

Scale by nominal GPA by gender

GPA by major

T-test

Analysis of Variance

Scale by scale Weight by height

GPA by SAT

Regression

Correlation

Evaluating an hypothesis

• Step 1: What is the relationship in the sample?

• Step 2: How confidently can one generalize from the sample to the universe from which it comes?

Evaluating an hypothesis

Relationship in Sample

Statistical Significance

2 ordinal vars. Cross-tab / contingency table

“p value” from Chi Square

Scale dep. & 2-cat indep.

Means for each category

“p value” from t-test

Scale dep. & 3+ cat indep.

Means for each category

“p value” from ANOVA

2 scale vars. Regression line

Correlation

“p value” from reg or correlation

Relationships betweenScale Variables

Regression

Correlation

Regression• Amount that a dependent variable

increases (or decreases) for each unit increase in an independent variable.

• Expressed as equation for a line –

y = m(x) + b – the “regression line”

• Interpret by slope of the line: m

(Or: interpret by “odds ratio” in “logistic regression”)

Correlation• Strength of association of scale measures

• r = -1 to 0 to +1

+1 perfect positive correlation

-1 perfect negative correlation

0 no correlation

• Interpret r in terms of variance

Mean&

Variance

Survey of Classn = 42

• Height• Mother’s height• Mother’s education• SAT• Estimate IQ• Well-being

(7 pt. Likert)

• Weight• Father’s education• Family income• G.P.A.• Health (7 pt. Likert)

Frequency Table for: HEIGHT  Valid CumValue Label Value Frequency Percent Percent Percent  59.00 1 2.4 2.4 2.4 61.00 2 4.8 4.8 7.1 62.00 3 7.1 7.1 14.3 63.00 3 7.1 7.1 21.4 65.00 5 11.9 11.9 33.3 66.00 3 7.1 7.1 40.5 67.00 4 9.5 9.5 50.0 68.00 5 11.9 11.9 61.9 69.00 1 2.4 2.4 64.3 70.00 6 14.3 14.3 78.6 71.00 1 2.4 2.4 81.0 72.00 4 9.5 9.5 90.5 73.00 3 7.1 7.1 97.6 74.00 1 2.4 2.4 100.0 ------- ------- ------- Total 42 100.0 100.0 Valid cases 42 Missing cases 0

Frequency Table for: HEIGHT  Valid CumValue Label Value Frequency Percent Percent Percent  59.00 1 2.4 2.4 2.4 61.00 2 4.8 4.8 7.1 62.00 3 7.1 7.1 14.3 63.00 3 7.1 7.1 21.4 65.00 5 11.9 11.9 33.3 66.00 3 7.1 7.1 40.5 67.00 4 9.5 9.5 50.0 68.00 5 11.9 11.9 61.9 69.00 1 2.4 2.4 64.3 70.00 6 14.3 14.3 78.6 71.00 1 2.4 2.4 81.0 72.00 4 9.5 9.5 90.5 73.00 3 7.1 7.1 97.6 74.00 1 2.4 2.4 100.0 ------- ------- ------- Total 42 100.0 100.0 Valid cases 42 Missing cases 0  Descriptive Statistics for: HEIGHT ValidVariable Mean Std Dev Variance Range Minimum Maximum N HEIGHT 67.33 3.87 14.96 15.00 59.00 74.00 42

mean

Variance

x i - Mean )2

Variance = s2 = ----------------------- N

 

Standard Deviation = s = variance

Frequency Table for: WEIGHT Valid CumValue Label Value Frequency Percent Percent Percent  115.00 1 2.4 2.4 2.4 120.00 1 2.4 2.4 4.8 124.00 1 2.4 2.4 7.1 125.00 4 9.5 9.5 16.7 128.00 1 2.4 2.4 19.0 130.00 6 14.3 14.3 33.3 135.00 4 9.5 9.5 42.9 136.00 1 2.4 2.4 45.2 140.00 3 7.1 7.1 52.4 145.00 2 4.8 4.8 57.1 150.00 3 7.1 7.1 64.3 155.00 2 4.8 4.8 69.0 160.00 6 14.3 14.3 83.3 165.00 2 4.8 4.8 88.1 170.00 1 2.4 2.4 90.5 185.00 1 2.4 2.4 92.9 190.00 2 4.8 4.8 97.6 210.00 1 2.4 2.4 100.0 ------- ------- ------- Total 42 100.0 100.0 Valid cases 42 Missing cases 0  Descriptive Statistics for: WEIGHT ValidVariable Mean Std Dev Variance Range Minimum Maximum N WEIGHT 146.38 21.30 453.80 95.00 115.00 210.00 42

mean

Relationship of weight & height:

Regression Analysis

“Least Squares” Regression Line

Dependent = ( B ) (Independent) + constant

weight = ( B ) ( height ) + constant

Regression line

Regression: WEIGHT on HEIGHT Multiple R .59254R Square .35110Adjusted R Square .33488Standard Error 17.37332 Analysis of Variance DF Sum of Squares Mean SquareRegression 1 6532.61322 6532.61322Residual 40 12073.29154 301.83229 F = 21.64319 Signif F = .0000 ------------------ Variables in the Equation ------------------ Variable B SE B Beta T Sig T HEIGHT 3.263587 .701511 .592541 4.652 .0000(Constant) -73.367236 47.311093 -1.551 

[ Equation: Weight = 3.3 ( height ) - 73 ]

Regression line

W = 3.3 H - 73

Strength of Relationship

“Goodness of Fit”: Correlation

How well does the regression line “fit” the data?

Frequency Table for: WEIGHT Valid CumValue Label Value Frequency Percent Percent Percent  115.00 1 2.4 2.4 2.4 120.00 1 2.4 2.4 4.8 124.00 1 2.4 2.4 7.1 125.00 4 9.5 9.5 16.7 128.00 1 2.4 2.4 19.0 130.00 6 14.3 14.3 33.3 135.00 4 9.5 9.5 42.9 136.00 1 2.4 2.4 45.2 140.00 3 7.1 7.1 52.4 145.00 2 4.8 4.8 57.1 150.00 3 7.1 7.1 64.3 155.00 2 4.8 4.8 69.0 160.00 6 14.3 14.3 83.3 165.00 2 4.8 4.8 88.1 170.00 1 2.4 2.4 90.5 185.00 1 2.4 2.4 92.9 190.00 2 4.8 4.8 97.6 210.00 1 2.4 2.4 100.0 ------- ------- ------- Total 42 100.0 100.0 Valid cases 42 Missing cases 0  Descriptive Statistics for: WEIGHT ValidVariable Mean Std Dev Variance Range Minimum Maximum N 

WEIGHT 146.38 21.30 453.80 95.00 115.00 210.00 42

mean

mean

Variance = 454

Regression line

mean

Correlation: “Goodness of Fit”• Variance (average sum of squared

distances from mean) = 454

• “Least squares” (average sum of squared distances from regression line) = 295

• 454 – 295 = 159 159 / 454 = .35

• Variance is reduced 35% by calculating from regression line

r2 = % of variance in WEIGHT

“explained” by HEIGHT

Correlation coefficient = r

Correlation: HEIGHT with WEIGHT   HEIGHT WEIGHT HEIGHT 1.0000 .5925 ( 42) ( 42) P= . P= .000 WEIGHT .5925 1.0000 ( 42) ( 42) P= .000 P= .

r = .59

r2 = .35

HEIGHT “explains” 35% of variance in WEIGHT

Sentence & G.P.A.

• Regression: form of relationship

• Correlation: strength of relationship

• p value: statistical significance

grade point average

grade point average

4.003.903.803.753.703.403.333.203.00

Fre

qu

en

cy

7

6

5

4

3

2

1

0

Statistics

grade point average23

1

3.5752

.35057

.12290

Valid

Missing

N

Mean

Std. Deviation

Variance

G. P. A.

jail sentence

jail sentence

18.0012.0011.009.006.004.003.002.00.00

Fre

qu

en

cy

7

6

5

4

3

2

1

0

Statistics

jail sentence24

0

5.1250

5.44788

29.67935

Valid

Missing

N

Mean

Std. Deviation

Variance

Length of Sentence

grade point average

4.24.03.83.63.43.23.02.8

jail

sen

ten

ce

20

10

0

-10

Scatterplot: Sentence on G.P.A.

Regression Coefficients

Coefficientsa

17.853 12.097 1.476 .155

-3.534 3.368 -.223 -1.049 .306

(Constant)

grade point average

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: jail sentencea.

Sentence = -3.5 G.P.A. + 18

grade point average

4.24.03.83.63.43.23.02.8

jail

sen

ten

ce20

10

0

-10

Sent = -3.5 GPA + 18

“Least Squares” Regression Line

Correlation: Sentence & G.P.A.

Correlations

1 -.223

. .306

23 23

-.223 1

.306 .

23 24

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

grade point average

jail sentence

grade pointaverage jail sentence

Interpreting Correlations

• r = -22 p = .31

• r2 = .05

G.P.A. “explains” 5% of the variance in length of sentence