Logistic Regression

Linear Regression

Purchases vs. Income

R Graphical Parameters

Purchase Dataset

• Conceptually, if a person has greater income, the probability that he or she will purchase is greater than if the person has less income.

Categorical with a Linear Model

Residual vs .Fitted

Categorical Dependent Variable

Binary Data

Don’tPrefer

Log is better at representing the data

Logistic Regression

bXaY

YODDS

ˆ1

ˆlnln

19237.025.1 XeODDS

Percent Yes --> PNo 165 49%

Yes 170 51%335

Odds Ratio 51% 1.0349%

Log Odds Ratio (Logit)Const Beta

Ln (Odds) 1.25 0.92

Exp(Beta) = OddsExp(0.92) = 2.52

Odds * Exp(Beta) 2.59

Calculated% 72%

Original Percentage 51%

Delta 21%

The original classification table is put in here to get the Ns as well as to get the original percent among the respondents

The original percent is turned into a probability

The Average Odds is then multiplied by the Exp of the Beta.

Which is then turned back into a percentage

The original percentage is subtracted from the predicted percent to determine the change

~1:1 Ratio for getting a No or Yes

Logit ModelIncludes Log;So Need to

Convert to Odds

2.52 vs. 1.03

Odds = P / (1-P)Odds – (Odds*P) = POdds = P + Odds*POdds = P(1 + Odds)P = Odds / (1 + Odds)

Delta from the Average Odds

100%-72% = 28%

72% / 28% = 2.6

The Regression Beta is then converted to Odds.

Mathematics

Logistic Regression

Measure of Goodness• R^2 ranges from 0 to 1.0, and can be considered as

a percentage of variability. An R 2 of 1.0—or 100%—means that 100% of the variance in the dependent variable can be explained by variability in the independent variable or variables.

• We use the log likelihood as our criterion for the “best” coefficients.

• The closer to 0.0 a log likelihood:• the better the fit• the closer you’ve come to maximizing the estimate of

the likelihood.

• Probability of No Purchase: • Person who did not purchase has a 0 on the Purchased variable• Predicted probability of 2% that he will purchase

• Probability of Purchase: • Person who did purchase has a 1 on the Purchased variable• Predicted probability of 94% that this person will purchase

• The probabilities are of two different events: • No Purchase and Purchase• In the first case, it’s 98% that he doesn’t purchase, and he doesn’t.