Contexts in Which Best and Worst CBC are Most Valuable: Application to School Choice

Joel Huber: Duke UniversityNamika Sagara: Duke University

Angelyn Fairchild: Research Triangle Institute

Why Study School Choice?

• School choice is an increasingly difficult reality for parents.

• It is possible to mimic aspects of the actual choices

• There are many analyses of actual school choices but few published conjoint studies

• Positive and negative reaction to features makes school choice ideal for Best/Worst choice based conjoint

Our process

• Identified attributes differentiating public schools in actual choices

• Pretested these to make sure we covered the most important ones with 4 continuous attributes and 4 binary ones

• Built a fixed B/W CBC design • Ran the study on a national sample of 150

parents with a child entering grades 6-11.

Our Sample• Sample: Parent of a child age 11 to 17 attending

public school– 57% Female– 59% Caucasian– 59% had at least some college– Median income was between $50,001 and $75,000

• “Think about your youngest child that is more than 11 years old.”– Age and gender– School grade– Select five important characteristics for child’s school (e.g., close to home)

An Important One-time Decision

• “Suppose you just moved to a new area where families are able to choose which school they would most like their children to attend.”

• Introduction to attributes and practice questions– We defined ranges of each attribute and asked relative

importance– We built up gradually to more complex, realistic choices

Ultimate choice task

Attribute 1: Distance

Travel time to school

Bus Ride 5 min 15 min 30 min 45 min

“About how many minutes does it take for your child to get to school now?”

Attribute 2: Academic Quality

School Attributes

Travel time 5 min 15 min 30 min 45 min

% under Grade Level 15% 25% 35% 45%

“At your child's current school, what percent do you think are below grade level?”

Warm up Choice 1“Next we will ask you to choose between two schools with different travel time and percent of students below grade level.

Imagine that only two school options are available for your child, and the schools are the same except for the differences shown below.” 

Attribute 3: Income

School Attributes

Travel time 5 min 15 min 30 min 45 min

% under Grade Level 15% 25% 35% 45%

% Economically Disadvantaged 10% 30% 50% 70%

“At your child’s current school, what percent of students are economically disadvantaged?”

Attribute 4: Diversity

School Attributes

Travel time 5 min 15 min 30 min 45 min

% under Grade Level 15% 25% 35% 45%

% Economically Disadvantaged 10% 30% 50% 70%

PercentMinority 25% 40% 55% 70%

“At your child's current school, about what percent of students are minorities?”

Warm up Question 2

Featured Programs

International Baccalaureate (IB) Program

Featured Programs

International Baccalaureate (IB) Program

Science, Technology, Engineering and Math (S.T.E.M)

Featured Programs

International Baccalaureate (IB) Program

Science, Technology, Engineering and Math (S.T.E.M)

Expanded Arts Program

Featured Programs

International Baccalaureate (IB) Program

Science, Technology, Engineering and Math (S.T.E.M)

Expanded Arts Program

Promote Sports Teams

One of 8 Best/Worst Choice Tasks

“Best” Schools

15 25 35 45 5 15 30 45 10 30 50 70 25 40 55 70

-150

-100

-50

0

50

100

150

% Below Grade Level

Travel Time

Arts Sports STEMIB%

Minority

% Economically

Disadvantaged

“Worst” Schools

15 25 35 45 5 15 30 45 10 30 50 70 25 40 55 70

-150

-100

-50

0

50

100

150

% Below Grade Level

Travel TimeArts

% Economically Disadvantaged

Sports STEMIB%

Minority

What is important in a school?

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

Academic quality

Travel time

Income

% Minority

Arts

STEM

Sports

What is important in a School?

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

Academic quality

Travel time

Income

% Minority

Sports

IB

Arts

STEM

Who cares?

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

Academic quality

Travel time

Income

% Minority

Sports

IB

Arts

STEM

Educated Parent

Nonwhite

Older child

Lower incomePart time

Who cares?

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

Academic quality

Travel time

Income

% Minority

Sports

IB

Arts

STEM

Educated Parent

Nonwhite

Older child

Lower incomePart time

Employed full time

Higher income

Younger Child

White

Less educated parent

What we learned

• By building complexity gradually it is possible to generate reliable responses for a difficult and important choice

• 8 choices is sufficient to separate those with quite different values

• B/W provides both insight into what is desired and feared, and generates stable individual estimates

Simple individual level estimation

• All the analysis used standard Sawtooth Software HB analysis

• Can we generate a simplified model that can allow feedback to subjects on the fly?

• Concert 4-level variables to linear, producing 4 linear, 4 binary variables from 8 Best and 8 Worst choices

• This follows from work by Saigal and Dahan (Sawtooth Software Proceedings 2012)

Individual Linear Choice Using Simple Vector Product

• Choice vector Y has 32 items, 4 for each choice set, code Best as a 1, Worst as -1 and Zero otherwise

• Design matrix X(32,8) is zero centered within each choice set

• B-hat = (X’X)-1X’Y• Since we use a fixed design X we multiply

(X’X)-1X’ (8 x32) by Y(32x1) to get B(8x1)

Correlation between HB and linear choice model

% Under Grade Sports STEM Arts Minority Bustime IB Poverty0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Correlation of HB and Linear with Self Explicated

Academic Bustime STEM Poverty Arts Minority IB Sports0.00

0.10

0.20

0.30

0.40

0.50

0.60

Effective use of Best/Worst Choice Based Conjoint

• Best/Worst is appropriate for choices with levels people avoid or fear

• Where there is both attraction and avoidance, combining Best + Worst choices results in better results

• Analyzing Best + Worst with a linear model generates results reasonably close to HB analysis

Thank you!

What is important in a school?

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

Academic quality

Travel time

Income

Income

% Minority

Sports

IB

Arts

STEM

STEM

Correspondence between HB and linear estimates

HB Importance Estimates