Choice-Based Conjoint Workshop

October, 2010 With information provided by

Sawtooth Software

Outline

• Introduction to Conjoint Analysis• Formulating Experiments• Conjoint Methods -types• Use CBC software• Introduction to CBC analysis

• Introduction to Conjoint Analysis– Identify Your Goal– Design your experiment – 5 stages– Interpreting part-worths and importance

– A brief introduction to market simulations

Different Perspectives, Different Goals

• Public wants all of the most desirable features of environmental assets at lowest possible cost

• Providers want to maximize welfare by:1) minimizing costs of providing features 2) providing products/services that offer greater overall value than other alternatives

Demand or Preference Side of Equation

• Focus first on demand/preference side of the equation

• After figuring out what consumer wants, next assess whether it can be built/provided in a cost- effective manner

Products/Services are Composed of Features/Attributes

• Olive Oil– Source, Price, Aroma, Size

• Forest Harvesting ProgramLive trees after harvesting, Cost , Dead trees after harvesting, % of forest set aside from harvest

• Plastic Bag Management– Cost/tax, % wildlife impact, durability, waste

Breaking the Problem Down

• If we learn how consumer values the components of a product, we are in a better position to design those that improve profitability

• If we learn how the public values the components of environmental goods and services, we are in a better position to design those goods and services to maximize societal welfare

How to Learn What Customers/Public Want?

• One way: Ask Direct Questions about preference– What Brand do you prefer?– How much would you pay for it?– What color do you prefer– What size of container would you like?

• Answers often trivial and unenlightening (e.g. respondents prefer low fees to high fees, medium size than large etc..)

How to Learn What Is Important?

• One way: Ask Direct Questions about Importances– How important is it that you get the << >> that you want?

Stated Importances

• Importance Ratings often have low discrimination:

Stated Importances

• Answers often have low discrimination, with most answers falling in “very important” categories– If they were not important, we probably wouldn’t have

included them in the research!

• Answers sometimes useful for segmenting market, but still not very actionable– We still don’t exactly what product they want

What is Conjoint Analysis?• Research technique developed in early 70s

• Measures how buyers value components of a product/service bundle

• Dictionary definition-- “Conjoint: Joined together, combined.”

• Marketer’s catch-phrase-- “Features CONsidered JOINTly”

How Does Conjoint Analysis Work?

• We vary the product/service features (independent variables) to build many (usually 12 or more) product concepts

• We ask respondents to rate/rank or choose among a subset of those product concepts (dependent variable)

• Based on the respondents’ evaluations of the product concepts, we figure out how much unique value (utility) each of the features (attributes) added

• (Regress dependent variable on independent variables; estimated betas equal to part worth utilities.)

Important Early Articles

• Luce, Duncan and John Tukey (1964), “Simultaneous Conjoint Measurement: A New Type of Fundamental Measurement,” Journal of Mathematical Psychology, 1, 1-27

• Green, Paul and Vithala Rao (1971), “Conjoint Measurement for Quantifying Judgmental Data,” Journal of Marketing Research, 8 (Aug), 355-363

• Johnson, Richard (1974), “Trade-off Analysis of Consumer Values,” Journal of Marketing Research, 11 (May), 121-127

• Green, Paul and V. Srinivasan (1978), “Conjoint Analysis in Marketing: New Development with Implications for Research and Practice,” Journal of Marketing, 54 (Oct), 3-19

• Louviere, Jordan and George Woodworth (1983), “Design and Analysis of Simulated Consumer Choice or Allocation Experiments,” Journal of Marketing Research, 20 (Nov), 350-367

Traditional, Less-Effective Questions

• How important is horsepower to you in a vehicle?

• How important is fuel efficiency to you in a vehicle?

• Which is more important to you, horsepower or fuel efficiency?

What’s So Good about Conjoint?• More realistic questions:

Which product would you prefer . . .- 210 horsepower or - 140 horsepower- 17 MPG - 28 MPG

• If choose left, you prefer Power. If you choose right, you prefer Fuel Economy

• Rather than ask directly whether you prefer Power over Fuel Economy, we present realistic tradeoff scenarios and infer preferences from your product choices

What’s So Good about Conjoint? (cont.)

• When respondents are forced to make difficult tradeoffs, we learn what they truly value

• These values (utility scores) are associated with specific and actionable attribute levels relevant to the problem at hand

Building a Model

Building a Model• Inputs:

– Attributes– Levels– Respondents– Prior Knowledge– External Data– Experimental

Design– Conjoint Method

• Outputs:– Utility Scores for each

level– Importance Scores for

each attribute– Ability to perform

Simulations

Defining Attributes

• Attributes are independent aspects of a product or a service (Brand, Price, Size, Color etc.)

• How many attributes?• -Depends on research objectives

– One rule of thumb was that no more than 6 0r 7 attributes is too much• May cause respondents to simplify, looking only at 2-3 most important

• Attributes should be independent, mutually exclusive– Brand, quality and product life expectancy may all measure the same thing

• Each attribute has varying degrees, or “levels”– Cost: $1, $2, $3

– Biodiversity Loss: 10, 50, 100

• Each level is assumed to be mutually exclusive of the others (a program has one and only one level of that attribute)

Rules for Formulating Attribute Levels

• Attributes are assumed to be mutually exclusive– Attribute: Add-on features– Level 1= Sun roof– Level 2= GPS system– Level 3=DVD player– If you define levels in this way, you cannot

determine the value of providing 2 or 3 of these features at the same time (or none of them)

Solutions

• 8 level Attribute:• Features– None– Sunroof– GPS system– DVD Player– Sunroof, GPS– Sunroof, DVD– GPS, DVD– Sunroof, GPS, DVD

• 3 Binary Attributes:– Sunroof:

• None• Sunroof

– GPS System• None• GPS

– DVD Player• None• DVD Player

Rules for Formulating Attribute Levels

• Levels should have concrete/unambiguous meaning

• “very expensive” vs “ costs $575”• “weight: 5-7 kilos” vs “ weight 6 kilos”

• -One description leaves meaning up to individual interpretation, while the other does not

Rules for Formulating Attribute Levels

• Don’t include too many levels for any one attribute– The usual number is about 3-5 levels per attribute– Make sure levels from your attributes can

combine freely with one another without resulting in utterly impossible combinations (very unlikely combinations OK)

Attribute Examples

Cost$1$2$3

BrandAB C

ColorRedBlackBlue

…or graphics as well can be levels.

Suggestions for Determining Which Attributes & Levels to Include

• Talk to all stakeholders

• Focus Groups

• Search of competitors websites, sales materials

Other Inputs into the Model

Conjoint Utilities (Part Worths)

• Numeric values that reflect how desirable different features are:

Feature UtilityVanilla 2.5Chocolate 1.8

25¢ 5.335¢ 3.250¢ 1.4

• The higher the utility, the better

Interpreting Conjoint Utilities

• Interval scaled data (no ratio operations!)• You cannot compare one level from one attribute

with one level from another attribute, since conjoint utilities are scaled to an arbitrary constant within each attribute (often zero-centered)

• You CAN compare differences between two levels of one attribute versus two levels of another attribute (an addition operation)

Conjoint Importances

• Ratio scaled data• Measure of how much influence each attribute has on people’s

choices• Best minus worst level of each attribute, then percentaged:

Vanilla - Chocolate (2.5 - 1.8) = 0.7 15.2%25¢ - 50¢ (5.3 - 1.4) = 3.9 84.8%

----- --------Totals: 4.6 100.0%

• Importances are directly affected by the range of levels you choose for each attribute

Market Simulations

• Make alternative program/services scenarios and predict which program/services respondents would choose

• Accumulate (aggregate) respondent predictions to make “Shares of Preference” (some refer to them as “market shares”)

Market Simulation Example

• Predict market shares for 35¢ Vanilla cone vs. 25¢ Chocolate cone for Respondent #1:

Vanilla (2.5) + 35¢ (3.2) = 5.7Chocolate (1.8) + 25¢ (5.3) = 7.1

• Respondent #1 “chooses” 25¢ Chocolate cone!

• Repeat for rest of respondents. . .

Market Simulation Results

• Predict responses for 500 respondents, and we might see “shares of preference” like:

• 65% of respondents prefer the 25¢ Chocolate cone

35%

65%

Vanilla @ 35¢

Chocolate @ 25¢

So you want to do a conjoint…..

Step 1: Begin With the End in Mind

• What is the objective of the research?– How much will public be willing to pay for biological

control feature?– Will farmers switch and adopt a different varieties?

• The better you define the root problem, the better your research will be!

Step 2: Plan Your Analysis

• Identify how clients need to use data

• Deliver analysis plan to clients as part of research proposal– Makes sure that objectives and deliverables are clear

Step 3: Define Attributes and Levels• How many attributes? Depends on research

objectives– More than 6 attributes may cause respondents to

simplify, looking only at 3-4 most important

• Attributes should be independent, mutually exclusive– Brand, quality, product life expectancy may all

measure the same thing

Rules for Formulating Attribute Levels

• Levels are assumed to be mutually exclusive

Attribute: Add-on features

level 1: Manuallevel 2: Biological Controllevel 3: Chemical

– If you define levels in this way, you cannot determine the value of providing two or three of these features at the same time (or none of them)

Rules for Formulating Attribute Levels

• Don’t include too many levels for any one attribute– The usual number is about 3 to 5 levels per attribute

• One temptation is to include many levels for price, so we can estimate people’s preferences for each– Better approach usually is to interpolate between fewer

more precisely measured levels for “not asked about” prices

• Cover the range of probable values

Rules for Formulating Attribute Levels

• Make sure levels from your attributes can combine freely with one another without resulting in utterly impossible combinations (very unlikely combinations OK)

Representing Levels• Text

– “High Performance Sports Car”• Pictures / Graphics

• Sample boards– Allows respondents to touch or feel samples for tactile attributes

(towel softness, greeting card paper quality, etc.) • “Null” Level – Has Stereo vs. __________

Low

Medium

High

Step 3: Choose a conjoint methodStep 4: Identify Research Constraints• Sample issues• Sample Population >200• Length of survey (how long can I keep their

attention)• Fielding issues• Budget• Client sophistication

Attributes Program A: Program B:Cost $.50 $.25Color Red Silver

Size .33 l .75 l

Sugar level Diet/light Regular

Example of a Pair of Soda Product Profile Scenarios

How to tell if you are in over your head…

• You should be okay if…– Small number of attributes– Attributes freely combine with one another– Large sample, even after adjusting for subgroup

analysis– Your client can describe the analysis, attributes in

one paragraph or less, and you can then explain it to a six year old with little difficulty!

Section 2

• Intro to Choice-Based Conjoint (Discrete Choice Modeling)

Setting Up CBC Interview (Definitions)

Concept 1 Concept 2 Concept 3 Concept 4

How many concepts per task?

How many tasks per survey?

Task

How Many Concepts per Task?

• Generally, 2 to 5 concepts are used

• Attribute text length, graphical representation affect the decision

How Many Tasks per Survey?• Respondents are expensive to recruit. It makes sense to

ask respondents multiple choice tasks.

• Respondents take about 7 minutes on average to answer 20 tasks (~20 seconds per task)

• CBC is very flexible in terms of how many tasks to include. Minimum is just one task! (but you’ll need huge sample size, and will face limitations in analysis)

• Typical choice is 12 to 18 choice tasks

Test CBC Design

• What is a Design? The sum total of information about the attribute levels being shown in the CBC tasks across all respondents. (The independent variable matrix)

• If you use ANY prohibitions, or use few questionnaire versions, you MUST test your design

• Failure to test the design can invalidate your study

• CBC/Web automatically tests your design when it generates the design file--pay attention to the test!

The “None” Concept• Pros:

– Respondents aren’t forced to choose a product concept that they really don’t like

– Lets you capture information about whether respondents (or segments) are more or less interested in buying the product concept

• Cons:– Choices of “None” provide much less information for estimating

utilities than other choices (reduces the effectiveness of your sample size)

– “None” utilities and Shares of Preference for “None” are difficult to interpret

Design Stage of CCE

Exercise

• Decide on topic• Discuss how you are going to decide the

attributes and the levels• Begin to think about attitudes• Assign team members tasks for the above• Be efficient