at the Sawtooth Software Conference · at the Sawtooth Software Conference The Sawtooth Software conferences are renowned for their practical, practitioner-oriented focus and depth

CORPORATE RESEARCHERS SUMMIT at the Sawtooth Software Conference

Case studies and practical training by corporate researchers, for corporate researchers

September 23-27, 2019 San Diego, California

2019 Sawtooth Software Conference Page 1 of 24

Corporate Researchers Summit at the Sawtooth Software Conference

The Sawtooth Software conferences are renowned for their practical, practitioner-oriented focus and depth in the fields of choice/conjoint analysis, segmentation, and data collection/analysis. It is a forum to exchange ideas, network, and learn about quantitative methods in marketing research.

Companies represented at the Corporate Researchers Summit include AT&T, Bose, Google, Google Drive, Jewelers Mutual, Lifetime Products, Mozilla, Nu Skin International, Polynesian Cultural Center, Procter & Gamble, Riot Games, Sylvan Learning, Twitter, Universal Orlando, Washington Dept. of Fish & Wildlife, and Zappos.

Corporate Researchers Summit Held on September 25–27, this newly-designed event features

case studies and practical training by corporate researchers, for corporate researchers!

Learn how your peers are leveraging best-practice survey techniques and analytics to optimize combinations of features and product offerings.

Gain insights from experienced corporate researchers regarding communicating findings and gaining the trust of the C-suite.

See how best-worst scaling (MaxDiff) resolves the problems of standard ratings scales, leading to greater insights with smaller sample sizes.

Listen how others like you are using trade-off techniques (e.g. conjoint analysis and MaxDiff) to more effectively segment the market and create products and services to increase share and profit.

How can you improve your advertising and messaging? Experienced researchers show you how.

This is the perfect event for corporate researchers who gravitate toward quantitative marketing research methods, especially conjoint analysis, MaxDiff, advertising messaging, and market segmentation.

It provides accessible, practical sessions led by your peers for networking and enrichment.

Optimize Your Experience This year we are offering three parallel conference tracks on

September 25–27: one for corporate researchers, one for methodologists and academics, and the other for those who want to learn more about specific techniques and software. Stick with one track, or customize your experience by switching to and from the other tracks. And don't forget about our in-depth pre-conference tutorials on September 23-24!

Registration

Fee Schedule Early Registration

Normal Registration

Conference (Wed-Fri) $1500 $1650

Pre-Conference: 2-Day Workshop $1250 $1450

Pre-Conference: 1-Day Workshop $700 $800

Pre-Conference: Half-Day Tutorials $350 each $400 each

Visit https://www.sawtoothsoftware.com/summit to complete your registration. Your registration for the conference, workshops, and/or tutorials is not considered complete until payment has been received by Sawtooth Software.

Cancellation charges are:

$100 if cancellation is made before August 15, 2019

$300 if cancellation is made on or after August 15, 2019

Full fee if cancellation is made after September 9, 2019

Substitutions of registered attendees may be made up to the start of the general session on Wednesday.

Academic discounts are available for qualifying full-time students and full-time faculty. Contact [email protected] to see if you qualify.

Hotel Information The conference will be held September 23-27, 2019 at the Hilton

San Diego Resort & Spa:

Hilton San Diego Resort & Spa Hotel 1775 E Mission Bay Dr San Diego CA 92109-6801

To get the special Sawtooth Software room rate of $209, simply request a room during your online registration process. Or, call +1 619 276 4010 before August 20.


Corporate Research Track Wednesday, September 25, 2019

During the morning we are combining the Corporate Research Track with the Methodology Track for a special session featuring a collection of presentations specifically chosen to appeal to both audiences.

Conference Registration WEDNESDAY | 7:00 AM - 5:00 PM

Breakfast WEDNESDAY | 7:00 AM - 8:25 AM

Welcoming Remarks WEDNESDAY | 8:30 AM - 8:50 AM

Bryan Orme, Sawtooth Software, Conference Moderator

See the Methodology & Analytics Track for a description of the morning presentations.

Lunch WEDNESDAY | 12:00 PM - 1:30 PM

New Research Tools for Product Optimization and Idea Screening

WEDNESDAY | 1:30 PM - 1:45 PM

Bryan Orme, Sawtooth Software

Corporate researchers make up a critical and growing segment of our users. Bryan discusses the state of Sawtooth Software: new breakthroughs and also advances in our tools for product optimization and idea screening, including multi-objective product optimization and Bandit MaxDiff.

How Zappos Uses Conjoint Analysis for Superior Insights and Customer Segmentation


Sourav Bhuyan, Zappos

Sourav will showcase how CBC studies have provided insights at Zappos. He describes the impact it brought at Zappos with regards to understanding customer needs and preferences. There is no fun in a presentation without a case study! Sourav will go over some of the CBC studies conducted at Zappos pertaining to value propositions and retail offering. He will also discuss the organizational impact of such CBC studies across silos that help bring researchers and stakeholders closer to the customers.

Conjoint Analysis throughout the Product Development Cycle


Tracey Di Lascio-Martinuk, Bose

Research and development is at the heart of our work at Bose, and we frequently use data obtained through conjoint analysis and market simulation to inform and support key business decisions. This talk summarizes and illustrates three stages of product development where conjoint analysis provides valuable insights, then shares a case study of using conjoint analysis and market simulation to explore the impact of adding a new product to one of our sub-category portfolios.

Refreshment Break WEDNESDAY | 3:00 PM - 3:30 PM

Connecting People with Public Lands Using MaxDiff Insights


Cynthia Wilkerson, Washington Department of Fish & Wildlife

Rebecca Elmore-Yalch, Northwest Research Group

Carey Evenson, C+C

The Washington Department of Fish & Wildlife (WDFD) conducted research to guide the development of a branding and messaging platform aimed at increasing public awareness of the conservation and recreation resources managed by WDFW and the benefits for Washington residents and public land users. MaxDiff was used to measure the importance of 14 core benefits. Four core brand pillars were identified, and a comprehensive brand platform and messaging playbook was developed. This exemplifies the integration of research insights and strategy development.


The Implicit and Explicit Effectiveness of Advertising on Twitter


Jeff Zifrony, Twitter

Michelle Grushko, Twitter

While brand surveys in certain situations serve as a great tool in evaluating advertising effectiveness, we’ll demonstrate the significant limitations of their accuracy when it comes to assessing campaign impact on driving actual purchase behavior. In this talk we’ll walk through the innovative way we set out to advance lower funnel measurement for Insurance advertisers on Twitter, through a conjoint analysis and a simulated buying experience, and what this analysis means for marketers.

Reception WEDNESDAY | 6:00 PM - 7:00 PM

Thursday, September 26, 2019

Conference Registration THURSDAY | 7:00 AM - 5:00 PM

Breakfast THURSDAY | 7:00 AM - 8:25 AM

The Joys and Sorrows of the Corporate Researcher

THURSDAY | 8:30 AM - 9:15 AM

Christopher Chapman, Google

After nearly 20 years at Microsoft and Google, Chris has learned a few lessons. He will describe how corporate research often fails, and what it takes to get it right: the questions you ask, how research is positioned, how to use vendors more effectively, and why to keep more research in house. Chris shares career successes and fails, and will take Q&A. Bring your questions about corporate research in tech!

How to Quickly Capture the User's Voice to Inform Product Strategy


Julia Barrett, Google Drive

As a User Experience team, it's our responsibility to ensure the solutions we pursue address high-priority problems for our target users. We modeled this by leveraging the MaxDiff methodology to answer the question: "What are the biggest challenges we should be solving for knowledge workers?” Insights from this project informed Google Drive's 2019 planning and this approach shifted how we think about our mission as Apps UX Researchers: We help everyone fall in love with problems!

Refreshment Break THURSDAY | 10:00 AM - 10:30 AM

Fleeting Frustration or Fatal Flaw? Using Choice Models to Predict Customer Behavior and Prioritize Problems


Kegan Clark, Riot Games

With a multitude of possible problems with your product to focus on fixing, it can be difficult to know which problems matter, and actually predict adverse outcomes (e.g. customers leaving your service). We used MaxDiff to get a sensitive read on how a wide variety of frustrations with our product stack rank, and then used utilities to figure out which frustrations actually predicted a subsequent decline in use of our product using behavioral data.

Best Practices for Market Segmentation


Keith Chrzan, Sawtooth Software

Effective segmentation involves separating respondents into groups of people that are similar to one another, but different from members of other groups. This turns out to be surprisingly hard to do well given the kinds of data marketing researchers work with. Based on over 30 years of experience and hundreds of segmentation studies, Keith offers best practices to help researchers avoid building marketing strategies on foundations of imaginary segments.


Using ACBC to Design Online Promotional Messaging for Lifetime Sheds

THURSDAY | 11:15 AM - 12:00 PM

Bob Goodwin, Lifetime Products

Bob Goodwin lays out the rationale, design, and insights of a multi-phase research suite (two conjoint studies supported by focus group projects) into improving online promotion of Lifetime Products’ Shed category. He explains that, with today’s ever-growing Ecommerce channel, the Company is trying to explore ways to better communicate product features online, especially when many of those features – including robustness of resin construction and unique wall textures – are best understood by physical contact.

Lunch THURSDAY | 12:00 PM - 1:30 PM

Credit Union Analytics Use Cases THURSDAY | 1:30 PM - 2:00 PM

Greg Nolder, Experienced Credit Union Analyst

We created the analytics function at a credit union from scratch over ten years ago and have executed projects from interest rate and credit risk to marketing analytics. We got effective and actionable results from techniques as simple as spreadsheet moving averages up to conjoint analysis and machine learning algorithms like random forests. This is an overview discussion of some interesting case studies, learning and tools from that work.

Managing Your Online Research Community for High ROI

THURSDAY | 2:00 PM - 2:30 PM

John Voda, AT&T

John managed an online research community of 500 small/midsize business decision makers for eight years, and conducted over 700 studies. He’ll share insights regarding the pros and cons of an online community versus other research methods, plus best practices for establishing and managing a community to maximize your investment, including:

identifying research topics and study sponsors,

options for analyzing and reporting,

organizing files and processes, and

sharing research beyond study sponsors.

Create Digital Ads Guaranteed to Stop People in Their Tracks (or Scroll)


Rebecca Cunningham, Sylvan Learning

Having just completed a very successful segmentation of parents, Sylvan Learning wanted to dig further into the types of ad copy and digital imagery that would motivate our new personas to stop and click. We used MaxDiff to identify clusters of “ad preference” within each new consumer segment. We learned that some moms pay attention to humor, some want you to tug at their heartstrings, and others prefer a straightforward appeal. We then coupled our new understanding of self-stated ad preference with measured clicks to understand why some ads performed well and others got optimized out.

Refreshment Break THURSDAY | 3:00 PM - 3:30 PM

The Peaks and Pitfalls of MaxDiff at Procter & Gamble


David Hengehold, Procter & Gamble

Megan Peitz, Numerious

Hitch a ride as we explore the maximum difference scaling landscape at P&G. This easy yet powerful research method is just the ticket for product benefit/claim screening and package designs in the CPG industry. See how we use MaxDiff in sensory testing and demo development. When we’ve got to measure a large number of concepts, adaptive methods get us where we need to go. At the end of the journey, segmentation and portfolio optimization let us discover a line-up we can call our own. Along the road, we’ll try to avoid picking up bad panelists, missing out on a good view of the stimuli or letting heavy “anchors” weigh us down. Hop in! At the end, we'll use Sawtooth Software's Discover MaxDiff to see which pitfalls most concern researchers like you!

Lessons Learned from my Years in Sawtooth Software Technical Support


Brian McEwan, Sawtooth Software

CBC and MaxDiff are premier tools for understanding customers and predicting what they’ll do. Sawtooth Software’s tools make it quite easy to implement choice models, but there are some mistakes we see researcher’s committing over and over. Brian will give practical tips for avoiding these troubles based on his experience in Sawtooth Software technical support.


Using MaxDiff to Capture Market Segments of Hawaii Visitors


Delbert M Nebeker, Polynesian Cultural Center

Learn how MaxDiff and Latent Class helped us discover new insights into Hawaii visitors. Hawaii tourists have expectations for their visit and what they want to do. We used a Travel Styles Inventory and MaxDiff from Lighthouse Studio to capture the importance 1,220 visitors placed on 27 statements about their interests and intents. Then with Latent Class we identified 3 market segments based on similarity of interests. Marketing strategies and messaging can target these segments.

Reception THURSDAY | 6:00 PM - 7:00 PM

Friday, September 27, 2019

Conference Registration FRIDAY | 7:00 AM - 12:00 PM

Breakfast FRIDAY | 7:00 AM - 8:25 AM

Using MaxDiff to Serve Users and Protect the Internet

FRIDAY | 8:30 AM - 9:00 AM

Tyler Downer, Mozilla

Follow the journey of MaxDiff methodology at Mozilla, from early experimentation to its current place as a standard tool in the Firefox User Research team. As an organization that protects the agency of the user, Mozilla uses MaxDiff methodology as a critical step in building features and products that serve the user, not external interests. Learn how MaxDiff methodology challenged our assumptions and helped build a better internet.

A Quest for a Deployable Churn Model FRIDAY | 9:00 AM - 9:30 AM

Loretta Chan, Jewelers Mutual

Customer churn/attrition is an age-old problem. Therefore, churn modeling itself is nothing new. With the age and ubiquity of this problem, have we seen all possible solutions out there? With the advancement of AI in the last decade (e.g. Gradient boosting, neural network, etc.), do we have more and/or better tools to tackle the problem? We will explore these questions in the quest of a deployable churn model for our commercial line insurance clients.

Finding the Next Big Thing at Universal Parks & Resorts

FRIDAY | 9:30 AM - 10:00 AM

Jennifer Avery, Universal Parks and Resorts

Aaron Hill, Sawtooth Software

Keeping customers happy is a challenge, especially when you’re on the cutting edge of the ultra-competitive theme park industry. Jennifer and Aaron recently conducted research to find out what the “next big thing” should be at Universal’s theme parks. They combined MaxDiff with an adaptive traditional conjoint analysis exercise to determine which ideas were best, and how each idea might impact the bottom line.

Refreshment Break FRIDAY | 10:00 AM - 10:30 AM

Using Adaptive Choice-based Conjoint Analysis to Unravel the Determinants of Voter Choices

FRIDAY | 10:30 AM - 11:15 AM

David Bakken, Foreseeable Futures Group

Gretchen Helmke, University of Rochester

Mitchell Sanders, Meliora Research

In recent years, political scientists have experimented with aggregate choice-based conjoint analysis (CBC) to make causal inferences about electoral choices. For the same reasons that CBC appeals to marketers, the method offers an efficient way to decompose into their contributing factors the choices that political actors (including voters) make in the political arena. Our study demonstrates the advantages of applying Adaptive Choice-based Conjoint (ACBC) with disaggregate estimation to understanding the drivers of electoral choices.


The Challenge of Identifying Causality in Observational Data

FRIDAY | 11:15 AM - 11:45 AM | IN METHODOLOGY & ANALYTICS ROOM

Ray Poynter, The Future Place & Nottingham Trent University

Observational data can deceive, in terms of cause and effect, leading to the wrong outcomes. In the 1980s, observational data (medical records) seemed to show that Hormone Replacement Therapy helped prevent coronary heart disease, so HRT was prescribed more widely. But a later experiment showed it was not the case. In this session you’ll learn why and how observational data can mislead you and how survey-based research can complement both big and observational data.

Audience Finish Online Voting FRIDAY | 11:45 AM - 11:50 AM

Audience Q&A for Poynter presentation

FRIDAY | 11:45 AM - 11:50 AM

Closing Remarks and Best Paper Award

FRIDAY | 11:55 AM - 12:05 PM


Conference Adjourned FRIDAY | 12:05 PM


Pre-Conference Training Monday, September 23, 2019

CBC Hands-On Workshop (Day 1 of 2) INTRODUCTORY MONDAY & TUESDAY | 8:00 AM - 5:00 PM


Christina Hubner, Sawtooth Software

If you are relatively new to choice-based conjoint (CBC) or just getting started, join us for two days of hands-on practice with Lighthouse Studio, our flagship survey platform that can include the popular CBC component and market simulator. We’ll cover the main aspects of designing, programming, and analyzing CBC studies. You will have an opportunity to program CBC questionnaires individually as well as analyze data from a real CBC study in a team-oriented case study session. We’ll provide coverage of counting analysis, logit, latent class, and HB. The instructors will share best practices, pitfalls to avoid, and experiences based on many years of technical support and consulting.

Attendees receive an evaluation copy of the software that they may use for 90 days (for non-commercial studies and evaluation purposes only). Attendees also receive a free copy of the “Getting Started with Conjoint Analysis” book. Limited to 25 participants.

Becoming an Expert in Conjoint Analysis ADVANCED MONDAY | 8:00 AM - 5:00 PM



(This course intended for those who are highly proficient in conjoint/choice analysis and are looking for the practical advanced training to achieve the next level.)

Join Bryan Orme and Keith Chrzan for a full day of instruction, with topics selected from their book, “Becoming an Expert in Conjoint Analysis”. The instructors will share concepts and practical know-how (e.g. power tricks) that have taken them decades to learn. Attending this course and carefully reading the associated book will accelerate you toward expert status in conjoint/choice methods. This is the book that we wish had magically fallen out of the sky and onto our desks in the early part of our careers!

Topics include: advanced CBC designs, dealing with many attributes and levels, sample size decisions, part-worth utility estimation, willingness to pay, optimization using simulators, and situational choice experiments.

Running a Conjoint Project from Start to Finish INTERMEDIATE MONDAY | 8:00 AM - 12:00 PM

Remco Don, SKIM Group

Jeroen Hardon, SKIM Group

While theory of conjoint analysis is very important and much discussed at the Sawtooth Conference, we want to take the practical approach with this workshop and run you through an entire CBC study. We will go over each of the steps including data re-coding, counts, Logit, Latent Class and HB. During each of these steps we will show tips and tricks to get the most out of your data. How do you model the data, interpret the result and what are the key things to look out for? All of this will be discussed based on a real data set using Lighthouse Software.

Advanced MaxDiff Programming INTERMEDIATE MONDAY | 8:00 AM - 12:00 PM


In this tutorial, Brian will demonstrate how you can move beyond the basics of standard MaxDiff. Using programming exercises and examples in Lighthouse Studio, Brian will:

Demonstrate how to program direct and indirect Anchored MaxDiff

Show how you can reduce the number of items evaluated by a respondent via either Express or Bandit MaxDiff

Take advantage of Free Format capabilities to customize the display of your MaxDiff exercise

Reveal unconventional ways you can employ MaxDiff’s designs or calculations for non-MaxDiff purposes, including leveraging scores-on-the fly and Bandit MaxDiff


Experimental Conjoint Solutions ADVANCED MONDAY | 1:00 PM - 5:00 PM


Kevin Lattery, SKIM Group

Sometimes standards are not enough to answer research questions at hand and one has to deviate from the roads most traveled. The trainers would like to get you on board for this 4-hour tutorial on advanced applications in CBC and MaxDiff. A number of challenging and interesting extensions of choice modeling will be discussed, including:

Custom design techniques combining Lighthouse and Excel and discuss the implications of these customizations

Showing different coding methods for faster processing

Combining multiple MaxDiff studies in one Latent Class estimation

Combining MaxDiff and CBC: different ways of data augmentation

Duct-tape solutions for the red-bus-blue-bus problem

Tuesday, September 24, 2019

CBC Hands-On Workshop (Day 2 of 2) INTRODUCTORY TUESDAY | 8:00 AM - 5:00 PM



If you are relatively new to choice-based conjoint (CBC) or just getting started, join us for two days of hands-on practice with Lighthouse Studio, our flagship survey platform that can include the popular CBC component and market simulator. We’ll cover the main aspects of designing, programming, and analyzing CBC studies. You will have an opportunity to program CBC questionnaires individually as well as analyze data from a real CBC study in a team-oriented case study session. We’ll provide coverage of counting analysis, logit, latent class, and HB. The instructors will share best practices, pitfalls to avoid, and experiences based on many years of technical support and consulting.

Attendees receive an evaluation copy of the software that they may use for 90 days (for non-commercial studies and evaluation purposes only). Attendees also receive a free copy of the “Getting Started with Conjoint Analysis” book. Limited to 25 participants.

Situational Choice Experiments and Tree-Based Models ADVANCED TUESDAY | 8:00 AM - 12:00 PM


In some cases the best way to model choices involves using characteristics of the chooser or of the choice situation as predictors. This is in contrast to choice-based conjoint wherein the characteristics of the choice alternatives are the predictors via a conditional multinomial logit (MNL) model. Situational choice models use an unconditional MNL model, which we will introduce in this session. We supplement the theory of Situational Choice Experiments with practical how-to examples using experimental design and modeling capabilities available in Sawtooth Software products like Lighthouse Studio, CBC/HB, CBC Latent Class and Menu-Based Choice (MBC). Case studies showing the diversity of Situational Choice Experiments will round out the session.

After a brief introduction to the theory and history of decision tree analysis we will cover the primary tree models in use today, including the ensemble analysis called Random Forests. Using publicly available data sets we will illustrate each tree method, supplying the R code needed to produce the trees, example outputs and a software demonstration using readily available (and free) software. Attendees should be able on leaving the session to make their own trees and forests, to diagnose the quality of their models, to classify new cases and to know which methods to use under which circumstances.

Predictive Modeling Using Microsoft Azure ADVANCED TUESDAY | 8:00 AM - 12:00 PM

Joseph Retzer, ACT Market Research Solutions

Ewa Nowakowska, EY

Routine market research predictive modeling seldom requires complex deep learning solutions though it often benefits from developments originating in today’s AI revolution. This tutorial reviews some of the most effective machine learning predictive algorithms currently in use. It also demonstrates their implementation in Microsoft’s cloud-based service “Azure ML Studio”.

The presentation begins with simple, easily interpretable, tree-based methods. Next, models which combine multiple trees (ensembles), for improved stability and accuracy of predictions, are covered. These ensemble methods will include bagging, random forests, boosting and extreme gradient boosting (XGBoost). Fundamental underlying machine learning concepts related to ensemble predictive modeling will be demonstrated. Practical demonstrations will also be presented using tools requiring no prior programming expertise.

Key Take-Aways

Attendees will gain insight into general ensemble modeling using a variety of approaches e.g., random forests, bagging, boosting, XGBoost

XGBoost, a state-of-the-art ensemble predictive model, will be described and compared to other top contenders (e.g. random forest analysis).

2019 Sawtooth Software Conference Page 9 of 24 A demonstration of modern model pipeline building and

cloud platform deployment using Microsoft Azure ML Studio will also be presented.

Advanced LHS Programming ADVANCED TUESDAY | 8:00 AM - 12:00 PM

Justin Luster, Sawtooth Software

Zachary Anderson, Sawtooth Software

Lighthouse Studio is a powerful application that has been designed to be very flexible. Custom code can be added to modify the appearance and functionality of your surveys allowing you to do amazing things. In this tutorial we will learn about how to incorporate the following into your Lighthouse surveys:

HTML

CSS

JavaScript

jQuery

Perl

Learning a little bit about these technologies will greatly enhance your ability to create surveys that your customers will love. This session will introduce you to many options and the instructors will also share code and .ssi project file samples.

Getting to Know Sawtooth Software’s Choice Simulator INTRODUCTORY TUESDAY | 8:00 AM - 12:00 PM


Water Williams, Sawtooth Software

Market simulators (choice simulators) are often considered the most valuable deliverable and output of a conjoint analysis study. Come to this four-hour introductory tutorial to learn how it works and to accelerate your understanding of this valuable analytical tool. We’ll take a balanced approach that first lays the groundwork in terms of understanding why market simulations are so useful, teaches you the math involved, and finally demonstrates how to conduct market simulations using Sawtooth Software’s Choice Simulator (Windows desktop) tool.

Intro to Menu-Based Choice (MBC) INTERMEDIATE TUESDAY | 1:00 PM - 5:00 PM


Menu-Based Choice (MBC) is a relatively new and flexible choice modeling approach for solving a variety of multi-check (combinatorial) menu-selection problems. Examples include: choosing options to put on an automobile, selections from a restaurant menu, banking options, configuring an insurance policy, purchasing bundled vs. a la carte services including mobile phones, internet, and cable. Using the MBC software requires expertise in terms of experimental design for conjoint exercises, developing MBC surveys using advanced HTML/Javascripting, and some data processing to prepare the files in .csv-format for analysis within MBC. MBC has a built-in simulator and can also automatically export Excel simulators.

This tutorial is intended for those with a strong background in discrete choice and econometric modeling. The training will be part conceptual, and part of a demonstration of the MBC software platform.

How to Find and Communicate the Story in the Data INTRODUCTORY TUESDAY | 1:00 PM - 5:00 PM

Ray Poynter, FuturePlace

Most people agree that the key to communicating research insight is to use storytelling. Storytelling helps transform information into insight, and insight into impact. In this four-hour tutorial, Ray Poynter discusses how to set about finding the story in the data, how to craft the story, and how to communicate it in the most effective way.

Conducting Segmentation to Solve Business Problems INTERMEDIATE TUESDAY | 1:00 PM - 5:00 PM

Michael Patterson, Radius Global Market Research

This four-hour training session will be designed to cover all aspects of a segmentation study from typical business questions through to implementation and roll-out. Particular emphasis will be placed on how to conduct a segmentation study using best practices including detailed coverage of different analytic techniques. The training will walk participants through analysis phases via R code that will give them practical experience in working through the various steps involved in a segmentation study. Finally, participants will be given guidance on making the results as actionable as possible by learning about various implementation/dissemination strategies.

Overview of Segmentation, including Unsupervised and Supervised

Criteria for judging a segmentation study (e.g., meaningful, differentiated, measurable, etc.)

Typical steps in a segmentation study

Typical inputs used in segmentation

Analytic Approaches: detailed discussion of various techniques covering what it is, how it works, strengths, shortcomings

Determining the number of clusters

Typing/classification tools (including MaxDiff)

Walk through analysis using simulated data – use different techniques, different scaling/processing approaches, compare results


MaxDiff, Latent Class Segmentation, and TURF Optimization INTRODUCTORY TUESDAY | 1:00 PM - 5:00 PM


MaxDiff (best-worst scaling) has proven to be one of the best survey-based approaches for developing needs-based segmentations. Key reasons include MaxDiff’s ability to capture much more information per unit of respondent effort than other techniques, and its freedom from scale use bias. Cluster analysis may be applied to individual-level MaxDiff scores, but Latent Class is also strong approach, leading to score estimation and market segmentation in a single-step procedure that many have argued is more statistically defensible. Megan will demonstrate how to interpret latent class results and how they can lead to important discoveries for driving marketing strategy. Taking a step further than segmentation analysis, an optimization approach called TURF (Total Unduplicated Reach & Frequency) helps you find the combinations of items that best motivate or reach respondents. TURF may also be used for paring down complex product lines, where there are too many offerings that are not actually optimally efficient for the organization.


Methodology & Analytics Track Wednesday, September 25, 2019

Conference Registration WEDNESDAY | 7:00 AM - 5:00 PM

Breakfast WEDNESDAY | 7:00 AM - 8:25 AM

Welcoming Remarks WEDNESDAY | 8:30 AM - 8:50 AM

Bryan Orme, Sawtooth Software, Conference Moderator

A Comparison of PC and Mobile Interviewing Modalities

WEDNESDAY | 8:50 AM - 9:25 AM

Deb Ploskonka, Cambia Information Group

Karlan Witt, Cambia Information Group

With the continued growth of mobile devices as a means of taking online surveys, we discuss results from tests of two types of questions that cause high drop off rates and/or result in the use of simplification techniques for mobile device users. We present outcomes from varied approaches to both question types, on both mobile and PC, from two cultures, and recommend best practices.

A Researcher's Guide to Studying Large Attribute Sets in Choice-Based Conjoint



Mike Serpetti, Gongos

Dan Yardley, Gongos

While researchers (and their clients) continue to push the envelope on the number of attributes to include in a choice-based conjoint, there is no clear answer on what to do as the number of attributes increases (>6). Design techniques including Partial Profile (PP) and Adaptive Choice-Based conjoint (ACBC) offer solutions, but past research has yet to crown a winner. Join us as we explore and validate the PP and ACBC techniques compared to a Full-Profile Choice-Based Conjoint (CBC) with real respondents testing up to 20 attributes across multiple design scenarios.

Refreshment Break WEDNESDAY | 10:00 AM - 10:30 AM

What South African Medical Students Value in a Rural Internship


Maria Jose, University of Cape Town

Amarech Obse, University of Cape Town

Mark Zuidgeest, University of Cape Town

Olufunke Alaba, University of Cape Town

To address healthcare inequality among rural populations effective health worker recruitment policy is vital. A study was conducted in South Africa that identified the rural health facility attributes valued by final year medical students when choosing their first work placement. Through literature review, focus group discussions and a discrete choice experiment, novel rural health facility attributes were unearthed, and commonly used recruitment incentives effectiveness dispelled. Choice analytics as a tool for job seekers is posited.

Leadership Qualities: Preferences from the Millennial Generation


Ronald Mellado Miller, UVU

Cray Daniel Rawlings, UVU

Christina A. Hubner, Sawtooth Software

Utilizing characteristics gathered from business researchers and evolutionary psychology, business students were surveyed to see which characteristics they would prefer to have in a CEO in a TURF analysis was run. Surprising differences include that those with less work experience desire a friendlier CEO while those with more desire greater honesty and personal characteristics. Additionally, females desired a CEO who could bring them personal fame much less than did males. Other interesting findings were found.


Choice-Based Conjoint Meets Virtual Reality

WEDNESDAY | 11:30 AM - 12:00 PM

Alexandra Chirilov, GfK

Virtual Reality (VR) becomes more and more accessible to a larger audience, thus there is a heightened need to understand if VR may improve the way we conduct marketing research (and conjoint studies in particular) and how we can benefit from producing more realistic experiences. We recently conducted one of the most comprehensive Virtual Reality Enhanced Interview (VREI) studies known with more than 250 VR untrained participants, to evaluate the effects of different study environments (VREI vs. CAPI) on the respondents’ behavior. Virtual Reality offers us plenty of new opportunities, accompanied by some challenges.

Lunch WEDNESDAY | 12:00 PM - 1:30 PM

Too Much Information?: The Curious Case of Augmented MaxDiff


Jackie Guthart, Radius Global Market Research

Curtis Frazier, Radius Global Market Research

Raman Saini, Radius Global Market Research

Augmented MaxDiff has become an effective and increasingly used variant for dealing with large numbers of attributes. While Augmented MaxDiff is efficient and effective at recovering individual level preferences, there is a little talked about issue in the design setup and estimation of utilities. In this presentation, we will illustrate the issue of “too much information” and discuss design decisions to mitigate utilities estimation problems.

Can We Use RLH to Assess Respondent Quality?


Marco Hoogerbrugge, SKIM Group

In CBC studies, the Root Likelihood (RLH) is often a used as a sufficient indicator of the quality of the conjoint data. In the presentation we will discuss why this is not good enough, we will present an improved procedure for assessing the quality of the conjoint data (conceptually) and we will compare different variants of the new procedure.


Bandit MaxDiff: The Effects of Design Parameters on Hit Rates in Diverse Datasets


Nico Peruzzi, Elucidate

Exploring changes to design parameters in Bandit MaxDiff yields practical recommendations regarding study length, degree of cognitive load, and considerations regarding study content. Will Bandit MaxDiff be robust to many changes that could improve respondent experience without reducing hit rate?

Trees, Forests and Situational Choice Experiments



Joseph Retzer, ACT

Situational choice experiments model questions eliciting a multichotomous response to a single profile using unconditional logit; for example, in pharmaceutical research a multi-attribute profile might describe a patient and the choice might be among several fixed therapy alternatives. Using data from 12 situational choice experiments we compare out-of-sample predictive validity, using 10-fold cross-validation, of unconditional logit and several decision tree models, including CART, conditional inference trees and random forests.

Optional Clinic: Product Optimization Search in the Sawtooth Software Market Simulator



Walt Williams, Sawtooth Software

Market (Choice) simulators are practical and powerful tools for making predictions and optimizing products/services to increase share and profit. Spend an hour with us as we show off the sophisticated optimization search capabilities of our capable Sawtooth Software Choice simulator. Topics include share, revenue, and profit search using exhaustive, greedy, and genetic search algorithms. This is the simulator that is integrated within Lighthouse Studio and may also be shared with clients as a Windows-based standalone simulator tool.


Optional Clinic: Powerful and Friendly Online Discover for CBC and MaxDiff




Sawtooth Software's Discover platform is the perfect resource for those who need to create simple choice-based conjoint, MaxDiff, and general interviewing surveys. While not as robust as Lighthouse Studio, it provides an easy-to-use web-based interface that is perfect for streamlined survey creation and analysis. While originally designed for those who are new to discrete choice surveys, this tool is becoming increasing popular with experienced users. Come join us to learn what Discover can do for you!





Examining the No-Choice Option in Conjoint Analysis


Maggie Chwalek, Ohio State University

Roger A. Bailey, Ohio State University

Greg M. Allenby, Ohio State University

In this paper, we examine the effect of providing respondents with information about the prices and features of outside goods on conjoint preference estimates. Using a national sample of respondents in the tooth whitening category, we find that conjoint estimates are surprisingly robust to the information provided about the outside no-choice options.

Modelling Stockpilable Product Purchase Decisions Using Volumetric Choice Experiments


Richard T. Carson, University of California, San Diego

Towhidul Islam, University of Guelph

Jordan J. Louviere, University of South Australia

Many products are easily stored. Changes in price and other attributes can induce consumers to change the quantities purchased of such products. A volumetric choice experiment approach is used to look at the quintessential stockpilable commodity, canned tuna using 120,000 quantity choices (defined by price, brand, size, and other attributes) by IRI consumer panel respondents. Fitting a multilevel mixed-effects negative binomial regression provides estimates for over 100 own (brand by size) and cross price elasticities.


Conjoint Meets AI THURSDAY | 10:30 AM - 11:15 AM

Peter Kurz, bms marketing research + strategy

Stefan Binner, bms marketing research + strategy

Artificial Neuronal Networks (ANNs) can be used to approximate any data generating process due to the universal approximation theorem. When generating experimental designs for complex choice models ANNs are therefore able to solve the trade-off between statistical error and measurement error, which is still unsolved in most design algorithms. The paper shows how to generate experimental designs using ANNs and elaborates on ANNs pros and cons in choice modelling.


Globally Adaptive D-Efficient DCE Designs


Marcel Jonker, Duke Clinical Research Institute, and Erasmus Choice Modelling Centre

In contrast to Sawtooth Software's ACBC, which is an adaptive orthogonal design approach, globally adaptive discrete choice experiment (DCE) designs use Bayesian methods to “learn” about individual respondent’s preferences during the survey administration and create a D-efficient DCE design on-the-fly. The approach is computationally intensive but has nice statistical properties, can incorporate (but does not require) user-specified Bayesian priors, and seems to perform well in both statistical simulations and real-life applications.


Predicting the (Unobserved) Predictable


Tom Gardner, Adelphi Research

Michelle McNamara, Adelphi Research

In market research, we routinely collect wave data where similar questions are every wave. At Adelphi, we present a method where we have used data from previous waves to predict the responses of a test wave. We taught a neural network to use a subset of attributes to predict the responses to the remaining attributes. When tested on the next wave of data, the model was able to accurately predict the responses.

Can I Reduce the Number of Conjoint Tasks and Still Get Good Quality Data?


Chris Moore, Ipsos MORI – UK

Ioannis Tsalamanis, Ipsos MORI – UK

Market research is becoming an increasing agile environment where everything needs to be quicker and more cost effective. At the same time, increasing use of mobile devices to answer surveys has led to the need to reduce questionnaire length to avoid respondents dropping out. However, there has been little work to understand how we can reduce the time needed to conduct a conjoint exercise. This paper will examine whether we can use imputation methods to reduce the number of tasks we need to ask respondents yet still obtain similar results.

Combining CBC and Dynamic Choice Models for More Accurate Forecasting


Faina Shmulyian, SKIM Group, US

In categories like Tech and Innovation, Travel, or Luxury Goods consumers often spend a significant amount of time considering products or offerings before making a purchasing decision. During this time, consumers' preferences can be affected by advertisement, social media, socio-economic processes, or family and friends. We will outline and discuss an approach combining a standard Choice-Based Conjoint and Dynamic Choice Modelling allowing adjusting for the external effects and resulting in more accurate forecasting.


Data Fusion: A Flexible HB Template for Modeling Structures across Multiple Data Sets


Kevin Lattery, SKIM Group

Data Fusion analyzes multiple sets of data in relation to each other. We describe three general analytical approaches to data fusion: multi-stage, data stacking/augmentation, and structured modeling. These approaches can be used independently or jointly across diverse data fusion contexts. These data fusion contexts include common applications like “Anchored MaxDiff”, “Dual Response None”, and ACBC. We describe the data stacking in these common applications and provide an alternative structured model using a flexible hierarchical Bayes template.

Segmenting Choice and Non-Choice Data Simultaneously: Part Deux


Tom Eagle, Eagle Analytics of California

Jay Magidson, Statistical Innovations

A commonly used method to develop segments based on both choice and non-choice data is to cluster HB derived individual-level choice parameters simultaneously with the non-choice variables of interest. However, the resulting segments are affected by the particular rescaling applied to raw HB parameters. We introduce a new Scale-Adjusted Latent Class (SALC) alternative for rescaling HB parameters, and compare simultaneous segmentations of Best-Worst choice and attitudinal non-choice data, based on SALC vs. other rescaling approaches.


Optional Clinic: Bandit MaxDiff: Results and How Easy It Is to Program in Lighthouse Studio



Cameron Halversen, Sawtooth Software

When you are studying 50+ items in a MaxDiff and when the main goal is to identify the top few winning items, a new adaptive flavor of MaxDiff can be 2x to 4x more efficient than standard MaxDiff. That means a savings of 50% to 75% on your data collection costs. Come see how easy it is to set up a Bandit MaxDiff study within Lighthouse Studio.

Optional Clinic: Latest Features in the Discover Platform



Derrick Hyde, Sawtooth Software

If you haven't recently used Sawtooth Software's Discover platform for survey authoring and analysis, you are missing out on some great new features and tools. Every few months, we update this web-based software with cosmetic enhancements as well as behind-the-scenes algorithms that allow you to create better surveys. Come join us to learn more about this resource and its constantly-expanding collection of features.





Understanding Consumer Preferences: A Comparison of Survey and Purchase-Based Approaches

FRIDAY | 8:30 AM - 9:00 AM

Bradley Taylor, GfK

James Pitcher, GfK

Can Point of Sale (POS) data tell us anything about consumer preferences? We apply machine learning algorithms to POS data to obtain utility rankings, attribute importance and price elasticities, and compare them with the results of a CBC exercise across 15 technology and durables product categories within 7 countries; 68 cells in total. Where do the approaches complement each other and where do they show us something different or interesting?

Modeling Crowded Pipelines in Pharmaceutical Markets

FRIDAY | 9:00 AM - 9:30 AM

Doug Willson, ZS Associates

Pharmaceutical manufacturers are often interested in forecasting demand for new products in future markets with large numbers of new competitors, some of which may be quite similar. Conventional modeling approaches using the HB multinomial logit are known to perform poorly in these situations. The paper investigates new tools for dealing with product similarity and product availability in choice models with large numbers of new products.

Maximizing the Impact of OOH Advertisement Using Discrete Choice Modeling & Text Analytics

FRIDAY | 9:30 AM - 10:00 AM

Rajat Goel, StatWorld Research Solutions

Rachin Gupta, StatWorld Research Solutions

Visual memories are the strongest memories! With this in mind, the idea of this paper is to use the technique of Discrete Choice Modeling and Text Analytics to figure out the best structure of an Out-of-home (OOH) advertisement that can have the maximum impact on the viewers. The paper will focus on the importance of OOH advertisement in marketing, how the ad assessment is done typically, the drawbacks in generally used methods and how our approach overcomes these drawbacks to make the OOH ad more impactful and actionable for the marketers.



Using Adaptive Choice-based Conjoint Analysis to Unravel the Determinants of Voter Choices

FRIDAY | 10:30 AM - 11:15 AM

David Bakken, Foreseeable Futures Group

Gretchen Helmke, University of Rochester

Mitchell Sanders, Meliora Research

In recent years, political scientists have experimented with aggregate choice-based conjoint analysis (CBC) to make causal inferences about electoral choices. For the same reasons that CBC appeals to marketers, the method offers an efficient way to decompose into their contributing factors the choices that political actors (including voters) make in the political arena. Our study demonstrates the advantages of applying Adaptive Choice-based Conjoint (ACBC) with disaggregate estimation to understanding the drivers of electoral choices.

The Challenge of Identifying Causality in Observational Data

FRIDAY | 11:15 AM - 11:45 AM

Ray Poynter, The Future Place & Nottingham Trent University

Observational data can deceive, in terms of cause and effect, leading to the wrong outcomes. In the 1980s, observational data (medical records) seemed to show that Hormone Replacement Therapy helped prevent coronary heart disease, so HRT was prescribed more widely. But a later experiment showed it was not the case. In this session you’ll learn why and how observational data can mislead you and how survey-based research can complement both big and observational data.



FRIDAY | 11:45 AM - 11:50 AM


FRIDAY | 11:55 AM - 12:05 PM




Tools & Techniques Track Wednesday, September 25, 2019

A Programming Showcase: Delighting Clients and Respondents with Interactive Surveys


Saurabh Aggarwal, Knowledge Excel Services


Join this interactive showcase on the power of programming surveys in Lighthouse Studio. We’ll share innovative approaches to make Surveys fun for respondents. Examples include:

Interactive MaxDiff with hundreds of items

A Gamified Automotive CBC

Virtual Shelf Pricing studies

Mock-Amazon.com conjoint exercise

Van Westendorp & Gabor–Granger using Chatbots

Using AI to convert responses into emotions

Concept test using Mock-Facebook page

And much more…

Whether you are Researcher or Programmer, relatively new or experienced, this session will have interesting case studies for everyone.


Hack Your Lighthouse Studio Experimental Designs



Sawtooth Software’s Lighthouse Studio provides very good automated experimental design capabilities for MaxDiff and CBC. But we often find situations in practice where we need to customize the designs in special ways. Fortunately, you can “hack” (access and modify) Lighthouse Studio’s experimental designs to implement nearly whatever you have in mind to do. Jeroen will discuss reasons for customizing designs and will demonstrate how to export the designs to .CSV file, strategies for modifying those designs, and how to import them back to Lighthouse Studio.

What Do We Mean by AI in Market Research?


Ray Poynter, FuturePlace

There seem to be two unhelpful trends in terms of Artificial Intelligence and market research. The first is to name a wide variety of things as AI simply to make them sound more modern, relevant or intellectual. The second, contradictory, trend is dismiss anything that is currently possible as ‘not really AI’. During this breakout session we will explore the best way to use terms like machine learning, expert systems, and AI to help people leverage the benefits of these approaches.

Optional Clinic: Product Optimization Search in the Sawtooth Software Market Simulator



Walt Williams, Sawtooth Software

Market (Choice) simulators are practical and powerful tools for making predictions and optimizing products/services to increase share and profit. Spend an hour with us as we show off the sophisticated optimization search capabilities of our capable Sawtooth Software Choice simulator. Topics include share, revenue, and profit search using exhaustive, greedy, and genetic search algorithms. This is the simulator that is integrated within Lighthouse Studio and may also be shared with clients as a Windows-based standalone simulator tool.





Sawtooth Software's Discover platform is the perfect resource for those who need to create simple choice-based conjoint, MaxDiff, and general interviewing surveys. While not as robust as Lighthouse Studio, it provides an easy-to-use web-based interface that is perfect for streamlined survey creation and analysis. While originally designed for those who are new to discrete choice surveys, this tool is becoming increasing popular with experienced users. Come join us to learn what Discover can do for you!






List Building and Quota Control in Lighthouse Studio Surveys



List building is a valuable tool for any Sawtooth Software user's toolkit. Join our breakout session for instruction, or a review, on how to build lists for piping in answers from previous questions or creating your own creative concoctions. We will also explore how to build and use quotas in your surveys to obtain answers from a certain quantity of qualified respondents.

Why, When, and How to Use Conjoint Analysis in Your Research



Join us as we discuss best practices for the entire conjoint analysis project life-cycle — from proposal to deliverable — including key questions to ask your clients, pitfalls to avoid, and what to look for in the results. We’ll even cover strategies for reporting that enhance presentations and delight your clients. For both newbies and pros alike!


Customizing your Lighthouse Studio Surveys Using CSS and Javascript



You can become a much more powerful Lighthouse Studio user if you understand Cascading Style Sheets (CSS) and JavaScript. CSS allows you to change the appearance of whatever is displayed on the device. JavaScript allows you to modify the functionality or behavior your survey. Come learn a bit of these programming languages and instantly create more dynamic and powerful surveys!


Using Conditional Display and Conditional Pricing in CBC Studies



Conditional display lets you display a graphic (or text or video clips) that is based on a combination of levels presented in a product concept. Conditional pricing allows you to change the prices shown for product concepts based on another attribute, such as brand. This session will explore how to use these powerful features to make your discrete choice exercises easier to complete and more meaningful.

Enhancing Your Surveys Using the Question Library


Nathan Bryce, Sawtooth Software

Zachary Anderson, Sawtooth Software

Do you have a bank of questions that you use repeatedly in your surveys? Or have you created a customized question with HTML, JavaScript, or CSS and you want to reuse it in other surveys? Or do you wish you had access to a library of customized questions that others have written, such as star ranking, highlighting, autocomplete, calendar widgets, image pop-ups, or recording the latitude and longitude of a respondent? Attend this session to learn how to use the time-saving Question Library feature of Lighthouse Studio and the corresponding Community Question Library on the Sawtooth Software website.

Linking Sawtooth Software with Other Sample and Survey Providers



At Sawtooth Software, we like to make it known that we work well with others. That's why we have made it possible to link our surveys with other sample and survey providers. Come learn the specific steps to making that seamless transition.



Tips and Tricks for Programming Excel Market Simulators


Kenneth Fairchild, Sawtooth Software

Excel conjoint simulators are easy to share, client friendly, and customizable delivery vehicles for conjoint simulators. In this breakout we will cover essential simulator calculations, customized calculations such as weighting and segmentation, style considerations, and other customizations. We will also look at previous work done with advanced simulators and talk about the challenges to implement them.

“Help Me, Sawtooth Software Forum, You’re My Only Hope!”


Paul Moon, Paul Moon Consulting

The “Sawtooth Software Forum” is an online gathering of users and Sawtooth Software employees to provide questions and answers, often after Sawtooth Software offices are closed for the day. Paul Moon, an active Forum contributor, reviews the positives of being an active contributor, and the most common questions and answers seen on the Forum. The focus of this breakout session is directed on general survey programming questions using Lighthouse Studio.

General Session Ends

Optional Clinic: Bandit MaxDiff: Results and How Easy It Is to Program in Lighthouse Studio



Cameron Halversen, Sawtooth Software

When you are studying 50+ items in a MaxDiff and when the main goal is to identify the top few winning items, a new adaptive flavor of MaxDiff can be 2x to 4x more efficient than standard MaxDiff. That means a savings of 50% to 75% on your data collection costs. Come see how easy it is to set up a Bandit MaxDiff study within Lighthouse Studio.





If you haven't recently used Sawtooth Software's Discover platform for survey authoring and analysis, you are missing out on some great new features and tools. Every few months, we update this web-based software with cosmetic enhancements as well as behind-the-scenes algorithms that allow you to create better surveys. Come join us to learn more about this resource and its constantly-expanding collection of features.





How to Develop Product and Positioning Concepts to Identify Winners in Testing

FRIDAY | 8:30 AM - 9:00 AM

Michaela Mora, Relevant Insights, LLC

Creating effective product and positioning descriptions for testing, either through conjoint analysis or monadic testing, is critical for the success of a test. Often clients know so much about their products that it becomes difficult to describe them in simple and relevant terms. In this breakout session, we will discuss the structure of product and positioning concepts, their difference, how to write key elements, and research approaches that can help to develop and evaluate them.


Practical Application of Conjoint WTP (Willingness to Pay)

FRIDAY | 9:00 AM - 9:30 AM

Ula Jones, Kantar

Irene Torres, Kantar

For hardware manufacturers, it’s critical to contrast the costs of components with consumers’ willingness to pay (WTP). We plan on reviewing a case where we collaborated with clients to better inform their retail pricing. To accomplish this goal, we use calibrated WTP to ensure that their retail pricing is better aligned with the cost of components.

Sensitivity Analysis and Optimization Search in Sawtooth Software’s Market Simulator

FRIDAY | 9:30 AM - 10:00 AM

Walter Williams, Sawtooth Software

The Sawtooth Software Choice Simulator, integrated within Lighthouse studio and also available as a standalone desktop tool, provides sophisticated sensitivity and optimization search capabilities. Walt steps you through the reasons for using sensitivity and optimization within choice simulators, then demonstrates how to do them using the tool. You’ll be impressed by the simplicity and power of the Sawtooth Software Choice Simulator.


Which Conjoint Method (or MaxDiff) Should I Use?

FRIDAY | 10:30 AM - 11:15 AM


This session will introduce the many conjoint and discrete choice analysis options offered by Sawtooth Software and help you determine when it is appropriate to use each method. Example case studies will illustrate various outcomes achieved with different conjoint and MaxDiff approaches.

Avoiding Common Pitfalls in Conjoint Analysis

FRIDAY | 11:15 AM - 11:45 AM


Come take advantage of decades of experiencing working with Sawtooth Software customers from our technical support team to learn about common pitfalls and how to avoid them. This class is geared towards beginners and those with a few studies under their belts. We will cover topics from attributes and levels, experimental designs, to fielding your survey and running analysis.



FRIDAY | 11:45 AM - 11:50 AM


FRIDAY | 11:55 AM - 12:05 PM




Sawtooth Software Conference At-a-Glance Pre-Conference Sessions Monday, September 23, 2019

Start Time Room 1 Room 2 Room 3 Room 4

7:00 AM Breakfast (for those attending AM tutorial sessions this day)

8:00 AM CBC Hands-On Workshop* (1 of 4)

Becoming an Expert in Conjoint Analysis*** (1 of 2)

Running a Conjoint Project from Start to Finish**

Advanced MaxDiff Programming**

12:00 PM Lunch (for those attending either AM or PM tutorial sessions this day)

1:00 PM CBC Hands-On Workshop* (2 of 4)

Becoming an Expert in Conjoint Analysis*** (2 of 2)

Experimental Conjoint Solutions***

5:00 PM Adjourned

*Introductory **Intermediate ***Advanced

Tuesday, September 24, 2019

Start Time Room 1 Room 2 Room 3 Room 4 Room 5

7:00 AM Breakfast (for those attending AM tutorial sessions this day)

8:00 AM CBC Hands-On Workshop* (3 of 4)

Situational Choice Experiments and Tree-Based Models***

Predictive Modeling Using Microsoft Azure***

Advanced LHS Programming***

Getting to Know Sawtooth Software’s Choice Simulator*

12:00 PM Lunch (for those attending either AM or PM tutorial sessions this day)

1:00 PM CBC Hands-On Workshop* (4 of 4)

Intro to Menu-Based Choice**

How to Find and Communicate the Story in the Data*

Conducting Segmentation to Solve Business Problems**

MaxDiff, Latent Class Segmentation, and TURF Optimization*

6:00 PM Dinner Reception (for attendees registered for the Wed-Fri conference sessions)

*Introductory **Intermediate ***Advanced

https://www.sawtoothsoftware.com/2104#p2



































Sawtooth Software Conference At-a-Glance General Sessions Wednesday, September 25, 2019

Start Time

Methodology & Analytics Track Ballroom 1

Corporate Research Track Ballroom 2

Tools & Techniques Track Room 3

7:00 AM Breakfast

8:30 AM Welcome Address

8:50 AM A Comparison of PC and Mobile Interviewing Modalities

9:25 AM A Researcher's Guide to Studying Large Attribute Sets in Choice-Based Conjoint

10:00 AM Refreshment Break

10:30 AM What South African Medical Students Value in a Rural Internship

11:00 AM Leadership Qualities: Preferences from the Millennial Generation

11:30 AM Choice-Based Conjoint Meets Virtual Reality

12:00 PM Lunch

1:30 PM Too Much Information?: The Curious Case of Augmented MaxDiff

New Research Tools for Product Optimization and Idea Screening (1:30–1:45) How Zappos Uses Conjoint Analysis for Superior Insights and Customer Segmentation (1:45–2:15)

A Programming Showcase: Delighting Clients and Respondents with Interactive Surveys (1 of 2)

2:15 PM Can We Use RLH to Assess Respondent Quality?

Conjoint Analysis throughout the Product Development Cycle

A Programming Showcase: Delighting Clients and Respondents with Interactive Surveys (2 of 2)

3:00 PM Refreshment Break

3:30 PM Bandit MaxDiff: The Effects of Design Parameters on Hit Rates in Diverse Datasets

Connecting People with Public Lands Using MaxDiff Insights

Hack Your Lighthouse Studio Experimental Designs

4:15 PM Trees, Forests and Situational Choice Experiments

The Implicit and Explicit Effectiveness of Advertising on Twitter

What Do We Mean by AI in Market Research?

5:15 PM Optional Clinic: Product Optimization Search in the Sawtooth Software Market Simulator


6:00 PM Reception

https://www.sawtoothsoftware.com/2104#g1





https://www.sawtoothsoftware.com/2104#r1






https://www.sawtoothsoftware.com/2104#b1






















https://www.sawtoothsoftware.com/2104#c1






Sawtooth Software Conference At-a-Glance General Sessions Thursday, September 26, 2019

Start Time




7:00 AM Breakfast

8:30 AM Examining the No-Choice Option in Conjoint Analysis

Keynote: The Joys and Sorrows of the Corporate Researcher

List Building and Quota Control in Lighthouse Studio Surveys

9:15 AM Modelling Stockpilable Product Purchase Decisions Using Volumetric Choice Experiments

How to Quickly Capture the User's Voice to Inform Product Strategy

Why, When, and How to Use Conjoint Analysis in Your Research


10:30 AM Conjoint Meets AI Fleeting Frustration or Fatal Flaw? Using Choice Models to Predict Customer Behavior and Prioritize Problems (10:30–11:00) Best Practices for Market Segmentation (11:00–11:15)

Customizing your Lighthouse Studio Surveys Using CSS and JavaScript (1 of 2)

11:15 AM Globally Adaptive Efficient D-DCE Designs Using ACBC to Design Online Promotional Messaging for Lifetime Sheds

Customizing your Lighthouse Studio Surveys Using CSS and JavaScript (2 of 2)

12:00 PM Lunch

1:30 PM Predicting the (Unobserved) Predictable Credit Union Analytics Use Cases Using Conditional Display and Conditional Pricing in CBC Studies

2:00 PM Can I Reduce the Number of Conjoint Tasks and Still Get Good Quality Data?

Managing Your Online Research Community for High ROI

Enhancing Your Surveys Using the Question Library

2:30 PM Combining CBC and Dynamic Choice Models for More Accurate Forecasting

Create Digital Ads Guaranteed to Stop People in Their Tracks (or Scroll)

Linking Sawtooth Software with Other Sample and Survey Providers

3:00 PM Refreshment Break

3:30 PM Data Fusion: A Flexible HB Template for Modeling Structures across Multiple Data Sets

The Peaks and Pitfalls of MaxDiff at Procter & Gamble

Tips and Tricks for Programming Excel Market Simulators

4:15 PM Segmenting Choice and Non-Choice Data Simultaneously: Part Deux

Lessons Learned from my Years in Sawtooth Software Technical Support (4:15–4:30) Using MaxDiff to Capture Market Segments of Hawaii Visitors (4:30–5:00)

“Help Me, Sawtooth Software Forum, You’re My Only Hope!”

5:15 PM Optional Clinic: Bandit MaxDiff: Results and How Easy It Is to Program in Lighthouse Studio


6:00 PM Reception



































































Sawtooth Software Conference At-a-Glance General Sessions Friday, September 27, 2019

Start Time




7:00 AM Breakfast

8:30 AM Understanding Consumer Preferences: A Comparison of Survey and Purchase-Based Approaches

Using MaxDiff to Serve Users and Protect the Internet

How to Develop Product and Positioning Concepts to Identify Winners in Testing

9:00 AM Modeling Crowded Pipelines in Pharmaceutical Markets

A Quest for a Deployable Churn Model Practical Application of Conjoint WTP (Willingness to Pay)

9:30 AM Maximizing the Impact of OOH Advertisement Using Discrete Choice Modeling & Text Analytics

Finding the Next Big Thing at Universal Parks & Resorts

Sensitivity Analysis and Optimization Search in Sawtooth Software’s Market Simulator


10:30 AM Using Adaptive Choice-based Conjoint Analysis to Unravel the Determinants of Voter Choices

(Attend either Methodological or Breakout Track—We’ve selected presentations to appeal to everyone)

Which Conjoint Method (or MaxDiff) Should I Use?

11:15 AM The Challenge of Identifying Causality in Observational Data

(Attend either Methodological or Breakout Track—We’ve selected presentations to appeal to everyone)

Avoiding Common Pitfalls in Conjoint Analysis

11:50 PM Closing Remarks

12:00 PM Conference Adjourned





























Documents

at the Sawtooth Software Conference · at the Sawtooth Software Conference The Sawtooth Software conferences are renowned for their practical, practitioner-oriented focus and depth