Embed Size (px)
Citation preview
The impact of choice environment on choice behavior
Studio GerARTGerard Loosschilder
ONCE UPON A TIME …
… there was a Princess by the name of Astrid. Astrid was preparing for a lifetime of governing as Queen Astrid, …
… but for now, she liked to spend her time traveling and seeing the world.
Also, she was highly engaged in good causes and animal welfare.
ONE DAY ….
Animal Welfare Conference
Princess Astrid received a letter from the Animal Welfare Association. They invited her as a keynote speaker to their Global conference…
… which would take place in the beautiful city of Istanbul, Turkey.
Astrid happily accepted the invitation. She got to work on it straight away. She booked her own travel as a part of the adventure. She consulted a booking site to find a hotel.
Call to action
Hotel Chain, Type & Style
Distance to city center
Placement of 50 hotel on the results page - top to bottom Review Score on cleanliness, staff and
facilities; result in a mean score and a label
Including room price per night
The booking site contained several helpful features to help her search a hotel room. 50 rooms were listed top to bottom and Astrid had to scroll to see the lower entries. Also, the brand of the hotel chain, the type and style of rooms available were listed along with their distance to the
city center, review scores, room price, and a call to action that suggested that Astrid had to act fast.
Astrid was distressed. As a princess, she got easily stuck with all of these options….
For example, usually her personal assistant would take care of all of her travel. Astrid did not know about things like hotel brands, room styles and types, …
Also, Astrid did not understand the meaning of the distance to a city center, because her security detail would take care of travel on site, …
Her domestic staff did her cleaning so Astrid did not understand room reviews and ratings on such concepts as “cleanliness, staff and facilities” …
Astrid was seriously challenged to book her flight and hotel and she ended up what many visitors do: click on one of the first hotels that appeared in her booking app.
When the day had come, Astrid went on the flight to Istanbul …
And she ended up in a terrible hotel, which made her cry her heart out.
MEANWHILE @ THE BOOKING SITE
We meet Oscar, revenue manager at the booking site that Astrid used …
Astrid’s bad room experience is a side effect of the way the site, and Oscar, work.
Oscar knows that consumers tend to gravitate to the top for their choices and leave the rest alone ….
… so Oscar adapted the fee structure to charge the highest commission fees for the top entries.
By playing the game of placing hotels strategically in top positions and charging fat commission fees for it, Oscar produced record-breaking revenues.
It went sometimes at the expense of visitors like Astrid, who pay the price if margin is spent on booking fees instead of room quality.
CHARLOTTE, INTERACTION DESIGNERIntroducing
Charlotte recently started at the booking site as an interaction designer
Charlotte likes working at the booking site because she likes traveling herself. She wants to do well to her fellow travelers, and give them access to great rooms and deals.
When Charlotte started working on the interaction design of the booking site, she was surprised at the rudimentary design. She wondered where the support functions were.
Filter functions on price and ratings
Sort functions on price and rating
She decided to give the site some extra functionality to reduce the cognitive burden on the consumer: sort and filter functions. The goal was to drive choices to entries towards
the bottom of the page and thereby achieve a better distribution of choices.
Charlotte explained her work to Oscar who listed attentively first, but once he understood what the sort and filter functionalities could do…
… he got upset because it would disrupt his revenue model.
They got into a fight …
… but then they decided to be professional about it and they agreed on a multivariate A/B test to see the effects of the changes.
BACK TO ASTRIDOne year later
Meanwhile, Astrid had gotten the hang of traveling. She was now a member of a ring of celebrities of NGOs and good causes.
She was invited back to Istanbul for the next installment of the animal welfare conference.
She got to work on it straight away. Astrid went back the booking site, but now armed with more knowledge and skill.
She was surprised to learn that the booking site now contained functions that she used to set bottom limits for prices and ratings. After all, she was still a spoilt princess.
Because she really wanted to explore the city, she now also explored other amenities such as the distance to the city center.
Along with the brand, type and style of hotel.
So that by the time she went to Istanbul …
She knew that she would end up in a nice hotel, and she was satisfied with her choices and the way she had gotten there.
THE RESULTS ARE IN!Meanwhile, back in the office …
Of those having the functions
available
67% uses sort and/or filter
functions at least once across the four tasks
47%uses the filter
function at least once
42%uses filter on price
27%uses filter on rating
40%uses the sort
function at least once
34%uses Sort on price
11%uses Sort on rating
33%does not use the
functions, not even once
To see how the sort and filter functions were used, Charlotte created a plot. She learned that 67% of her visitor used the sort or filter functions; 33% did not. It was disappointing. 47% uses a filter function at least once; 40% used the sort function, so filtering is more popular than sorting. 42% filtered on price versus 27% on rating, so it
is more popular to filter or sort on price than on rating. Charlotte did not know if these numbers are high or low; at least she had some benchmark numbers and time will tell.
5047444138353229262320171411852
-2% 0% 2% 4% 6% 8% 10% 12%
Ideal situation
Likelihood of choosing a room
Posi
tion
on th
e re
sults
pag
e
Ideally, the choices of rooms would be uniformly distributed across the search result page; at 50 entries, each entry would have an equal 2% chance of being chosen.
5047444138353229262320171411852
-2% 0% 2% 4% 6% 8% 10% 12%
Situation before redesign
Likelihood of choosing a room
Posi
tion
on th
e re
sults
pag
e
Yet, before the change and the addition of the sort and filter functions, there were many people like Astrid and choices on the booking site are skewed towards the top, and surprisingly, to the bottom. At the bottom there is
also an option to choose none of the hotel rooms. The visitor had to scroll all the way down to find it.
5047444138353229262320171411852
-2% 0% 2% 4% 6% 8% 10% 12%
Sort & filter made available
Likelihood of choosing a room
Posi
tion
on th
e re
sult
page
To her positive surprise, Charlotte observed that after she had added the sort and filter function, the distribution of choices was flatter than when they were not available.
5047444138353229262320171411852
0% 2% 4% 6% 8% 10% 12%
If sort & filter are used
Likelihood of choosing a room
Posi
tion
on th
e re
sults
pag
e
The effect was strongest among those who actually used the sort and filter function: the ideal flat distribution was approximated!
5047444138353229262320171411852
0% 2% 4% 6% 8% 10% 12%
If sort & filter are not used
Likelihood of choosing a room
Posi
tion
on th
e re
sults
pag
e
The effect was offset by those who did not use the sort and filter functions, and whose choices skewed even more strongly towards the top entries.
5047444138353229262320171411852
0% 2% 4% 6% 8% 10% 12%
All compared
Likelihood of choosing the room
Posi
tion
on th
e re
sult
page
In all, Charlotte had gotten the effect that she was looking for: the distribution of choices was flatter after the redesign than before it,
and the result of those using the functions, were encouraging enough to stay on this path.
Top box satisfaction
if sort and filter are ...
Not available:
56%
Available:
62%
Also, Charlotte was happy to learn that visitors were happier with the task after the redesign than before it. After the redesign, there was 62% top-box satisfaction, versus 56% before the redesign.
OSCAR STRIKES BACKBut then,
Average room price
if filter is...
Not available€123
Available€116
Not used€126
Used€102
He noticed that the average price of the room booked was down from 123 Euros to 116 Euros after the redesign and the introduction of the sort and filter functions.
This was due to those using the functions at an average room price of 102 Euros.
Use of the none option, if filter
was
Not available:12%
Not used: 12%
Used: 17%
Also, the site had started to suffer more from what we may call the empty basket syndrome: visitors leaving the site or the choice task without making a choice, or here, using the none option. It was up from 12% when the filter function was not available or used, to 17% if the filter was used.
WHO DID WIN?So what?
It is in Astrid’s favor that room choices were much better distributed across the results page if the functions were added, which could drive up the real estate value of the page. Also, satisfaction was up which may mean that visitors are more likely to come back to the
site. That’s two points for Astrid. However, the average room price booked is going down and the number of people not chosing a room, goes up. Both result in a significant loss of revenue. That’s two points for Oscar.
Flatter distribution of choices Higher task satisfaction
Lower room prices Higher drop-out rates
And because we don’t know if Astrid and other people are more likely to come back to the booking site, we discount the last point for Charlotte. Based on this test, we declare Oscar the winner,
much to the dismay of Charlotte and Astrid.
Flatter distribution of choices Higher task satisfaction
Lower room prices Higher drop-out rates
WHAT’S IN IT FOR YOU?Dear Colleagues
By bringing it to you as a story, I may have been more engaging, you may have understood the results of the study better, you may remember it better and you may be
more likely to act upon it. But that’s only a part of the deal here.
Dude it’s just a story
Stories are usually about change. It is embedded in storytelling formats: once upon a time; every day, one day, and then…, until ... The change has happened and a new
situation kicks in. Stories are a great way to engage people and to inspire the change.
CHANGEAHEAD
Change involves stakeholders, and not all stakeholders may take the change lightly because of different vested interests. Research-based stories are a good way to explore and share the implications for the stakeholders and
support their informed decision making, taking everyone’s interests into account.
Stakeholderthe market
Stakeholder, protagonist
Stakeholderantagonist
Also, purchase environment matters a lot and can change the results of a study. In our case, it was the addition of two user interface functions, sort and filter. It is a challenge to implicit assume of conjoint analysis that the
purchase environment does not have an impact. It shows that we should conduct our choice exercises in a virtual environment representative of the future environment to improve external validity.
This is what we do in our initiatives in which we play and experiment with e-commerce environments. We believe it is the future of conjoint to replicate consumer behavior in these kinds of environments.
Experiment with usGerard Loosschilder, Paolo Cordella,
Jean-Pierre van der Rest and Zvi Schwartz
