Satisficing & Social Desirability Bias

Justin WheelerVP, Product Innovation

Are In-Context Respondents More “Honest”?

Prior Research Demonstrates…

1

• Some exercises more accurate on mobile (e.g. ranking)

• Mobile response quality at least as good as desktop/laptop

• Mobile data quality superior to face-to-face interviews (brand tracking)

• Mobile respondents not influenced by interviewer

• Mobile In-Context respondents exhibit better recall of ads seen during shopping

• Mobile In-Context respondents more confident in providing accurate answers about shopping trip

The Study

2

SatisficingSocial

Desirability Bias

Our Partner

3

Paul J. Lavrakas, PhD

Former Chief Methodologist for Nielsen

Former Professor: Ohio State UniversityNorthwestern University

Voluntary research partner for uSamp

Hypothesis & Definitions

4

Mobile In-Context respondents are less likely to exhibit Satisficing & Social Desirability Bias in survey responses, compared to online, Out-of-Context respondents

SatisficingA cognitive heuristic whereby respondents perform the least amount of mental work to find a reasonable, “good enough” answer.

Social Desirability BiasThe tendency of respondents to provide the answer they believe the Interviewer wants to hear or that will make the respondent “look better”.

The Study: A Dining Experience

5

Which survey setting is more reliable?

6

This In-contextvs.

Validation they were there to dine

7

Yes! Um….no!

We were right

8

Online respondents did exhibit satisficing

andsocial desirability

behaviors not exhibited by Mobile-In-Context

respondents

Satisficing behaviors among online respondents

9

4.91

7.21

0

1

2

3

4

5

6

7

8

Total Visits

Median # Total Visits to Restaurant

Mobile Online

Social desirability behaviors among online respondents

10

Mobile Online

Median number of minutes looking over the menu

2.76 3.62

Mean number of others who dined with me

2.24 2.44

Hindsight degraded satisfaction metrics, which is incongruous with eating there more often with more people

11

55

56

63

63

45

44

46

46

51

51

32

34

0 10 20 30 40 50 60 70

Menu Variety

Types of Choices

Quality of Food

Quantity of Food

Noise Level

Value for Money

% Very Satisfied

Online Mobile

Mobile Online

Very/somewhat likely to return to restaurant

89% 82%

Conclusion

12

In-Context = less satisficingbetter access to memories

Hindsight can alter perceptions

Less social desirability bias

Postscript

13

Does this mean Online respondents are lying?

Online respondents are as honest as they can be

[Mobile] In-Context respondents have better access to the truth:

feelings, memories & experiences

Thank You