Upload
merlien-institute
View
1.461
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Mobile surveys - The 5th methodology: a new paradigm or a strategic enhancement?Michael Francesco Alioto - Vice President, Marketing Sciences - Gongos ResearchComparing smartphone-based and online survey data: assessing research-on-research done with Best Buy. Developing effective best practices for designing mobile-based quantitative research. Understanding when to use a mobile/online hybrid approach to surveys.
Citation preview
Mobile Surveys | The 5 th Methodology: New Paradigm or Strategic Enhancement ?
M a r k e t R e s e a r c h i n t h e M o b i l e W o r l d 1 9 - 2 0 J u l y 2 0 1 1 : A t l a n t a
M i c h a e l F r a n c e s c o A l i o t o V i c e P r e s i d e n t , M a r k e t i n g S c i e n c e s
Surveying consumers in
their natural environment
What Role Will Mobile Play in the Future ?
Critical Study Objectives:
• Understand/confirm current mobile survey usability—
integrative hybrid or next survey platform?
• Confirm consumers’ usage of mobile devices in their
daily lives—communication and environment
• Confirm if surveys can be completed within an integrated
platform with similar results
• Compare and contrast smartphone and online surveys—
scale usage, results, incidence rates, survey length, etc.
• Assess potential to analyze data utilizing more advanced
analytical techniques (e.g., MaxDiff) on mobile devices
Path to understanding
research design
A Four-Phase Process
Approaches Toward Analysis:
Usage & Lifestyle
Flash Polls &
Dialogues
n=496
Desk Research
Review literature, conventional thoughts, and
secondary data; establish baseline
Survey Design
Comparison
Smartphone survey vs. online version
n=1,006
(495 smartphone/
511 online)
Survey Results
Comparison
Confirm data comparability and
scope across platforms
Advanced Statistical Analysis
Confirm
comparability of more complex quantitative techniques
PHASE ONE: PHASE TWO: PHASE THREE: PHASE FOUR:
In partnership with
Macro Trends • Two schools of thought: strategic enhancement—
online supplement vs. platform evolution—
5th Methodology
• Tendency toward hybrid platforms and data
integration, but results and strategy not conclusive
• Suggests major limitations for mobile survey
component of hybrid platform—device/technology,
research design elements
• Academic community mostly silent on the issue
• 5th Methodology as mainstream future platform—
will be driven by behavior and mobile device
technological improvements
Current understanding
and benchmarking
Desk Research: Key Findings
Micro Trends • Data comparability between online and mobile
survey platforms debatable— supplement vs.
independent platform
• Critical limitations concerning mobile research
identified: e.g. survey length, exhibits, types of
questions, scales, incentives
• Mobile segments illustrate personal differences:
Millennials, Multicultural
• Online sample providers just beginning to
understand/get comfortable with mobile
surveys (incidences, costing, viability)
PHASE ONE:
The mobile device
l ifestyle integration
Usage & Lifestyle: Key Flashpoll Findings
Smartphone Usage/Integration:
• Placing phone calls, texting, sending e-mails, and
Internet searches are in the top five usage
categories—ideal for communication
• Supports desk research concerning smartphones
as “connectivity to the world”
• Smartphone usage is well integrated into daily
lifestyle (home, work, vehicle)
• Smartphones used extensively while shopping
• Smartphones used to conduct a wide variety of
tasks, many times simultaneously
• Smartphones used throughout the day/within
periods of “constant interruption”
Smartphone Lifestyle Usage
PHASE TWO:
19%
24%
35%
61%
62%
70%
94%
Other
At school
Internet café/Coffee house
While shopping
In the car
At work
At home
Survey subjects:
Internet Sites Visited, Grids
Store Display, Image Testing
Mobile Preferences, MaxDiff
Concept Evaluation
Demographics
Online Survey Assessment
Mobile Survey Assessment
Best Buy Segmentation
1 5
Agree Completely
Disagree Completely
Scales types:
Multiple Mention
Single Response
Three-point, fully-anchored
Five-point, fully-anchored
Five-point, end-anchored
Five-point, semantic differential
Six-point, fully-anchored
PHASE THREE: Sett ing the stage
quantitative survey
Questionnaire and Design: Key Elements
Design elements:
Response Rates
Incentive Structure
Survey Timing
Open-Ended Questions
Respondent Assessment (Worldview)
Technology Limitations
Key Comparisons
quantitative survey
Design Elements: Key Findings
PHASE THREE:
Research Results
No difference between
platforms
Higher incentives not required
for smartphones
Not as restricted as thought
Minimal verbatims possible
Different, but thought to be
converging
Limited by device
Traditional Thoughts
Significantly lower for
smartphones
Higher incentives required
for smartphones
Restricted for smartphones
Difficult for smartphones
Much different for
smartphone users
Limited significantly
RESPONSE RATES
INCENTIVES
TIMING
VERBATIMS
WORLDVIEW
TECHNOLOGY
Image Affinity: 3-Point Scale Respondents were asked how much they liked a
store display using a 3-point scale, where:
1 = Dislike it
2 = Feel neutral about it
3 = Like it
9% 11% 10%
66% 49% 49%
25% 40% 41%
Online: NoSmartphone
(A)
Online: HaveSmartphone
(B)
Smartphone(C )
Like it
Feel neutral about it
Dislike it
B,C
A A
Like vs. Dislike
Comparison of Results
quantitative survey
Questionnaire Design: Key Findings
PHASE THREE:
Significance testing at the 95% confidence level
n=330 n=174 n=487
“Using the scale below, overall, how much do you like this store display?”
Concept Test: 5-Point Fully Anchored
“Which of the following statements best describes how much you like or dislike this product/idea?”
Respondents were asked to rate the
concept on a 5-point anchored scale
Comparison of Results
quantitative survey
Questionnaire Design: Key Findings
PHASE THREE:
8% 3% 2%
7% 2% 7%
47%
31% 24%
28%
40% 43%
11% 24% 24%
Online: NoSmartphone (A)
Online: HaveSmartphone (B)
Smartphone (C )
5 Strongly like it
4 Somewhat like it
3 Neither like it nor dislike it
2 Somewhat dislike it
1 Strongly dislike it
B,C B
B,C
B,C
B,C B,C
B,C A
B,C
A A
C
A
C
C
A
A
A,B
A
B,C
T2B = 64%
A A
T2B = 39%
T2B = 67%
B
A
Significance testing at the 95% confidence level
n=335 n=176 n=495
Display Appearance: 5-Point End Anchored
8% 8% 7%
18% 14% 16%
31% 24% 21%
27% 33% 33%
15% 20% 23%
Online: NoSmartphone
(A)
Online: HaveSmartphone
(B)
Smartphone(C)
Agree Completely
4
3
2
Disagree Completely
C
“The table is inviting and I would want to look at the products displayed.”
Respondents were asked to rate the
display table on a 5-point scale, with:
1 = Disagree Completely
5 = Agree Completely
Comparison of Results
quantitative survey
Questionnaire Design: Key Findings
PHASE THREE:
Significance testing at the 95% confidence level n=300 n=174 n=487
Clearly View Image:
Semantic Differential Mean Summary
Respondents were asked to rate the image on
four 5-point differential semantic scales.
Online: No Smartphone Online: Have Smartphone
Smartphone
Sample Size n=312 n=174 n=487
(1) Organized vs. (5) Messy 2.26 2.18 2.30
(1) Appealing vs.(5) Unappealing 2.65* 2.40 2.53
(1) Stands out vs. (5) Blends in 2.72 2.57 2.68
(1) Clear vs. (5) Confusing 2.31 2.28 2.33
* Significantly Higher than 1 other group
Comparison of Results
quantitative survey
Questionnaire Design: Key Findings
PHASE THREE:
Significance testing at the 95% confidence level
“Below is a list of attributes with differing meanings. For each pair, please select the one attribute that best describes the store display.”
8% 12% 13%
33% 26% 21%
19% 17%
15%
11% 10%
8%
25% 30%
36%
4% 6% 8%
Online, NoSmartphone (A)
Online, HaveSmartphone (B)
Smartphone (C)
Memory
Features
Size
Battery
Price
BrandB,C
A
B,C
B,C
B,C B,C
A
B,C B,C
A,B C
A
C
C
A
A
A
A,B
Respondents completed a MaxDiff exercise where they ranked various mobile phone features, including: brand, price, battery life, size, features (e.g. camera, video) and memory.
Quantitat ive comparisons
statistical analysis
Advanced Analytics: Key Findings
PHASE FOUR:
MaxDiff Analysis: phone
feature rankings
Significance testing at the 95% confidence level
n=335 n=176
n=495
“Of the attributes listed below, which one is most important when choosing a mobile phone?”
Survey Enjoyment Willingness to Participatein Future
Smartphone/Online Survey
Willingness to go toSpecified Location/Store to
Participate
Fair Incentive
68%
90%
73% 69%
76%
99%
66% 66%
Mobile Online
NOTE: Top two boxes on a five-point scale.
Survey comparisons
respondent participation
Survey Satisfaction: Key Findings
PHASE THREE:
Smartphone: n=495 On-Line: n=511
LIKES DISLIKES
Anywhere at anytime
• “I don't have to be at home or in front of the computer. I can do this anywhere.”
• “Ease of being able to complete the survey from anywhere. Not needing to be tethered to my desktop.”
Convenience
• “I always have my phone on and with me. Not so for the computer. So, convenience.”
• “It's convenient, and I can get them done quickly because I always have my phone with me.”
Easy to do, easy to access
• “Easy to complete and does not require more effort than a survey taken on a computer.”
• “First time taking survey on mobile – it was easy to follow and easy to get thru, not confusing in the least.”
Fun, different, new
• “It was fun and easy to answer the questions. Normally I take surveys on my laptop, so using a different method such as a mobile phone was interesting.”
• “It was new, interesting, and fun!”
Difficulty typing/Small keyboard
• “Don't like typing on my phone cause the keyboard is too small.”
• “Excessive typing can be a problem. I'd prefer longer, essay response answers for the computer.”
Pages load slowly/Too long/Drained battery
• “Pages load slowly, takes longer than computer.”
• “Too much time, wastes battery life.”
• “Doing anything on the web on my phone takes longer.”
Screen too small, survey content too much for screen
• “The screen on my phone is too small to see the questions and the answers at the same time.”
• “My blackberry is too small to comfortably take the survey.”
• “The survey was made bigger than my screen and it was hard to select my choices at times.”
Too much scrolling
• “It takes so long, and I have to scroll left and right, up and down to read the question.”
Prefer participating from computer
• “It’s easier to do most things on my computer.”
PHASE THREE: Mobile Survey
respondent participation
Survey Satisfaction: Key Verbatims
Crit ical Findings
quantitative survey
Design Elements – Scale Types
Response rates are similar across platforms. Males trend directionally higher.
Smartphone respondents will not require higher or significantly different incentives.
There are both technology limitations and different “worldviews” for smartphone users, but this should dissipate in the near future.
Smartphone platforms can viably support 5-point fully-anchored scales, but 3-point anchored and 5-point end-anchored scales are recommended.
Scale usage does not differ significantly, supporting the use of a hybrid methodology and the integration of mobile and online platforms.
Statistical techniques have similar results across survey platforms, supporting the smartphone platform for in-store, real-time data collection.
• Difficulty typing/Small keyboard
• “Don't like typing on my phone cause the keyboard is
too small.”
• “Excessive typing can be a problem. I'd prefer longer,
essay response answers for the computer.”
• Pages load slowly/Too long/Drained battery
• “Pages load slowly, takes longer than computer.”
• “Too much time, wastes battery life.”
• “Doing anything on the web on my phone takes longer.”
• Screen too small, smaller text, survey content too much
for screen
• “The screen on my phone is too small to see the
questions and the answers at the same time.”
• “My blackberry is too small to comfortably take the
survey.”
• “The survey was made bigger than my screen and it
was hard to select my choices at times.”
• Too much scrolling
• “It takes so long and I have to scroll left and right, up
and down to read the question.”
• Prefer participating from computer
• “It’s easier and faster to do most things on my
computer.”
Smartphone platform
critical implications
Smartphone surveys are a viable option to supplement studies with hard-to-reach segments of the population
Data and scale usage is comparable across platforms, supporting the use of hybrid designs
Smartphone users today tend to be more tech-savvy with unique shopping, usage and behavior preferences regardless of platform
Smartphone platform may more readily support specific subgroup segments, i.e. Hispanic, Millennials, as well as global markets
Mobile-enabled surveys are ideal for in-store shopping and exercises, real-time assessment, and interrupted surveying
The “Strategic Enhancement” of today could well be
the next paradigm—or the 5th methodology—of
tomorrow.
• How will the marketing research industry prepare for this
evolution?
• Does it reduce barriers to reaching specialized segments
(Millennials, Hispanics, global markets?)
• Will this offer greater authenticity to marketing
research (rational/emotional surveying). Are
smartphone devices more conducive for lifestyle-based
interviewing?
• Does it support the evolution of the marketing research
discipline from the current scientific/data-based structure
to a humanistic/decision-making approach?
Mobile Research
considerations for the future
Michael Francesco Alioto, Ph.D.
Vice President
Marketing Sciences
Gongos Research
Greg Heist
Vice President
Research Innovation
Gongos Research
Mobile Surveys | The 5 th Methodology: A New Paradigm or a Strategic Enhancement ?
q&a | discussion
Presented at:
Market Research in the Mobile World 2nd International Conference | July 19 & 20, 2011 Atlanta
Organized by: Thank you to sponsors:
LinkedIn Group: Mobile MR
Upcoming Merlien Events: Merlien.org GreenBook Directory: GreenBook.org Market Research Blog: GreenBookBlog.org New Qual Blog & Directory: NewQualitative.org
Get Inspired. Stay Informed. Cut Through the Clutter. Sign up for our free email newsletter