1

AMA Iowa Experience EventFebruary 5, 2016

SURVEY DESIGNAvoiding the Common Pitfallsthat Can Sabotage Your Data

Presented by:

2

3

Types of Research

4

Types of Research

Qualitative Research

Quantitative Research

• In-person Focus Groups

• Phone Surveys

• Online Chat Focus Groups

• Mail Surveys

• Bulletin Board Focus Groups

• Online Surveys

• IDI’s – individual interviews

• Omnibus Surveys

• Dyads, Triads• Online Communities• Journals, Diaries• Usability Testing• Mock Juries• Ethnography

(observational)

5

Consider the respondent’s point of view

Survey Design

Will they be able to answer your questions?

Will they want to answer your questions?

Will they be motivated to answer all your questions?

6

Survey Design

7

Do not havemutually exclusive responses

Mistake #1

Good:What is your age?____ Under 20____ 20 – 29____ 30 – 39____ 40 – 49____ 50 +

Bad:What is your age?____ Under 20____ 20 – 30____ 30 – 40____ 40 – 50____ Over 50

8

Asking double-barreled questions

Bad:How satisfied are you with the pay and work benefits of

your current job?

Good:• How satisfied are you with the pay of your current job?

• How satisfied are you with the work benefits of your current job?

Mistake #2

9

Not providing all answer options

What type of vehicle do you own?____ Van____ SUV____ Sedan

____ Truck____ Convertible____ I do not own a vehicleOther _________________

Mistake #3

10

Not providing all answer options

What is the main reason why you attended AMA Iowa’s Q&AMA event?

____ The event sounded interesting____ I wanted to see what a trivia night was like____ Networking opportunities

Other (please specify)________________________

Mistake #3

11

Questions are not clear and precise,have ambiguity

In a typical month, how often do you shop at a mall?

Ambiguous:____ Never____ Occasionally____ Sometimes____ Often____ Regularly

Mistake #4

Good:____ Not at all____ 1 or 2 times____ 3 or 4 times____ More than 4 times

12

Using absolutes in questions

Mistake #5

Bad:Do you always eat breakfast?____ Yes____ No

Good:How many days a week do you usually eat breakfast?____ Every day____ 5-6 days____ 3-4 days____ 1-2 days____ I usually don’t eat breakfast

13

Questions that are leading or biased

Bad: Is Colgate your favorite toothpaste? Yes or NoGood: What is your favorite toothpaste brand? __________

Bad:The Mac operating system rarely gets infected by viruses and therefore the Department of Education should only purchaseMac computers. Yes, No, Maybe

Good:The Department of Education should only purchase Mac computers. Yes, No, MaybeWhy do you say this? _______________________________

Mistake #6

14

Using Loaded Questions

Bad: What kind of wine do you like to drink?

Good: Do you like to drink wine?

____ Yes____ No

If Yes, what kind of wine do you like to drink?(could be open-ended or drop-down menu)

If No, why do you not like to drink wine?

Mistake #7

15

Confusing rating scale questions;Reversing rating scales

Q1: Please rank your overall satisfaction with Company ABC on a scale of 1 to 5.

1 2 3 4 5

Q2: Please rank your satisfaction with Company ABC’s customer service staff on a scale of 1 to 5.

5 4 3 2 1

Mistake #8

16

Confusing rating scale questions;Reversing rating scales

Q1: Please rank your overall satisfaction with Company ABC on a scale of 1 to 5, where 1 is Not At All Satisfied and 5 is Very Satisfied.

1 2 3 4 5

Q2: Please rank your satisfaction with Company ABC’s customer service staff. Not At Not Very All Satisfied Satisfied Neutral Satisfied Satisfied

Mistake #8

17

Not using languageyour respondent understands

Using too much jargon, slang, acronyms

Bad: What marketing and research organizations are you a member of? (Please select all that apply.)

____ AMA____ DMA____ MRA____ NAMA____ PRSAOther _____________

Mistake #9

Good:Need to spell out acronyms

18

Open-ended questions thatdo not have parameters

Bad:According to you, what are the factors that contribute to the success of a company in the poor economic conditions?

Good:In your opinion, what are the key factors that contribute to the success of a manufacturing company in the present recessionary economy?Please briefly describe up to three.1. ________________________________________________ 2. ________________________________________________3. ________________________________________________

Mistake #10

19

Asking too many open-ended questions

Creates fatigue and disinterest in completing survey

Adds unnecessary length to survey

Mistake #11

20

Pre-survey design, identify your plan for analysis

Don’t let them off the hook too easilyfrom answering a question

Proof, PROOF, Proof

Test, TEST, Test

Organize survey to minimize bias

Design Tips

21

Q & A