28
Questionnaire Questionnaire Design Design

Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Embed Size (px)

Citation preview

Page 1: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Questionnaire DesignQuestionnaire Design

Page 2: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

“It is not every question It is not every question that deserves an answerthat deserves an answer.”

Publius Syrus(Roman, 1st century B.C.)

Page 3: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Ask a Sensitive Question, Get a Sensitive Answer

• Survey researchers believe that the responses that people give are valid.

• Care must be taken with sensitive questions.

• Researchers must take care in asking relevant questions in ways that produce the most truthful results.

Page 4: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Basic Considerations in Questionnaire Design

• Questionnaire design is one of the most critical stages in the survey research process.

– A questionnaire (survey) is only as good as the questions it asks — ask a bad question, get bad results.

– The questions must meet the basic criteria of relevance and accuracy.

Page 5: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

What Should Be Asked?

• Questionnaire Relevancy– All information collected should address a research question

that helps the decision maker in solving a current marketing problem.

• Questionnaire Accuracy– The information is valid; it faithfully represents reality.

• Questionnaires should use simple, understandable, unbiased, unambiguous, & nonirritating words.

• Questionnaire design should facilitate recall & motivate respondents to cooperate.

• Proper question wording & sequencing to avoid confusion & biased answers.

Page 6: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Major Decisions in Questionnaire Design

What should be asked?How should each question be phrased?In what sequence should the questions be

arranged?What questionnaire layout will best serve

the research objectives?How should the questionnaire be pretested?

Does the questionnaire need to be revised?

Page 7: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Phrasing Questions• Open-ended questions

–Aka essay questions, short-answer questions

• Fixed-alternative questions–Aka closed or closed-ended questions

Page 8: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Open-Ended Response Questions

– Pose some problem & ask respondents to answer in their own words.

– Advantages:• Particularly beneficial in exploratory research, especially when

the range of responses is not known.• Identify which words & phrases people spontaneously give.

• Valuable at the beginning of an interview.

– Disadvantages:• High cost of administering open-ended response questions.

• The possibility that interviewer bias will influence the answer.

• Bias introduced by articulate individuals’ longer answers.

Page 9: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Example of Open-Ended Response Question

Page 10: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Fixed-alternative Questions– Questions in which respondents are given specific,

limited-alternative responses & asked to choose the one closest to their own viewpoint.

– Advantages:• Require less interviewer skill

• Take less time to answer

• Are easier for the respondent to answer

• Provides comparability of answers

– Disadvantages:• Researcher may be unaware of all potential responses

• Tendency of respondents to choose more prestigious or socially acceptable alternative

Page 11: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Example of a Fixed-Alternative Question

Page 12: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Types of Fixed-Alternative Questions

• Simple-dichotomy (dichotomous) Question– Requires the respondent to choose one of two alternatives (e.g., yes or no).

– Example:Did you make any calls with your home (landline) phone during the 7 days?

_____ Yes _____ No

• Determinant-Choice (multiple-choice) Question– Requires the respondent to choose one response from among multiple

alternatives (e.g., A, B, or C).

– Example:

Page 13: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Types of Fixed-Alternative Questions (con’t.)

• Frequency-determination Question– Asks for an answer about general frequency of occurrence (e.g., often,

occasionally, or never).

• Checklist Question– Allows the respondent to provide multiple answers to a single question by

checking off items.

• Scale– Likert, Semantic Differential, Stapel, etc.

Page 14: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Phrasing Questions for Self-Administered,Telephone, & Personal Interview Surveys

• Influences on Question Phrasing

– Means of data collection — telephone interview, personal interview, self-administered questionnaire — will influence question format & question phrasing.

• Questions for mail, Internet, & telephone surveys must be less complex than those used in personal interviews.

• Questionnaires for telephone & personal interviews should be written in a conversational style.

Page 15: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Best Question Formats Vary by the Interview Medium

Page 16: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Guidelines For Avoiding Mistakes

• Simpler is better

• Avoid leading & loaded questions

– Leading question – directs respondents to an answer you want them to give

– Loaded question – suggests a socially desirable answer or is emotionally charged.

Page 17: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Guidelines For Avoiding Mistakes (con’t.)

• “Avoid” ambiguity: Be as specific as possible

– Probably /Definitely; Sometimes/Always, etc.

• Avoid double-barreled items

– Double-barreled question – may induce bias because it covers two or more issues at once.

– Do you think the President is responsible for the federal government shut-down and the currently rising gasoline prices? Yes No

• Avoid making assumptions

– Given Macy’s skill-level at gift wrapping, …….

• All-inclusive response alternatives

• Avoid taxing respondent’s memory

Page 18: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Avoid Common Wording Mistakes in Questionnaire Design

Page 19: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Order Bias• Question Sequence

– Order bias• Bias caused by the influence of earlier questions in a questionnaire or by an

answer’s position in a set of answers.

– Funnel technique• Asking general questions before specific questions in order to obtain

unbiased responses.

• Randomized Presentations– Used in electronic questionnaires, but rarely used in printed

questionnaires due to coding difficulties.

• Randomized Response Techniques– Randomly assigning respondents to answer either the question of

interest (embarrassing) or a mundane & unembarrassing question.

Page 20: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Survey Flow• Survey flow

– The ordering of questions through a survey.

• Breakoff– A respondent who stops answering questions before reaching the

end of the survey.

• Filter question– A question that screens out respondents who are not qualified to

answer a second (or follow-up) question.

• Branching– Directing respondents to alternative portions of the questionnaire

based on their response to a filter question.

Page 21: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Survey Flow for Eurocar Tour de France Sponsorship

Page 22: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Telephone Questionnaire with Skip Questions

Page 23: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Survey Technology

• Physical Features– Heat map question

• A graphical question that tracks the parts of an image or advertisement that most capture a respondent’s attention.

– Status Bar• A visual indicator that tells the respondent what portion of the survey he

or she has completed.

• Prompting– Informs the respondent that he or she has skipped an item or

provided implausible information.

• Piping Software– Allows question answers to be inserted into later questions.

Page 24: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Tracking Points of Interest Using a Heat Map

Page 25: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Illustration of Status Bar & Prompts

Page 26: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Pretesting & Revising Questionnaires

• Pretesting Process– Seeks to determine whether respondents have

any difficulty understanding the questionnaire

– whether there are any ambiguous or biased questions.

• Preliminary Tabulation– A tabulation of results of a pretest to help

determine whether questionnaire will meet the objectives of the research.

Page 27: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

I Give Up! Large portion of respondents give up

before finishing & abandon the survey (break-offs).

Guidelines Visually appealing & easy to read Fewer questions per page (no more than 20) 4 pages maximum for consumers 6 pages maximum for business leaders Question order (funnel vs. reverse-funnel) Sensitive questions & open-ended questions

encourage break-offs &/or item non-response Sophisticated samples increase response rate Make information requests legitimate Pretesting is important (pretest EVERYTHING)

Page 28: Questionnaire Design. It is not every question that deserves an answer “It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century

Questionnaire Reproduction(if mail survey)

Professional appearanceBooklet format for long

questionnairesPlace directions as close to

questions as possibleExpect Expect reproduction errors