Chapter 8

Designing the Survey Questionnaire

Copyright © 2013 by The McGraw-Hill Companies, Inc. All rights reserved.McGraw-Hill/Irwin

8-2

Questionnaires in Marketing Research

• Questionnaire: A formal, structured response framework consisting of a set of questions and scales designed to generate primary data.• Can be administered orally or textually (i.e.

“written”)

8-3

Questionnaires in Marketing Research – Two Main Kinds

• Descriptive survey questionnaires collect data that can be turned into knowledge about consumers, companies, markets, etc.

• Predictive survey questionnaires predict changes in attitudes and behaviors and are used to test research hypotheses.

8-4

Steps in Questionnaire Design

8-5

Step 1: Confirm Research Objectives

• Example of Research Objectives:– To collect data on selected customer demographic

characteristics and lifestyle dimensions of our different market segments

– To identify preferred banking services, as well as attitudes and feelings toward those services

8-6

Step 2: Select Appropriate Data Collection Method

• We will develop and administer an internet survey with the following sections:– Section I: Banking services– Section II: Lifestyle dimensions– Section III: Banking relationships– Section IV: Demographic characteristics

8-7

Step 3: Develop Questions and Scaling

• Question formats– Unstructured questions: Open-ended questions

allow respondents to reply in their own words – no pre-determined choices imposed by the researcher (like an essay test!)

– Structured questions: Closed-ended questions that require the respondent to choose from a pre-determined set of responses or scale points (pretty much any scale we have looked at so far)

8-8

Step 3: Develop Questions and Scaling

• Bad questions: Any questions that prevent or distort the fundamental communication between researchers and respondents

• A question is bad when it is:– Unanswerable– Leading (loaded)– Double-barreled

8-9

Step 3: Develop Questions and Scaling

• Other considerations:– Use simple words; avoid technical words or jargon– Make questions as concise as possible– Avoid using qualifying phrases (always,

sometimes, never, etc.) in question stems, but they’re generally ok in response scales

– Ensure response categories are mutually exclusive– Ensure question and corresponding scale

descriptors are meaningful to respondents and “jive” with each other.

– Grammar, diction and spelling must be flawless!!!

8-10

Step 3: Develop Questions and Scaling

• Sensitive questions: Questions about income, sexual beliefs or behaviors, medical conditions, financial difficulties, drug/alcohol consumption, or other questions that respondents might skip, answer falsely or otherwise take offense to should appear near the end of the survey.

8-11

Step 3: Develop Questions and Scaling

• Skip questions: Used if the next question (or set of questions) should be responded to only by respondents who meet a previous condition

• Much easier to implement in web/digital surveys

8-12

Step 4: Determine Layout and Evaluate Questionnaire

• Introductory section: Gives the respondent an overview of the research• Re-mention the purpose of the research (if indicated)• Set any necessary ground rules• Provide general instructions

• Screening questions: Used to disqualify inappropriate respondents • This should usually/mostly be done before the formal

survey is administered• Research questions section: Focuses on the main

research questions of interest

8-13

Step 4: Determine Layout and Evaluate Questionnaire

• Ensure instructions are clear for each section• Arrange questions in a logical order.– Keep questions using the same response formats in close

proximity if possible.– Start with easy questions and ask the more difficult

questions later in the survey.

• Be mindful of coding needs when choosing response formats, especially if manual entry of data will occur.

• The entire survey should be as short as possible without sacrificing the integrity and amount of data collected – it’s ultimately a judgment call!

• End with a thank-you statement.

8-14

Step 4: Determine Layout and Evaluate Questionnaire

• Response order bias: Occurs when the order of the questions, or of the closed-end responses to a particular question, influences the answer given

• Common methods variance (CMV): Falsely correlated answers due to a subject’s need to appear consistent across a series of related questions.

8-15

Step 5: Obtain Initial Client Approval

• Provide copies of the questionnaire to all parties involved in the project

• Client may identify any overlooked items or other problematic aspects of the survey (i.e. inaccurate representations, offensive questions, etc.)

• What if client makes a stupid suggestion!?

8-16

Step 6: Pretest, Revise, and Finalize the Questionnaire

• Final evaluation of the questionnaire is obtained from a pretest– Helps the researcher determine:• How much time respondents will need to complete the

survey (Is survey too long?!)• Whether to add, delete or revise instructions, questions

and scales that are biased or poorly written. • Where did subjects get confused or misled?

• Helps with how to frame aspects of the cover letter (see subsequent slides)

8-17

Step 7: Implement the Survey

• Administer the survey to the “real” (i.e. non-pretest) sample.

• Keep correcting any survey shortcomings / errors that are revealed during implementation without changing the fundamental nature of the survey so that results remain comparable across the changes.• Good + Bad Data is usually better than All Bad

Data

8-18

The Cover Letter / Invitation to Participate

• Cover letter / Invitation:• Designed to enhance respondent willingness to

accurately and faithfully complete and/or return the survey in a timely manner

8-19

Guidelines for Developing Effective Cover Letters

8-20

Collecting Data

• Interviewer instructions: Used to train interviewers how to:– Select prospective respondents– Screen them for eligibility– Conduct the actual interview

• Screening questions• Quotas: A tracking system that collects data

from respondents and helps ensure that subgroups are represented in the sample as specified

8-21

Collecting Data: Interviewer Instructions

• Used to train interviewers how to:– Select prospective

respondents– Screen them for

eligibility– Conduct the

actual interview

8-22

Collecting Data: Contact Records

• Contact Record: Gathers basic summary information about an interviewer’s performance efficiency