Upload
augustus-griffin
View
230
Download
0
Tags:
Embed Size (px)
Citation preview
CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED
QUESTIONS
CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED
QUESTIONSDamon BurtonDamon Burton
University of IdahoUniversity of Idaho
What is the difference between open- and closed-ended questions?
QUESTION-TYPE DEFINITIONS
•Open-Ended Questions – are questions in which the choice of responses are not specified and respondents must generate their own responses.
•Close-Ended Questions – are items in which responses choices are specified and respondents must choose one of
the available alternatives.
TYPES OF OPEN-ENDED QUESTIONS
• Descriptive Questions – in which respondents are asked to provide in-depth information on the topic of the question.
• Numerical Response Questions – respondents must provide numerical data such as dates, frequency, monetary value, count, amount or scalar value.
• List-of-Items Questions – respondents must provide a list such as grocery stores they frequent, brands of clothing, or major purchases they’d like to make.
We’re going to discuss a number of different question types and show you how to best write each type of question.
NUMERICAL RESPONSE GUIDELINES
1.Ask for the specific unit desired in the question stem (5.1).
2.Provide answer spaces that are sized appropriately for the response task (5.2).
3.Provide unit labels with the answer spaces (5.1).
4.Specificity enhances response accuracy.
5.1 NUMBER BOX QUESTIONS
5.2 IMPACT OF ANSWER SPACES ON RESPONSES
DESCRIPTION & ELABORATION GUIDELINES
1.Provide extra motivation to respond (5.3).
2.Provide adequate space for respondents to completely answer the question (5.3).
3.Use scrollable boxes on web surveys (5.3).
4.Consider programming probes to open-ended responses in internet surveys (5.4).
5.3 DESCRIPTION & ELABORATION QUESTIONS
= 4/DAY
5.4 PROBING OPEN-ENDED QUESTIONS
You’ll probably use more closed- than open-ended questions, so here are guidelines for how to write good items.
CLOSED-ENDED GUIDELINES
1.State both positive and negative sides in the question stem when asking either/or questions (5.5).
2.Develop lists of answer categories that include all reasonable possible answers (5.6).
3.Develop lists of answer categories that are mutually
exclusive questions (5.7).
5.5 INCLUDE POSITIVE & NEGATIVE SIDES
= 4/DAY
5.6 MUTUALLY EXCLUSIVE
CATEGORIES
= 4/DAY
5.7 ORDERING OF RESPONSES
= 4/DAY
CLOSED-ENDED GUIDELINES
4. Maintain spacing between answer categories that is consistent with measurement intent (Figure 5.8).
5. Ask respondents to rank only a few items at once rather than a long list (Figure 5.9).
• Ranking questions are difficult to understand and complete correctly.
• Only have respondents rank top 3-4.
• Pair all possible options and calculate overall rankings during analysis.
5.8 SPACING RESPONSE OPTIONS
EVENLY
= 4/DAY
5.9 CLOSED-ENDED WITH UNORDERED
RESPONSES
= 4/DAY
CLOSED-ENDED GUIDELINES
6. Avoid bias from unequal comparisons (Figure 5.10).
7. Randomize response options if there is concern about order effects (Figure 5.11).
8.Use forced-choice questions instead of check-all-that-apply questions (Figure 5.11).
5.10 AVOIDING UNEQUAL COMPARISON
BIAS
= 4/DAY
5.11 UNORDERED RESPONSE OPTIONS
= 4/DAY
5.12 COMPARISON OF RESPONSE RATES
= 4/DAY
CLOSED-ENDED GUIDELINES
9. Use differently shaped answer spaces (circles vs squares) to help respondents distinguish between single- and multiple-answer questions (Figure 5.13).
5.13 QUESTION TYPE & SHAPES OF ANSWER
SPACES
= 4/DAY
How do the type of analyses you plan to do influence your response choices?
ORDINAL CLOSED-ENDED GUIDELINES
10. Choose appropriate scale length of 4-5 categories (Figure 5.14).
• Typically you must have a minimum of 5 choices on a Likert scale to assume interval data and utilize inferential statistics.
• Even versus odd-number scales are based on whether you want to allow respondents
to choose a neutral response.
5.14 SCALER QUESTIONS AND RESPONSE
OPTIONS
= 4/DAY
ORDINAL CLOSED-ENDED GUIDELINES
11. Choose direct or construct-specific labels to improve understanding (Figure 5.15).
12. Provide scales that approximate the actual distribution of the characteristic in the population. • Typically respondents assume that the
midpoint response category represents the middle of the distribution in the population.
How does the reason for conducting the survey influence response options (e.g., academic research versus consumer satisfaction)?
ORDINAL CLOSED-ENDED GUIDELINES
13. Provide balanced scales where categories are relatively equal distance apart conceptually (Figure 5.16).
14. Consider how verbally labeling and visually displaying all response categories may influence answers (Figure 5.17). • Dillman suggests that most surveyors prefer fully
labeled scales that range from 2-5 choices.
• Conversely, researchers tend to prefer numerical scales of 5-9-point Likert scales (see examples).
5.16 EVENLY-SPACE CONCEPTUAL CATEGORIES
= 4/DAY
5.17 VERBAL VERSUS NUMERICAL LABELS
= 4/DAY
ORDINAL CLOSED-ENDED GUIDELINES
15. Carefully evaluate the use of numeric labels and their impact on measurement (see examples).
• Numbers reinforce the interval nature of data.
• Dillman contends that numbers add to processing time, but this depends on whether questions use same response categories or change categories from question to question.
• Number consistently across questions.
ORDINAL CLOSED-ENDED GUIDELINES
16. Align response options vertically in one column or horizontally in one row and strive for equal distance between
categories (Figure 15.18).17. Place nonsubstantive options at the
end of the scale and separate from substantitive options (Figure 15.19).
• Be very careful about using “don’t know” or “no opinion” categories.