Copyright 2000, Richard B. Warnecke QUALITATIVE APPROACHES TO QUESTIONNAIRE DESIGN: METHODS AND...

QUALITATIVE APPROACHES TO QUESTIONNAIRE DESIGN:

METHODS AND APPLICATIONS

RICHARD B. WARNECKE, Ph.D. UNIVERSITY OF ILLINOIS AT CHICAGO

MINORITY HEALTH PROJECT

UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL

JUNE 15, 2000

ANSWERING QUESTIONS

INTERPRETATION

RECALL

JUDGMENT FORMATION

RESPONSE FORMATTING

EDITING

LABORATORY METHODS

FOCUSED INTERVIEWS THINK-ALOUDS PARAPHRASING CONFIDENCE RATINGS RESPONSE LATENCY SORTS

INTERPRETING THE QUESTION

LITERAL MEANING

INTENDED MEANING

RESPONSE ALTERNATIVES

INTERPRETATION PROBES

PARAPHRASE QUESTION. DEFINE A WORD OR PHRASE. WHAT DOES A RESPONSE MEAN TO

YOU? BOUNDARIES (WHICH OF THE

FOLLOWING WOULD ALSO BE …..?) LABEL PREFERENCE INTERPRETATION DIFFICULTY

WHAT TYPES OF PHYSICAL ACTIVITY OR EXERCISE DID YOU PERFORM DURING THE

PAST MONTH?

________________________________________________

WHAT DID YOU THINK WE MEANT WHEN WE SAID “PHYSICAL ACTIVITY?

WHICH, IF ANY, OF THE FOLLOWING WOULD YOU (ALSO) CONSIDER TO BE

PHYSICAL ACTIVITY? WHAT ABOUT . . .

WALKING,

HOUSEWORK, SUCH AS CLEANING THEBATHROOM OR KITCHEN,

WORK-RELATED ACTIVITY, SUCH AS CONSTRUCTION

YARD WORK, SUCH AS MOWING THE LAWN OR RAKING LEAVES?

LOGISTIC REGRESSION ANALYSIS OF PHYSICAL ACTIVITY QUESTION MEANING PROBES

(UNSTANDARDIZED COEFFICENTS REPORTED)

Behavioral frequency 0.22* 0.02 0.07 0.12Male (1=Yes)

Male -0.91*** -0.36** 0.41* -0.43**(1 = Yes)

African American 0.07 0.042** -0.01 0.59**(1 = Yes)

Mexican American 0.48 0.25 0.36 0.43(1 = Yes)

LOGISTIC REGRESSION ANALYSIS OF PHYSICAL ACTIVITY QUESTION MEANING PROBES

(UNSTANDARDIZED COEFFICENTS REPORTED) cont’d.

Puerto Rican 0.26 0.38* -0.09 0.34(1 = Yes)

Age 0.08** 0.01 -0.00 0.00

Education 0.12 0.10 -0.12 -0.03

Model chi-square 30.11*** 18.42** 8.21 18.00**

Sample Size 365 372 371 371

RECALLING AND RETRIEVING

RECALL PRIOR INFORMATION COMPUTE ESTIMATE USE A SCHEMA (A PATTERN OF

ASSOCIATIONS AROUND WHICH THINGS ARE RECALLED, I.E. PAP SMEAR AS PART OF A PHYSICAL EXAM).

RETRIEVAL PROBES RECALL STRATEGY RECALL INTERVAL SEARCH STRATEGY (PROXIMAL, DISTAL,

ANYWHERE) LONG TERM RECALL--LINK TO EVENTS TO

HELP REMEMBER RECALL FRAME OF REFERENCE--WHAT

KINDS OF THINGS HELPED YOU REMEMBER?

HOW MANY TIMES PER WEEK OR PER MONTH DID YOU TAKE

PART IN (FIRST ACTIVITY MENTIONED)?

_______________ TIMES PER WEEK

____________________ TIMES PER MONTH

HOW DID YOU REMEMBER THE NUMBER OF TIMES YOU DID THIS? (PERFORM PHYSICAL

ACTIVITY)

IN A TYPICAL WEEK, HOW MANY TIMES DO YOU TALK ON

THE TELEPHONE WITH FAMILY, FRIENDS, OR NEIGHBORS?

______ TIMES PER DAY

______ TIMES PER WEEK

IN RECALLING THIS INFORMATION ABOUT HOW MANY TIMES YOU TALK ON

THE TELEPHONE, DID YOU SPECIFICALLY THINK ABOUT THE MOST RECENT WEEK

OR DID YOU AVERAGE TOGETHER EXPERIENCES YOU’VE HAD OVER A

NUMBER OF WEEKS, OR DID YOU THINK ABOUT IT IN SOME OTHER WAY?

THE MOST RECENT WEEK AVERAGE WEEK OTHER (SPECIFY) __________________ DON’T KNOW

JUDGMENT FORMATION

FIT A REFERENCE PERIOD

FIT A DEFINITION

RELATE TO QUESTIONNAIRE CONTEXT-- ORDER EFFECTS

JUDGMENT PROBES JUDGMENT EXPLANATION

– HOW ARRIVED AT ANSWER JUDGMENT CERTAINTY

– SCALE OF CERTAINTY (1-7) ANCHORING STRATEGIES

– COMPARATIVE ATTITUDE CRYSTALLIZATION

– THOUGHT ABOUT BEFORE– FREQUENCY OF PREVIOUS THOUGHT– DISCUSS WITH OTHERS

HOW LIKELY DO YOU THINK IT IS THAT YOU WILL HAVE SERIOUS

HEALTH PROBLEMS FROM SMOKING? DO YOU THINK IT

IS…..

VERY LIKELYSOMEWHAT LIKELY

SOMEWHAT UNLIKELYVERY UNLIKELY

DEFINITELY NOT POSSIBLE?DON’T KNOW

BEFORE WE ASKED YOU THE QUESTION ABOUT SERIOUS

HEALTH PROBLEMS, HAD YOU THOUGHT ABOUT IT BEFORE?

ON A SCALE FROM 1 TO 7, WHERE 1 IS ONLY RARELY AND 7

IS VERY OFTEN, HOW OFTEN HAVE YOU THOUGHT ABOUT IT

BEFORE?

1 2 3 4 5 6 7

ONLY VERYRARELY OFTEN

ORDER EFFECTS-I

MINIMIZING ORDER EFFECTS– GENERAL BEFORE SPECIFIC (FUNNEL)– SPECIFIC BEFORE GENERAL

(INVERTED FUNNEL)– USE ORDER TO REDUCE EFFECTS OR

ITEM NON RESPONSE

ORDER EFFECTS -2 DEVELOP A LOGICAL FLOW TO HELP

RESPONDENT

– SEGMENT BY TOPIC– ASK ABOUT RELATED TOPICS

TOGETHER– ASK RECALL FROM MOST RECENT TO

LEAST RECENT– USE TRANSITIONS TO CHANGE TOPICS

ORDER EFFECTS-III ESTABLISH RAPPORT

– START WITH EASY, NON-THREATENING QUESTIONS

– DON’T START WITH DEMOGRAPHICS END WITH THEM

– DON’T START WITH KNOWLEDGE AND AWARENESS

EXAMPLE OF USE OF ORDER TO RESOLVE CONFUSION

ORIGINAL QUESTION: HOW SATISFIED ARE YOU WITH YOUR RELATIONSHIP WITH YOUR SPOUSE/SIGNIFICANT OTHER?

PROBE: WHAT PARTS OF THE RELATIONSHIP DID YOU THINK ABOUT?

THE PROBLEM QUESTION WORKED WELL IN

ENGLISH. HOWEVER IN SPANISH THE WORD

FOR RELATIONS IS THE WORD FOR SEXUAL RELATIONS

SOLUTION

REVERSED ORDER OF QUESTIONS. ASKED ABOUT SATISFACTION WITH SEXUAL RELATIONS FIRST.

THEN ASKED ABOUT OTHER ASPECTS OF MARITAL RELATIONSHIP.

FORMATTING A RESPONSE

RESPONSE CATEGORIES

RATING SCALES

RESPONSE SET– EXTREME CHOICES– ACQUIESCENCE

AfricanAmerican(N=102)

MexicanAmerican(N=106)

PuertoRican(N=84)

Whitenon-Hispanic(N=101)

Mean** 7.78 8.09 8.69 6.7

Standarddeviation

4.65 3.99 4.20 4.42

**F-ratio p< .01.

Note: Means with different superscripts differ significantly(p< .05) on the basis of multiple range test.

Mean Group Responses to Measure ofExtreme Response Style

AfricanAmerican(N=102

PuertoRican(N=84)

Whitenon-Hispanic(N=101)

Mean** 10.00 9.96 9.43 8.76

Standarddeviation

2.84 2.91 3.12 2.71

**F-ratio p< .01.

Note: Means with different superscripts differ significantly (p< .05) on thebasis of multiple range test.

Mean Group Responses to Acquiescence Scale

AfricanAmerican(N=110)

PuertoRican(N=87)

Whitenon-

Hispanic(N=108)

Mean** 5.57 5.92 5.47 4.85

Standarddeviation

2.47 2.30 2.16 2.29

**F-ratio p<.01.

Note: Means with different superscripts differ significantly (p<.05) on the basis of multiple range test.

Mean Group Responses to the Marlowe-CrowneSocial Desirability Scale

SCALE OF SATISFACTION WITH QUALITY OF LIFE

QUALITY OF LIFE SCALE WITH RANGE OF VALUES.– VERY SATISFIED– SATISFIED– SOMEWHAT SATISFIED– A LITTLE SATISFIED– NEITHER SATISFIED NOR DISSATISFIED– A LITTLE DISSATISFIED– SOMEWHAT DISSATISFIED– DISSATISFIED– VERY DISSATISFIED

ASSESSMENT

SATISFACTION RANGE

VERY 100-60

SATISFIED 100-50

SOMEWHAT 90-45

A LITTLE 70-30

DISSATISFIED RANGE

VERY 0-80

DISSATISFIED 0-80

SOMEWHAT 10-54

A LITTLE 10-53

SOLUTION

USED A BAR GRAPH WITH SINGLE SCALE WHERE BLANK EQUALS “NOT AT ALL SATISFIED” AND 6 EQUALS “VERY SATISFIED.”

SIMPLIFIED RESPONSE ON ONE DIMENSION

AVOIDED LABELS THAT WERE CONFUSING.

EDITING A RESPONSE

SOCIAL DESIRABILITY SELF PRESENTATION EFFECT OF INTERVIEWER

– GENDER– AGE– CULTURE/ ETHNICITY

RESPONSE EDITING PROBES

PERCEIVED SOCIAL DESIRABILITY PERCEIVED EMBARRASSMENT PERCEIVED HONESTY PERCEIVED INTERVIEWER EFFECTS

– ETHNICITY– AGE – GENDER

INTERVIEWER EFFECTS: AGE ORIGINAL QUESTION: HAVE YOU EVER

SMOKED MARIJUANA, HASH OR HASH OIL OR TAKEN THEM IN SOME OTHER WAY?

PROBE: DO YOU FEEL THIS IS A QUESTION THAT A PERSON YOUR AGE WOULD BE COMFORTABLE OR UNCOMFORTABLE TALKING ABOUT TO A SURVEY INTERVIEWER OF THE SAME AGE OR WHO WAS MUCH YOUNGER OR OLDER?

Interviewer age

Respondent age

Younger Older (18-29) (30-50)

Same age 77.5 78.6

Younger orOlder

30.3 54.6

Projected Comfort Level Discussing Marijuana Use with HypotheticalInterviewer of Same Age/Different Age

INTERVIEWER EFFECTS: GENDER

ORIGINAL QUESTION:DURING THE PAST WEEK, HOW

OFTEN HAVE YOU HAD CRYING SPELLS? PROBE:

DO YOU FEEL THIS IS A QUESTION THAT (RESPONDENT’S GENDER) WOULD BE COMFORTABLE TALKING ABOUT TO A SURVEY INTERVIEWER OF THE SAME SEX? OF THE OPPOSITE SEX?

Interviewergender Male Female

Same gender 34.1 93.3

Opposite gender 28.1 41.3

Projected Comfort Level Discussing Crying Spells with Hypothetical Interviewer of Same Sex/Opposite Sex

INTERVIEWER EFFECTS: ETHNICITY

ORIGINAL QUESTION: DO YOU THINK OF YOURSELF AS HETEROSEXUAL, HOMOSEXUAL OR BISEXUAL?

PROBE: DO YOU FEEL THIS IS A QUESTION THAT (RESPONDENT’S ETHNIC GROUP) WOULD BE COMFORTABLE DISCUSSING WITH AN INTERVIEWER OF THE SAME (ETHNIC GROUP)? A SURVEY INTERVIEWER OF A DIFFERENT ETHNIC GROUP?

Comfort LevelAfricanAmerican(N=101)

PuertoRican(N=82)

WhiteNon-Hispanic(N=99)

Topic: SexualorientationInterviewerrace/ethnicity: Samerace/ethnicity

72.3 64.0 65.4 82.8

Differentrace/ethnicity

61.0 51.5 54.9 71.4

Projected Comfort Level of Respondents by Topic, By Cultureof Hypothetical Interviewer, And By Culture of Respondent

(Percent Comfortable)

Copyright 2000, Richard B. Warnecke QUALITATIVE APPROACHES TO QUESTIONNAIRE DESIGN: METHODS AND...

Documents

Bruce A. Miller*, Richard B. Emlet

B. Lungsi Sharma1 , and Richard B. Wells - Department of …rwells/techdocs/Mathematical or… · · 2017-10-30B. Lungsi Sharma1 , and Richard B ... (1999). The Feeling of What

Affidavit of Richard Hutchings Exhibit B - PwC

Extreme Heat and Human Health Richard B. Rood and Marie S. O’Neill Richard B. Rood 734-647-3530 rbrood@umich.edu

Datum: 25. Mai 2010 Referent: Dipl.-Ing. Carsten Warnecke Exportinitiative Energieeffizienz

Richard B. Wells CURRICULUM VITAE - uidaho.edu B... · 2019-12-24 · Richard B. Wells CURRICULUM VITAE University of Idaho NAME: Wells, Richard B. rwells@uidaho.edu DATE: Dec 24,

Christoph Warnecke „HR als Business-Partner“ · Christoph Warnecke „HR als Business-Partner“ 12 Ø Das Kerngeschäft des Unternehmens verstehen und “die gleiche Sprache”

Bachelorarbeit 2017 Sabrina Warnecke - MOnAMi · 2019. 8. 21. · Warnecke, Sabrina: Nachhaltigkeit als Element der Produktpolitik in der Fast Fashion Branche Sustainability as an

CLOSURE OF ILEOSTOMY IN ULCERATIVE COLITIS* RICHARD B

Warnecke, Jörn; Käpylä, Petri J.; Käpylä, Maarit J.; Brandenburg, … · Warnecke et al. 2013b, for the implementation and the proﬁle) above fractional radii r/R = 0.985 and

Richard B. Freeman · Labor Regulations, Unions, and Social Protection in Developing Countries:¸˛Market distortions or Efficient Institutions? Richard B. Freeman NBER Working Paper

Alexander Johannes Warnecke Degradation Mechanisms in NMC

Richard Nixon and the Rise of Conservatism b Kennedy -Nixon: 1960 b 1968- Richard Nixon back again! b Vietnamization of the War in SE Asia b Supressing

Strategy for Responsible Peatland Management Susann Warnecke IPS Communications Manager 11 January 2011

Richard B Belzer - The Report on Carcinogens

Schmeduling Jeffrey B. Liebman and Richard J. Zeckhauser ...aida.econ.yale.edu/~shiller/behmacro/2004-11/schmeduling-zeckhauser.pdf · Schmeduling Jeffrey B. Liebman and Richard J

Urteilsheuristiken Verarbeitung sozialer Informationen Leitung: Dr, Rainer Roth Anna Christin Warnecke

Richard P. Feynman, Robert B. Leighton

CENCI GOEPEL + JENS WARNECKE - Lightmark · Photographic art: The colours of darkness Ruth Hoffmann Cenci Goepel and Jens Warnecke travel around the world,turning night into day

JOHN A. PEARCE II & RICHARD B ROBINSON, Jr