Embed Size (px)
Citation preview
1
Questionnaire Design
K Alpers, C Campese, P McKeown, V Bremer, V Prikazsky
EPIET Introductory Course 2010Lazareto, Menorca
2
What is a questionnaire?
• A tool to
– collect answers to questions
– collect factual data
• A series of written questions
in a fixed, rational order
3
A well designed questionnaire
• Gives accurate and relevant information
to your research question
• Minimises potential sources of bias
As simple and focused as possible
4
Bias
Systematic difference in the response measurement
• Recall bias – Cases more likely to remember than controls
• Observer bias – Different interviewers – different interpretations– Different interpretation of similar questions
• Non-response bias– telephone interviews: more females, elderly
5
How to reduce bias
• Structured questionnaire • Ensure high response rate• Random choice of interview partners
(next birthday)• Training of interviewers
6
Advantages of questionnaires
• Can reach a large number of people• Relatively easy and economic• Relate directly to study question
• Provide quantifiable answers
• Relatively easy to analyse
7
Disadvantages of questionnaires
• Provide only limited insight into a problem– the range of possible responses is limited – the question maybe misleading
• Varying response – Unclear question can lead to
• misunderstanding • misinterpretation
• Do not allow for mistakes– must be right from the beginning – missing data hard to chase
8
Types of Questionnaires
• Interviewer-administered– face to face
9
Types of Questionnaires
• Interviewer-administered– face to face – telephone
10
Types of Questionnaires
• Interviewer-administered– face to face – telephone
• Self-administered– by post
11
Types of Questionnaires
• Interviewer-administered– face to face – telephone
• Self-administered– by post – email/Internet
12
Self-administered questionnaire• Advantages
– cheap and easy to administer– preserves confidentiality– completed at respondent's convenience– not influenced by interviewer
13
Self-administered questionnaire• Advantages
– cheap and easy to administer– preserves confidentiality– completed at respondent's convenience– not influenced by interviewer
• Disadvantages– low response – questions can be misunderstood– no control by interviewer– only literate persons– time delay (post)– e-mail/internet: need computer access/software
14
Interviewer-administered questionnaire
• Advantages– participation of illiterate people
– clarification of ambiguities
– quick answers
15
Interviewer-administered questionnaire
• Advantages– participation of illiterate people
– clarification of ambiguities
– quick answers
• Disadvantages– interviewer bias– needs more staff resources– only short questionnaires possible
• especially on telephone
– difficult for sensitive issues
16
Before starting to design a questionnaire
Write a study protocol!
• define objectives• define study questions
Risk factors for being stung by jellyfish at Lazareto (Menorca), October 2010
Study protocol
Prepared by cohort 16
Menorca, 7 October 2010
17
Study protocol and questionnaire
Definition of objectives
List of variables to collect
Analysis plan
Study protocol
Design questions
Control and reading
Pilot test
questionnaire
The question must answer your objectives
The question collects appropriate information for
analyses
18
Use existing questionnaires
• As an inspiration• Don´t need to re-invent the wheel• Have been tested
• BUT: adapt to present situation!
19
Questionnaire needs to be adapted to study population
• Know the respondents– occupation – sensitive issues– education– ethnic group – language
20
What makes a well designed questionnaire?
• Good appearance (easy for the eye)
• Short and simple
• Relevant and logical
⇒ High response
⇒ Easier to collect
to summarize
to analyse
21
ARZTFRAGEBOGEN Aktenzeichen:____________ Name, Vorname: _____________________ Geb.: ___.___._____
Masern-Ausbruch Nordfriesland Juni 2001 Dr. Gerard Krause Abt. Infektionsepidemiologie Robert Koch-Institut Seestrasse 10 13353 Berlin
Vom Befrager einzutragen: - Datum der Befragung: ___.___._____ - Initialen des Befragers (Vor-+ Nachname): ___ ___ - Name des Arztes: ________________________________
Guten Tag, mein Name ist ___ (bitte Namen nennen)___.
Ich bin Mitarbeiter im Kreisgesundheitsamt in Husum
oder Als Mitarbeiter des Robert Koch-Institutes unterstütze ich das Kreisgesundheitsamt Husum.
Wir verzeichnen derzeit im Landkreis Nordfriesland seit Anfang September 2005 eine ungewöhnliche Häufung von Masernerkrankungen. Wir möchten uns zunächst herzlich dafür bedanken, dass Sie uns
Ihren Patienten ___(Name des Falles wie in Kopfzeile)___, geboren am __(Datum wie in Kopfzeile)__,
als Masernfall gemeldet haben. Wir führen derzeit eine epidemiologische Untersuchung durch um diese Masernhäufung eindämmen zu können. Das Erfassen von Masernkomplikationen ist hier besonders wichtig. Um den von Ihnen gemeldeten Fall korrekt einordnen zu können, möchten wir noch kurz einzelne Informationen abklären. 1. Hatte Ihr Patient eines der mehrere der folgenden Symptome? Generalisierter Hautausschlag > 3 Tage: (1) Ja* (2) Nein (3) Unbekannt * Wenn ja: Wann begann der Hautauschlag?
(4) Datum genau bekannt: ___.___.2005 (5) Datum nicht genau bekannt: Auschlag begann zwischen ___.___.2005 und ___.___.2005. 2. Welche der folgenden Labornachweise liegen Ihnen zu diesem Patienten vor?
(Mehrfachnennungen möglich)
(1) Virusisolierung (2) Nukleinsäurenachweis (PCR) (3) IgM (4) IgG mit 4-fachem Titeranstieg (5) Antikörpernachweis mit 4-fachem Titeranstieg
3. War Ihr Patient gegen Masern geimpft? (bitte eine Angabe pro empfohlene Impfung)
Erste Masernimpfung:
Zweite Masernimpfung: Dritte Masernimpfung:
(1) Nein (2) Ja* (nicht dokumentiert) (3) Ja* (Impfausweis)
(1) Nein (2) Ja* (nicht dokumentiert) (3) Ja* (Impfausweis)
(1) Nein (2) Ja* (nicht dokumentiert) (3) Ja* (Impfausweis)
Basic Rules
• On first page– Return address– Study title
• On all pages– Identifyer– Page numbers
• Numbered questions• Instructions
(in bold or italic)
22
Structure of a questionnaire
1 Interview introduction
2 Identification
3 Instructions on how to answer
4 Questions
5 Conclusion
23
1 - Introduction
• Covering letter/ interview introduction
– Who are you / you work for
– Why are you investigating
– Where did you obtain the respondent’s name
– How and where can you be contacted
– Guarantee of confidentiality
– Length of interview (be honest)
⇒ Usefulness of study should be clear to all respondents
24
Example of introduction
• Good morning ,
My name is ..... , I work for ……..
You may have been already informed that a survey on risk factors for
being stung by a jellyfish will be done this week in Mahon. This study
has been approved by the national ethical committee. Only anonymous
data will be analysed. You have been randomly selected to participate
in this study. Your participation is volontary. The interview is about 10
minutes long.
I will start with the first question .....
25
2 - Identification
• Each questionnaire/interviewed person must be identified – Example : I_I I_I_I I_I_I
interviewer N° district N° person N°
• Demographic data : age, sex …
• Identifiers kept separately from names (anonymous)
26
3 - Instructions
• Guide for Interviewer/Respondent
• Give clear instructions about how to answer the
questions
– Use different fonts (e.g. italics)
• Give instruction which questions can be skipped
-> where to jump to
Example: I will ask you a few questions about your
relationship with jellyfish.
You will answer by yes or no
If no, go to question 27
27
4 - Questions
• Content
• Order
• Format
• Problems and pitfalls
• Coding
28
Content of questions
• Clear focus on research question
– avoid sidetracking
– avoid unnecessary information
29
Question order
• Decide on the order of items/questions – easy difficult– general particular– factual abstract
• Group questions by topic
• Be aware of ordering effects
• Don’t put the most important items last• Where to place sensitive questions?
30
Starting questions
– “Door-opener”– Simple– Closed format – Relevant to main subject– Non-offending– Neither demographic nor personal questions
31
Format of questions
• Adjust to responding audience– professionals vs. public– middle class vs. prisoners
• Keep sentences simple and short
• Define key words (“fully vaccinated”)• Remember options
– “don’t know” – “don’t want to answer”
32
Format of questions
7 - Do you like to go swimming and do you mind to
be stung by jellyfish?
Yes No
• Ask for one information at a time• Answer options
– Mutually exclusive and exhaustive– Vertical order of answer options
33
Be accurate8 – What is the jellyfish situation?
Good Bad
Versus
8 - How often did you see jellyfish during the last week?
Once Twice Three times or more Never Don´t know
34
Be appropriate
9 – Are jellyfish a useless nuisance? Yes No
versus
9 - How often have you been stung by jellyfish?
Once a day or more Once a week Less than once a week Don´t know
35
Be objective10 - Do you agree that jellyfish are annoying?
Yes No
versus
10 - According to you, jellyfish in Lazareto are… not at all a public health problem not a major public health problem a potential public health problem an important public health problem no opinion/don´t know
36
Be simple
11- Did you see more than an average of 33 jellyfish/m2 salt water surface on more than 3 occasions that you went swimming in the morning last week?
Yes No
versus
11- Have you seen jellyfish on more than 3 mornings last week?
Yes No Don´t know
37
Two main question formats
• Closed format forced choiceYes Always No Sometimes Don’t know Never
• Open format free text12 – What did you do to avoid being stung by jellyfish? Please describe : __________________________________________________________________________________
38
Advantages of open questions
• not directive • detailed and unexpected answers possible
– > allow exploration of issues to generate hypotheses
• useful for exploring knowledge and attitudes• qualitative research
• focus groups
• trawling questionnaires
39
Disadvantages of open questions
• interviewer bias
• time-consuming
• coding problems
• difficult to analyse
• difficult to compare groups
40
Advantages of closed questions
• Straightforward response• Simple • Quick • Less discrimination against less verbally
expressive people • Easy to code, record, analyse • Easy to compare • Easy to report
41
Disadvantages of closed questions
• restricted number of possible answers• Possible loss of additional information
Compromise– insert field
“others,
if yes specify : __________”
42
Examples for closed questions
2 - What is your age in years? ___ years
3 - How long have you stayed at Lazareto? ___ days
4 - What is your sex (gender)? Male Female
5 - Did you go swimming on 4 October 2010? Yes No Don’t know
43
14 - Which of the following beaches have you visited during your stay in Menorca?
Lazareto beach Cala’n Porter Rafalet Macarella Sa Mesquida
Closed questions: Checklist
44
14 - Which of the following beaches have you visited during your stay in Menorca?
Lazareto beach Yes No Don´t know
Cala’n Porter Yes No Don´t know
Rafalet Yes No Don´t know
Macarella Yes No Don´t know
Sa Mesquida Yes No Don´t know
Checklist
45
Rating scale
15 – How often did you see jellyfish during the past weeks?
Always Sometimes Seldom Never
Mornings Lunchtime Evenings
46
Rating scaleNumerical16 - How severe was your pain after you were
stung? (please circle)
1 2 3 4 5 6 7 Not painful at all Very painful
Analogue17- How severe is your pain (put the tick on the line)
0 10
47
Likert Scale• Rensis Likert, 1903-1981• Psychometric scale• Five (or more) ordered response levels
18 – Jellyfish also have the right to swim in the Mediterranean sea
I strongly disagree
I disagree
I neither agree or disagree
I agree
I strongly agree
48
Problems and pitfalls
• Avoid questions that ask two things at once - you won’t know which part people are answering:
19 - Have you seen or been stung by jellyfish?
• Ambiguity.....
20 - Do you swim a lot?
49
Problems and pitfalls• Avoid jargon/abbreviations/slang
21- Should jellyfish victims receive a PEP?
(post exposure prophylaxis)
• Avoid not mutually exclusive options
22 - What is your age ? 16-20 20-25 25-30 35-40
50
Problems and pitfalls
• Avoid leading questions
23 - Do you think we should forbid EPIET-fellows to go swimming in the mornings?
• Avoid making the questionnaire too long
• Typographical / spelling errors
51
Coding
• Answers can be pre-coded • Quicker and easier data entry
• Examples:
Male 1 Ill 1 Female 0 Not ill 0 Don’t know 9 Don’t know 9
Lazareto beach 1 Harbour 2 Es Castel 3Mahón 4
52
5 - Conclusion
• Don’t forget to thank the interviewed persons
• Tell them when the results will be available and where
53
Example of conclusion
This is the end of the interview. Thanks for answering this questionnaire. The result of this study will be available on the website epiet.org in six months.Any others questions you wish to ask ?
We are still in a very early phase of our investigation. Would you agree to be contacted again in case further questions arise?
In some outbreak investigations you might want to add:
54
Questionnaire validation
• Use or adapt existing questionnaires – validated (and possibly harmonised)
• New questionnaires– not validated – need to be tested (pilot)
55
Pilot your questionnaire!
• Pilot with a group of people
-> similar to your target subjects
• Highlight problems before starting– Misunderstandings– Look for alternative wording– Final polishing
56
Stages in designing questionnaire
Definition of objectives
List of variables to collect
Analysis plan
Study protocol
Design questions
Control and reading
Pilot test
Questionnaire
The question must answer your objectives
The question collects appropriate information for
analyses
57
Presentation and layout
• Clear and consistent • adequate space to answer
– large font size– appropriate page breaks
• avoid • messy layouts• too many and fancy logos
• printing questionnaire on coloured paper may help
58
Questionnaires for outbreak investigations
• Exploratory– Generate hypotheses– Detailed– Only some cases– Based on existing questionnaires– Open questions
• Analytical study– Testing hypothesis– Focus on possible vehicles
59
Summary
A well designed questionnaire:
• collects appropriate data
-> to answer your research question
• minimises potential sources of bias
-> increases the validity of the replies
• Will much more likely be completed
