Survey Research: Design, Samples and Response Mayyada Wazaify,
PhD Reference: 1. Smith F. Survey Research: Design, Samples and
Response In: Research Methods in Pharmacy Practice. 1 st Ed,
2002:pp:1-31 2. The School of Pharmacy. Queens University of
Belfast. An Introduction to Research Methods in Pharmacy Practice.
2000 3. Abudahab S. Clinical Research Course/ Faculty of
Rehabilitation Sciences. University of Jordan. 2009
Slide 2
Slide 3
Slide 4
Slide 5
Survey verb (of a person or their eyes) look carefully and
thoroughly at (someone or something), especially so as to appraise
them. noun a general view, examination, or description of someone
or something.
Slide 6
Surveys are appropriate when the question requires obtaining
information FROM subjects, rather than measuring performance
Slide 7
Introduction What are surveys? A survey is composed of a series
of questions that are posed to a group of subjects, and may be
conducted as an oral interview or a written questionnaire In
clinical research, surveys are used to: Describe attitudes and
values Describe levels of knowledge or experience Describe current
practices Describe characteristics of specific groups Assess
outcomes related to function and quality of life (Standardized
questionnaires e.g. SF-36)
Slide 8
Response Rate Ideally 66% In interviews >> questionnaires
Drop and pick Q >> than postal Q Face-to-face interviews
>> telephone interviews Can be increased if: Questionnaire is
well-designed Enclose a friendly cover-letter/cover
memo/introduction (background of the survey, subjects help is vital
to the success of the project) or send an advance letter Enclose a
stamped envelope Explain how respondents have been selected
Slide 9
Slide 10
Interviews and Questionnaires Interviews Face-to face or over
the phone Face-to-face tends to be more effective Major
disadvantage: cost and time Structured vs. Semi structured vs.
In-depthSemi structured Structured: consists of standardized set of
questions that will be asked The three main pitfalls that can occur
during qualitative research work are: - equipment failure, -
environmental hazards and - transcription errors (Easton et al.,
2000).
Slide 11
Exercise-1: give examples on structured, semi-structured and
in- depth interviews relating to each of the following topic: -
Barriers and facilitators of attending CE courses - Identifying and
treating drug misusers and abusers in community pharmacy
Slide 12
Face-to-face interviews
Slide 13
Advantages Can provide explanations if required Better response
rate Can explore issue in-depth Gain trust and confidence
Opportunity to probe Disadvantages Questionnaire may influence the
interview Time consuming expensive
Slide 14
Interviews and Questionnaires Cont. Questionnaires Structured
surveys that are self-administered using pen and paper Advantages:
More efficient than interviews Data can be gathered from a large
sample over wide geographical distribution in relatively short time
Standardized everyone is exposed to the same questions in the same
way Provide anonymity Disadvantages: Misunderstanding or
misinterpreting questions or response choices Unknown accuracy or
motivation of respondent
Slide 15
Self-report Respondents report of their attitudes, knowledge,
practices, etc. Recall bias is of concern when the respondents are
asked to remember past events You need to understand your target
population and the respondents abilities to answer your questions
Do you know what is proxy report ?
Slide 16
Slide 17
Slide 18
Slide 19
Design of Surveys The research question: what do you want to
assess? Guiding questions Delineate what the researcher is trying
to find out Reflect descriptive interests or expected relationships
among variables How much do mothers of children with asthma know
about the asthma? How did mothers of children with asthma cope with
their children special needs?
Slide 20
Design of Surveys Cont. The research question Cont. Hypotheses
Mothers of children with asthma will demonstrate more knowledge
about asthma more than mothers who do not have any child with
asthma Questionnaire outline Listing of all items needed to answer
guiding questions and/or hypothesis Often more than one item is
needed to address each question and/or hypothesis Demographic
information (income, ethnicity, marital status)
Slide 21
Design of Surveys Cont. Review of existing instruments Apply or
adapt already existing instruments Designing the instrument Write
all the questions that reflect the purpose of the study Group and
organize questions to reflect each category or topic Watch the flow
of your questions Questions should proceed from general to specific
Less sensitive questions should precede more sensitive ones Make it
as friendly as possible, the less complicated the more responses
you will get
Slide 22
Design of Surveys Cont. Preliminary drafts Panel of colleagues
for review (Face Validity) Constructive criticism Ongoing process
may take several times Pilot testing and revisions Small
representative sample (5-10 individuals) Interview respondents to
determine where questions were unclear or misleading Monitor
missing questions and inconsistencies Monitor time Length of
questionnaire In case of interviews --- the interviewers should be
trained on how they present the survey, how questions are asked,
and how probing follow-up questions are used (if they are to be
allowed)
Slide 23
Design of Surveys Cont. Administration of the Survey As much as
possible, a probability sample should be selected For interview
surveys limited within a local geographic area Introduce the Survey
include a Cover letter The purpose and importance of the survey
Indicate why the respondents has been chosen for the survey
Confidentiality How will the data be used How long it will take
Thank the respondents for their participation Sign the letter NO
longer than one page
Slide 24
Slide 25
Slide 26
Constructing Survey Questions Open-Ended: Useful for probing
respondents feeling and opinions, without biases and limits from
the researcher Advantages: Allows a wide variety of responses Good
for exploring a topic Does not superimpose answers and expectations
Disadvantage: Can be difficult to summaries, code and analyse
Unpopular if included in a self-administered Q Time-consuming to
answer
Slide 27
Constructing Survey Questions Closed-ended Identification
questions Selection questions Yes/No questions Advantages: Easier
to analyze Good if questionnaire is long Better is motivation of
respondent is low Does not descriminate against less articulate
subjects Disadvantages: Narrows the answer Restricts spontaneity of
respondents Good design of question is important to avoid ambiguity
e.g. Two-way question Likert scale Semantic differential scale
Check-lists Ranking
Slide 28
Examples: Close-ended questions Format For questions that in on
a continuum, provide range of responses (3-5) How important do you
think it is to have an orientation program for first-year students?
a- Very important b- Important c- somewhat important d- Not
important e. Unsure
Slide 29
Constructing Survey Questions Cont. Close-ended questions Cont.
Format Cont. When a series of questions use the same format, a grid
or checklist can provide a more efficient presentation. For each of
the following questions, please indicate your level of satisfaction
in the first-year students orientation program: Not satisfied
Somewhat satisfied SatisfiedHighly satisfied Unsure Topics
discussed Time of the orientation Method of presentation
Slide 30
Constructing Survey Questions Cont. Close-ended questions Cont.
Format Cont. Rank-order questions. The following are the topics
discussed in the orientation. Please order them in terms of
importance to you, from 1 (most important) to 5 (least important)
______ Using the library ______ Dinning options on campus ______
Sports centers and sports activities on campus ______ Campus
security ______ Time management
Slide 31
Constructing Survey Questions Cont. Close-ended questions Cont.
Format Cont. Branching 1- Did you attend the opening ceremony of
the orientation? a. No Skip to Question 4 b. Yes 2- Do you believe
the ceremony was well-organized? a. No b. Yes Skip to Question 4 3-
what are the reasons that made you feel that the ceremony was not
organized? Please list as much as possible
Slide 32
Constructing Survey Questions Leading questions They are worded
so that they are not neutral; they suggest what answer should be
given and indicate the questioners own point of view. In
questionnaires, leading questions must be avoided as they will bias
the responder to give the answer that the questioner wants. e.g. As
a caring parent, would you not agree that you should provide your
family with a private health care? YES.. NO
Slide 33
Wording of Questions Simple, Simple, Simple Sentences should be
concise and grammatically correct Common language for the lowest
educational level that might be encountered The shorter the better
BUT keep in mind clarity Avoid idioms or sublet cultural
expressions / Avoid Jargon The Response Rate (e.g. not too many
open-ended) Provide definitions Avoid double-barrelled or double
negative questions
Slide 34
Scales Purpose: to distinguish among people who demonstrate
different intensities of the characteristics that is being measured
Types Categorical: based on nominal measurements Used for variables
such as gender, ethnicity, religion, etc. Continuous: represent
characteristics that exist on a continuum Ordinal: variables are
collapsed into ranks (mild, moderate, severe) Summative: presents a
total score with all items contributing equal weight to total
Cumulative: demonstrate an accumulated characteristic, with items
representing an increasing amount of the attribute being
measured
Slide 35
Scales Cont. Likert Scale Summative scale Most often used to
assess attitudes or values Includes 5 categories: Strongly Agree,
Agree, Neutral, Disagree, Strongly Disagree Modified to have 7
categories: Somewhat Agree, Somewhat Disagree. Or 4 categories:
eliminating Neutral Each choice along the scale is assigned a point
value Actual values are not important as long as there is
consistency Positive items are coded 5 through 1 while negative
items are coded 1 through 5 Overall score is computed by adding
points for each item Items are measuring the same thing Each item
reflects EQUAL element of the characteristic being studied
Slide 36
Scales Cont. Semantic Differential Measure the individual
feelings about a particular object or concept based on a continuum
that extends between two extreme opposites Pairs of words that
reflect opposite feelings 7-point scale is used; middle
representing neutral position Different from Likert scale: 2
extremes are labeled Continuum is not based on agree/disagree
Assessed by assigning a value from 1-7 to each of the spaces within
each adjective pair (1 negative and 7 positive) Alternate the
position of negative and positive values Slow Fast
Slide 37
Slide 38
Slide 39
Slide 40
Scales Cont. Visual Analogue Scale (VAS) Simplest method to
assess the intensity of a subjective experience A 100mm line is
usually drawn, with word anchors on either end that represent
extreme of a characteristic Intermediate levels along the line are
NOT defined Respondents are asked to mark along the line
corresponding to their perceived level for that characteristic VAS
measures the distance of the mark from the left hand anchor in
millimeters No Pain Pain as bad as it can be
Slide 41
Scales Cont. Guttman Scales Cumulative scale Presents an
increasing intensities of the characteristic being measured One
dimension within a set of responses If we have 6 items that
represent the same dimension, the respondent who agrees with item 2
will also have agreed with item 1. Only ONE way to achieve the same
score in a Guttman score Each item is scored as 1 = Agree or 0 =
disagree 1- I believe that this country should allow more
immigrants in. 2- I would be comfortable with new immigrants moving
into my community. 3- It would be fine with me if new immigrants
moved onto my block. 4- I would be comfortable if a new immigrant
moved next door to me. 5- I would be comfortable if my child went
to school with a new immigrant. 6- I would permit a child of mine
to marry an immigrant.
Slide 42
Analysis of Survey Data Collate responses and enter then into a
computer Each item on the survey is a data point, and must be given
a variable name (usually item number) Researcher should go through
each survey, making sure that the respondents has correctly filled
out the survey Code closed ended questions Descriptive statistics
are used to summarize responses Quantitative data: averages and SD
Categorical: Percentages Should always be reflected to the total
sample size Cross-tabulation Chi-square analysis
Slide 43
Slide 44
Slide 45
Slide 46
Exercise-2: How to code open-ended questions in surveys?
Slide 47
e.g. How would you like the service provided by your community
pharmacy improved? 1. Give your customers somewhere to sit 2.
Provide a quiet area to discuss health issues privately 3. Lower
prices 4. More opportunity to talk with pharmacist 5. Faster
dispensing of medicines 6. Less out of stock medicines which we
have to call back for 7. More counter staff 8. Wider aisles to
allow prams 9. Friendly, more helpful staff 10. Wider product range
Cluster the responses into the following 4 categories: Improved
dispensing service 4,5,6 Improved shop design 1,2,8. Increased
product range/Lower price 3,10. Better trained staff 7,9. Consider
the following answers which were obtained from the above survey
open-ended question.
Slide 48
Informed Consent Obtain a consent before you collect any
dataconsent
Slide 49
Exercise-3: In surveys related to events, collecting data can
be conducted by 2 methods: self-completion by participants (e.g.
diaries) or by the researcher (e.g. direct observation of the
practice setting or retrospective analysis of records). List down
the main advantages and disadvantages of each method