- 1. Rebecca Brody, PhD, RD, CNSCAssistant Professor Graduate
Programsin Clinical Nutrition
2. Session Objectives1. Understand sources of error in survey
research2. Explore types of survey methodologies (web-based,
telephone, mail, mixed mode)3. Determine best practices for survey
design (who, what, where, when, and how) 3. Your readings for the
week are posted in thesession (under Lessons Table SurveyResearch)
along with your discussionquestions. Some questions are targeted to
studentsplanning to conduct survey research If you are conducting
survey research,Dillmans Internet, Mail and Mixed ModeSurveys The
Tailored Design Method, 2009by Don Dillman is a valuable resource
as ishis website: http://www.sesrc.wsu.edu/dillman/default.ASP 4.
DescriptiveExperimentalDescribe Explore cause and populations
effect ExploratoryFind relationships Surveys and
QuestionnairesPortney, 2009 5. WHO to survey WHAT to survey about
What questions will you ask? HOWto survey What method is best for
your sample interview, mail, internet, mixed modes? WHENto survey
Timing is critical need to consider your population (i.e. educators
may be off during the summer so not the best time to survey this
group) WHEREto survey 6. Coverage errorSampling errorNon-response
errorMeasurement error 7. Target PopulationCoverage ErrorSampling
FrameSampling ErrorSampleNon-response Error RespondentsMeasurement
Error Survey ResultsAdapted from Sampling Frames and Coverage
Errorwww.idready.org/courses/2005/spring/survey_SamplingFrames.pdf
8. The sampling frame excludes or erroneously includes a certain
segment of your population Omission = Undercoverage Duplication or
wrongful inclusion = Overcoverage Thesurvey method selected
minimizes the possibility of a certain group responding Web-based
survey sent to elderly subjects without computer skills Mail survey
sent to dietetic internship alumni (10 years post graduation) using
an address list from the year they graduated 9. The difference
between the sample and the targetpopulation due to characteristics
represented in thesample i.e. Sample of 100 RDs exploring personal
weight perception.Sixty RDs responding to the survey were obese. Is
thisrepresentative of the RD population? Can make a sample
unrepresentative of the targetpopulation Causes Chance Sampling
bias The tendency to favor selection of subjectswith certain
characteristics due to a poor sampling plan.
http://www.socialresearchmethods.net/tutorial/Mugo/tutorial.htm
andhttp://www.socialresearchmethods.net/kb/sampstat.php 10.
Responders to your survey are different than non-responders Types
of Non-Response Errors Complete: Survey fails to measure some of
thecharacteristics of the sample. Partial: Specific information is
not completed byrespondents. Must design & test surveys well to
reducepartial non-response errors Causes of Non-response error
Chance (random error) Respondent is unable or refuses to
participate in the survey Sampling bias Non-response bias check 11.
Dueto poor survey design and question construction Participant
misinterprets a question Unclear definitions or survey instructions
Poorly designed survey scales Example: Likert scale with 11
possible options and little differentiation between points on the
scale) Respondents tendency to exaggerate or underplay events
Report they are taller and thinner than actual Respondents answer
in a more socially desirable way Interviewer bias (too friendly,
not friendly enough) if delivering the survey in-person. Must be
trained! 12. Must consider: Do you want a generalizable sample or a
targeted sample Access to the sample Email list versus listserve
Cost to access sample Availability of the sample How representative
is the sample i.e. A list of Dietitians in Nutrition Support DPG
members may include clinicians, managers, entry level RDs, advanced
level RDs, specialists in adult, pediatric nutrition support, etc.
Do you want all of these characteristics represented or do you need
a more targeted sampling frame? 13. Interview In-person Telephone
Internet (Skype / Elluminate) Paper and Pen Mail Survey Electronic
SurveyWeb-based Email survey Mixed Mode Survey Combination of
methods (i.e. paper survey with optionfor web-based version) 14.
Advantages Disadvantages Limit labor required May not be most Easy
access to large effective way to sample reach certain Respondents
have populations time to consider Need accurate answersmailing
addresses High response rates Cost of supplies Time for
surveyreturnFowler, Survey Research Methods, 4th Ed 2009 15.
Advantages Disadvantages Easy to access large Coverage error
populations Sampling error Reduced cost Measurement error Reduced
time for data Non-response error entry Computer illiteracy Reduced
error in data High # of non- entry deliverable emails Less time to
conduct Computer security survey(fire walls, spam, etc) Flexibility
in design Response rates lower Ease of administrationthan mail
surveys Skip logic Hoonakker and Carayon, 2009 16. Avoid mailing
during holidays Thanksgiving and Christmas Select mail out dates
based on characteristics of your populations Educators may not be
available in the summer Families with children may vacation in the
summer Survey of high school coaches should be done during school
is in session 17. Letter of Introduction Purpose of survey Why
participant selected Who is conducting the research All of the
elements of consent For mail surveys Self addressed stamped
envelope included Coding of envelope to survey to participant Name
and address of where to return survey to within survey (in case
they loose the envelope) Dillmans Tailored Design Method 18.
Tailored design is the development of surveyprocedures that work
together to form thesurvey request and motivate various types
ofpeople to respond to the survey by establishing trust and
increasing the perceived benefits of completing the surveywhile
decreasing the expected costs of participation. (Dillman, 2009,
p38)Dillman D, Smyth JD, Melani Christian L. Internet, mail, and
mixed mode surveys.The tailored design method, 3rd Ed. John Wiley
& Sons, Inc. 2009. 19. Scientific approach designed to reduce:
Coverage, sampling, non-response, andmeasurement errors Customized
survey procedures based on: Survey type Sample Budget and time 20.
Implementation1. Brief pre-notice letter or e-mail sent a few days
before survey arrives2. Survey mailing Includes cover letter,
survey, stamped & addressedreturn envelope, and incentive (if
desired)3. Thank you postcard sent ~ 1 week later as a reminder to
complete survey4. Second survey mailing for non-responders 2-4
weeks after previous mailing5. Final contact in different delivery
mode 2-4 weeks after previous mailing (i.e. e-mail, certified mail)
21. To: [email protected]: Tuesday, October 16, 2007 4:01
PMSubject: Please take our surveyYou have been selected to
participate in a brief research survey being conducted as part of
aMasters thesis. I would greatly appreciate your input. Please
respond by the end of the month(10/31/07).Your responses to this
survey will be used to evaluate the current utilization and
implementation oftechnology, internet applications, computer
software/programs for education (Blackboard, WebCt),etc. in the
Didactic curriculum. All results will be used for research purposes
only and by submittingthis survey, you are agreeing to these
terms.Internet applications refer to Blogs, Web and Pod casts,
Online discussion groups, socialnetworking sites, interactive
multimedia, Wikis, website development and maintenance, etc.
thatare used for educational purposes.Thank you,xxxx, Graduate
student at University XClick on the following link to take the
survey:
