Embed Size (px)
Citation preview
Collecting Information via the Web
Stephen PorterDirector of Institutional Research
Michael RoyDirector of Academic Computing Services
Wesleyan University
Advantages of web surveys
Low costs - only software, no printing, envelopes or postage needed.
No data entry costs. Minimal data entry errors. Easy to correct problems during survey
administration. Much quicker completion time for project. Response rates are comparable.
Population It’s important to keep in mind your survey
population. Do you have good email addresses for sample? Does everyone use email software that allows
hyperlinking? Do some people not use email at all?
Students (especially grad) may not use institution assigned emails. Do you collect home email addresses?
Some low-tech faculty don’t use email or check infrequently.
Security and web surveys
Security can be a big issue – you have two choices:
Ask for name and other identifying information such as userid and password. Multiple responses can be eliminated. Responses can be linked to institutional data. No evidence this affects response rates, but be
aware many people are nervous using SSN’s on the web.
Important to state that only aggregate data will be released.
Security and web surveys
Allow anonymous responses. In theory, anyone can answer the survey. Multiple responses will vary with topic and
survey length. Cannot use incentives. Research indicates that promises of
anonymity and/or confidentiality do not affect response rates except for sensitive topics such as sexual behavior.
Security and web surveys
Sending out id numbers and passwords can be cumbersome.
It’s possible to embed an id number and password in a unique hyperlink that is emailed to each survey participant.
This approach has worked well for us, but make sure participants know that the survey is not anonymous.
Survey structure
Keep it simple: avoid complicated designs with lots of colors.
Make the survey one (screen) page in length. For drop down boxes, make sure first response
category is not the visible line of the box. Keep the survey as short as possible. See Dillman (2000) pp. 352-401 for a good set
of detailed recommendations.
Survey administration
The Dillman (2000) “tailored design” approach Pre-notice, survey, reminder, follow-up survey,
and so on. For web surveys it is simply multiple emails. Considered the best method possible for
surveying - used for example by the Census Bureau.
If you do only one survey mailing, you WILL get a low response rate.
With multiple emails you need to take out respondents from your email list.
Survey administration
Response Rate for 2002 Senior Survey (Web Version)
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
4/8/02
4/9/02
4/10/02
4/11/02
4/12/02
4/13/02
4/14/02
4/15/02
4/16/02
4/17/02
4/18/02
4/19/02
4/20/02
4/21/02
4/22/02
4/23/02
4/24/02
4/25/02
4/26/02
4/27/02
4/28/02
4/29/02
4/30/02
5/1/02
5/2/02
5/3/02
5/4/02
5/5/02
5/6/02
Cu
mu
lati
ve r
esp
onse
rat
e as
of
mid
nig
ht
Initialemail
First reminderemail
Second reminderemail
Third reminderemail
Survey administration
The email should be concise, with hyperlink to survey visible when participant opens the email.
Avoid sending emails on Monday. We usually wait a week between reminder
emails. How many? 2 or 3 reminders are fine,
especially if you allow participant to opt out. Offering a web option via “snail mail” does not
seem to work.
For more information
Dillman, Don A. (2000) Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons.
National Computer Systems, Inc. (NCS) has a great website with a sample size calculator, survey research glossary and a set of short reports on various survey research topics:
http://www.ncs.com/research-notes/ Talk with your institutional research or
assessment office – no need to reinvent the wheel.
