Psychological Science on the Internet:

Designing Web-Based Experiments From the Ground Up

R. Chris FraleyUniversity of Illinois at Urbana-Champaign

Introduction

• During the last few years an increasing number of psychologists have begun to use the Internet to collect empirical data.

• The Internet has the potential to be useful for two reasons:

• Integrated data management– Whether you collect data from undergraduate subject

pools, community samples, or elsewhere, you can use the same platform for research design, collecting data, providing feedback, and managing data.

• Open laboratory– Participants can take part in your research any time of the

day.– If desired, you can open your lab doors to people from

across the world.

Our plan

• My objective today is to illustrate some of the research techniques that can be implemented using the Internet.

• Outline:

1. What kinds of research can be done using the Internet?2. How to obtain access to a web server.3. How to create a basic HTML web page that can be used to

collect data.4. How to create Perl/CGI scripts to process and save those

data.5. How to do advanced things, such as randomize

stimuli/questions, randomly assign people to conditions, and present items/stimuli on separate pages.

6. A discussion on ethics, sampling concerns, and informed consent.

Part 1Basic concepts and what can be done

How the Net Works

• Most web pages are written in a language called HTML (hypertext markup language).

• HTML files exist on a web server. When a person types in the URL for a specific file in his or her web browser, the person is essentially retrieving that pre-existing file from the web server.

How the Net Works

web server

locates requested HTML file and sends it to user’s computer

web user

types a URL into the browser

web user

the browser renders the HTML file as a web page

Static vs. Dynamic Web Pages

• The obvious limitation of using HTML pages in this manner is that the user’s web experience is static—he or she will see the same page each time he or she goes to that address.

• It is possible, however, to program the server to deliver dynamic content—content that varies from one situation to the next or from one person to the next.

• This kind of interactivity is necessary in order to produce the kinds of web pages that allow for creative, complex research designs.

Dynamic Web Programming and CGI

• This kind of interactivity can be accomplished by writing programs, called CGI scripts, that run on the web server.

• CGI scripts are often written in Perl, PHP, or other languages. (I’ll expand on some technical details later.) What is important to note right now is that those languages are available for free and, with a bit of practice, are relatively easy to use.

Dynamic Web Programming and CGI

web server

• locates requested CGI script.

• Performs the commands and creates a new HTML file.

• Sends HTML file to user’s computer

web user

types a URL into the browser

web user

the browser renders the HTML file as a web page

What Can Be Done with CGI Scripts?

• By using CGI scripts, it is possible to accomplish a number of feats that are impossible using standard web pages.

– save data– randomize the order of questions– random assignment to conditions– track participants over time– implement complex branching patterns– customize text and images– create customized feedback for users

• Let’s discuss some of these points in more detail.

Automatically Save Responses to a Data File

• One of most time-consuming aspects of conducting paper-and-pencil research is data entry.

• It is easy to write CGI scripts that save the data automatically.

– save as a comma-delimited text file (that can be easily imported into SPSS or MS Excel)

– save directly into a database (such as MySQL)

snapshot of a comma-delimited text file

Provide Participants with Customized Feedback

• One of the most valuable features of conducting research over the Internet is that it gives you a means to provide instant feedback to the research subject.

– plot the person’s scores– create bar charts or tables that summarize the data

(e.g., means, SDs, correlations) from the broader sample automatically

Substitution

• One of the perks of using dynamic programming is that it allows for substitution.

• In other words, you can take information that has been provided by the user (or computed by some other part of the script) and substitute that into the web page that is sent to the user.

My wife’s name has been substituted into the web page, making the questions tailored to me.

Randomization of Stimuli Order

• In most questionnaire-based research, it is not unusual for researchers to present questions in the same order for each research participant.

• In fact, many researchers who have placed their questionnaires on-line have followed this practice.

Randomization of Stimuli Order

• There are obvious limitations of using the same presentation order for stimuli.

• Most importantly, there may be systematic order effects that impact responding.

• By randomizing the order in which items/stimuli are presented, it is possible to minimize these kinds of problems.

• It is possible to randomize the order of stimuli quite easily using a CGI script.

Random Assignment to Conditions

• It is also possible to use these same randomization techniques to randomly assign participants to experimental conditions.

• Importantly, it is easy to implement this in a way that precludes research subjects from knowing that they are in but one of many possible cells of a research design.

• Example: • http://www.web-research-design.net/cgi-bin/APS

2006/experiment1.pl

Measure Response Times

• It is also possible to write CGI scripts that assess the amount of time that a user spends on a web page.

Measure Response Times

• It is not advisable to assess response times for decisions that can be made within a few seconds (e.g., lexical decision tasks) because it can take up to three seconds for a simple HTML page to fully load.

• However, by using multimedia applications, such as Macromedia’s Flash, it is possible to assess response times at a more microscopic level.

Measure Response Times

• Example lexical decision application

• http://www.web-research-design.net/APS2006/lexicaldecision.swf

Use Complex Branching Patterns

• One limitation of paper-and-pencil questionnaires is that it is difficult for participants to follow complex branching operations.

– There may be certain questions that should be skipped if the participant is not married, for example.

– If you want to tailor the stimuli to a specific user (e.g., a person’s ideographically unwanted traits), it is difficult to do so with a paper-based test.

• Again, CGI scripts can be used to implement these conditional patterns.

Track Participants Over Time

• It is often desirable to follow a group of people across time to study continuity and change in attitudes, moods, behavior, and traits.

• The Internet provides a useful way to track data across time because people can

– (a) log-on at their convenience– (b) all submissions can be time-stamped

automatically by the web server– (c) the person’s progress can be easily tracked

across multiple sessions

Learning How to Conduct Research over the Internet

• How do you do this stuff?!

Learning How to Conduct Research over the Internet

• The first thing to note is that these techniques are relatively easy to learn—even if you have very little experience in programming.

Learning How to Conduct Research over the Internet

• What you need:

– a computer with an Internet connection (dial-up modem or high-speed)

– a web server• Professional hosting sites (e.g. , Netfirms)• Create your own (e.g., spare PC, Apache

software)

– software that is freely available over the Internet• text editor (e.g., EditPlus)

Learning How to Conduct Research over the Internet

• There are a number of web sites available that provide tutorials on designing HTML and CGI files.

– HTML• http://www.w3schools.com/html/

– CGI• http://www.cgi101.com/class/• http://hotwired.lycos.com/webmonkey/98/47/index2a.html

– Both HTML and CGI• http://www.web-research-design.net/ • [Note: You can download a ton of free research-

relevant scripts at this site]

Learning How to Conduct Research over the Internet

• There are also some useful books.

Guilford Press Prentice Hall

Learning How to Conduct Research over the Internet

• Finally, there are a number of professional services for hire that can design web studies for you.

– Knowledge Networks• http://www.knowledgenetworks.com/

– Survey Monkey• http://www.surveymonkey.com/