A low-cost follow-up resource: using the World Wide Web to maximize client location efforts

A Low-Cost Follow-Up Resource: Using theWorld Wide Web to Maximize ClientLocation Efforts

LORA L. PASSETTI, SUSAN H. GODLEY, CHRISTY K. SCOTT,AND MELISSA SIEKMANN

ABSTRACT

Many evaluations depend on follow-up interviews, and low attrition rates can help strengtheninternal and external validity. The process of locating participants can consume significantevaluation resources. The World Wide Web is an inexpensive tracking tool offering cost-effective and time-saving access to several information networks and agencies that may provideinformation on program participants. The use of the World Wide Web as a tracking tool isillustrated by describing its application during follow-up work in the evaluation of a substanceabuse treatment program for pregnant and postpartum women. This article serves as a guide tolocation-related links and their potential applications with a variety of participant groups.

INTRODUCTION

In longitudinal evaluations involving interviews with participants, one of the most criticalsteps is locating program participants at designated times for follow-up assessment. Withoutadequate follow-up, evaluators cannot measure long-term outcomes, treatment effectiveness,and change over time. High attrition rates at follow-up can threaten the internal validity ofa study and also can compromise generalizability of results due to nonresponse bias (Cottler,

Lora L. Passetti ● Lighthouse Institute, Chestnut Health Systems, 720 W. Chestnut, Bloomington, IL 61701; Tel: (309)827-6026; Fax: (309) 829-4661; E-mail: [email protected]. The project that implemented these location methods wasfunded by the Center for Substance Abuse Treatment (CSAT) of the Substance Abuse and Mental Health ServicesAdministration (SAMHSA) through the U.S. Department of Health and Human Services (DHHS). The authors thank theagency administrators, evaluation and project staff, interviewers, trackers, and clients who made follow-up possible. We alsowould like to acknowledge the advice, support, and work of Mike Boyle and Deborah Oberg of Fayette Companies andMelinda Ferguson and Angela Carey of CHS. The opinions expressed in this document are solely those of the authors anddo not represent official positions of CSAT or any other governmental agencies.

American Journal of Evaluation, Vol. 21, No. 2, 2000, pp. 195–203. All rights of reproduction in any form reserved.ISSN: 1098-2140 Copyright © 2000 by American Evaluation Association.

195

Compton, Ben-Abdallah, Horne, & Claverie, 1996; Desmond, Maddux, Johnson, & Confer,1995; LaPorte, McLellan, Erdlen, & Parente, 1981; Ribisl, Walton, Mowbray, Luke, David-son II, & Bootsmiller, 1996; Wright, Allen, & Devine, 1995). Although no absolutelynecessary follow-up percentage rate exists, the best plan is generally to conduct as manyinterviews as possible (LaPorte et al., 1981).

One common barrier to high follow-up rates is the difficulty of locating people who haveagreed to participate in follow-up interviews (Desmond et al., 1995; Ribisl et al., 1996;Streissguth & Giunta, 1992). Maintaining contact with participants in longitudinal studiespresents several challenges to evaluators, especially when participants abuse substances,suffer from mental illness, or become involved with the criminal justice system. Studyparticipants can be difficult to contact due to high residential mobility, homelessness, death,disability, illegal activities, incarceration, residence in mental health or substance abusetreatment agencies, lack of telephone or driver’s license, illiteracy, aliases, name changes,false locator data, and unreliable information from contacts (Ribisl et al., 1996; Ziek,Beardsley, Deren, & Tortu, 1996). Location work can become frustrating, time-consuming,costly, and of limited productivity.

Effective location methods, creativity, and persistence can improve follow-up partici-pation rates. Traditional methods for tracking study participants include obtaining detailedlocating information during recruitment, mailing fliers and correspondence to updatedaddresses, searching treatment centers or the criminal justice system, accessing agencyinformation (e.g., driver’s license facilities or Social Security Administration), searchingtelephone directories, calling family and friends, and making field visits until the individualis found or considered unreachable (Cottler et al., 1996; Desmond et al., 1995; LaPorte et al.,1981; Ribisl et al., 1996). Many methods are costly, with resources needed to coverpersonnel, travel, phone bills, postage, and incentives. When multiple strategies are at-tempted with one participant, significant resources may be required to obtain high follow-uprates on even small sample sizes.

The purpose of this article is to provide information about the World Wide Web(WWW) as an inexpensive tool for supplementing existent participant location methods,making the overall follow-up initiative more efficient. This resource is not addressed in mostarticles and manuals on tracking participants. In the remainder of this paper, we discuss theadvantages and limitations of using the WWW, describe WWW tracking methods, andpresent one case study.

MAXIMIZING THE WWW AS A LOCATION TOOL

For tracking work, the WWW offers advantages to the enterprising and resourceful user. Itprovides easy access to certain location-related information networks and to contact pointsfor various agencies. Several databases can be searched for free as often as needed. Somelinks mentioned in the next sections will contribute to finding particular study participantsmore than others. Relevant links will be determined by participant characteristics for a givenproject. The following discussion describes various web-based services and their possibleapplications for tracking a variety of groups.

Links to all sites mentioned in this paper and additional sites offering similar servicesare accessible through the “Research Tools” area of Chestnut Health Systems’ web page athttp://www.chestnut.org/LI/downloads/link_lists/fluplink.html. Sites were chosen, first, on

196 AMERICAN JOURNAL OF EVALUATION, 21(2), 2000

their frequency of use, accuracy, or perceived helpfulness during the study described laterand, second, to provide users with a variety of links to choose from when deciding whichones suit their needs. This page will be maintained to update addresses, add links, and reflectfurther input from evaluators who use this site.

Residential Telephone Number and Address Directories

Using these directories is similar to looking through printed phone books, searchingcomputerized phone disks, or calling directory assistance. When a person’s name andpresumed city and/or state of residence are entered, the telephone number and address of apotentially matching record are provided. A related search may produce phone numberslisted to individuals with the same last name in a specified area, which may be useful inlocating relatives for whom participants remembered only a city of residence but not thespecific name under which the phone was listed. One search on the Internet could yield thesame data as three calls to directory assistance.

Web sites that offer these services areBigfoot (bigfoot.com/), WhoWhere? (www.whowhere.lycos.com/wwphone/phone.html), andWorld Pages (www.worldpages.com/whitepages/). The “Help” or “Frequently Asked Questions” sections of these sites mayprovide tips for finding people with common names or for narrowing and broadeningsearches (Borasky, 1998). For example,Yahoo! People Search(people.yahoo.com),Switch-board (www.switchboard.com/bin/cgiqa.dll?MG5), and Infospace(in-132.infospace.com/_1_107051060_info/reverse.htm) lend help in refining searches or will describe who may notbe found in their databases.

Beyond this application, the WWW allows users to search for someone based on anaddress without a phone number or a phone number without an address or name. Sites suchas Infospaceoffer reverse look-ups, similar to criss-cross directories, where a tracker canenter information obtained and try to trace it. One example might involve a collateral who didnot know a participant’s most recent phone number but provided an address. Entering thataddress at a web site may yield a matching phone number.

As many trackers realize, much of this process assumes that phones are listed inparticipants’ names. When working with women, adolescents, substance abusers, the men-tally ill, or homeless individuals, this often is not the case. These situations require creativityfrom research staff who may need to trace family members, guardians, friends, significantothers, or anyone else who participants have given trackers permission to contact.

Trackers need to be aware that unlisted phone numbers cannot be accessed nor canphone and address directories keep up with someone who moves frequently. Databases maybe updated every few months, yearly, or at random times depending on the organizations thatmaintain them. Information can be outdated (Borasky, 1998), though a thorough review ofwhite pages sites will often reveal how frequently a database is updated.

Realistically, the WWW may often be more useful for tracking people who knowparticipants than the participants themselves. This approach may seem counterintuitive, butmuch location work involves letters and calls to relatives, friends, and agency staff whomparticipants have listed as contacts for a new address. Frequently, these collaterals havephones that show up on searches, and if participant-provided contact information is incom-plete, a name search might turn up leads. Especially when participants have unstable livingarrangements, social networks can play an important role in location efforts (Ziek et al.,1996).

197Low-Cost Follow-Up Resource

Telephone and Address Directory Aids

Sites in this category may complete partial phone numbers or rectify errors in mailingaddresses for study participants. Area codes can be matched with towns and states.Fast AreaCode Look-Up(www.555-1212.com/area_codes.cfm) is one site that provides this informa-tion. Similar procedures can identify the proper zip code for an address or discover whatgeographic location it covers. This feature is helpful for letters returned with the designation“Incorrect Zip Code.” TheUnited States Postal Service ZIP14 Code Lookupweb site(www.usps.gov/ncsc/lookups/lookups.htm) is an excellent resource for clarifying zip codes.

Another service of the WWW can help reveal why a letter is returned stamped“Insufficient” or “Incorrect Address.” TheNational Address and Zip14 Browser(www.semaphorecorp.com/cgi/form.html) allows trackers to view all addresses near a target,to track whether or not the address goes to an apartment building, and to look at cities andstreets with similar names. Something as simple as a missing apartment number or misnamedstreet could be the reason for returned mail.

Yellow Pages and Government Agencies

Study participants may maintain contact with local or government agencies, such aschild welfare, homeless shelters, mental health centers, or the Department of Public Aid.Participant birth, marriage, divorce, and death records are kept by other organizations. Someweb sites have compiled links to such agencies, which typically can be searched by state andcounty. Sites may list an organization contact, and if applicable, the cost of requestinginformation. Family Treemaker (www.familytreemaker.com/00000229.html),InfospaceGovernment Listings(www.infospace.com/_1_4JO9TJ1032FL8Y9_info/index_govctr.htm),and United States Vital Records Information(vitalrec.com/index.html) are helpful startingpoints to locate links.

Selected sites, like Social Security Death Index(www.ancestry.com/ssdi/advanced.htm), provide direct access to data. Users need to review any web site carefully tounderstand why individuals are included or excluded from an agency’s database. Forexample, theSocial Security Death Indexwill not include people who never claimed theirdeath benefits from the Social Security Administration or people whose paperwork containederrors during processing.

Another method of finding agency contact information is through yellow pages sitesarranged by organization name, location, or category.Infospace Yellow Pages(www.infospace.com/_1_4JO9TJ1032FL8Y9_info/index_y/w.htm) andInfoUSA—Nationwide Yel-low Page Search(www.lookupusa.com/) allow yellow page searches. Some services, in-cludingYahoo! People SearchandWho Where?, offer white and yellow pages searches fromthe same site. Quick access to these numbers can be helpful to trackers who cover a widegeographic area and do not possess all needed phone books.

Trackers should be aware that some agencies found on the web charge fees for theirdata, but it is advisable to determine whether other sites share the same information at nocost. Not all records, such as driver’s license information, can be obtained through theInternet. In those cases, the project budget dictates whether or not money is available to payfor information.


Criminal Justice System

Although links to correctional facilities are not available everywhere, a few state prisonsallow WWW users to search inmate rosters. In Illinois, the database can be searched by birthdate, last name, or inmate number. Because this data set is public information, it is possibleto identify which prisons participants reside in, whether they are on parole, the charges forwhich they are serving, and their expected dates of release and discharge. Physical descrip-tions and county of arrest are provided. This search capability is allowed in a limited numberof states, but it may be worth periodic checking to see if a state adds this feature to its website. Corrections Connection(www.corrections.com/andmore/state.html) contains links tostate prison web sites and may be checked with theFederal Bureau of Prisons—InmateInformation (www.bop.gov/inmate.html) page for contact information and Internet-accessible inmate data.

Media

Local newspapers may provide participant information, particularly if deaths, criminalactivity, major accidents, or lawsuits are involved. Depending on the media agency’s website, trackers may search newspapers on-line for noteworthy events. A name can be enteredfor a match in stories or classifieds where certain criminal actions, legal activity, or deathnotices are reported. This information can guide trackers to jails, prisons, hospitals, or deathindexes.AJR News Link(ajr.newslink.org/) is one starting point for links to newspapersaround the country.

One limitation of media links is that not all newspapers are accessible via the WWW,and not all on-line newspapers have searchable archives. If a newspaper lacks searchablearchives and the exact date an incident occurred is unknown, the only way to locate articleswould be to browse old issues. A second limit is that some newspapers charge fees to accesstheir database or to request an article once it has been identified (Borasky, 1998). Usersshould read all conditions stated on the web site.

Mapping Addresses

A unique feature of the WWW is its ability to produce maps of any address in the UnitedStates for use during home visits. No matter what the location, a map with “zoom” functionscan be obtained. Most sites, such asMap Blast!(www.mapblast.com/mblast/index.mb) andWorld Pages(www.worldpages.com/maps/), allow the map’s center to be shifted.

Some issues to consider when using on-line maps is that one-way streets are often notindicated, and it may be wise to compare the same map produced by different sites to checkfor inaccuracies. Additionally, maps from WWW sites offer a picture of a circumscribedarea, but the more detail requested in the map, the less area can be viewed at once.

WWW Search Engines

To identify sites relevant to a study, evaluators can use search engines. These enginesare designed to search the WWW by keyword, and results can be refined until the desired webpage is found (if it exists). For example, if trackers wish to find the phone number of the jailserving Rockford, Illinois, but do not know the name of the county, they would want a web


page that matches Illinois cities with their counties. In this case, trackers would most likelyenter “Illinois and County and City and Search” as keywords at a site such as Go.com(www.go.com).

Users need to enter several terms or utilize several search engines routinely beforediscovering the most appropriate links. The WWW is in a state of constant growth andchange. No engine retrieves every web page (Brake, 1997), and different engines point todifferent sites when identical search terms are entered (Borasky, 1998). In fact, studiesdemonstrate that no search engine indexes more than approximately one-third of the index-able WWW (Lawrence & Giles, 1998). The more engines are searched, the more uniquedocuments are returned.

According to Lawrence and Giles (1998),HotBot(www.hotbot.com/), followed byAltaVista (www.altavista.com/), covers the highest percentage of estimated web pages in exis-tence (34% and 28%, respectively), whileLycos (www.lycos.com/) covers approximately3%. Other sites, likeMetaCrawler (www.metacrawler.com/index.html), allow the user tosearch multiple engines concurrently. However, in a review of WWW search engineevaluation studies, Chu and Rosenthal (1996) found that results conflict over which enginescover more pages and search more precisely. One engine may rate highly in one area ofperformance, such as ease of use, while rating lower in another area, such as WWWcoverage. Selecting a search engine to use depends on user preference, on which enginesperform better on desired types of searches, and on which aspects of performance are mostimportant for locating sites useful to a particular sample.

Another issue involves the large quantities of pages that may be reported in response toa query. Some documents could be reported because the keyword appeared in a contextirrelevant to the intended search. Other unwanted pages will appear because some enginesrequire partial matches to keywords (Lawrence & Giles, 1998). Users often sift throughmultiple pages of search results to find what appear to be the most useful links (Borasky,1998; Kolata, 1998). Many engines, such asAlta Vista, Infoseek, Metacrawler, andNorthernLight (www.northernlight.com/), advise on refining searches. For links to the sites above andothers, refer to Chestnut Health Systems’ web page at www.chestnut.org/LI/downloads/index.html.

WWW USE FOR TRACKING IN AN EVALUATION STUDY

Search procedures described in this article were used to help locate pregnant and postpartumwomen participating in a residential substance abuse treatment demonstration project. Theproject required six-month post-discharge follow-up for a cross-site evaluation. At projectadmission, all participants signed an informed consent that allowed collection of locator data.Permission was given for trackers to contact the collaterals listed on a “Future ContactForm,” as well as hospitals, shelters, and treatment agencies if participants could not bereached at home addresses for the follow-up interview. The consent form stated thatcollaterals would be asked for an address and phone number and would be told only that thewoman was participating in a “health study.”

Women admitted to the project (n5 387) resided in approximately twelve Illinoiscounties, scattered throughout the state and covering urban and rural areas. Of the 387participants, 63% were African American and 36% were White; 89% were unemployed atadmission to treatment; and 69% had never been married. The majority (84%) received less


than $500 income per month, and 55% of women reported attaining less than a 12th-gradeeducation. A large geographic catchment area, participants’ substance abuse, unstable livingarrangements, and the possibility of surname changes were some challenges confronting thefollow-up team.

Tracking staff consisted of two part-time research team members in a central Illinoisoffice and two contractual outreach workers, one in central and the other in northwesternIllinois. The two office workers performed the bulk of searching through mail, telephone, andthe WWW, whereas the outreach workers conducted field visits for hard-to-reach participantsin their cities.

Over the course of 32 months in this ongoing project, 222 women were due forfollow-up. Other women admitted to the project were still in treatment, had not reached thesix-month anniversary of their discharge date, or were participants in the project beforefollow-up was established. Of the 222 women due for follow-up, the participant location ratewas 93% (i.e., 206). Interviews were successfully completed with 188 out of the 222 women(85%). In addition to the 16 women who could not be located, one woman refused to beinterviewed, one woman had died, and 16 women were located but did not keep appointmentsor return calls. Most interviews (68%) were administered over the telephone; the remaining32% were completed in-person.

Documenting the effectiveness of the WWW is difficult within the context of this studyfor several reasons and is an avenue for further research. Calculating location rates ofparticipants with and without the aid of the WWW is impossible since few participants weretracked before the Internet was implemented as a resource. Confounding the issue, manytracking strategies were used in conjunction with the web to locate participants. The WWWprovided valuable information without locating participants—for example, by showingparticipants were not in prison or by providing phone numbers of all homeless shelters in anarea. A record review of a subsample of 50 participants showed that use of the WWW wasrecorded in 29 or 58% of cases; WWW assistance was not needed for the remaining 21participants because they had not moved or were located easily through the contact infor-mation provided at intake into the study.

For those cases where the WWW was utilized, trackers viewed the web as a convenientresource for white and yellow pages information, for zip code and postal service information,and for mapping addresses over a large geographical area. It proved important to locationwork even when it provided collateral data rather than directly locating participants. Withinthe limitations noted, use of the WWW as a locating tool was a welcome supplement tofollow-up efforts.

CASE STUDY

One case study from the evaluation project in which these techniques were used illustratesthe utility of the WWW in helping to find participants. Identifying information and detailshave been altered or removed to maintain confidentiality.

Martha

Martha was an African American mother of six in her late 20s diagnosed with cocainedependence. Upon admission to the study, she granted follow-up trackers permission to


contact her mother, her child protective services caseworker, and hospitals, shelters, ortreatment organizations for location information. A staff member used the WWW to verifyher mother’s phone and address by typing in her phone number at thePC411site.

After remaining in residential substance abuse treatment for approximately three weeks,Martha left against staff advice. At that time, she lived in an urban area with a domesticpartner and her two youngest children. She did not have a phone. Within two months afterdischarge, Martha called evaluation staff to update her address and phone number after oneletter and one phone message left with her mother. She eventually returned a pre-paidpostcard confirming her six-month follow-up interview appointment and was sent a reminderletter one week in advance of the agreed-upon date. When a staff member called her mother’shouse to conduct the interview, her mother stated that Martha was admitted into the IntensiveCare Unit of the hospital the previous day due to domestic violence injuries.

Two weeks later, her mother reported that Martha went to a safe house after her releasefrom the hospital. She agreed to pass along a message, but did not report the name of theshelter. The tracker searchedAmerican Yellow Pagesfor battered women’s shelters inMartha’s city and surrounding areas. Three agencies were contacted, all of which allowedmessages to be posted for residents without confirming or denying her presence. Follow-upstaff asked only to leave a note for Martha containing the project’s toll-free phone number.No response was received.

The tracker then accessedAmerican Yellow Pagesto find the phone number for childprotective services in Martha’s county and contacted her case worker. The case workerrevealed that Martha admitted herself recently to residential substance abuse treatment in aneighboring town, and the phone number was obtained from the same web site. The treatmentcenter was contacted, and the interview was completed over the phone.

LIMITATIONS OF THE WWW AS A TRACKING TOOL

Every tracking method has limitations, and the WWW is no exception. Users must evaluatewhich sites provide the best information for their sample because this will vary from projectto project, state to state, and agency to agency. For tips on how to evaluate the quality of aweb site and its content, refer toEvaluating Quality on the Net(www.tiac.net/users/hope/findqual.html),EvaluatingWebResources(www2.widener.edu/Wolfgram-Memorial-Library/webevaluation/webeval.htm), orThinking Critically about World Wide Web Resources(www.library.ucla.edu/libraries/college/instruct/web/critical.htm).

One limitation inherent in the World Wide Web is that web addresses and sites do notremain constant. A page may exist one day but not the next, and web addresses may changewithout notice (Brake, 1997). Links to revised addresses are sometimes given for a time, butthere is no guarantee that a web address will be current. Consequently, trackers may need tosearch for web resources that change their addresses or lose a resource completely.

Additionally, the WWW is a relatively new strategy for supplementing the location ofparticipants. It can occasionally be used successfully alone but is most often effective whenused in combination with other tracking procedures. As with traditional tracking methods, nosingle technique can be used to find every participant or collateral, and one strategy is rarelyused by itself. For example, it is usually not sufficient to identify a collateral’s new phonenumber through a telephone directory. A phone call to the collateral will most likely befollowed by calls to a participant’s new phone number, mailings, and/or field visits. Many


participants require repeated contacts via different methods (Ziek et al., 1996). Even thoughno strategy works for every participant, almost all strategies work for a subset of the sample(Wright et al., 1995).

CONCLUSION

The World Wide Web can supplement and complement other location methods in additionto potentially saving time and money. When trackers require resources other than the initiallocating information provided by participants, the WWW and a little creativity might provideuseful clues. Knowing how to search the web to help locate study participants is an importanttool for enhancing evaluation practice. Avenues for future research include collectingevidence about the most and least useful web sites for locating participants and gathering dataabout which sites may be more effective in finding special populations such as minorities,women, mentally ill individuals, and adolescents. An updated list of the tools discussed inthis article will be maintained at http://www.chestnut.org/LI/downloads/link_lists/fluplink.html, and input from other evaluators is welcome.

REFERENCES

Borasky, D. V. (1998, March/April). Researching people on the net.On-Line, 22(2), 55–59.Brake, D. (1997, June 28). Lost in cyberspace: Large areas of the World Wide Web are ignored by the

search engines that most surfers rely on. Should they be doing better?New Scientist, 2088(12),12–13.

Chu, H., & Rosenthal, M. (1996). Search engines for the world wide web: A comparative study andevaluation methodology.Proceedings of the ASIS annual meeting, 33, 127–135.

Cottler, L., Compton, W., Ben-Abdallah, A., Horne, M., & Claverie, D. (1996). Achieving a 96.6percent follow-up rate in a longitudinal study of drug abusers.Drug and Alcohol Dependence,41(3), 209–217.

Desmond, D. P., Maddux, J. F., Johnson, T. H., & Confer, B. A. (1995). Obtaining follow-up interviewsfor treatment evaluation.Journal of Substance Abuse Treatment, 12(2), 95–102.

Kolata, G. (1998, April 9). It’s confirmed: Web’s size bogs down searches.New York Times, p. 3.LaPorte, D. J., McLellan, A. T., Erdlen, F. R., & Parente, R. J. (1981). Treatment outcome as a function

of follow-up difficulty in substance abusers.Journal of Consulting and Clinical Psychology, 49(1),112–119.

Lawrence, S., & Giles, C. L. (1998, April 3). Searching the world wide web.Science, 280(5360),98–100.

Ribisl, K. M., Walton, M. A., Mowbray, C. T., Luke, D. A., Davidson II, W. S., & Bootsmiller, B. J.(1996). Minimizing participant attrition in panel studies through the use of effective retention andtracking strategies: Review and recommendations.Evaluation and Program Planning, 19(1),1–25.

Streissguth, A. P., & Giunta, C. T. (1992). Methodological issues in epidemiological, prevention, andtreatment research on drug-exposed women and their children.NIDA Research Monograph, 117,137–154.

Wright, J. D., Allen, T. L., & Devine, J. A. (1995). Tracking non-traditional populations in longitudinalstudies.Evaluation and Program Planning, 18(3), 267–277.

Ziek, K., Beardsley, M., Deren, S., & Tortu, S. (1996). Predictors of follow-up in a sample of urbancrack users.Evaluation and Program Planning, 19(3), 219–224.


Documents

A low-cost follow-up resource: using the World Wide Web to maximize client location efforts