Notes from the Field Stories from the Field: Reflections on ...jsherwan/pubs/stories-itid09.pdfNotes from the Field Stories from the Field: Reflections on HCI4D Experiences Abstract

101

Yaw [email protected] StudentComputer Science andEngineeringUniversity of WashingtonBox 352350Seattle, WA 98195 USA(206) 616 1843

Thomas N. [email protected] StudentGeorgia Institute of TechnologySchool of InteractiveComputing85 5th StreetAtlanta, GA 30332 USA

Divya [email protected] CandidateUniversity of California,BerkeleyBerkeley Institute of Design360 Hearst Memorial MiningBldg.Berkeley, CA 94720 USA

Jahanzeb [email protected] FacultyCarnegie Mellon UniversityLanguage TechnologiesInstitute5000 Forbes Ave.Pittsburgh, PA 15213 USA

Yael [email protected] EntrepreneurConstituyentes #117Col. San Miguel ChapultepecDel. Miguel HidalgoC.P. 11580Mexico City, Mexico

Stories from the Field ANOKWA, SMYTH, RAMACHANDRAN, SHERWANI, SCHWARTZMAN, LUK, HO, MORAVEJI, DERENZI

Notes from the Field

Stories from the Field: Reflectionson HCI4D ExperiencesAbstract

Human-computer interaction for development (HCI4D) requires considerabletime in the ªeld interacting with users. While this is true for most HCI work,ªeldwork in developing regions presents unique challenges due to differencesin culture, language, ethnicity, and socioeconomic status. As a group of nineHCI4D researchers, we have adopted a systematic approach to reºect on thechallenges we have encountered in the ªeld. Arising from this exercise arethree contributions: The ªrst is our research method itself, which uses a mix ofqualitative and quantitative instruments to elicit and synthesize individual ex-periences. The second, intended for beginning researchers, is a set of lessonslearned and suggested strategies for navigating the unique challenges ofHCI4D research. The third, intended for the HCI4D community at large, is acritical reºection on the ªeld itself, inspired by our ªndings. Topics covered in-clude the incentives and agendas of the research world, the importance ofmanaging expectations, the nature of “participation” in HCI4D, and theconºict between research and development more generally.

IntroductionThe potential for information and communication technologies (ICTs) toplay a vital role in international development has been recognized widely.The UN’s Millennium Development Goals make speciªc mention of ICTs,and a ºourishing body of interdisciplinary research has sprung up aroundthe idea, often referred to as ICTD.

This paper sits within a newer subªeld of ICTD research, often dubbedHCI4D (human-computer interaction for development). That subªeld isconcerned speciªcally with the relationship between humans and technol-ogy in the context of international development, ranging from lower-levelinterface design issues to higher-level social interactions. In fact, the truescope of HCI4D as a ªeld is still being negotiated, as its literature andcommunity both continue to evolve.

Nonetheless, it is safe to say that a large portion of HCI4D research isdistinguished by the introduction of new technologies to developingregions as part of its research methodology. This is in contrast to muchdevelopment research, which studies existing technologies and practiceswithout intervening, and ICTD research, which does not necessarily studythe users or the process. It is the former, interventionist style of researchwhich focuses on users and process with which this paper is concerned.As we shall demonstrate, such an approach poses unique challenges.

These challenges may manifest as practical considerations. For exam-ple, one discussion in this paper suggests techniques for eliciting criticalfeedback, despite a tendency in some cultures to be overly polite, espe-

© 2009 USC Annenberg School for Communication & Journalism. Published under Creative Commons Attribution-NonCommercial-Share Alike 3.0 Unported

license. All rights not granted thereunder to the public are reserved to the publisher and may not be exercised without its express written permission.

Volume 5, Number 4, Winter 2009, 101–115

102 Information Technologies and International Development

STORIES FROM THE FIELD

Rowena [email protected] EngineerDimagi Inc.529 Main St., Suite 606Charlestown, MA 02129 USA

Melissa [email protected] CandidateUniversity of California,BerkeleySchool of Information102 South Hall, #4600Berkeley, CA 94720 USA

Neema [email protected] StudentLearning Sciences andTechnology DesignStanford Center forInnovations in LearningWallenberg Hall450 Serra Mall, Bldg. 160Stanford, CA 94305 USA

Brian [email protected] StudentUniversity of WashingtonComputer Science andEngineeringBox 352350Seattle, WA 98195 USA

cially to foreigners. But the need for such maneuvers may hint at deeperissues. Considering the same example, is it possible that the trouble ingetting at the truth could be born of the distances in culture, socioeco-nomic status, ethnicity, and language between researcher and participant?

This is not to say that HCI4D is a doomed endeavor born of a tragicallyºawed model. Indeed, compelling research has been produced, and somemodest successes have been reported. Still, the realities of the ªeld sug-gest two explorations that must be undertaken at this early stage. Theªrst is a practical discussion of how best to manage the unique challengesof working in the ªeld, shedding light on techniques, methods, and bestpractices that can lead to better development outcomes. The second is acritical reºection on the ªeld itself, including the relationships, practices,and motivations that underlie it.

This paper draws on the experiences of nine HCI4D researchers (whoare also the authors) in making contributions to both of these explora-tions. What began as a search for practical techniques gave way to thesort of critical reºection described above, and the results of both arereported here. We submit that our ªndings are of interest to others work-ing in the ªeld: both new students seeking to improve their practice, aswell as more experienced researchers interested in the evolving discussionto which we aim to contribute.

We also offer a third contribution in our methodology itself, which webelieve to be novel. Having met at an international HCI4D workshop, theauthors soon recognized that we had much to learn from each other byway of our diverse experiences in the ªeld. Informal discussions led to amore systematic method for sharing and analyzing those experiences. Thedetails of that method are described in this paper, and we encourageother groups of researchers to replicate our efforts and contribute to thisimportant discourse.

This paper is organized as follows: We set the context by reviewingrelated work and describing the methods we used in gathering and ana-lyzing a set of short stories representative of our ªeld experiences. Wethen present the stories, which we have grouped into themes thatemerged from our analysis. The subsequent discussion features a reviewof some of the most evocative stories, and a look at how they contributeto broader discussions in HCI4D. A critical review of our method itself isalso offered. We conclude with a summary of our contributions.

Related WorkAs a ªeld of research, ICTD is relatively young. Within the body of workpublished in the area, we ªnd that many publications focus mainly on theanalysis of results, or on the characteristics and performance of ªnishedartifacts, rather than on the experience of gathering or producing thoseresults. The latter is the stated purview of this paper.

However, it must be noted that outside the ªeld of ICTD, much hasbeen written about the practice of research in developing regions, particu-larly in the social sciences. Several texts stand out as seminal. Devereuxand Hoddinott (1993) offer advice on choosing partners, learning lan-guages, interacting with governments, and compensating participants

ethically in the course of doing ªeldwork in develop-ing countries. Scheyvens and Story (2003) also focuson ªeldwork, offering practical information, areview of ethical and personal challenges, and dis-cussions on research design and methods. Mean-while, Desai and Potter (2006) provide a more basicguide to development research, one that is intendedfor beginning students.

All these works are ªlled with indispensableinformation, and many readers of this paper are welladvised to consult at least one of them. Absentfrom these volumes, however, is a speciªc focus onthe introduction and study of technological artifactsinto the ªeld, an occurrence central to much HCI4Dresearch. This paper takes that focus. At the begin-ning of our ªndings, we argue more speciªcally howour research moves beyond the reports listed above.

Other related writings can be found in the HCIliterature. Early work in that community investigatedcross-cultural interaction design. Nielsen has been aleader in this area, and his edited collection (Nielsen,1990) is a foundational work. In other work, Russoand Boor (1993) present a checklist of consider-ations for cross-cultural design, including “sugges-tions for an effective international productdevelopment life-cycle.” Chavan (2005) focuses onmethods and tools used in the cross-cultural designprocess, describing several innovative evaluationmethods developed for use in India.

There are also several notable works in HCI thatdiscuss ªeld experiences. One early design brieªngreºects on the experience of designing a mobilehealth application for users in rural India (Grisedale,Graves, & Grünsteidl, 1997). Participatory designstudies, such as those by Braa, Montiero, andReinert (1995) and Korpela, Soriya, Olufokunbi,Onayade, Davies-Adetugbo et al. (1998), discusshow partnerships can be leveraged in the designprocess.

Later work by Parikh, Ghosh, and Chavan (2003)shares some lessons learned, remarking, for exam-ple, that “we soon found out . . . we would have tohang around long enough to no longer be noveltiesfrom the big city.” Chetty, Tucker, and Blake (2004)describe the circumstances leading to their idea toestablish “human access points” to facilitate interac-tions between developers and community members.In each of these works, the focus lies mainly withresearch results, with ªeld experiences being men-tioned only in passing.

More recently, several papers have speciªcally

examined practical design issues. Ramachandran,Kam, Chiu, Canny, and Frankel (2007) present tech-niques for gathering requirements from local stake-holders in developing regions during the designprocess. They discuss ªve lessons learned duringªeld studies in Uganda and India. Similarly, Kam,Ramachandran, Raghavan, Chiu, Sahni and Canny(2006) provide recommendations for carrying outparticipatory design with school children in develop-ing regions. Maunder, Marsden, Gruijters, and Blake(2007) present several tools and techniques compris-ing a candidate UCD4Dev (user-centered design fordevelopment) methodology, based on a criticalexamination of traditional UCD methodologies asapplied to developing world contexts. While of greatutility to HCI4D researchers, these papers are con-cerned primarily with the design process, which isoften only one component of a successful project.This paper takes a broader view.

Another body of work looks at challengesbeyond the design process. For instance, Brewer,Demmer, Ho, Honicky, Pal, Plauché et al. (2006)describe a set of challenges of technology researchin developing regions, including technical, environ-mental, and cultural challenges. This work serves asan excellent reference for a broad set of potentialobstacles to research. It focuses mainly on technicaland environmental challenges, though it does fea-ture some discussion on so-called “cultural” chal-lenges. Surana, Patra, and Nedevschi (2008) alsofocus on technical challenges. Chetty and Grinter(2007) report a series of practical lessons learned inthe course of an HCI4D project in rural South Africa.Finally, Schwartzman and Parikh (2007) describetechniques for building rapport with rural stake-holders, designing relevant solutions, and overcom-ing evaluation challenges.

Our paper reinforces and builds on this latter lineof work by examining the speciªc challenges ofHCI4D in greater depth, and by employing a novelmethodology to achieve a synthesis of a range ofexperiences.

MethodThis paper is an account of the experiences of nineNorth American graduate students who self-identifyas HCI4D researchers. We gathered by chancethrough a meeting at an HCI4D-themed workshopduring the ACM CHI conference in 2008.

We all have signiªcant ªeld experience, with

Volume 5, Number 4, Winter 2009 103

ANOKWA, SMYTH, RAMACHANDRAN, SHERWANI, SCHWARTZMAN, LUK, HO, MORAVEJI, DERENZI

durations ranging from a few months to a fewyears. Our ªeld locations have included China,Ghana, Guatemala, India, Liberia, Mexico, Nigeria,Kenya, Pakistan, Rwanda, Tanzania, Thailand, andUganda. Our research domains are as different ashealth care, education, agriculture, infrastructure,and peace and reconciliation.

Over ªve months, we participated in weekly dis-cussions about our research activities via e-mail, tele-phone, and in person. Throughout these discussions,we shared anecdotes of our ªeld experiences andremarked on the trends we saw in some of ourreports, as well as on the surprises resulting fromothers. The ensuing discussions provided us with arich qualitative background to frame our ªndings.

As we began to formalize our inquiry, a sharedblog was created, and each author was asked tocontribute a minimum of three short stories repre-senting surprising or important experiences learnedfrom our ªeldwork.

Two qualitative coding exercises then followed.The ªrst was carried out individually, and the secondwas conducted as a group during a teleconference.We coded the stories according to their salient fea-tures, drawing on principles of grounded theory(Glaser & Strauss, 1967). Several stories were thenadded in the process, as we were reminded of rele-vant experiences. In total, 55 stories were sharedand synthesized into the 19 ideas.

To gain a deeper understanding of the relevanceand applicability of these ideas, we issued an anony-mous questionnaire to each author. For each idea,every author rated their agreement with the follow-ing statements on a ªve-point Likert scale, rangingfrom “strongly disagree” to “strongly agree.” Wealso included an option for “not applicable.”

The statements were:

1. I agree with this idea.

2. This idea has been or could be applicable tomy HCI4D project(s).

3. This idea has been or could be useful to myHCI4D project(s).

4. This idea has been or could be important tothe overall success of my project(s).

5. This idea is widely applicable to HCI4D proj-ect(s).

For each idea, we also asked ourselves if andhow we would express the idea differently, whatexperiences we had to support or discount each

idea, whether this idea was a surprise, and whetheror not we considered the idea to be standard meth-odology. Additional comments were allowed.

Three of the authors where charged with collat-ing the anonymous results. Each idea was catego-rized as one of the following:

A. Most Agree—Responses agree, with up to 2undecided

B. Most Disagree—Responses disagree, withup to 2 undecided

C. Most Undecided—Responses undecided,with up to 2 differing

D. Divided—Half agree, while the other halfdisagrees

E. Outliers—One or two responses vary fromthe other responses

F. Ignore—Half agree, while the other half areundecided, or half disagree, and the otherhalf are undecided

The “Most Agree” ratings were mostly unani-mous. There were no “Most Disagree” or “MostUndecided” ratings.

Ratings were brought back to the group, wherethey were used as guides to qualitative data thatwas interesting fodder for discussion and reºection.We examined contentious (Divided or Outliers)ideas, as well as those that were universally consid-ered “important to overall success,” “useful,” and“widely applicable” for common themes.

Finally, the results of the questionnaire were com-bined with the 19 ideas in identifying the dominantthemes in our data.

In the next section, we describe the ideas andthemes we identiªed, and we recount the storiesthat deªne them.

FindingsFive themes emerged from our coding and groupingexercise. In this section, we present a synthesis ofour stories, organized by those themes. Whereappropriate, we discuss differing opinions of eachidea in our conversations.

We distinguish the ªndings in this portion of thearticle from previous work as follows: The ªrst sec-tion covers challenges encountered during actualuser studies and is most obviously unique to HCI4Dresearch. The second section consists of storiesrelated to choosing users for HCI4D research pro-



jects. While any research with human subjectsinvolves the selection of participants, the stories wereport revolve around the speciªc challenges of user-centered design.

The third, fourth, and ªfth sections deal withcomplications arising from the introduction of newtechnologies as part of our research. The prospect ofnew technology can raise the expectations ofresearch partners, also heightening the chances ofdisappointment. We discuss strategies for dealingwith this in the third section. Many of the systemswe study need localized content, such as voicerecordings, translations of text, or iconic imagery,and producing that content is often one of the mostdifªcult aspects of the research. The fourth sectionpresents stories related to that challenge. Mean-while, successful systems are sometimes deployedfor use beyond the life of the study. The ªfth sectionreviews complications arising from that eventuality.We feel that none of these issues is adequatelytreated in previous ªeldwork literature, which, as wehave argued before, lacks speciªc focus on theintroduction and study of technological artifacts andtheir impact on users.

Studying UsersUser studies are at the heart of HCI research.Researchers need users to help design, pilot, and

evaluate systems to meet andsuccessfully alleviate a technologi-cal need. HCI4D researchers havethe same responsibilities, but withmore constraints. Differencesacross user groups come in allshapes and sizes, so scaling userstudies to larger populations ishard, even without introducingnew, confounding factors. It isoften impossible or undesirable toªnd any sort of lab-like settingfor interviews, and backgroundnoise, interrupting bystanders,and technical problems due topower and other failures are prac-tically guaranteed. There are alsocultural challenges that can pre-vent access, understanding, orappeal to users.

For example, one authoremphasized being mindful of

local cultural norms when in Pakistan.

Pakistani communities, as well as many othercommunities in the developing world, are veryconservative. Therefore, it was decided early onthat all user studies would be conducted with atleast one female facilitator to help female usersfeel comfortable in the presence of the male re-searcher.

This author (shown in Figure 1) found that, giventhe limited availability of HCI female practitioners inPakistan, working with women from within thecommunity was a good choice.

In the same story, the author found that the localfacilitators were not only beneªcial in making studysubjects more comfortable, they also proved usefulin generating some of the content of the studythemselves.

We purposefully left parts of the user study de-sign (e.g., the introduction text) partially incom-plete, and asked the facilitators to help completethem—this helped build ownership of the userstudy process.

The author would further explain some of themethodology and reasoning behind certain ques-tions, but the author would then allow the facilita-tors to word the questions as they saw ªt. Sincefacilitators speak the local language both literally



Figure 1. In Pakistan, user studies with female users were conducted withthe assistance of a female facilitator .

and ªguratively, their participation improved theunderstanding and effectiveness of the interviews.

As noted earlier, much of the daily work in stud-ies happens in open, public spaces, where everyspectator is free to participate, creating challengesfor HCI4D researchers. One of our authors, whenconducting a study of mobile phone usage in ruralLiberia, was fortunate to have a way to manage thedisruptions.

The research inevitably attracted large crowds,which for the most part were cooperative, but of-ten an outspoken onlooker would demand toknow what the study was about, what the moti-vations were, or what was in it for them or theircommunity. Thanks to the presence of the twouniversity students that I hired as assistants, I wasable to speak to such inquisitors off to the sidewhile the study continued, thus avoiding damag-ing interruptions.

In addition to disruptions by curious individuals,one might also experience disruptions due to poweroutages, noisy environments, and a variety of otherunpredictable factors. Therefore, one cannot be pre-pared to handle all of these in advance. One authorsuggested that maintaining a ºexible attitude aboutinterruptions and learning to persevere in new situa-tions is possibly the more valuable idea.

A few of the stories touched on the difªculty ofascertaining truth and the importance of gatheringdata from multiple sources. During a study, someusers were often eager to please researchers, bothbecause of their perceived difference in status, andbecause of their curiosity about the various gadgetsbeing tested. As one author discovered, it was notalways advisable to take what participants said atface value.

My collaborator ªnally took me aside and ex-plained that because I was a woman, and espe-cially because I was a “white” woman, regardlessof whether they were actually listening to me, un-derstanding what I was saying, or anything, theywould always tell me exactly what they thought Iwanted to hear (which was generally “yes”).

The author clariªed that there seemed to be amystique to being labeled a “technologist,” whichonly added to the assumed status of being a “richwhite woman from America.” To that end, therewas a limit to the value of the feedback participantsshared.

The quest for accurate feedback led to storiesabout less orthodox methodology. For example, oneof the authors found it difªcult to get honest feed-back from participants, and so advocatedeavesdropping.

During a user study, I was being followed by agroup of coffee producers that were curious tosee the system in action, when I overheard a ladycoffee producer say “everything now is made us-ing machines instead of paper.” I really wantedher to expand on that comment. However, theminute that I got closer to her she stopped talkingand went into polite mode.

While all authors agreed that this might be auseful source of information, many raised concernsthat this would be a breach of ethical behavior.

Another strategy for eliciting feedback was toget groups talking as shown in Figure 2. One of theauthors realized she was getting positive feedbackfrom two users during a private interview. They feltthe device she was proposing was great and theywould use it exactly how she wanted them to, butupon leaving the study, she discovered somethingfar more interesting.

I returned back to the porch where a very opin-ionated husband was observing his wife tryingout the device. He didn’t want to actually use thedevice but he had a lot of questions about thestudy, my intentions, and what the device would



Figure 2. In a group, users can enthusiastically helpwalk each other through difªcult tasks, and their dis-cussions can help the researcher understand howbetter to design the device or user interface for thenext iteration.

ever be used for. People started gathering and theopinions started spreading like wildªre. My hostswere asking and answering questions, smallgroups were talking, people wanted to try out thedevice. I was gathering data without even havingto ask any questions. I became an observer of anhonest group discussion about technology.

By conducting the study in an open environmentand being open to spectators, this author was ableto gather more opinions about the technologyunder question.

Choosing UsersBefore users can be studied and interacted with,they must be recruited. This section presents severalstories related to that task.

For the sake of time and expense, authors havetried different strategies (with limited success) to cir-cumvent the inaccessibility of international users.One tried interviewing illiterate children from Cali-fornia’s Central Valley instead of India, while anotherworked with immigrant African doctors in theUnited States instead of those in Ghana.

While these [African-American] doctors did makea signiªcant contribution to our understanding of

the medical practice and to theevolution of our design, at thesame time it became clear afterour ªrst round of ªeldwork thatthere were distinct differences be-tween the American-Ghanaianmedical community and the Gha-naian medical community. Manyof the barriers faced by the Gha-naian doctors in Ghana were notanticipated by the American doc-tors, despite the freshness of theirexperience in-country.

This “substitution” extendedeven within-country: workingwith secondhand data from moreaccessible research partners andexpatriates instead of ªrst-handdata garnered directly from ruralhealth workers or school teach-ers. Ultimately, the authors allagreed that, while substitutioncould guide some high-level deci-sions, most design decisions werebest made working directly withthe target community.

A similar story suggested changing users entirely,rather than substituting. One author was able toalleviate her accessibility problems after she began aproject with an organization in a remote area ofGuatemala.

Every trip took four days of traveling, most ofwhich was spent under stress either due the roadconditions or the crime rate in and around Guate-mala City.

The author later realized that, in her case,remoteness (as seen in Figure 3) was not a prerequi-site to accessing her target user community. Sheswitched to a closer partner, allowing her to visitevery few weeks, far more frequently than with theremote partner.

The ideal approach of working directly with usersin context was not without its problems. One authorshared a story of a shifting user base while design-ing an education application.

We came home and designed for a particular lit-eracy level and then when we returned, [we] hadto work with other users that had a completelydifferent baseline. We ended up doing a whole



Figure 3. Remoteness of users can create difªculty. One author had to driveand walk for eight hours to get to her test site. Another idea drawn fromthe stories was the notion that, even while ªxing one variable (e.g., liter-acy), users still demonstrate a large range of abilities within that variable. Inthis story, a system was being designed for doctors, but the author sooncame to realize other signiªcant variables.

lot of rapid prototyping and iterative design thatwe might have avoided if we had really “known”our users and planned to meet them again.

This author concluded that one must chooseusers available for the lifetime of the project, but inour discussions, that particular strategy was seen asoften unrealistic. A number of authors reported hav-ing users come and go during the lifetime of theirprojects without adverse effects. Yet another authorsaw this idea as a symptom of a larger problem withrigid technology and poor study design, not a prob-lem with the particular target group selected.

We found a signiªcant divergence in issues sur-rounding access to equipment in urban versus ru-ral areas. Doctors in rural hospitals typically sharedone computer, but the shortage of doctors en-sured that that computer was highly available.Doctors in larger hospitals, while they had moreand better equipment, often had to compete withmany other doctors for time to use that equip-ment.

Most authors found this idea an obvious obstacleto plan for. One went further and noted that whileassuming homogeneity was a known problem, it stilloccurred and thus found the idea a useful reminder.

Managing ExpectationsThroughout the life of an HCI4D project, it is tempt-ing to over-promise to project stakeholders. Manyauthors spoke of similar experiences managingexpectations with partners, the target community,and themselves. This section extends earlier work bySchwartzman and Parikh (2007) that only focusedon users’ expectations.

After speaking with a number of organizationsabout a promising idea, one author realized that hisproject scope was too large and the research contri-bution too small, so the idea had to be put on hold.Unfortunately, on his next research trip, when hehad a more modest idea, he came across a partnerfrom the previous project. The balance betweenbuilding excitement with partners and being realisticwas hard to maintain.

My heart sank when I heard him say, “Oh, I re-member you from last time.” It wasn’t maliciousbut it reminded me how I had promised a lot thatI could never deliver.

All authors agreed that researchers need to con-trol the hype surrounding their projects and otherHCI4D projects, as well.

Many of our authors mentioned the importanceof clarifying the researchers’ limitations, especiallywhen it comes to the budget. Even if the partneragency has worked with research organizationsbefore, the nature of that collaboration could still befundamentally different, as one author discovered.

Our ªrst partner was a research organization, andhad dealt with research projects for a long time.Sounds like there should be no problem, right?Wrong! The research projects they worked withhad much larger budgets, and involved large datacollection from the community, employing localcommunity members for this work.

In this case, the target community looked towardthe research organization to provide jobs, while theresearch organization expected its partner to providethe manpower. This mismatch in expectationsproved to be disastrous, especially because it tookseveral months to understand, by which time expec-tations were already set.

In another case, the partner’s expectations werekept low. This author’s research group was unsurewhether funding would be available to continue asummer project in Guatemala past the pilot stage.

From day one, my advisor told me to be very clearwith what was needed to move the project for-ward. It is two years later and we haven’t beenable to come back. However, we communicate of-ten and are just waiting to get enough resourcesto return.

In this case, setting clear expectations and main-taining open communication channels ensuredongoing support from the partner agency, evenafter two years of inactivity.

One author highlighted the fact that researchersneeded to be aware of the limitations of their ownsmall-scale projects, and that even if a product iswell accepted, the resources might not be availableto support immediate widespread adoption. Asidefrom keeping his partner’s expectations under con-trol, he wished he had managed his own, as well.

Our prototype was robust enough to deploy butnow we have scores of installations around thecountry and cannot keep track of updates, feed-back, or bugs from the ªeld.

By allowing the prototype to be deployed muchmore broadly than originally planned, the authorunwittingly increased his workload a great deal, yet



did so without the mechanisms in place to reallylearn from his scaling up experience.

Another author discussed the difªculties of doingwork in a community that had already participatedin a research project, as well as the importance ofclosing projects well, so as to keep the door openfor future researchers. She discussed how partnerswho have had other, possibly negative, partnershipsin the past may have some reservations and requiregreater encouragement.

We interviewed doctors about what their hopeswere for “telemedicine” but found a few whowere quite bitter about their previous experiences.Many of these consisted of one-off, short-termdeployments that had subsequently broken downunannounced. These included closed systems withunpredictable availability, and even, in one case, aconsultation network which speciªcally forbadethe re-use of any information for research or fur-ther publication by the doctors.

Similarly, another author had to get her projectapproved by the entire board of directors becausethe social scientist that had worked with her partnerpreviously had greatly misrepresented them in animportant publication. This required translating allher articles and getting clearance prior topublication.

In these cases, authors stressed that simply beingtruthful was not enough; the authors and even theirpredecessors had to be explicit and clear about thebeneªts and risks the projects could involve. Oneauthor went even further, to argue that researchershave an obligation to be gracious, sending thank-you notes and being available for follow-up.

Developing ContentUltimately, technology is a conduit for content—it isa means to an end, and not an end in itself. In theauthors’ collected and disparate experiences, thesource, quantity, and kind of content vary tremen-dously in HCI4D projects. This wide range of experi-ence also prompts a great divergence in the authors’views on this topic, yielding no clear consensus onmany of the questions.

For example, one author felt that common-sensewisdom would dictate using only pre-existingcontent from domain experts. However, anotherpointed out that, sometimes, appropriate pre-existing content does not exist, and where it does,

it might not be enough, since new technology (as anew medium) often requires its own speciªcchanges to the content for optimal presentation.

After many months of searching, we came to theconclusion that no one had designed textualhealth information for use by low literate commu-nity health workers, because no current technol-ogy existed which could make such contentuseful.

Another author thought it was important to lookto the users as content developers. He shared astory about relying on local cultural elements in theclassroom:

In our classroom instruction system, we havesome default PowerPoint templates that teacherscan overwrite to teach their class. We saw that. . . they . . . spend a signiªcant amount of timemaking the content speciªc to their students’lives. For example, teaching English words forfamily members used photos of people who werefrom the local country. In a session where instruc-tors could make content about any topic, therewere a couple of decks devoted strictly to the lo-cal king.

Two authors felt that it was important forresearchers to realize the difªculty in ªnding andworking with qualiªed content development expertsin HCI4D projects, and for researchers to budgettime accordingly for this purpose in their research.One author stated:

By its nature, our content work had to be iterativeand interactive—as the technologist, it wasdifªcult for me to develop a shared vocabulary ofconcepts very quickly with the content developeras we had very different disciplines, and so thisprocess took a signiªcant amount of time, andnotably, signiªcant shared time.

Some authors found that the ofªcial languagewas not the optimal choice, while another authorfound that even languages suggested by the localNGO partner, or by end-users themselves, turnedout to be sub-optimal:

I tested an information access system in Sindhiwith health workers who natively spoke Balochi.When asked if they would prefer a Balochi-speak-ing system, each participant replied that Sindhiwas ªne. On deeper probing, each respondent re-vealed a different reason as to why they said this,



ranging from peer pressure, to a misunderstand-ing of what Balochi dialect was being proposed,to a preference to learn Sindhi because of otheradvantages it brought.

Clearly, researchers need to consider carefully thechoice of what language to use.

Deploying TechnologyManaging relationships with stakeholders during theadoption phase encompasses practical consider-ations about the technology and deployment, aswell as making the deployment relevant to the user’scontext and social identity. Because the goal ofHCI4D research is development, HCI4D researchmust continue past evaluating the purely technicalcontributions. Planning for adoption, ownership,and long-term use of the proposed solution plays acritical role in ensuring that the technologyaddresses the development goals for which it isdesigned.

One strategy that all our authors found promis-ing was to create a social identity around technol-ogy, drawing from the principles of productbranding. For example, one of the authors produceda reporting system for the Ghanaian judicial service,which he jokingly called the “Judicialyzer.” To hissurprise, the name quickly took hold:

The staff would routinely refer to their work as“Judicialyzing.” A “Judicializer” poster appeared atthe ofªce one day. . . . even the Chief Justiceknew the word.

The staff was proud to beassociated with the system, andhaving a product with a concreteand unique identity made it easierto solicit cooperation from leader-ship in the judicial service.Another author deploying a med-ical record system in Rwanda, aswell as yet another working withcommunity health workers in Tan-zania reported similar experiencesafter printing T-shirts and distrib-uting baseball caps emblazonedwith the product’s name (seeFigure 4).

What is interesting to notehere is that the names and identi-ties produced may have very littleto do with the product’s speciªc

function; the focus is instead on fostering a progres-sive attitude toward process improvement, as well ason creating a sense of pride and unity among users.Users need not feel like the authorities are forcingstrange new policies on them, but rather, that theyare part of a social movement in which they andtheir friends voluntarily choose to engage.

Another deployment strategy involved design forappropriation and local ownership. Techniques toapply this method ranged from designing icons andavatars that users could customize to loading stringsfrom a ªle that could be changed on the ºy for easylocalization. When one of our authors was scoutingfor local talent to customize and maintain his appli-cation, he found their choice of platform and lan-guages to be outside the scope of localprogrammers.

The combination of Java, Tomcat, and MySQL wasa bit much for the programmers. Tiered architec-ture, model-view-controllers, and object-orienteddevelopment were just out of the question.

In this case, the team worked to build a nationaltraining program that taught the necessary skills.Additionally, they created a programming interfaceto enable a more familiar scripting language.

One story demonstrates the potentially disruptiveeffects a new technology can have. While deployinga lab system in Ghana, one author reported that thesystem played an unexpected role in allowing thehospital management to better track fraudulent



Figure 4. Two examples of branding. On the left, a data ofªcer in Rwandawearing a shirt, hat and lanyard for OpenMRS, an open-source medical re-cords project. On the right, a community health worker in Tanzania wearingattire from BRAC, a multi-country development NGO.

transactions. Users certainly did not mention theirmisconduct during the design exercises.

A few technicians, ªnding one of their sources ofincome blocked, started selling fake lab results tothe patients, resulting in misdiagnosis and mis-treatment. The fraud was exposed, and the tech-nicians were ªred, including one that was aboutto retire, who would now be retiring with nobeneªts.

This story is a poignant reminder of the need forsensitivity in our research practice.

DiscussionIn the previous section, we presented a number ofstories gathered from ªeld experiences in diverselocations and domains. For us, sharing stories was alearning experience, invoking discussion about simi-larities and conºicts, as well as a reºection on ourroles and relationships as researchers. This discussionwas aided by an anonymous questionnaire, asdescribed in our methodology section. In the follow-ing section, we present some of the more interest-ing ideas and trends that arose from ourdeliberations.

A Review of the MethodAs stated at the outset of this paper, we hope thatother groups of researchers will ªnd the methodused in this paper appealing, and adapt it to theirown purposes. Here we offer some reºections onthe method.

Perhaps the method’s strongest feature is thatthe stories provided by the participants provide asuccinct but concrete grounding for all the analysisand reºection that follows. Several authors foundthat simply writing their initial stories led to remem-bering others and stimulated insights, which theythen offered for discussion.

The method is “mixed,” in that it features bothqualitative and quantitative elements. The qualitativestage of the method allows attention to all stories,no matter how unique or idiosyncratic, and preventsover-generalization. On the other hand, the quanti-tative survey stage ensures that dominant view-points are given their due emphasis.

On a more practical note, much of the process iscarried out asynchronously, which accommodatesthe tight schedules and diverse time zones. This isespecially attractive to those working in develop-ment where geographic dispersion of colleagues is

the norm. Also, the short format of the storieskeeps the body of data rich, but manageable.Finally, we found that carrying out the process drewus closer as a group, because it offered us a chanceto learn about each other’s experiences and perspec-tives. Beyond plain nicety, we submit that this sortof bonding is important to the construction of aresearch community, especially one as young asHCI4D. We continue to meet every few weeks todiscuss many of the issues we raise in this paper.

A potential drawback of the method’sasynchronicity is the lack of face-to-face contact.However, we feel that we were able to minimizethose effects by using a multi-modal communicationstrategy. All teleconferences were accompanied bytext-based chat and a shared blog, to which all par-ticipants could make reference. Voice and chat wereboth recorded for later reference. We also used sev-eral channels to collect feedback and criticism,including the blog, a mailing list, the survey tool,and the collaboration features of Microsoft Word.Each of these communicative channels played animportant role.

One aspect of the method that could have beenimproved in our particular case was the randomnessand size of our sample. Our group was formedbased on an incidental meeting at a conferenceworkshop, and all the authors came from Westerneducational institutions. A future study could reachout to members of the growing HCI4D communitiesin other regions.

Incentives and AgendasTwo ideas from our exercise provoked thought anddiscussion on the motivations of the researcher in atypical HCI4D project. These were the suggestionthat working with less remote users can saveresources, and the preference to choose users whowill be available throughout the lifetime of a project.Both ideas seem to be born at least partially of adesire for efªciency in the pursuit of research goals.In our questionnaire, this was problematic for someauthors, who argued that an emphasis on efªciencymight hinder the work from reaching its most inter-esting or deserving audiences.

Our ªeld stories are also revealing in what theydo not discuss. No stories dealt with the process ofselecting a worthwhile problem to research or iden-tifying the objectives of a project. Instead, our sto-ries focused on issues that arise after thetechnological artifact is already under development:



choosing users to test it, developing content for it,managing expectations around it, and so on.

These issues—arguably emphasizing efªciencyand technological innovation over the needs ofstakeholders—seem to be a product of the exigen-cies of the research world. Our practical incentivesas researchers, to publish papers and satisfy ourfunders, unavoidably dictate where we focus ouractivities. Since our research community emphasizesfrequent publication and a technological focus,much of our work tends to adopt those priorities.

Expectations, RevisitedEarlier, we recommended managing our own expec-tations, as well as those of our project partners, toavoid disappointment. That recommendation engen-dered wide agreement among the authors—two ofthe ideas in that section were unanimously consid-ered “applicable to all HCI4D projects” in our ques-tionnaire.

But this begs the question: Why must expecta-tions be controlled in the ªrst place? The storiesthemselves suggest an answer. Most HCI4D projectsmust be limited in scope, short on resources, andmodest in scale. These constraints produce smallprojects that are usually unsustainable. It is thus nec-essary to be clear with partners and other stake-holders on the differences between this breed ofproject and the more common style of developmentproject to which they may be accustomed. Thisamounts to managing expectations. Even armedwith the most promising idea, we may not have theresources to turn our projects into ongoing nationaldeployments without strong support from our part-ners. The onus is thus on us to quash unnecessaryhype and maintain humility.

In addition to this responsibility to our partici-pants, we have a duty to the greater community ofICTD researchers to maintain good reputations withthe communities in which we work. Managingexpectations is an important part of this, as are stay-ing in touch with partners after the close of a proj-ect and respecting local priorities. Many other ideaspresented in this paper also apply. Failure to main-tain this positive image and clear differentiation canonly lead to greater difªculty for those wishing todo so in the future.

Participation and ControlThe adjective “participatory” is common in the ªeldof international development, and the ideas it repre-sents inform much of our work, consciously or not.

But in reviewing our experiences, we noticed thatthere exists a tension with respect to the level ofparticipation that our work has exhibited. Someideas seemed especially supportive of participationand local ownership, such as the emphases oninvolving local facilitators and designing for localownership, while others suggested a more asym-metric relationship between researcher and partici-pant, such as considering the practical advantagesof becoming an eavesdropper.

This inconsistency echoes a general ambivalencetoward participation in much of contemporarydevelopment practice. Dearden and Rizvi have com-mented at length on this, stating that, “as participa-tory approaches have been adopted by themainstream, recent dialogues have highlighted thecomplex, and often hidden, workings of power rela-tions in the practice of participation” (2008, p. 89).We ªnd that the stories in this paper have revealedsome of this complexity in our own practice. Wealso claim that this paper forms part of the criticalself-reºection that Dearden and Rizvi suggest.

Along a similar line, Michener (1998) presents adichotomy in which participation is viewed as eitherstrong, which involves partnership and shared con-trol, or weak, which involves only consultation.While examples of weak participation abound in ourwork, it is arguable that none of it attains the stron-ger variant. We suggest that this is the case formuch of HCI4D—the value of participation of thetarget user community is recognized (as in our ideaemphasizing involving local facilitators), but, asalluded to previously, involvement of that commu-nity in the conception and design of the project israrer. Even the oft-used phrase “target user commu-nity” is suggestive of an asymmetry.

However, that strong participation is not perva-sive in our work is perhaps not surprising. Kimaroand Titlestad (2008) have argued that, in low-income countries, achieving meaningful participa-tion in the design of computer systems can be chal-lenging due to limited technical skills. They advocatean approach dubbed “participatory customization,”in which, by focusing on the critique of an existingsystem instead of the design of a new one, therequirement for technical skills is softened. Unfortu-nately, this approach may not be suitable for HCI4Dresearch agendas, which usually expect a greater de-gree of novelty.

While this article takes no speciªc stance onwhether participation is an appropriate philosophy



for HCI4D, we submit that some of our ideasdescribe unique ways to support participation (albeitthe weaker variety) in a high-technology context.Involving facilitators in the design of a user study isa good way to teach HCI principles. Assistance withthe design of content for a system may be a step-ping stone to more in-depth involvement in thedevelopment process for local enthusiasts. Finally,technical participation can be encouraged by choos-ing technologies, such as programming languages,that will enable those with moderate skills toengage with and maintain a system. All of theseideas suggest a bootstrapping approach, in which apart of the goal of early research is to build localcapacity, hopefully leading to future autonomy.

Research vs. DevelopmentThe previous three sections have discussed issuesarising from the stories we have presented: theimportance of research incentives and agendas, theneed to manage expectations of partners, and thechallenge of achieving participatory ideals. Uponreºection, it seems that each of these issues resultsfrom a central conºict between “doing research”and “doing development.”

Indeed, in our discussion of incentives and agen-das, mention of research agendas was made onlybecause they seem to conºict with the prevailingideal of development, by which recipients are cho-sen based on need and projects are conceived of incollaboration with stakeholders. Managing theexpectations of partners, as discussed in the Expec-tations, Revisited section, is only necessary becausea typical partner will have come to expect a particu-lar style of development project, which our research-driven style may not match, leading to disappoint-ment. Finally, the ideal of strong participationdescribed in the prior section is difªcult to achieve,in part because offering a share of project control toa non-research partner risks seeing that project strayfrom the all-important research agenda. All theseissues are clearly interconnected.

We see this conºict as a source of confusion forour ªeld, and we believe that further discourse isrequired to resolve it. One perspective in that dis-course sees HCI4D as basic research, claiming that,unlike typical development practice, the role ofHCI4D is to experiment with new ideas, allowingsuccessful ideas to be put into practice at a laterdate. This implies that early ideas may be unsustain-able, as in much experimental work. This perspective

therefore plays down the expectation of near-termdevelopment outcomes.

The other, more hopeful position cites the actionresearch tradition in sociology and expresses opti-mism about the potential to do development andresearch at once. But the mechanics of how thismight be achieved are still being negotiated. Luk,Ho, and Aoki (2008) offer one potential model. Hav-ing chosen only their application area (health care)they performed a needs assessment, during whichthe goals of their project were “co-articulated” byboth the researchers and the doctors with whomthey worked. That “co-articulation” produced aproject that satisªed a real need of the doctors, aswell as an interesting research subject. But thisapproach was not without its costs. They stressedthat multiple trips to the site of study are required tosupport such an approach. Shorter lengths of timein the ªeld require that projects be ready for datacollection as soon as a researcher hits the ground,making such co-articulation more difªcult, and con-sequently lessening the chance that the project willaddress a real need.

There are surely more examples of projects andapproaches in this spirit. Our ªndings suggest thatmore attention needs to be given to those that pro-duce success, and that researchers need to be clearwith themselves, their partners, and their researchcommunity about how their projects are positionedwith respect to the dichotomy of research anddevelopment.

ConclusionThe ªeld of HCI4D is young, distinct, and exciting. Inthis paper, through exploration of our experiences asresearchers, we have identiªed challenges in doingHCI4D research and offered practical strategies fordealing with those challenges. We are conªdentthat beginning researchers can learn from both ourmistakes and our successes.

We have also taken time to reºect on our ªeldand our work, and we have found that severalthings stand out. We have considered the incentivesand agendas of the research world and how theyrelate to HCI4D, the importance of managing theexpectations of our partners as well as our own, thenuanced nature of “participation” in HCI4D, andthe conºict between research and developmentmore generally.

We submit these lessons and reºections as a con-



tribution to the broader conversation about HCI4Dthat has begun in recent years. We are excited tocontinue participation in that discussion as our expe-rience grows and we work toward a deeper under-standing of our true role in the process ofinternational development. ■

ReferencesBraa, J., Montiero, E., & Reinert, E. (1995). Technol-

ogy transfer vs. technological learning: IT infra-structure and health in developing countries.Information Technology for Development, 6(1),15–23.

Brewer, E., Demmer, M., Ho, M., Honicky, R. J., Pal,J., Plauché, M. et al. (2006). The challenges oftechnology research for developing regions. IEEEPervasive Computing, 5(2), 15–23.

Chavan, A. (2005). Another culture, anothermethod. HCI International 2005, pp. 399–401,Montreal.

Chetty, M., & Grinter, R. (2007). HCI4D: HCI chal-lenges in the global south. CHI ’07: Extended ab-stracts on human factors in computing systems(pp. 2327–2332), San Jose, CA.

Chetty, M., Tucker, W., & Blake, E. (2004). De-veloping locally relevant software applications forrural areas: A South African example. SAICSIT’04: Proceedings of the 2004 annual researchconference of the South African institute of com-puter scientists and information technologists onIT research in developing countries. October 4–6,2004, 239–243.

Dearden, A., & Rizvi, H. (2008). Participatory IT De-sign and Participatory Development: A compara-tive review. PDC ’08: Proceedings of the eleventhconference on participatory designers, Oct 1–4,2008, 81–91.

Desai, V., & Potter, R. (2006). Doing development re-search. Thousand Oaks, CA: Sage Publications.

Devereux, S., & Hoddinott, J. (1993). Fieldwork inDeveloping Countries. Boulder, CO: LynneRienner Publishers.

Glaser, B. G., & Strauss, A. L. (1967). The discoveryof grounded theory: Strategies for qualitative re-search. Piscataway, NJ: Aldine Transaction.

Grisedale, S., Graves, M., & Grünsteidl, A. (1997).Designing a graphical user interface forhealthcare workers in rural India. CHI ’97: Pro-ceedings of the SIGCHI conference on humanfactors in computing systems. March 22–27,1997, pp. 471–478.

Kam, M., Ramachandran, D., Raghavan, A., Chiu, J.,Sahni, U., & Canny, J. (2006). Practical consider-ations for participatory design with rural schoolchildren in underdeveloped regions. Earlyreºections from the ªeld. IDC ’06: Proceedings ofthe Conference on Interaction Design andChildren, (pp. 25–32). Tampere, Finland.

Kimaro, H. C., & Titlestad, O. H. (2008). Challengesof user participation in the design of a computerbased system: The possibility of participatorycustomisation in low income countries. Journal ofHealth Informatics in Developing Countries, 2(1).http://jhidc.org/index.php/jhidc/article/view/9

Korpela, M., Soriya, H. A., Olufokunbi, K. C.,Onayade, A. A., Davies-Adetugbo, A., &Adesanmi, D. (1998). Community participation inhealth informatics in Africa: An experiment in tri-partite partnership in Ile-Ife, Nigeria. Proceedingsof Computer Supported Cooperative Work, 7,339–358.

Luk, R., Ho, M., & Aoki, P. M. (2008). Asynchronousremote medical consultation for Ghana. CHI ’08:Proceedings of the SIGCHI conference on Humanfactors in computing systems, pp. 743–752, April5–10, 2008. Florence, Italy.

Maunder, A., Marsden, G., Gruijters, D., & Blake, E.(2007). Designing interactive systems for the de-veloping world: Reºections on user-centred de-sign. ICTD2007: Proceedings of the IEEE/ACMInternational Conference on Information andCommunication Technologies and Development,pp. 321–328, December 15–16, 2007.Bangalore, India.

Michener, V. (1998). The Participatory Approach:Contradiction and Co-optation in Burkina Faso.World Development, 26(12), 2105–2118.

Nielsen, J. (1990). Usability testing of internationalinterfaces. In Designing user interfaces for inter-national use. Essex, UK: Elsevier Science Pub-lishers Ltd., 39–44.



Parikh, T., Ghosh, K., & Chavan, A. (2003). Designstudies for a ªnancial management system formicro-credit groups in rural india. CUU ’03: Pro-ceedings of the conference on universal usability.November 10–11, 2003. Vancouver, BC.

Ramachandran, D., Kam, M., Chiu, J., Canny, J., &Frankel, J. (2007). Social dynamics of early stageco-design in developing regions. CHI ’07: Pro-ceedings of the SIGCHI conference on humanfactors in computing systems. April 28–May 3,2007, 1087–1096.

Russo, P., & Boor, S. (1993). How ºuent is your inter-face? Designing for international users. Proceed-ings of the INTERACT’93 and CHI’93 conferenceon human factors in computing systems. April24–29, 1993, 342–347.

Scheyvens, R., & Storey, S. (2003). DevelopmentFieldwork: A Practical Guide. Thousand Oaks,CA: Sage Publications.

Schwartzman, Y., & Parikh, T. (2007). Establishingrelationships for designing rural information sys-tems. CHI ’07: Extended abstracts on human fac-tors in computing systems. April 28–May 3,2007, 1845–1850.

Surana, S., Patra, R., & Nedevschi, S. (2008). Beyondpilots: Keeping rural wireless networks alive. In J.Crowcroft and M. Dahlin (Eds.), Proceedings ofthe 5th USENIX Symposium on Networked Sys-tems Design and Implementation, (pp. 119–132).Berkeley, CA: USENIX Association. April 16–18,2008.



Documents

Notes from the Field Stories from the Field: Reflections on ...jsherwan/pubs/stories-itid09.pdfNotes from the Field Stories from the Field: Reflections on HCI4D Experiences Abstract