PAPERS CHI 97 * 22-27 MARCH 1997

Participatory Analysis:Shared Development of Requirements from Scenarios

George Chin Jr., Mary Beth Rosson and John M. CarrollComputer Science Department

Virginia Polytechnic Institute and State University

Blacksburg, VA 24061-0106 USA

+1 5402316931

chku$jcsgrad.cs. vt.edu; rosson@vt.edu; carroll@cs.vt.edu

ABSTRACTParticipatory design typically focuses on envisionment andevaluation activities. We explored a method for pushingthe participatory activities further “upstream” in the designprocess, to the initial analysis of requirements. We used avariant of the task-artifact fi-arnework, carrying out aparticipatory claims analysis during a design workshop fma project addressing collaborative science education. Theanalysis used videotaped classroom sessions as sourcematerial. The participant-teachers were highly engaged bythe analysis process and contributed significantly to theanalysis results. We conclude that the method has promiseas a technique for evoking self-reflection and analysis in aparticipatory design setting.

KeywordsParticipatory analysis, participatory design, scenarios, task-artifact fi-amework.

INTRODUCTIONSystem developers employ a variety of techniques to elicitrequirements from users; these include interviews,observations, and artifact analysis [8]. In these techniques,users typically play the role of informants, not initiators —they provide information, but do not do the analysis. Inthis sense, traditional forms of requirements analysisinclude but fail to empower the user during the mostformative stages of sotlware development. This can limitboth the quality of the requirements analysis itself, and theeffectiveness of user-developer collaboration over the entirecourse of a project. We are exploring participatory analysistechniques aimed at increasing user participation fi-om theearliest stages of requirements analysis.

Our analysis approach is built on the philosophies andobjectives of participatory design. These objectives includemutual learning between user and designer, application ofdesign tools familiar to users, envisionment of fhture worksituations, and grounding of analysis in the practice of theuser [3].

Permussiou[0 make digd:lllm-d copies of;lll or porl oflllis material Itirpersonal or classroom use IS gr;tnled Ivnhou( lie pmv!dcd that the copieswe noI nmic or dtstribu[ed l-orprofit or commcrcinl Id\aIIIage. IIIC copv-nght IWIIIX. Ilw Iitle ol’llw puhlicnlmn ;md ils dmte :ippwrr, mlcl IIOIICC is

giw!] 11111 copyrighl is hy prnnissim ol’tlm ,\Chl. iIIc. ‘I_o copy otlwrwise.to republish, 10 posl 011 scn,crs 01- 1{) rcdislrilullc 10 lists. requires specific

permission and{or UccCl l] 97, Allnnta ( i,\ I iS/\Copyright I ‘)97 F\Chl O-X979 I -X(12-9,97,03 ,$3.50

CAN TEACHERS WRITE CLAIMS?We are working with a scenari~based design flamewokcalled the task-artifact iiztmework (TAF, [2]). In thismethod, system development starts ti-om an analysis cfimplicit requirements embodied in user tasks. The TAFdescribes an iterative model of analysis and design: theanalysis of requirements scenarios guides the developmentof envisionment scenarios. In this framework, the questionof direct user participation in requirements analysisbecomes an investigation of users participating in theanalysis of their own usage scenarios. Can users contributeto the identification of implicit requirements embodied intheir own current practices, and if so, how?

Scenario-based design generally facilitates user participationin system development: scenarios are informal, evocative,work-oriented, can be sketchy or highly detailed, and ateequally accessible to various stakeholders in a design,Perhaps most important, the scenarios belong to the users,they describe and exemplify the users’ own practices. Thistransforms the user from the recipient or consumer of thesystem development process into an expert participant.

A variety of scenario-based design techniques have emergedthat allow users to participate directly in the design andformative evaluation of early prototypes. In Kyng’s [5]cooperative method, users and developers collaborativelyspeci~ and refine design details as users simulate workactivities by stepping through interaction scenarios usingmock-ups and prototypes, In PICTIVE (Plastic Interthcefix Collaborative Technology Initiatives through VideoExploration, [7]), users and developers collaborativelyconstruct informal prototypes using low-tech accessoriessuch as markers, Post-It notes, scissors, and tape.

In general, participatory design techniques have focused onsystem development activities that presuppose initialprototypes [5] or that address prototype development veryconcretely (for example, as user interhce design, Muller etal. [7]). Little research has been directed at techniques huser participation in earlier phases of design, such asrequirements analysis. Muller et. al. recognize thisdeficiency as they write, “the major failing of PICTIVE isthat it often encourages detailed design, without supportinga critical participatory analysis of higher-level issues.”

Although scenarios are most tlequently used for envisioningdesigus, it has been argued that they can play an integrating

162

CHI 97 * 22-27 MARCH 1997 PAPERS

role throughout the sofiware development process [1],including requirements analysis:

People using current technology can be directlyobserved to build a scenario description of the state-of-the-art and to ground a scenario analysis on whatsubsequent technology might be appropriate: therequirements scenarios embody the needs apparentin current work practice. [2, p. 7]

The TAF tries to push scenario analysis to Muller’s higherlevel. It attempts to articulate implicit relationshipsbetween features in a situation and consequences for users inthat situation as “claims” — for example, “a blinking iconquickly captures the attention of the user in the event of anemergency condition.” The featwe of this situation is ablinking icon; the consequence is quick notification.

This paper describes a case study that applies participatoryanalysis in the design of a collaborative learningenvironment for middle and high school students. Thus,our central question is whether public school teachem canparticipate in claims analysis.

PARTICIPATORY ANALYSIS IN THE TASK-ARTIFACT FRAMEWORKA basic notion of the TAF is that any particular systemdevelopment project takes place in a wider context cftechnology development in which human artifacts andhuman tasks co-evolve. The artifacts-in-use of currenttechnology embody affodances and constraints for humanactivity; at the same time, the tasks people engage inembody requirements for fhrther technology development.Understanding artifacts in terms of the scenarios of use theyenable and obstruct makes it possible to more deliberatelymanage the co-evolution of tasks and artifacts.

The TAF has four stages (see Figure 1):

● Scenario generation - A set of usage scenarios isdeveloped. Each scenario should exempli@ adistinctive and typical usage situation for the currently-employed artifact(s), or an especially significant usagesituation, such as error and recovery. The set dscenarios should provide reasonable coverage of alltypical uses of the currently-employed artifact(s).

. Claims analysis - Claims m causal relationshipsidentified in scenarios. A claim asserts that a givenf- of an artifxt in a situation of use can havevarious specific consequences for a user. Consequencesmay be either positive or negative for the user. Claimsare organized into claim schemas, grouping a f-with all of its identified consequences.

● Feature envisionment - New or refined features can beenvisioned by hill climbing from the claim schemas dcurrent features. Envisioned f-s should enhancethe positive consequences and mitigate the negativeconsequences of the current featwes.

● Scenario envisionment - Once new features have beenenvisioned, they are incorporated into new scenarios,Envisioned fkztures will typically modulate theactivities described in the original scenarios.Nevertheless, an envisioned scenario should follow the

patterns or steps of the original scenario upon which itis based as much as possible.

nowtoaturuwthSnluncmpros,n’a@t*Cons

Figure 1. Iterative analysis and design in the TAF.

The TAF is a cyclic integration of the processes of analysisand design. Claims analysis “seeks to get at just how theartifact suggests to users that they do something one way oranother, how it supports and fails to support their eff~and how it signals progress and error [2].” The products cfclaims analysis (i.e., claim schemas) organize analysis datain a way that is conducive to interpretation and designreasoning.

Design occurs through the envisiomnent of features and newscenarios. During design, we scrutinize our claims andreason about how to create new features or refine ouroriginal features to improve the consequences. Forintegration and validation of features, the envisionedfimtures are then attached to particular instances of use asthey are inserted into envisionment scenarios. The resultsof the design process are envisioned fkatures and scenarioswhich may again undergo claims analysis.

We extend the TAF by fully involving users in theanalysis and design process. We wanted to investigatewhether the TAF is accessible to persons who are nothuman-computer interaction (I-ICI) specialists. And moregenerally, we wanted to investigate whether requirementsanalysi=working horn scenarios and claimsrepresentations-could be carried out as a participatorydesign activity.

To ourselves, we made the argument that TAF is just anincrementally more systematic variant of the decisionprocesses people engage in day-to-day. When people shop,they petiorm claims analysis as they assess the features dproducts: how will this automobile do in the ice scenario,are the tires wide enough for the beach? Claims analysismerely asks that the reasons why a specific f- is goodor bad be explicitly elaborated.

Extending the TAF to a participatory approach had manyramifications. We needed to codifi “raw” user practice tomake it more discussible by developers and users. Thus,we collected a substantial sample of videotapedobservations of classroom situations, but made no a prioriidentification of artifacts and features. We wanted to &&rthe analysis of f-s and consequences in order to takefill advantage of the special expertise of those whounderstand the work context namely the users.

163

PAPERS CHI 97 * 22-27 MARCH 1997

CLASSROOM CASE STUDYOur research is performed as part of a larger educationaltechnology project funded by the National ScienceFoundation’s Networked Infrastructure for Education (NE)program. The project is coordinated by Virginia Tech andMontgomery County (VA) Public Schools, A primaryobjective of the project is to develop and evaluatecomputer-based collaborative learning tools andenvironments in support of middle and high school physicseducation. Participants of the project include teachers andstudents tiom four Montgome~ County schools andcomputer science and education researched ffom VirginiaTech. We decided to adopt a participatory designapproach, since we knew tiom prior work that the teacherswe am working with have a great amount of technicalknowledge that no one else on the project team did [6][9].

The requirements analysis case study is drawn fimm a two-week summer workshop which included all projectparticipants. Activities in the workshop includeddemonstrations and discussions of commercial and reseamhlearning environments, consideration of various proposeduser intert%ce metaphors, planning discussions for baselineand post-intervention evaluation of student skills and hclassroom projects and activities fbr the 1996-97 schoolyear. Time was specifically allocated during the workshopfor participatory analysis and design.

Participatory analysis and design took place over threesessions. Participatory analysis occurred over two one-halfday sessions. In these sessions, we analyzed videotaperecordings of classroom situations, gathered during thepreceding semester. Participatory design occurred for twohours in a third session. In the following, participatoryanalysis includes scenario generation and claims analysiswhile participatory design ~fa to feature and scenarioenvisionment (see Figure 1).

Participant RolesOur project team divided into three constituencies, whichcorresponded to three roles. The team included fburteachers (one was not present during the fmt analysissession). The teacher role contributed understanding ofclassroom activities and context. The teacher could assesshow elements of current and envisioned situations matchedclassroom needs.

The team also included four technologists, computerscience reseamhers who wm responsible for the technicalinfi-astructure fm the project (one of these team memberswas not present fbr the final design session). Thetechnologist role contributed an understanding of andinterest in computer technology and system developmen~including new technology not previously applied in aclassroom setting. The technologist could assess whatf-s of a design could realistically be developed andunder what constraints.

Finally, the team included four HCI designers (one HCIdesigner was also not present fm the final design session).The HCI designer role contributed an understanding of userinterface technology, and of the TAF method we wereapplying. The HCI designers were able to facilitate the

participation of both teachers and technologists as well asto participate themselves. Also present were two studentswho provided videotaping and recording support andoccasional contributions to the discussion; a public schooladministrator joined us for part of one analysis session,

In general, teachers played the teacher role, systemdevelopem played the technologist role, and HCIresearchers played the HCI designer role. During the courseof the analysis, however, participants sometimes changedtheir roles. One of the teachem was particularly interestedin computer technology, and sometimes played thetechnologist role. Some of the system developers were alsoexperienced university professors, and provided input intothe analysis process as teachers. HCI researchers were bothcollege professors and system developers; they contributedwithin those additional roles during the analysis.

We considered this fluidity of role assignment to be apositive factor. We had been concerned going into thisinvestigation about potential inequalities among the roles:it is ofien alleged within the CHI community that HCIdesigners m sometimes subordinated to technologists indesign contentions. We worried that this might be evenmore acute in collisions between teachem andtechnologists. None of this materialized. Perhaps becausemany of the people on our project team played multipleroles, there was obvious deference to principal roles: theteachers were the final authority on pedagogical activities,the technologists were the final authority on technology,and the HCI researched were the final authority on userinterface and design methodology.

Data Collection and Scenario SelectionThe classroom scenarios used as source material in theparticipatory analysis were extracted from videotapedobservations of classroom activities. Since our aim is todevelop a collaborative learning environment for scienceexperimentation, the focus of our observations was studentsworking on experiments together in groups. We alsoworded other activities related to the experimentsincluding teachers’ lectures, students forming into groups,class discussions, and group presentations.

We supplemented videotaped observations with other formsof data including fieldnotes taken during observations,videotaped interviews of students and teachers, andclassroom artifacts such as textbooks, lab instructions, labassignment sheets, data worksheets, lesson plans, andhomework problem sets. In the interviews, we askedstudents and teachers general, open-ended questions oncollaboration, experimentation, and pedagogy. Theinterviewees were encouraged to elaborate their views andto take the discussion in any direction they wished.Classroom observations, interviews, and artifacts werecollected over a period of three months horn the fburschools participating in the project.

Over the period of our data collection, we observed severallessons in each teacher’s classroom. From theseobservations, we identified key phases of science lessons atthe middle and high school levels. In the middle schools,a science lesson otlen consisted of many small, diverse

164

CHI 97 * 22-27 MARCH 1997 PAPERS

learning activities such as brainstorming, physicalexperiments, demonstrations, class discussions,roleplaying, and student presentations. In the highschools, the science lessons were typically focused on onecomplex, physical experiment. There, the lesson phaseswere geared towards the stages of an experiment such asequipment assembly and execution, data collection andanalysis, and lab reports.

We used these lesson phases as genres. For each geme, wesearched for rich episodes of interaction among the teacherand students. We also examined the videotaped interviewsto find topics and themes that were important to teachersand students, and then tried to find occurrences of thesethemes in the classroom videotapes. For example, oneteacher expressed a strong commitment to discoverylearning: we looked for and found evidence of this teachingstyle in the videos and selected the relevant episodes.

There were also episodes that we simply found interestingand worthy of analysis. For example, during a high schoolexperiment, a confrontation occurred between two male labpartners vying for control over the same piece of apparatus.A female lab partner resolved the conflict by suggesting andenforcing a turn-taking policy. We felt that this episodewas rich in the issues of gender, leadership, and conflict,and would incite good discussion and analysis.

Once a set of episodes was selected for participatoryanalysis, we transcribed the selected episodes. We nf&mdto our collected artifacts to fill in the details of the scenariossuch as the specific instructions that students were given fbran experiment, the homework and lab problems thatstudents were to address, and the worksheets students usedto organize their experiments.

Ultimately, we prepared two classroom scenario sets. Oneset described activities smounding a middle schoolexperiment on waves (Figure 2a). The other describedactivities from a high school experiment on inelasticcollisions (Figure 2b). Each scenario set consisted of oneoverview scenario and several more focused scenarios. Theoverview scenario described the chronological teachingactivities of the lesson as well as its pedagogical contextand goals. Focused scenarios narrated specific interactions,collaborations, and events occurring during the lesson.

A typical concern of designers using any scenario-baseddesign approach is whether selected scenarios providereasonable coverage of the tasks and artif%cts that occurwithin a system or situation. In these initial sessions,coverage was not a major concern. Our goal was to get theteachers interested and involved in the analysis by selectingscenarios that were motivating and revealing. We selectedscenarios that contained interactions and activities, thatwere interesting, and that would serve as a good source ddiscussion among the teachers and technologists. We willevaluate scenario coverage in future analysis sessions.

Participatory Analysis of Classroom ScenariosThe classroom scenarios were presented to participants intwo different forms. Prior to the participatory analysissessions, the scenarios were transcribed from the collecteddata and delivered to all participants in a textual form We

asked all participants to review the written scenarios inpreparation for the analysis sessions. We also preparedvideo presentations of the scenarios tiom our classroomvideotapes. These video scenarios were presented toparticipants at the beginning of the analysis sessions.

) Middle school lesson on waves

An overview of activities for learning about waves ispresented; these include teacher-student brainstorming, aphysicaI experimen~ group presentations, roleplaying,and a teacher demonstration.While a group of students is running an experiment, theteacher provides conceptual guidance.Students in a group assume different roles during anexperiment. The group experiences diftlculty incoordinating among the different tasks and roles.During group presentations, the teacher leads a hesitantgroup through its presentation.Teachers and students enter into class discussion. Thediscussion centers around a ball floating on the ocean.Student roleplay as particles in a liquid and a solid.The teacher demonstrates the speed of sound throughdifferent media using a musical tuning forks.

~) High school lab on inelastic collisionAn overview of activities associated with a lesson oninelastic collisions is presented. Activities include alecture, a physical experiment, and a lab quiz.

Members of a group coordinate to assemble the labequipment prior to the execution of the experiment.Through trial and error, a group makes several attempts atrunning the experiment before achieving a successfid run.Collectively, a group diagnoses a mechanical problem itencounters during a run of the experiment.A largely uninvolved group member forcefully raises anissue regarding the experiment with the rest of the group.In the midst of the experimen~ group members vie forcontrol over the direction of the experiment.

Figure 2. Classroom scenarios.

We had several motivations for using the video scenarios.First, we wanted to provide a concise record of thescenarios to jog participants’ memories. Second, in the(inevitable) event that a participant did not review thetextual scenarios, the video excerpts would quicklyfamiliarize the participant with the scenarios so that s/hewould be able to contribute to the discussion. Third, wewished to remind teachers that scenarios were simply theactivities of their everyday lives—not something foreign,analytically derived for the purpose of design.

When scenarios are transcribed tim videotapes or tlomnotes, they become abstractions of the raw data. Thetranscriber cannot capture all the details present in thesetting. As a result, written scenarios seem more abstractand symbolic, and less real than video scenarios. Whenteachers are presented with a video scenario, they see thescenario in the same form in which they experienced thescenario in their working lives. The scenario becomesmore tangible and more vivid.

165

PAPERS CHI 97 + 22-27 MARCH 1997

At the beginning of the participatory analysis sessions, wegave participants a brief introduction on how to performclaims analysis. We asked the participants to identifi anyfeatures of the classroom scenarios they found interesting.We described ‘feature’ as something that captures theobserver’s attention or something that afilcts the way aperson teaches or learns. For each feahue, we fiut.her askedparticipants to elaborate both what is good or desirable(“pros”) and bad or undesirable (“cons”) about that fmture.Finally, we asked participants to extrapolate beyond thecontext of the particular scenario when brainstorming aboutthe pros and cons of a feature.

During the introduction and throughout the participatoryanalysis sessions, we used terminology that was fiuniliar toall participants. For instance, we diained from usingtechnical terms such as “claims analysis:’ “claims,” and“consequences” and instead used more generic terms suchas “features” and “pros and cons.”

We urged two ground rules to the participants. First, weasked that the teachers “take center stage” in identifying thefeatures and the pros and cons from the classroom scenarios.Since the scenarios essentially recorded activities of theteachers and their students, the teachers were in the bestposition to understand and to reflect on the scenarios.Second, we asked that participants limit critiquing of ideascontributed by others. We wanted the analysis to be abrainstorming session with a fkee-flowing exchange of iderrsand opinions. We wanted participants to think broadly anddivergently about their activities and requirements. Issuesof feasibility, criticali~, and tradeofTs were to be addressedduring the envisionment stage later in the design process.By fostering an open environment where all ideas wtxwequally valuable, we hoped to encourage high engagementand involvement from all, but especially from the teachers.

As f-s and consequences were generated during theanalysis, they were transcribed onto large paper sheets.Each claim schema was transcribed onto a separate sheet dpaper. When the discussion of a claim schema wasseemingly complete, the paper sheet containing the schemawas taped to a wall of the room. ORen, the group wouldreturn to a claim schema to modi~ or extend it based ondiscussions evoked by other claims. Sometimes, the groupdeveloped several related claim schemas simultaneously ifthe issue under discussion was sufficiently complex orbroad. By the end of the analysis, the walls of the roomwere covered with paper sheets listing claim schemas.

From the onset of the participatory analysis session, theteachers were immediately able to generate claims. Theynaturally and openly spoke about the feature andconsequences that captured their attention in the classroomscenarios. In some cases, the teachem explicitly identifiedthe features and consequences. In other cases, the teacherswould enter into discussion about pedagogical issues suchas whether discovery learning is a valuable approach firscience or how groups should be formed to best supportindividual and group learning. The HCI designers(facilitators) allowed such discussion to take place, but triedto structure the discussion in the direction of generatingclaims. For example, the facilitators would ask questions

such as “what are the features of discovery learning?” or“how do groups form in the classroom?” Lengthy, opendiscussions of this kind stimulated the creation of manyrelated claims and claim schemas.

We illustrate the participatory analysis process through anexample taken directly tiom the analysis sessions.Participants were shown a video scenario of a group ofstudents collecting equipment and organizing themselves ata workbench. In the scenario, three of six studentscollaborated to assemble the equipment while the otherthree casually conversed amongst themselves. A f&iture weextracted from the scenario was labele~ “large group size”(see Figure 3). Eight of thirteen participants attending thefirst analysis session contributed to this claim schema.The cons listed for the large group size feature were directlyidentified tlom studying the video scenario. The pros,however, were not evident in the video scenario, but ratherwere developed by extrapolating beyond the scenario.

Feature: Large Group SizePros:

(pi) may provide greater input and knowledge thansmaller groups

(p2) may handle more challenging and complexexperiments

(p3) requires fewer workstations and equipment(p4) requires less grading by teacher if group is

graded as whole

(p5) may allow teacher to grade in greater depth sincethere are less groups

(p6) is easier for teacher to provide guidance to eachgroup since there are less groups

Cons:(cl) not everyone in the group maybe engaged (group

may be too large to keep everyone interested)

(c2) is easier for the dominant personalities to simplytake control of the experiment

(c3) demands greater accessibility to the equipmentsince the equipment must be shared among a largernumber of students

(c4) tends to produce subgroups - one subgroup maysimply take control of the experiment while theothers idly stand by

Figure 3. Claims for large group size.

The two participatory analysis sessions produced 32 claimschemas, 12 for the middle school scenarios, 20 fm thehigh school scenarios. The schemas fell into the generalcategories of science, lessons, experiments, physicalsetting, groups, student roles, learning styles, and studentpersonalities. Discussion of one feature or claim often leadto the discovery of others. For example, discussion of the“large group size” claim schema lead to the discovery (fclaims concerning “self-selected groups” and “persistentgroup composition.”

We videotaped the participatory analysis sessions. Fromthe videotape, we associated each claim and consequencewith the participant who originally identified it. A total d

166

CHI 97 * 22-27 MARCH 1997 PAPERS

32 features rmd 220 consequences were generated over thetwo analysis sessions. Of all the claim features genemted,48% originated from teachers (see Figure 4). Of all theconsequences generated, 48 °A also originated born teachers.The average time spent on the analysis ofa claim schemawasappro;imately13 minutes. -

60%

~ 50%coy 40%nEg 30%

o20%

z

# 10%

0%.

48% 48% ~ C!am Faetume

H C!aim COneequencss

36%34%

12%13%

Teechsre Technologists HCIDes@ws

Participants

Figure 4. Relative contributions to generation ofclaims and consequences by teachers, technologistsand HCI designers (contributions by student assistantsand school administrator are not shown).

We noticed that teachers we~ likely to contribute in thediscussion of a claim schema even if a teacher did notoriginate the claim. We examined the videotape to see fdrwhich f-s and consequences teachem had “significantinput.” We considered a participant to have significantinput if slhe originates, modifies, extends, or elaborates afeature or consequence. In the discussion of a lab notebookfeature, for example, each teacher provided significant inputinto the discussion by describing how students use labnotebooks in his/her class. General discussions wouldoften lead to a more rich and comprehensive view of thefeature or consequence. From our evaluation, we found thatteachers provided significant input to 88% of the f-and 680/0of the consequences.

Given that only three of thirteen participants on the fmt dayand four of thirteen participants on the second day wereteachers, we were encouraged by the amount of teacherparticipation occurring in the requirements analysis. Thelevel of participation we observed is good evidence thatteachers can actively contribute to requirements analysis tixnew educational technology, and, more specifically, thatthey can create and work with claims.

Envisionment of New TechnologyWe were pleased with the degree of participation in theanalysis sessions. However, it was also important to us toassess how usefid these sessions had been. We wanted toknow first whether the resuit of the analysis-the claimsschemas-would be usefid in driving design reasoning (seeFigure 1). We also wanted to determine whether teacherswould remain engaged and involved when we shifted thegroup’s focus tiom their current classroom activities to thedesign of new activities. The ensuing envisionmentsession allowed us to evaluate these questions.

At the beginning of the envisionment session, we providedthe following set of guidelines.

s envision new features that may accentuate the pros andde-emphasize the cons of a classroom feature

s consider technology features

● consider pedagogical features

● you may consider competing features that have similaror overlapping capabilities

s do not expect a one-to-one mapping betweenenvisioned features and classroom features

● focus on features we can control or affect

We asked the participants to envision two types of fixtures:technology and pedagogy. Technology features comprisethose associated with a computer or user interfiwe.Pedagogy features comprise those associated with teaching.When designing systems, we must consider theimplications that the system has on the activities itsupports. The activity and the system should not bedesigned in isolation tlom one another [6].

The teachers understood this dependency between activityand system. During a discussion on student roles duringparticipatory analysis, one teacher told the group that hisstudents typically fell into one of four roles during anexperiment-a doer, an equipment manager, a leader, and arecorder. Later, during participatory design, a “definedrole” technology f- was suggested. This feature wasintended to fix roles or tasks that students perform during ascience simulation. The teacher who originally describedthe four student roles realized that the roles were lessapplicable for experiments run on computers. After somethought, the teacher came up with three new student roles,“keyboarder,” “mouser,” and “observer.” The new rolesrelated to the sharing of the input devices to a computer.

Referring back to Figure 4, we may demonstrate howreasoning about the “large group size” claim contributed toa feature envisioned during participatory design. Based onother claims concerning group organization, a participantsuggested a new featmre, a “group selection iiamework.”This technology feature was described as a computer-mediated grouping tool that would allow students toorganize into groups with some teacher control. Byexamining the pros and cons of the “large group size”feature, the group began to elaborate the capabilities orfunctions of the proposed feature. For example, to mitigat~the con that a large group size (con ‘c2’ in Figure 4) mayencourage dominant personalities to take over, a participantsuggested that one function of the group selectiontiamework might be an ability to form groups based onstudent personality profiles. This way, the teacher coulddefine groups such that a dominant personality is neverpaired with a weak personality.

Details of all proposed f-s were discussed andelaborated in this way. Figure 5 summarizes the results rfenvisioning the group selection feature. Seven of thethirteen team members present fbr the design sessioncontributed to this envisionment.

167

PAPERS CHI 97 * 22-27 MARCH 199;

A total of 17 feature and 24 elaborations were envisionedduring the two hour design session. Of all new features,4 l% were proposed by teachers (see Figure 6). Of thefeature elaborations, 71YOoriginated from the teachers.

Feature : Group Selection Framework

That is based on the individual’s skills (abilitygroupings) and personalities

That allows for anonymous selection of ornegotiation for group members

That allows students some flexibility in selectinggroup members

That may be student-controlledThat allows students fiwn remote locations to

collaborate (may want to hide the fmt thatthey’re remote)

That is teacher-configurable

Figure 5. Envisionment of group selection fi-amework.

80%

70%

60%

50%

40%

30%

20%

1o%

o%

lwEnvisimsdFeatures

41%

Taachets Technologists I-ICI Oasiylers

Participant

Figure 6. Relative contributions to fixttmenvisionment and elaboration by teachem,technologists, and HCI designers (contributions bystudent assistants and school administrator are notshown).

The average time spent on the envisionment of a f-was approximately 6 minutes. As for the claims analysis,we examined the videotapes to count the number off-discussions in which one or more teachers participated wefound that teachers provided significant input to 90?40of thefeatures and 88’% of the elaborations.

Four of the thirteen participants attending the participatorydesign session were teachers. Again, we n encouraged bythe amount of teacher participation in the design activities.The level of participation demonstrates that teachers arequite capable of envisioning new features horn claims.

In fiwt we were surprised to see an apparent increase inteacher participation from analysis to design. We expectedlower teacher participation during the design phase becausethe other participants had greater experience in both usingand designing technology, which we assumed would aidthem in the envisionment of new technology.

One contributing factor may have been changes in groupcomposition from analysis to design. One of the teacherswho was relatively “vocal” was not present on the fwst d?the analysis sessions, but was present during the designsession, In contrast, one of the technologists who mademany contributions during analysis was not present duringthe design session. These changes alone could haveproduced the apparent increase in the relative degree cfteacher participation. However, it seems clear that teachersparticipated in design envisionment at least as effiively asthey had in analysis of their own classroom context.

DISCUSSIONWe have learned that participatory design is f% moreinvolved than merely inviting users to a design meeting.The users must feel engaged they must have a stake. Theymust have effective access to relevant information. Theymust have status, power, and scope of action sufficient toallow them to take positions and contribute to decisions[4]. And of course much about the context of technologydevelopment militates against any of this.

The principal insight (thus &) in our investigation is thedemonstration that the teachers were able to participate fullyin the initial requirements analysis — not as informants, orsubjects of analysis, but as analysts.

We believe that several factors contributed to this t%vorablequality of participatory interaction. The teacherscontributed to activities and decision-making tiom the vcxystart. We had spent months developing a workingrelationship with these teachers, recruiting them during theoriginal preparation of the grant supporting the project.The videotaped scenarios we used as raw material fm ourrequirements analysis were collected fimm the teachem’classrooms. This meant that team members had alreadyspent much time in an ethnographer’s role, observing andrecording classroom activities, interviewing teachers andstudents about class goals and experiences. Thus theteachers had been an integral part of the months of pmstructuring activities that took place befm the analysissessions.

Effective participatory design requires a commonenvironment-shared media of analysis and design as wellas shared terminology-in which the user and the developerequally participate. For our method the shared mediaincluded scenarios, claims, and the new features inspired bythe claims. We found that users and developers weRequally capable of working with these media.

The shared terminology that emerges in participatoryanalysis often relies extensively on the language of the user,because S/he is most able to identifi, label and confirmfeatures and consequences extracted fbm usage scenarios.For example, recall the four student roles of a doer, anequipment manager, a leader, and a recorder. The names dthese roles became elements in our analysis. Today theseterms have become part of the project’s shared vocabulary.

As designers, we also contribute to the shared terminology,when we introduce concepts associated with our analysisand design methodologies. As part of our participatorystance, we are carefhl to replace technical terms with mom

168

CH197 -k 22-27 MARCH 1997 PAPERS

generic ones (we use “features;’ “pros,” and “cons” ratherthan “artifacts:’ “claims,” and “consequences”). Despiteour caution, we now find that the teachers use words suchas “scenarios” and “envisionment” in normal conversation,In a conversation toward the end of the workshop, when adeveloper raised a question about an as-yet-undiscussedfeature, a teacher quickly responded, “I don’t know aboutthat I’ll have to see if I can come up with a scenario.”

Teachers equated scenarios to classroom activities. Thisdefinition was easy fbr teachers to apply as they talkednaturally about the activities that occur in their work. Aswe moved from analysis to design, the teachers’ view cfscenarios evolved to include envisioned classroomactivities as well.

Laughton described the problems of achieving symmetry incollaboration between the designer and the user [6]. Webelieve that our method successfully shifted power in thedirection of the user. Classroom scenarios established theclassroom context as the fires of analysis and design, andteachem were explicitly recognized as the owners of theclassroom activities. Because introducing new technologynecessarily modifies the underlying teaching activity, thedevelopers were placed in the position of “seekingpermission” from the teachers to make these changes.

During the analysis sessions, the developers sometimesseemed to respond to this shift in power by focusing firston the possible consequences of a technology feature tfinterest. For example, one developer repeatedly stressed aparticular negative consequence that specific teachingactivities require the teacher and students to be co-located.The technology feature that the developer had in mind wasvideo teleconferencing. As a participant in our analysisflamework, however, the developer did not propose thissimply as an interesting feature, but rather emphasized therationale in the current situation that motivated it. In thissense the role of the developer seemed to change somewhat,to that of a salesman who wanted to persuade the teachersthat his/her new and improved feature would make theteachers’ lives easier or better.

Participatory analysis relies on usage scenarios describingthe natural setting. This enables the users to immediatelyconnect the scenarios to their own image of their own work.Users develop a greater sense of ownership of the analysisand design because the activities are grounded in authenticactivities occurring in the user’s world (rather than fiurnabstract manifestations of those activities hidden in theworkings of a computer system). As expert participants cfthe work setting, the users naturally own the real-worldactivities. From this initial sense of ownership of theclassroom scenarios, the teachers were able to establish amore general sense of ownership throughout the designprocess as their practice was analyzed and re-designed.

Early on in the project, the teachers thought of requirementsanalysis and system design as processes that computerscientists performed in developing computer systems. Wetried to induce a more user-centered view by comectinganalysis and design to the classroom activities that teachersconduct and perform. In doing this, the teachem saw how

analysis and design would impact and modifi their ownwork. Grounding the analysis and design on classroomscenarios motivated the teachers to participate in thesoftware design process. Establishing the teachem as theowners of the classroom scenarios gave them the power andright to participate.

Thus far, we have carried out a single phase of participatoryanalysis followed by fmture envisionment, to develop aninitial set of requirements fm the collaborative learningsituation. Subsequent work will build in more detail onthe claims analyses produced, generating new scenarios thatexercise the f-es envisioned. Continuing within theTAF, these scenarios will be subjected to participatoryanalysis and tier envisionment, as part of the ongoingevolution of the teachers’ classroom activities.

ACKNOWLEDGMENTSThis research was supported in part by the National ScienceFoundation, under grants REC-9554206 and CDA-9303152.

REFERENCES1.

2.

3.

4.

5.

6.

7.

8.

9<

Carroll, J.M. (1995). Introduction: The ScenarioPerspective on System Development. In Carroll, J.M.(Ed.), Scenario-Based Design: Envisioning Work andTechnology in System Development, J. Wiley, NY, pp.1-17.

Carroll, J.M. and Rosson, M.B. (1992). Getting aroundthe task-artifact cycle: How to make claims and designby scenario. ACM Transactions on InformationSystems, 10(2), pp. 181-212.

Greenbaum, J. and Kyng M. (1991). Introduction:Situated Design. In Greenbaum, J. and Kyng M. (Eds.),Design at Work: Cooperative Design of ComputerSystems, Lawrence Erlbaum Associates, Hillsdale, NJ,pp. 1-24.

Kensing, F. and Munk-Madsen, A. (1993). PD:Structure in the Toolbox. Communications of theACM, 36(4), pp. 78-85.

Kyng, M. (1995). Creating Context fm Design. InCarroll, J.M. (Ed.), Scenario-Bcmed Design:Envisioning Work and Technolo~ in @stemDevelopment, J. Wiley, NY, pp. 85-107.

Laughton, Stuart. (1996). The Design and Use ofInternet-Mediated Communication Applications inEducation: An Ethnographic Stu&. Virginia Tech,Blacksburg, VA, Ph.d. dissertation.

Muller, M. J., Wildman, D.M., and White, E.A.(1993). ‘Equal Opportunity’ PD Using PICTIVE.Communications of the ACM, 36(4), pp. 54-66.

Preece, J. (1994). Human-Computer Interaction,Addison-Wesley, Wokingham, England.

Williams, M. (1994). Enabling schoolteachers toparticipate in the design of educational soflware.Proceedings of PDC ’94: Participatory Design (ChapelHill, NC, October), pp. 153-157.

169