Computer-Based Support for Curriculum Designers: A Case of

Computer-Based Support for CurriculumDesigners: A Case of Developmental Research

Susan McKenneyJan van den Akker

In this article, we explore the potential of thecomputer to support curriculum materialsdevelopment within the context of secondarylevel science and mathematics education insouthern Africa. During the four-year courseof the study, a computer program wasdeveloped named CASCADE-SEA, whichstands for Computer Assisted CurriculumAnalysis, Design and Evaluation for Science(and mathematics) Education in Africa. Bycarefully documenting the iterative process ofanalysis, prototype design, evaluation, andrevision, we sought insight into thecharacteristics of a valid and practicalcomputer-based tool that possesses thepotential to affect the performance of its users.The results of this study include theCASCADE-SEA program itself, which assistsusers in producing better quality materialsthan they otherwise might, while they alsolearn from the development process. Further,this research has contributed to thearticulation of design principles and relateddevelopmental research methods. This articlehighlights the research and development thattook place, and only briefly addresses the toolitself.

The research described in this article buildson a previous study that began to explore thepotential of the computer’s supportive role incurriculum development. The previous studywas initiated by the Department of Curriculumat the University of Twente together with theDutch National Institute for Curriculum Devel-opment (SLO). It has yielded a computer pro-gram called CASCADE: Computer AssistedCurriculum Analysis, Design and Evaluation,concentrating on formative evaluation for theSLO. Evaluation of this program has indicatedthat such a tool may offer much to the world ofcurriculum development, particularly withregard to the formative evaluation of classroommaterials (Nieveen, 1997). Findings indicate thatuse of the CASCADE program saved time andimproved consistency of formative evaluationplans and activities, while motivating users tocarry out evaluations that otherwise might havebeen omitted. The CASCADE-SEA study usesthe previous CASCADE findings, in the forms ofboth knowledge and product, as a launchingpad for continued investigation into computer-supported curriculum development.

The original CASCADE study suggested thatadditional fruitful applications might lie in con-texts outside that of the Dutch SLO, where cur-riculum materials are also being produced. Inthe late 1990s, collaboration between the Univer-sity of Twente and various curriculum reforminitiatives in the southern African region grew.The exchange of information, problems, andpotential solutions between Dutch and Africaninstitutions prompted a line of inquiry: Couldthe computer possibly offer valuable support toAfrican curriculum developers?

ETR&D, Vol. 53, No. 2, 2005, pp. 41–66 ISSN 1042–1629 41

AAH GRAPHICS, INC. / 540-933-6210 / FAX 540-933-6523 / 04-07-2005 / 19:20

SHAPING THE INQUIRY

Context

Within the latter half of the 20th century, manysouthern African countries achieved indepen-dence. Among other changes, independence hasoften brought about new curricula, new subjectsyllabi, and reform of teaching methodologies(such as a call for more learner-centered teach-ing). Considering that most countries in thisregion already suffer from a serious shortage ofqualified teachers and a severe lack of teachingand learning materials (Caillods, Göttelman-Duret, & Lewin, 1996), the challenges presentedby such changes are tremendous. Many curricu-lum development efforts have been initiatedthroughout the region to help cope with post-independence transformation. Experts in thearea of educational improvement in the thirdworld tend to agree that sustainable develop-ment can only happen when investments aremade into local human resources (capacitybuilding). This implies that investments aremade into the professional development ofteachers, teacher educators, and curriculumdevelopers, preferably in coordination withother major change components, such as curric-ulum redefinition and materials development,reform of the examination system, and improve-ment of preservice education.

Many countries in this region (Zimbabwe,Tanzania, Namibia, and the Republic of SouthAfrica, to name a few) employ the use ofregional resource centers to support profes-sional development of teachers. In most cases,such centers are staffed with specially trainedteachers who are responsible for inservice activ-ities. Often, these facilitator teachers coordinateregional projects, including the development oflesson plans and classroom materials. The cre-ation of classroom materials has been deployedto help teachers (a) improve their subject matterknowledge, (b) strengthen their basic teachingskills, and (c) begin to understand and imple-ment more innovative teaching methods. At thesame time, sharing the materials among otherteachers (regional or national colleagues) canbegin to fill a void of resources and assist in

implementing the new curriculum. Towardexploration of how the computer might supportthese processes, the next section addresses thetheoretical stimuli behind the investigation.

Conceptual Framework

Three key fields of study fuse to form the foun-dation of the CASCADE-SEA researchendeavor. These are the notions of (a) electronicperformance support systems (EPSS), (b) curric-ulum development, and (c) teacher professionaldevelopment. In this exploration, the first is seenas a vehicle to enhance the relationship betweenthe two latter themes. Each of these is addressedbelow.

EPSS

Much of contemporary thinking regarding thecomputer-based support of myriad task typesstems from the field of EPSS. According to Gery(1991, p. 24), the goal of EPSS is “to providewhatever is necessary to generate performanceand learning at the moment of need . . . what dis-tinguishes an EPSS from other types of systemsor interactive resources is the degree to which itintegrates information, tools and methodologyfor the user.” Yet consensus has not beenreached on the ideal balance of various elementsin support systems. Raybould (1990) distin-guished three similar, but different componentsof EPSS: (a) an advisory system, (b) an informa-tion base, and (c) learning experiences.Nieveen’s (1997) definition of EPSS included theintegration of job aids (including conceptual andprocedural information and advice), communi-cation aids, and learning opportunities.

In this study, the notion of electronic perfor-mance support is characterized by four mainelements: (a) advice, (b) tools, (c) learning op-portunities, and (d) communication aids. Advicerefers to tailor-made guidelines that are offeredto the user to help carry out a particular taskbased on what the system knows about theuser’s needs and context. In addition, advicealso includes more generic tips that could besupportive to the user, which are not necessarilybased on specific input. Tools in a support sys-


42 ETR&D, Vol. 53, No. 2

tem are elements that can assist the user to carryout a certain task. This category includes tem-plates (prestructured forms that the user needonly fill in to use), checklists (lists of things to door consider) and programs (additional softwarelinked or outside of the EPSS that can beaccessed to carry out a task). The learning oppor-tunities category refers to parts of the system thatallow users to extend their existing knowledge.This may relate to procedural or conceptualknowledge, and can be offered explicitly (for ex-ample, in the form of a tutorial or a help file) orimplicitly throughout the system (for example,by structuring activities in certain ways). Lastly,the communication aids category refers to thoseaspects of the program that facilitate and orstimulate dialogue (written or verbal, real timeor asynchronous).

Both advocates of the concept of EPSS andcreators of various support systems for curricu-lum development presume several advantagesof providing computer support. For example,effectively designed and implemented supportsystems have the potential to promote improvedtask performance, increased knowledge aboutthe task itself, and organizational learning(Gery, 1991; Nieveen & van den Akker, 1996;Stevens & Stevens, 1995).

Curriculum development

As a field of study, “it is tantalizingly difficult”to know what curriculum is, (Goodlad, 1994, p.1266). Although Taba’s (1962) definition of aplan for learning is generally accepted, disputeabounds with regard to further elaboration ofthe term (Marsh & Willis, 1995). However, it isgenerally agreed that such a plan at leastaddresses the aim, content, and organization ofthat learning (Walker, 1990). Like the notion ofcurriculum itself, the curriculum developmentprocess is multidimensional and complex. AsEisner (1994, p. 371) put it, “The process of cur-riculum development, like the process of doingquantitative empirical research, appears muchneater and much more predictable in textbookversions of curriculum development than it is inpractice.” Throughout this study, examinationof curriculum planning models took place togain insight on curriculum development pro-

cesses at two levels: (a) that of the researcher–developer creating a support system, and (b)that of the end user creating teacher guides. Inboth cases, a fundamental consideration in thedevelopment of curriculum is determining whowill be involved. In literature, and throughoutthis investigation, the idea that teachers embracethe role of curriculum maker has been promoted(Clandinin & Connelly, 1992; Eisenhart & Borko,1991; Lieberman, 1986; Noddings, 1986; Zumw-alt, 1988).

Teacher professional development

As previously mentioned, many countries in thesouthern African region have been facing con-siderable challenges in designing and imple-menting curriculum change. Continued profes-sional growth of teachers is widely accepted asan essential ingredient to any educationalreform (Black & Atkin, 1996; Fullan, 2001;Loucks-Horsley, Hewson, Love, & Stiles, 1998;van den Akker, 1996; Yager, 1994). So, in view ofthe stimulating nature of the activities involvedin materials production, the creation of class-room resources has been introduced into vari-ous inservice programs as a teacher devel-opment strategy. The notion of engaging teach-ers in materials development as an effectiveform of inservice is widely advocated (Ball &Cohen, 1996; Ben-Peretz, 1990; de Feiter, Vonk &van den Akker, 1995). Simultaneously, thisactivity fulfills the need in the curriculum devel-opment arena for new materials.

The relationships among the main ideas of

Figure 1 Conceptual model usedthroughout this study.


COMPUTER-BASED SUPPORT FOR CURRICULUM DESIGNERS 43

this study are illustrated in Figure 1. Here, onecan see that the arenas of curriculum develop-ment and teacher professional developmentoverlap in the process of creating exemplary les-son materials. Further, the notion of computer-based performance support, designed to assistand enhance that process, is also represented.The final element in the model is the supportstructure in which these activities take place.This study targeted regional teacher resourcecenters, although teacher training colleges anduniversities also served as support structures onoccasion. It should be noted that, in this model,the term support structure is not limited to physi-cal infrastructure. This concept also includes thelarger framework or program (for example, ateacher inservice program) that provides thefoundation for such physical and social facilitiesas teacher resource centers, teacher training col-leges, or university faculties.

Aim of the Study

Based on the notion that curriculum develop-ment and teacher professional development aretwo mutually enhancing processes, in this studywe set out to explore how the computer mightcontribute to and even enhance the synergy thatexists between them. The computer-based sup-port targeted a very natural crossroads: the cre-ation of exemplary lesson materials. Part of thereason for embarking on a study that exploresthe potential of the computer to contribute tocurriculum development and teacher develop-ment in southern Africa is the fact that little hasactually been done in this area. Although manystudies have examined inservice education, cur-riculum reform, or even the role of informationand communications technology (ICT) in educa-tion within this context, few, if any, have lookedat the use of performance support toward thecreation of exemplary lesson materials. At thesame time, research in other settings has con-firmed the notion that the arena of computer-supported curriculum development containsgreat potential to contribute to educationalimprovement (Nieveen, 1997; Nieveen &Gustafson,1999). The CASCADE-SEA researchaimed to generate outputs in the forms ofknowledge and product.

Knowledge refers to insight into a systematicapproach to the research and development of atool for the specified purpose. And the product isa software program that promotes improvedtask performance (better quality materials),improved curriculum design and developmentknowledge (teacher professional development),and organizational learning (among resourceteachers).

ABOUT THE TOOL

To better understand the research and develop-ment activities presented in this article, theremainder of this section presents a briefdescription of the resulting product: the CAS-CADE-SEA program. This software packageassists facilitator teachers through importantsteps in making exemplary, paper-based lessonplans and teacher guides that can then be usedby other teachers (usually colleagues in the sameregion). CASCADE-SEA guides users throughthe following key steps of curriculum develop-ment:

1. Rationale. (Why am I making materials? Whatdo I want to achieve with them?)

2. Analysis. (What kinds of materials do weneed? What are the problem areas?)

3. Design. (How can I best structure these mate-rials? What kinds of tips do I include?)

4. Evaluation. (Do they work as I had hoped?How can they be improved?)

The CASCADE-SEA program consists of twoelements: a Website and a CD-ROM.

Although the number of CASCADE-SEAusers with Internet access is rapidly increasing,many still work with the system in an off-linesetting. For this reason, the Website is a supple-ment to (and not a constituent part of) the mainprogram. It supports that to which the Internet isextremely well suited: communication. Whereasthe CD-ROM aids the materials designer inmaking personal decisions about how to create aseries of lesson plans (a teacher guide), theWebsite aims to foster communication betweenmaterials designers. This is done through vari-ous means, including a discussion forum and adatabase. The database contains a variety of


44 ETR&D, Vol. 53, No. 2

completed lesson plans, as well as buildingblocks for materials (clip art, activity ideas, etc.),and is linked with the Gateway to EducationalMaterials (GEM) consortium (for additionalinformation, visit http://www.thegateway.org/). Visitors are welcome to use what theyfind, and are also encouraged to contributeresources for biology, chemistry, physics, ormathematics. The site further contains genericsupport for developing lesson materials andinformation about the CASCADE-SEA project.For additional information regarding this com-ponent, refer to the site itself at: http://pro-jec ts . edte .utw ente .nl/cascade/seasi te/(McKenney, 2001b).

The CASCADE-SEA CD-ROM offers variousforms of support throughout the curriculumdevelopment process. The program asks users tothink about what they would like to achieve(why they are making materials, and what kindsof materials would be useful for that particularsetting). If the developer already has a basic

rationale in mind, then the program helps tomake this explicit and generates a “rationaleprofile” that may then be used in discussionwith codevelopers. Should users have difficultydetermining key issues related to the materialsto be developed, then CASCADE-SEA will rec-ommend that the analysis section be visited. Inthe analysis portion of the program, support isoffered in conducting a needs and context analy-sis, which will then aid in forming or reforminga rationale. Once the user has generated suffi-cient specifications regarding the kinds of mate-rials to be developed, the design phase supportsthe creation of these materials. It helps the userto map out a lesson series, build individual les-sons, and think about the layout of the pages.For users who have completed some of thedevelopment (ranging from rationale formationto a complete lesson series), support is alsoavailable for conducting a formative evaluationof that which has been designed so far. The eval-uation component is heavily based on the origi-

Figure 2 Main menu page from CASCADE-SEA CD-ROM.



nal CASCADE program, although it has beentranslated in terms of both language and con-text. Figure 2 shows the main menu page fromthe CASCADE-SEA program. It illustrates boththe procedural and conceptual model for curric-ulum development that is supported within thisprogram. Although the model itself (representa-tion and nomenclature) is unique, the ideasbehind this model are based on a synthesis ofdesign models described in McKenney, 2001a.

Support is offered throughout the program inall of the formats previously described (advice,tools, learning opportunities, and communica-tion aids). Two examples from each main areaare listed here. Tailor-made advice for materialsdesign and implementation is given at the end ofthe rationale component; this is based on userinput selections. Generic tips are given throughprintable guides that specify how to conductdata collection during analysis and evaluation.Internal tools include data collection instru-ments that can be used with or without alter-ations; they come with suggestions forcustomization. External tools include links tosimple drawing, word processing, and mind-mapping programs distributed along withCASACDE-SEA. Implicit learning opportunitiesare provided through the visual appearance ofthe main menu and submenus that suggest astructure for the materials development process.The tutorials offer explicit learning opportuni-ties through linear on-screen presentations.Written communication is stimulated by theopen source database that allows participants toshare work asynchronously, and verbal commu-nication is stimulated through checklistsdesigned to help materials writing teams plantheir efforts and allocate tasks within the group.

In addition to the four key steps, four addi-tional features are available: (a) help, (b) tutori-als, (c) an interactive agent, and (d) a toolboxcontaining additional resources (such as theexternal programs mentioned above). CAS-CADE-SEA was created using Authorware™(made by Macromedia) and comes with a usermanual in portable document format (PDF).Because of the detailed nature of the program(including approximately 250 different activityscreens), a comprehensive portrayal of the sys-tem would be inappropriate here. However,

additional information is available at:http://projects.edte.utwente.nl/cascade/seastudy/. Hereafter, this URL will be referred tosimply as the research Website.

RESEARCH DESIGN

Insights from relevant literature on curriculumdevelopment, teacher professional develop-ment, exemplary materials, existing supportstructures (such as teacher resource centers),and computer-based performance supportshaped both the CASCADE-SEA program andthe structure of the study. Some of these ideaswere articulated in the form of tenets that servedto guide research and development activities;they pertain to the following topics:

1. Local relevance: Any educational innovationmust be carefully examined and, if necessary,tailored or retailored for the context and cul-ture in which it will be implemented.

2. Collaboration: Design and development activ-ities related to an innovation must be con-ducted in collaboration with and not forthose involved.

3. Authenticity: Efforts must be based on a work-ing knowledge of the target setting and,where possible, research and developmentshould be conducted in naturally occurringtest beds.

4. Mutual benefit: A skillful attempt should bemade to combine research activities withmeaningful experiences for the participants.

5. Continuous analysis: Careful and regular anal-ysis and reanalysis of the risks and benefits ofthe innovation should be conducted in thelight of the target setting, with design anddevelopment decisions being taken accord-ingly.

Main Research Question

The research was further guided by the follow-ing main question: What are the characteristicsof a valid and practical support tool that has thepotential to affect the performance of (resource)teachers in the creation of exemplary lesson


46 ETR&D, Vol. 53, No. 2

materials for secondary-level science and mathe-matics education in southern Africa? This studyfeatured research and development activitiesthat generated successive approximations of asupport tool for the given context. The quality ofthe tool was evaluated in terms of three criteria:(a) validity, (b) practicality, and (c) impactpotential; these criteria were carefully defined.

Validity refers to state-of-the-art knowledgeoffered in an internally consistent fashion. In theCASCADE-SEA tool, such knowledge relatesspecifically to curriculum development andteacher development, and to contemporarythinking on how to support these processes.Internal consistency means that the content, sup-port, and interface elements should be alignedthroughout the various system components.Practicality relates to the way a tool “fits” withand contributes to the target setting. Elaborationof this aspect has been inspired by Doyle andPonder’s (1978) “practicality ethic,” which high-lights three important concepts: (a) The notion ofinstrumentality (“depicting real-world contin-gencies”) relates to the necessity to provide pro-cedural specifications for implementing thecreation of exemplary curriculum materials inan actual setting. (b) Congruence (a matchbetween proposed and prevailing conditions)refers to a fit with the way teachers usually con-duct curriculum development activities andteacher perceptions of the origins of the pro-posed innovation. (c) The cost:benefit ratio is cal-culated in terms of the time, effort, and financialresources that must be invested in order to gainreturns in time, effort, satisfaction, learning, andrecognition. Because this study focused on thedesign and development (but not the full imple-mentation) of a computer-supported curriculumdevelopment tool, no attempt was made toobtain conclusive evidence in terms of overallsystem effectiveness. However, researcherswere keen on learning about the impact potentiala system like this might have, if implemented ona full scale. Toward that end, indicators ofimpact potential were identified in terms of suc-cessfully yielding better quality materials, ascompared to those materials developed withoutthe aid of CASCADE-SEA, and offering a contri-bution to enhancing the professional develop-ment of the user.

Research Approach

Once the criteria for quality were established,their implications for program characteristicswere taken into consideration. Three traits weredefined: (a) the program’s content, (b) the sup-port offered to the user, and (c) the technicalinterface. Consideration of how to gauge qualityin each of three traits yielded a set of qualityaspects, presented in Figure 3. During designactivities, these aspects helped developers main-tain focus; during research activities, they pro-vided a framework for product evaluations.

Toward learning more about the characteris-tics of a tool that meets the desired criteriawithin the context of science education in south-ern Africa, a developmental research approachwas employed. Developmental research has beendefined as “the systematic study of designing,developing and evaluating instructional pro-grams, processes and products that must meetthe criteria of internal consistency and effective-ness,” (Seels & Richey, 1994). For additionalinformation on developmental research, seeRichey and Nelson (1996), van den Akker (1999),or Reeves (2000). This approach was selectedbecause of the opportunities it yields in terms ofdevoting attention to dynamic and complexeducational realities, in line with the aforemen-tioned tenets. For example, because of the lim-ited availability of guidelines for developingsuch tools, evaluation of successive approxima-tions of the desired tool was conducted to helpreduce uncertainties in design decision making.This also allowed for the much-needed continu-ous interaction with experts and practitioners ina variety of settings. The research approach inthis study may be more specifically labeled asformative research, since it involved the actualdesign and formative evaluation of a program.

Three Main Phases

Within this study, three main phases may be dis-tinguished: (a) needs-context analysis; (b)design-formative evaluation of prototype tools;and (c) a more summative assessment of thefinal product (including exploration of its valuefor other contexts). The main aim of the analysis



phase was to better understand the needs andcontext in which the proposed computer-basedtool for curriculum development could be put touse. This was achieved through literature studyand site visits. The second phase featured thedesign, development and formative evaluation

of four prototypes. The evaluation phaseexplored the potential impact of the CASCADE-SEA tool in terms of contributions to teacherdevelopment and curriculum developmentresulting from system use. Additionally, anexplorative query was conducted of other con-

Figure 3 Quality aspects for designing, developing and evaluating the CASCADE-SEAprogram.


48 ETR&D, Vol. 53, No. 2

texts and situations in which this tool, or arevised version hereof, might be useful. Figure 4displays these three main phases as well as thescope of the eight cycles that took place withinthe phases in terms of participants involved andtime spent.

Instruments

In total, 108 instruments were used in this study.Although variation exists among like kinds ofinstruments, so do similarities. For example,various interview schemes were designed togather information about the same aspects (e.g.,internal consistency of the program interface),while being used in different settings. In such acase, rather than develop completely newinstruments, researchers often tailored existingones. Additionally, instruments were improvedwherever possible, based on insights acquiredthrough previous uses. This approach hasresulted in “instrument families” containing likekinds of instruments, with related roots but alsocertain degrees of variation. All instruments

were based on the framework of quality aspectsdisplayed in Figure 3.

Six main families of instruments were distin-guished: (a) interview and walk-throughschemes; (b) questionnaires; (c) discussionguides; (d) observation and demonstrationschemes; (e) logbooks; and (f) document analy-sis checklists. The 10 interview schemes con-tained mixtures of opened and closed questionsand were used with participants not associatedwith the development process. Four of the 5walk-through schemes were designed for usewith members of the development team (1 wasused with an expert group, in a slightly differentfashion). These walk-through schemes con-tained screen shots or design specifications andserved as discussion tools during screeningactivities. Whereas most of the instruments weredesigned to collect qualitative data, the majorityof the 38 questionnaires also gathered quantita-tive data. The questionnaires contained openand closed questions, using various formatsincluding short answers, essays, concept map-ping, multiple choice questions, chart comple-tion and Likert scales. The 21 discussion guides

Figure 4 Display of the CASCADE-SEA study.



contained general questions that helped gentlyshape group talks, most of which were primarilyopen and informal in nature. The 7 demonstra-tion schemes were similar to the discussionguides, except that they contained questionsasked during interactive demonstration ses-sions, which made them usually more specific.In contrast to the other instruments, the 14observation schemes captured researcher per-ceptions of participant experiences duringhands-on sessions with CASCADE-SEA inmicroevaluations and tryouts. The formats ofthis type did vary somewhat, and may be sortedinto three categories: (a) Those mainly used tohelp interpret and confirm data from othersources within a circuit consisted of global ques-tions that helped structure researcher reflec-tions; (b) those mainly used to reconstruct theoverall atmosphere and experience took theform of a running summary; and (c) those usedto portray particular aspects of the hands-onexperience contained frameworks into whichobservation data were placed. Five logbookswere used to capture participant reflections inbrief, prestructured format. The 8 documentanalysis checklists contained open questionspertaining to the quality of either the CAS-CADE-SEA tool or the materials developedthrough its use.

Data Collection

Across the three phases of research, four basicstrategies were used: (a) screening, (b) expertappraisal, (c) microevaluation, and (d) tryout (cf.Nieveen, 1997). Screenings were conducted bythe developers and involved a comparison ofthat which has been developed and the desiredquality aspects. In the expert appraisals, expertswere solicited for feedback on products devel-oped, which ranged from global design ideas toworking prototypes. The microevaluation strat-egy was used to evaluate prototypes with smallgroups of users or experts, outside of theintended setting. Finally, a tryout meant that theprototype be tested by the target group in thetarget setting. Participants in the four strategiesbelonged to user groups (preservice teachers,inservice teachers, or curriculum developers)

and/or expert groups (science education, curric-ulum development, or computer-based perfor-mance support experts).

In line with the five tenets influencing thestructure of the study, the research activities thattook place were selected and carried out basedon deliberation between the researchers andparticipating organizations. Such collaborationwas considered to be the most effective way ofassuring that the research setting remainauthentic as well as relevant and beneficial tothe respondents involved. In settings whererepeated data collection took place (such asSouth Africa, Tanzania and Zimbabwe) this alsoafforded opportunities to learn jointly fromexperience and revise plans and approachesaccordingly.

Each time one of the four strategies was used,a data collection “circuit” was completed. Atotal of 34 circuits took place: 7 during the needsand context analysis; 23 during design anddevelopment; and 4 during the final evaluation.The scope of circuits varied. For example, cir-cuits ranged from half-day to month-longevents, involved 3–54 people and used one toeight instruments. Data collection took placeduring workshops, meetings, and interactivepresentations, most of which were connected toongoing professional development activities.Such events were held at teacher resource cen-ters, teacher training colleges, universities, cur-riculum development institutes, andconferences. The following text describes a typi-cal circuit, while Figure 5 offers an overview ofwhich strategies were used during the 33 cir-cuits that comprised the three main phases ofthe study.

Example circuit (14 in Figure 5). This try-out tookplace in cooperation with individuals associatedwith the Tanzanian Teacher Education Assist-ance in Mathematics and Science (TEAMS) pro-gram, which is based in the Faculty of Educationat the University of Dar es Salaam. The secondprototype was tested out during a week-longwriter’s workshop sponsored by the TEAMSproject. At this workshop, teachers who hadbeen hand selected (by their peers) as being par-ticularly motivated and competent, traveled toDar es Salaam from all over the country with the


50 ETR&D, Vol. 53, No. 2

goal of creating lesson materials. Throughoutthe week, groups of teachers worked to createlesson materials that could be photocopied andshared among the regions. A demonstration ofthe CASCADE-SEA program was given to allthe teachers (n = 34) and everyone was encour-aged to try it out during the week. Three work-stations were set up in the workshop rooms, and

assistance was available at all times. Teachers, insmall groups and individually, took time tobecome acquainted with the tool. Commentsand feedback were captured through a question-naire distributed at the end of the workshop.Additionally, data were collected through dis-cussions with TEAMS staff members and partic-ipants were observed using the program. Finally

Figure 5 Research activities overview.



the teacher guides created with CASCADE-SEAwere analyzed and compared to those madewithout aid of the program.

Data Analysis

The data collected from each circuit wereanalyzed and classified according to relevanceto either quality criteria (validity, practicality,and impact potential), additional related data, orother nonrelated data. Thereafter, quality-related data were further classified in terms ofthe domains previously presented: for validity,these were state-of-the-art knowledge and inter-nal consistency; for practicality, these wereinstrumentality, congruency, and cost; forimpact potential, these were better quality mate-rials and enhancing professional development.Next, data were classified according to the qual-ity aspects. As described in Figure 3, theseaspects related to program content, support, andtechnical interface.

Each time a data collection opportunityarose, researcher–developers weighed off per-ceived costs (time, finances, etc.) with estimatedbenefits (e.g., depth and validity of prototypefeedback), in accordance with the tenets thatguided this study. Many activities were eventu-ally conducted even when the anticipated bene-fit was fairly low, because (as long as the relatedcosts were also minimal) this was considered alow-risk method of exploring what situationswould actually yield fruitful data. To track thisaspect of the research process, each circuit wasalso evaluated for the weight of its data. Thedata weight of each circuit was rated twice: Thefirst rating represented the researcher’s percep-tions before the start of each circuit, and the sec-ond rating illustrated the researcher’s percep-tions after each activity was conducted. As pre-sumed, some activities that appeared less likelyto yield significant findings turned out to offermore than foreseen; the opposite is also true. Thedata weights were based on the researcher’sinterpretation of the salience and intensity of thedata, not particularly on the volume.

A separate summary was created for datarelating to each domain addressed in each cir-cuit (not all circuits addressed all domains).Every summary included a brief description of

that circuit’s activities, the data sources used inthat circuit, the sources used for that particulardomain summary, data weight before activitieswere conducted, and the data weight after theactivities took place. The data weights werereflected through grey-scale shading, indicatingthose with no, low, medium, and high contribu-tions. Further, the summaries were color coded(better visibility in the electronic version) as fol-lows: teal text relates to the rationale component;dark green text relates to the analysis compo-nent; blue text relates to the design component;pink text relates to the evaluation component;dark red text indicates that the data pertain tomore than one area of the program simulta-neously or to other parts of the system (not thecore components); and grey text relates to issuesoutside of the program. Figure 6 shows anexcerpt from a typical summary; it contains datarelating to state-of-the-art knowledge collectedduring Circuit 14. A total of 187 data summarieswere created. These documents are availableelectronically (PDF) at the research Website.

RESEARCH PHASE DESCRIPTIONS

This section features global descriptions of theresearch activities undertaken in each phase,outlining the participants, strategies, and instru-ments used during the various cycles (cycleswere represented as loops in Figure 4). The mainfindings and implications from each phase,which stem from a synthesis of the above-men-tioned data summaries, are also presented.Detailed information about each individual cir-cuit may be found in McKenney 2001a, alsoavailable in PDF at the research Website(McKenney, 2001c).

Phase 1: Analysis

Main purpose. The primary goal of the analysisphase was to obtain a working knowledge of thetarget setting, user group, and areas in which asupport tool may be put to work. As describedearlier, previous exploration into computer-based support for curriculum developersyielded a tool (CASCADE) that served as aspringboard throughout this study, especially in


52 ETR&D, Vol. 53, No. 2

the analysis stage. This phase consisted of twomain cycles. Data were collected especially per-taining to system validity, with tentative ideasabout practicality.

Methods. The needs and context analysis phasebegan with a cycle of literature review and con-cept validation activities. The concept validationcycle was composed of four data collection cir-cuits, with participants from the Netherlands,Lesotho, Tanzania, and Zimbabwe. The site visitcycle was composed of three circuits, with visitsto South Africa, Botswana, and Tanzania. Dur-ing the expert appraisals and microevaluationscarried out within this phase, an English versionof the original CASCADE program served as adiscussion tool. Initial findings during the con-cept validation cycle were presented to expertand potential user groups during the site visits.They offered feedback in the form of design

ideas and suggestions for cooperative activitiesduring the design and development phase.These ideas pertained to the validity and practi-cality of the proposed system, especially interms of which knowledge would be most rele-vant to include. In total, 18 science education,curriculum development, and/or performancesupport specialists took part in the concept vali-dation cycle. Their input was gathered throughone discussion guide and three demonstrationschemes. Thereafter, 54 participants in the sitevisit cycle (primarily experts, but also someusers) shared their insights via seven instru-ments: two interview schemes, three question-naires, and two discussion guides.

Findings and implications. Data collected duringthis phase, together with the simultaneous liter-ature review, contributed to the following majorinsights in terms of the validity of CASCADE-

Figure 6 Sample data summary: State-of-the-art knowledge from Circuit 14.



SEA. Namely, the program should (a) capitalizeon strengths of the existing (original CASCADE)system, particularly in terms of offering a clear,consistent structure and exportable examples,samples, and tools; (b) offer subject-specific aswell as generic support for the process ofdesigning lesson materials (particularly teachermaterials); and (c) be integratable in science andmathematics education professional develop-ment programs throughout the southern Afri-can region.

With regard to the practicality of CASCADE-SEA, data from this phase indicated that the pro-gram should (a) exploit and perhaps elaboratethe advantages of the existing CASCADE pro-gram, with particular regard to the step-by-stepguidance that increases instrumentality andlowers the cost threshold; (b) specifically targetfacilitator teachers working in resource centers,often sharing computers and jointly creatingexemplary lesson materials; and (c) operatewithin a Windows™ environment, and assumeonly basic computer literacy among its users.

Phase 2: Design and Development

Main purpose. The main aim of the design anddevelopment phase was to create and evaluatesuccessive approximations of the desired sys-tem. Through four iterative cycles of design,development, and prototype evaluation, theCASCADE-SEA tool evolved. The main criteriaupon which the four prototypes were evaluatedduring the design and development phase werevalidity and practicality, although the greatestemphasis was placed on practicality.

Methods. During the first of the four cycles, Pro-totype One was evaluated through 4 data collec-tion circuits, with participants from theNetherlands as well as from three universities inthe United States. Altogether, 50 participantswere engaged in the 3 expert appraisals and 1developer screening that constituted this cycle.Their input was gathered through 2 walk-through schemes, 1 interview scheme, 1 discus-sion guide, and 1 demonstration scheme.Prototype Two underwent 3 separate evalua-tions, which took place in the Netherlands,

Zimbabwe, and Tanzania. These activitiesinvolved a total of 63 people. Each took part ineither a developer screening (n = 4), a micro-evaluation (n = 25), or a tryout (n = 34). Datawere collected via 6 instruments: (a) 1 walk-through scheme, (b) 1 questionnaire, (c) 1 dis-cussion guide, (d) 1 observation scheme, (e) 1document analysis checklist, and (f) 1 demon-stration scheme. The third prototype cycle wascomposed of 10 data collection circuits, withparticipants from the Netherlands, Swaziland,South Africa, Zimbabwe, Tanzania, and Nami-bia. Prototype Three was used in 1 developerscreening, 4 expert appraisals, and 5 micro-evaluations throughout the 10 circuits thatinvolved 169 individuals, most of whombelonged to either preservice, inservice or cur-riculum developer user groups. A total of 43related instruments was used: 1 walk-throughscheme, 4 interview schemes, 19 questionnaires,5 discussion guides, 3 demonstration schemes, 5observation schemes, 3 logbooks, and 3 docu-ment analysis checklists. Prototype Four wasevaluated during 6 circuits, with activities in theNetherlands, Namibia, South Africa, andZimbabwe, with 140 participants. Here too, ablend of users and experts was involved,although users composed the majority. Duringthe 1 developer screening, 1 expert appraisal, 1microevaluation, and 3 tryouts, data were col-lected via 25 instruments: 1 walk-throughscheme, 3 interview schemes, 10 questionnaires,4 discussion guides, 5 observation schemes, 1logbook, and 1 document analysis checklist.

Findings and implications. Throughout the evo-lution of the various prototypes, formative eval-uation data informed development of theprogram as well as elaboration of ideas on whatthe program should actually aim to do. Forexample, determining exactly how users’ coreideas (i.e., their rationales for making materials)should be tied into the ideas in other areas of theprogram was a topic that evolved throughoutthis phase. The rationale component started outin Prototype One as mostly an organizationalarea primarily addressing general in formationabout a materials development project, andgrew to be both visually and operatively the hubof the program in Prototype Four. Further, ear-


54 ETR&D, Vol. 53, No. 2

lier notions about possibly making CASCADE-SEA into an “expert” system were eventuallydiscarded in lieu of developing a program that,in addition to supporting the complex task ofcurriculum development, also strives to contrib-ute to its user’s professional development. Forexample, the decision was made to endeavor tomake the process of curriculum developmenttransparent and even inviting. This meant thatbeing able to generate materials quickly and eas-ily was not an exclusive priority. Explaining tointerested users why certain processes are rec-ommended and how they might be carried outalso became important. Facilitating user learn-ing about the complex process of curriculumdevelopment was supported by illustrating howand why decisions in one area of the programinfluenced advice given in other areas of theprogram.

With regard to maximizing the potential ofstate-of-the-art of ICT, a number of opportuni-ties existed. Yet deciding how to take advantageof them (if at all) was linked to practical matters.For example, initial concern was expressed as towhether or not users would feel comfortablesharing their work. Latter evaluations showedthat participants greatly appreciated the oppor-tunity to share with colleagues, even asynchro-nously. This insight led to the question ofwhether or not (and to what extent) CASCADE-SEA should be available via the Internet. And, inturn, this led to further questions such as:Should additional resources be incorporated tofacilitate sharing through this medium? Andwhat about including information pertaining tothose individuals who contribute, so that teach-ers may contact them if they desire additionalinformation? Emerging questions such as thesedemonstrate just how connected the aspects ofvalidity and practicality can be when conduct-ing developmental research. The following sec-tion examines other issues that may have startedout as what-to-address questions and eventuallyevolved into how-to-do-that discussions.

Throughout the evolution of the various pro-totypes, insight was gained toward optimizingthe practicality of the program. Formative evalu-ation data improved understanding about waysto structure the program, how the programcould or should be used, and enhancements

from a technical standpoint. In terms of thestructure of the program, the visual representa-tion had an impact on user interpretation andunderstanding with regard to the curriculumdevelopment process. This was accomplishednot only by presenting steps to carry out (e.g.,main menu), but also by encouraging users toindicate when they were not prepared to per-form certain tasks.

Because of repeated participant requests, theuser group was extended to include preserviceteachers during formative evaluation activities(although not in development efforts). This gaverise to the realization that CASCADE-SEA mightbe useful to both preservice teachers and facilita-tor teachers engaged in professional develop-ment activities. But whether used withpreservice or inservice teachers, the usefulnessof the program would, in part, be determined byits availability. That is, the limited access ofusers to computers must be addressed for train-ing to make sense. Obviously, computer avail-ability is a prerequisite for initial workshops totake place. But after that, participants must stillbe able to access the program regularly andoften if it is to offer any added value. Anotherinsight with regard to extending the CASCADE-SEA user group stems from the way it is struc-tured. From the very first prototype,participants clamored about the importance ofexamples. Decisions on how to incorporateexamples into the program were linked to per-ceptions regarding CASCADE-SEA’s validity.Namely, opinions were mixed with regard towhether or not to include subject matter sup-port. At the same time, it seemed as though,when it came to examples, more was always bet-ter. Further, those examples should be as close tothe user’s own field of expertise as possible.Hence, the decision was made to structure theprogram with generic rather than subject-spe-cific guidelines and advice, but to offer subject-specific examples as often as possible. Those fewparticipants involved in formative evaluationactivities who did not benefit from the relevantcontent-based examples, still made use of thegeneric support. They also recommended that,by adding more examples, the program (andthus the user group) could be extended to othergrade levels and other subject areas.



Insights related to technical and interfaceaspects pertained primarily to screen design andoffering users extensive opportunities for sup-port through additional software. Although cer-tain zones were distinguished in the interfacestarting with the first prototype (navigation,instructions, interaction, etc.), the practicality ofthe interface design improved as a result of for-mative evaluation feedback. For example, on-screen instructions were further divided intoprocedural and content-related information.Improved understanding with regard to whatCASCADE-SEA should actually aim to do (gen-erated mostly through validity-related data)served to sharpen ideas on where CASCADE-SEA should ’leave off’ in terms of offering sup-port. This, in turn, made it easier to judge whattypes of external programs would be usefuladditions to the CASCADE-SEA suite.

Data collected during the design phaseshowed that CASCADE-SEA does have thepotential to positively affect user performance,but it should be noted that this aspect is difficultto measure through the relatively short-term(one week or less) activities that were under-taken during these circuits. Participants judgedmost of the materials produced with CAS-CADE-SEA to be of equal or better quality thanthose produced without the computer. They did,however, comment that the materials could beimproved by offering additional support toteachers in terms of describing what learnersshould be doing during lessons (as opposed tomostly providing guidelines for what theteacher should do). In addition, the vast majorityof participants shared the opinion that CAS-CADE-SEA has the potential to contribute to theprofessional development of its users. However,most participants involved during the designphase had little previous exposure to any formof computer-supported curriculum develop-ment. Without any similar experience for com-parison, it may have been difficult for theseindividuals to imagine what CASCADE-SEAcould be missing, or how it might be improved.One the other hand, participants were quite ableto comment on the overall quality of the work-shops in which CASCADE-SEA was used.Although no great surprise, these results implythat the ability of CASCADE-SEA to contribute

to professional development is partially depen-dent on the way in which it is implemented.

Phase 3: Semisummative evaluation

Main purpose. Because the end evaluation wasprimarily summative in nature but did maintaina number of formative evaluation elements, theterm semisummative is used. Although the finalphase did consider validity and practicalityissues, the main aim of the semisummative eval-uation was to determine whether the system cre-ated possessed the potential to affect theperformance of its users. Particular attentionwas given to the quality of materials developedwith the aid of the system when compared tomaterials developed without computer-basedsupport. Further, the professional developmentof users resulting from interaction with the pro-gram was also examined. This phase addition-ally touched on generating ideas for continuingthis line of inquiry in the future, and exploredpotential uses for CASCADE-SEA outside therealm of the original intentions.

Methods. The semisummative evaluation wascomposed of two cycles: (a) a final evaluation(three circuits) and (b) an exploratory query (onecircuit). Data were collected during this phasethrough a full-day expert appraisal, a week-longmicroevaluation and a month-long tryout. Thethree final evaluation circuits involved 45 partic-ipants, from the Netherlands, Tanzania, andNamibia. Their responses were captured via 14instruments: 4 questionnaires; 3 discussionguides; 3 observation schemes; 1 logbook; and 3document analysis checklists. The query wasconducted with two goals in mind. To a lesserextent, this activity contributed to (a) answeringthe main research question; but the foremostaim was (b) exploring options for follow-upresearch. The query circuit received input from34 interested parties from North America,Europe, Africa, and Asia, whose ideas weregathered through one questionnaire.

Findings and implications. All of the users andmost of the experts were enthusiastic about thevalidity of the program. However, the degree to


56 ETR&D, Vol. 53, No. 2

which the program may be labeled valid ismuch more difficult to pinpoint. For example,participants encountered state-of-the-art knowl-edge inside CASCADE-SEA, as evidenced bytheir classification of the program as systematic,comprehensive, rich, clear, thought-provoking,and logical. But aspects listed as strengths bysome were considered by others to beweaknesses. For example, the enormous amountof information contained in the program wasapplauded by the users and the majority ofexperts, yet a few experts were concerned thatCASCADE-SEA could be overwhelming andconfusing. Further, a few experts from the fieldof curriculum expressed concern about the factthat CASCADE-SEA offers only one curriculumdesign and development paradigm. One evenstated that CASCADE-SEA does not reflectstate-of-the-art knowledge in the curriculumdomain, because in her opinion, it was “not con-structivist.” Additionally, a few science educa-tion experts in Circuit 32 questioned the validityof selected subject matter content. Participantopinions also varied, though not as emphati-cally, in terms of the internal consistency of theprogram. Whereas nearly all participants weresatisfied with this aspect in relation to the inter-face and support, opinions diverged withrespect to the content of the program. Some par-ticipants appreciated the interconnectedness ofthe content in the various components, but themajority found this aspect to be present yetweak. Although determining where the validityof CASCADE-SEA should be placed on a qualitycontinuum remains difficult, the participantreactions indicate that validity of the supportand interface are subject to less dispute than thevalidity of the program content.

The semisummative evaluation of the practi-cality of CASCADE-SEA yielded a number ofobservations about instrumentality, congruence,and cost. Generally speaking, the program wasviewed to be quite practical. With regard toinstrumentality, participants generally appreci-ated the guidance offered by the program,although some concern was expressed (mostlyby experts in curriculum de velopment andteacher professional development) that CAS-CADE-SEA could offer too much step-by-stepguidance. Whereas most participants expressed

satisfaction in terms of the pace with which theycould use the system, there were concerns thatthe computer’s advice would be over-bearing.To a few users and even fewer experts, the levelof English was seen to present an overly difficultchallenge. Most participants felt that the pro-gram was congruent with the needs and wishesof the target group, and many emphasized theimportance of using the program within a train-ing setting. For example, 16 out of the 19 partici-pants in Circuit 31 indicated that they would beinterested in using CASCADE-SEA in futureworkshops. Opinions were more mixed withregard to the costs associated with using the pro-gram, in particular, time investment. About halfof the participants found that CASCADE-SEAshortened the length of time they would other-wise invest, whereas the other half indicated theopposite to be true, primarily because the pro-gram inspired them to be more thorough thanotherwise would be the case. Although sugges-tions were given for improvements, more partic-ipants were consistently satisfied with thepracticality of the support and the interface. Andeven though their reactions were not alwaysunanimous concerning the degree to whichCASCADE-SEA could be labeled practical, theoverall consensus on practicality was more con-sistent when compared to validity.

The data collected during this phase yieldedthe overall conclusion that CASCADE-SEA doespossess the potential to have a positive impacton the performance of its users, but that poten-tial is strongly influenced by how the system isimplemented and personal characteristics ofthose using it. The vast majority of participantsfelt that using CASCADE-SEA could help usersto create better quality materials than theywould on their own. The structured nature ofthe program was seen to help participants artic-ulate (in the form of procedural specifications)useful guidelines for the materials user; the lay-out of the materials created with CASCADE-SEA was judged easy to use. Expanding theexamples (particularly in the database) is onearea in which improvement was repeatedly rec-ommended. With regard to enhancing theprofessional development of users, most partici-pants emphasized that the potential does exist,although a few (mostly experts, as well as a few



users) raised concerns that the program couldmake things too easy for the user and either sti-fle creativity or encourage laziness as a result.The value of the program as a learning tool forpreservice or inservice education was not dis-puted. But participants from both user andexpert groups emphasized that impact potential(on the quality of the materials created and thelearning that takes place) strongly depends onhow the system is used.

The final research cycle gathered ideas forfuture uses of the CASCADE-SEA tool (or analtered version). While 24 of 29 query partici-pants indicated that they would be interested inusing CASCADE-SEA in their own settings,only half of them thought that implementing theprogram into their current infrastructure wouldbe feasible. Participants would (like to) use theprogram in a variety of ways, much broaderthan those intended by the developers. The mostfrequently cited uses were:

• As a practice tool to teach computer literacy.

• For macrolevel curriculum development.

• As an example of interactive educationalmedia.

• As a model and/or tool for classroom actionresearch.

• As an example of how curriculum improve-ment can be approached in developing coun-tries.

• As originally intended, for creation of exem-plary lesson materials.

DISCUSSION

Reflections on the findings

Quality criteria. Much of the data collectedrelated to multiple quality criteria (validity,practicality, and impact potential) simulta-neously. However, during analysis, data wereclassified into only one area according to themost, but not the only, relevant relationship tothe aforementioned criteria. A number of meritsand faults may be associated with the decision totry to sharpen distinctions that are, naturally,more blurred.

Decomposition of the notion of quality was

seen as a route to being able to articulate itsessence. So, when the quality criteria domainswere examined in light of the content, support,and interface traits, the resulting quality aspects(indicated in Figure 3) helped to shape an imageof the desired system. Further, the quality cri-teria helped in terms of understanding the datacollected. In particular, they helped to clarifyand highlight emergent patterns. In this sense,the quality criteria served their purpose.

However, reflection on the findings showsthat they might have served their purpose toowell. That is, careful study of how those desiredaspects would actually take shape revealed that,in a few ways, they were in competition witheach other. Take, for example, program content.When it comes to making decisions about thecurriculum design and development knowledgethat should be included in CASCADE-SEA, anumber of aims were pitted against each other:

• Toward state-of-the-art knowledge (validity),curriculum design and development knowl-edge should be included, but the question ofhow much state-of-the-art knowledge is leftover to practicality, which says:

• Toward optimizing user cost (practicality),the program should include enough of whatthe users need and not bog them down withunnecessary steps, but:

• Toward supporting professional develop-ment (impact potential), the program shouldhelp users to think about materials develop-ment in a more systematic and thoroughfashion.

Depending on the user’s perspective, offeringcontent that stimulates the user to be more thor-ough than usual could alternatively be experi-enced as food for thought or as too muchunnecessary information.

It appears that conflicts such as these areinherent in the structure of the quality aspects, asthey were defined in this study. However, it wouldbe unfair to say that the ramifications of their fric-tion were thoroughly understood at the time theinvestigation (and in particular, the main researchquestion) was conceived. As these implicationsemerged, they served as perpetual reminders toweigh available options carefully before makingthe unavoidable trade-off decisions.


58 ETR&D, Vol. 53, No. 2

Determining the validity, practicality, andimpact potential of CASCADE-SEA was not astidy as trying to answer a yes–no question.Understanding the program’s quality really per-tains less to a black or white issue and more toassessing where, on a grey scale, the programshould be located. But such a location is in flux,depending on which quality aspects are beinginspected, and which ones are given priority.Moreover, these issues are dependent on who islooking to carry out what tasks under which cir-cumstances. As a result, these are reflections ofpersonal value judgments, more than indica-tions of any objective facts. During evaluationactivities, participants gave their opinions on thediffering quality criteria separately. For exam-ple, they rarely reflected on validity issues influ-encing practicality ones and, as one wouldexpect, they usually evaluated the programagainst the backdrop of their own, individual,needs and interests.

Target context. To design a good quality systemfor a particular setting, a working knowledge ofthat setting is needed, as well as an understand-ing of what good quality is for that situation. Theprevious section examined the notion of qualityas addressed in this study. This section contin-ues the discussion with a look at contextual fac-tors that influenced trade-off decisions. For eachprogram trait (content, support, and interface)judgments had to be made when it came to pri-oritizing state-of-the-art (a validity issue) versusstate-of-practice in the target context (a practical-ity issue).

For example, in terms of program content,CASCADE-SEA does not explicitly address con-structivism, which could be an important valid-ity aspect for some people. But, some expertsargue that radical constructivism (in the shortterm) can actually do a disservice to Africaneducation (Taylor, 2000). They cite a combina-tion of factors to be the cause, such as disputedramifications and benefits of this approach anddebate on how to implement its methods, and itsreliance on a well-educated teaching force (cf.Sanders, 1999). The choice to build CASCADE-SEA as a teaching and learning methods “agnos-tic” was predicated on the supposition that itshould speak to the highly varied needs of the

majority, not the specialized preferences of thefew.

In addition to influencing program content,contextual factors also shaped the way supportwas designed inside the system. In particular, itwas designed for integration into a professionaldevelopment program, not to be used in isola-tion. For example, a number of participantslamented the fact that CASCADE-SEA allowedusers too much freedom, but others were con-cerned that the computer had too much controlover the materials development process. Interms of being able to make a valuable contribu-tion to professional development, CASCADE-SEA supports users during curriculumdevelopment processes, not through prescrip-tion but by stimulating deliberate consideration.Hence, support offered inside the tool wasdesigned to encourage reflection and group dis-cussion (in person, not only via the computer).Opting for a program that relies on the localinsights of those using it in a professional devel-opment setting (as opposed to a self-guidedapplication) was based on a belief that thiswould, ultimately, be more appropriate andeffective for the target context. Given that partic-ipant perceptions are an influential ingredient inprogram impact (Guskey, 2000), such anapproach seemed to be worthwhile, because itallowed for tailor-made use (i.e., integration intoexisting frameworks), and it stimulated would-be users to consider how to maximize the poten-tial benefits of the program within their ownsettings.

In terms of the program’s interface and tech-nical complexion, numerous clashes betweenstate-of-the-art and state-of-practice are obvious,given the fact that technological infrastructure isnotoriously weak in developing countries. Forexample, the poor or out-dated facilities avail-able at many resource centers provided a soundargument for building a technically “lite” ver-sion of CASCADE-SEA. On the other hand, therealization that the final version of the programwas to be ready a full four years after theresearch was initiated provided an argument toconsider state-of-the-art now versus anticipatedstate-of-practice in four years. This allowed formore technically demanding aspects to beincluded than otherwise might have been advis-



able. Still, certain options (such as an Internet-dependent version, as opposed to the resultingInternet-supplemented one) were ruled outbecause of contextual constraints. Even in set-tings where infrastructure is not a barrier,experts have found that, when it comes to EPSS,the fact that something is within the realm ofpossibilities does not guarantee its feasibility inpractice (Miller, 1997). In terms of the interfaceand technical aspects of CASCADE-SEA, vari-ous types of factors (infrastructure, know-how,etc.) within the target context instigated continu-ous Can it be done? . . . Should it be done? delib-eration during design and developmentactivities.

In this study, some elements of practicalityactually turned out to be prerequisites to deter-mining aspects of validity. For example, deci-sion-making regarding state-of-the-art curricu-lum development knowledge (a validity issue)to be integrated into the system was stronglyinfluenced by understanding of target user per-ceptions and habits (a practicality issue). This iscontrary to the assumption made at the begin-ning of the research that practicality could bestbe studied after validity issues were clarified.This assumption steered a shift in emphasisfrom validity to practicality to impact potentialduring data collection activities. Fortunately,however, the methods used allowed for com-pensation in this area, as the integrated nature ofprototypes (and the evaluation thereof) empha-sized assessment of the system as a whole. Infact, because validity and practicality wereassessed simultaneously, this process high-lighted areas of discord by bringing potentialconflicts between them to the forefront of discus-sions.

Revisiting the Research Approach

A developmental research approach was usedthroughout this study because of the opportuni-ties it offers in terms of devoting attention to theuncertainties and complexities of educationalrealities. Although the overall benefits of thisapproach are perceived to have outweighed therisks, thoughtful contemplation of the executionof this study would not be complete without dis-

cussing the enduring struggle between thechallenges and opportunities presented by it.Three main points are addressed in this regard:(a) multiple roles played by the main researcher;(b) dilemmas associated with authentic researchsettings; and (c) the emergent (adaptive) designof the study.

Many roles. The study was both challenged andenriched by the fact that the main researcherwas (simultaneously) also the designer, devel-oper, programmer, and (in most cases) evaluatorof the CASCADE-SEA program. Further, it wasthe same individual who (usually) served as thefacilitator of workshops and activities in whichparticipants became acquainted with the soft-ware. This conscious decision to take on multi-ple roles was based primarily on the perceivedbenefits associated with a constant overview ofdevelopments and their implications. For exam-ple, a constant overview of the various aspects ofthe study (research history, future directions,developer intentions, etc.) allowed theresearcher–evaluator to network toward addi-tional testing situations during field work. Addi-tionally, the practicality and impact potential ofthe CASCADE-SEA tool was improved by help-ing the developer to understand the context bet-ter. This was even the case in situations wheredirect prototype feedback was minimal, non-committal, or for other reasons weak, whichwould have rendered the findings less relevantfrom the evaluator standpoint, and might other-wise have gone unused.

Another decision that helped to maximizethe potential benefits of multiple roles was theselection of a friendly, easy-to-learn environ-ment for developing the software. The front-endinvestment in training the researcher to useMacromedia’s Authorware™ programmingenvironment (and having a programming coachon call) paid off by allowing for truly rapid pro-totyping to take place, both in the researcher’smind and electronically. One of the most sub-stantial contributions this decision yielded per-tains to problem solving in designing theCASCADE-SEA program. Generally speaking,software developers are acutely more aware oftechnical options available than are typical edu-cational designers, so much so in fact, that agreat deal of their tacit knowledge often remains


60 ETR&D, Vol. 53, No. 2

inaccessible to other members of a developmentteam. This was certainly the case at the start ofthe CASCADE-SEA endeavor. Nevertheless, byschooling the researcher–designer in how (andwhy) certain technical aspects are structured,ideas for support were approached creatively,from the various perspectives. Further, theseideas were able to evolve quicker and moreappropriately by actually creating, testing, andrevising them immediately, with the input of themultiple roles.

Cognizant of the threats to the study associ-ated with the decision to place the mainresearcher in so many roles simultaneously, twomain precautions were taken to mitigate them:(a) vigilant segregation of data and subjectiveinferences; and (b) triangulation. Recordingimpressions through photographs, audio- andvideotapes, developer logbooks, and field note-books helped to interpret the data, but thesesources were not interspersed with the data.Additionally, three forms of triangulation wereachieved: data, method, and investigator. Vary-ing the sources of data (time acquired, locationsof activities, and people involved) yielded multi-ple opportunities to examine similar phenom-ena. This mitigated the threat of introducingrandom errors due to drawing premature con-clusions based on unique instances. Differingmethods of data collection (e.g., through inter-views, observations) resulted in data summarieswhose sources had been analyzed and thencross-checked, to validate the findings. Wherepossible, data analysis and interpretation wereconducted together with other individuals(research assistants, developer group, and criti-cal friends), to reduce the threat of personal bias.This also reduced the chance of systematic error.

Real-world research settings bring real-world com-plications. The research approach used in theCASCADE-SEA study allowed data collectionactivities to take place in cooperation with natu-rally occurring test beds. The benefits of con-ducting parts of the study in authentic settingsseem obvious: The more genuine the researchsituation, the more genuine the research resultswill be. To be sure, these benefits outweigh therisks when it comes to investigations such as thisone. However, deeper understandings come atthe potential cost of losing control over data col

lection rigor. When researcher interests are nolonger the only ones at stake, compromise isimminent.

For example, participants in earlier circuitslearned about the CASCADE-SEA program, andraised the issue of exploring the potential role ofthis program in preservice education. Addingpreservice teachers and associated counterpartsto the sample group resulted in the collection ofunanticipated types of information that turnedout to be quite useful. However “populationvalidity” (cf. Bracht & Glass, 1968) was conse-quently threatened. Given the fact that general-ization to a particular population or situationwas not a high priority, and that deeper contex-tual understanding was considered more valu-able, this seemed an acceptable concession.Nonetheless, it does typify the kinds of dilem-mas faced by those who prioritize the mutualbenefit of research activities (for researchers andparticipants). At the same time, it emphasizesthe need for researcher–participant creativity inthe selection of activities that meet the needs ofall parties in volved.

In addition, real-world investigation venuesalso place certain limitations. Particularly whena “cultural stranger” (cf. Choksi & Dyer, 1997 inThijs, 1999) attempts to carry out research in aforeign setting (as was the case with most cir-cuits of data collection activity), the degree towhich an outsider can conduct meaningfulresearch must be addressed. In many situations,participants are hesitant to be completely openwith researchers from different cultural con-texts. Toward earning participant trust andbuilding an understanding of the context, theimportance of collaboration and mutually bene-ficial activities cannot be over emphasized; theseare the two main avenues available to aresearcher who prioritizes the insider perspec-tive. This is not to say that being an outsider iscompletely without advantages. In some situa-tions, it actually allows for a degree of objectivityand, along with that, a freedom (or forgiveness)for honesty that is not permitted to those withina particular group.

Adaptability. The adaptability built into theapproach used throughout this study has of-fered many possibilities for understanding anddealing with the myriad complexities inherent



to the nature of the CASCADE-SEA investiga-tion. While developmental research offers manybenefits and opportunities, such an emergentapproach also demands that attention be givento the resulting consequences in terms of how astudy is designed, as Smith indicated:

Whether emergent programs are more developmental,driven by the attainment of programmatic goals, ormore adaptive, responsive to certain pressures forcingprogrammatic change, they are characterized by rapid,often dramatic, changes which make the creation andimplementation of stable evaluation designs very diffi-cult. Successfully evaluating such programs requires acertain degree of flexibility in the design and conductof the evaluation study. (1990, p. 209)

Within the framework of a systematic approach,this study incorporated the necessary and suffi-cient conditions for enabling the reflective pro-cesses that fed its evolution.

The flexibility provided by multiple perspec-tives allowed fortuitous data collection opportu-nities to be utilized. Even though the structure ofdata collection activities was subject (if not open)to local influences, the basic sequence of explor-ing validity, practicality, and impact potentialremained the same. Not only was this adaptabil-ity valuable because of the additional data thatwere collected, it also proved insightful in termsof learning about what additional kinds of datacollection opportunities could be useful. Forexample, researcher expectations regarding thesalience and intensity of some data collectionactivities were quite low, especially in such non-committal settings as brief (half-day) demon-strations or discussions. But by having theflexibility to try them out some of these activitiesproved to be much more useful than anticipated.Although rarely sufficient to stand alone with-out the input of longer term, hands-on partici-pant experience, data collected in these sessionsoften served a confirmative purpose highlight-ing emerging patterns in the findings. Similarly,these types of sessions often informed one per-spective (hat) better than another. And finally,because priority was given to the quality of thefeedback (as opposed to the quantity), even briefsessions allowed particularly insightful individ-uals to share their ideas.

Circuit 26 was notably illustrative in this

regard. It was entirely unplanned (it took placewhile the researcher was in Namibia for anotherpurpose) and seemed, at the start, to have morepotential toward public relations than towardthe collection of useful data. However, the dis-cussion that took place during that two-hourseminar turned out to be one of the deepest,most insightful sessions in the entire study.While it yielded relatively little data in terms ofdetailed prototype feedback (making this circuitless relevant for the evaluator hat), this circuitcontributed significantly to the evolving concep-tualization of what the program should (andshould not) aim to do and why, thus being quiterelevant from the developer viewpoint.

Conclusions

The CASCADE-SEA study was designed toexplore the potential of the computer to supportscience and mathematics education materialsdevelopers in southern Africa. Through thedevelopment of a valid and practical system,this study has shown that the computer doeshave the potential to positively affect curriculumdevelopment and teacher development by sup-porting the creation of exemplary lesson materi-als in the aforementioned context. Usersgenerally produce better materials than theyotherwise would, and learn from this processbecause of the program’s combination of the fol-lowing traits:

Content: CASCADE-SEA systematicallystructures the materials development processand illustrates its iterative nature through analy-sis, design, and evaluation activities that areguided by an explicit rationale.

Support: CASCADE-SEA blends generic andtailor-made advice, internal and external tools,implicit and explicit learning opportunities, andwritten and verbal communication aids to assistthe user throughout the materials creation pro-cess.

Interface: CASCADE-SEA offers the contentand support through a direct, consistent, andforgiving visual and technical representation,which grants the user both flexibility and controlover the process.

This study has produced both substantive


62 ETR&D, Vol. 53, No. 2

and procedural outputs (cf. van den Akker,1999). Substantive outputs include the CAS-CADE-SEA program itself, as well as designprinciples pertaining to the characteristics of thetool; procedural outputs include insights on asystematic approach to the research and devel-opment of a tool for the specified purpose. Thetool was described, in brief, at the beginning ofthis article. The design principles, which werenot previously addressed in this article becauseof length restrictions, are briefly described here.They have been articulated on three levels ofabstraction: (a) foundational tenets, (b) develop-ment guidelines, and (c) product specifications.As mentioned in the section on Research Design,foundational tenets were formulated in accor-dance with the overall aims of this study. Thesame themes also influenced the design of theprogram. These relate to the issues of local rele-vance, collaboration, authenticity of the researchsetting, intensifying the synergy that shouldexist between curriculum development andteacher development, and the importance ofcareful, continuous analysis and evaluation ofthe target setting. Further, reflection on researchexperiences together with related theories andmodels (pertaining to curriculum developmentin general and the creation of exemplary lessonmaterials in particular) generated fewer abstractdevelopment guidelines for creating the desiredprogram. The most concrete layer was the prod-uct specifications, which represented developerideas on how to achieve the characteristics pre-scribed by the foundational tenets and the devel-opment guidelines for the content, support, andinterface of the program. Detailed discussion ofthe design principles (as well as the CASCADE-SEA program) may be found in McKenney(2001a). Together with the CASCADE-SEA pro-gram itself, these design principles speak to themain research question insofar as they illustratethe characteristics of a computer-based supporttool for secondary-level science and mathemat-ics education in southern Africa that is bothvalid and practical, with the potential to have apositive impact on the performance of its users.

The aforementioned tenets influenced thedesign of the CASCADE-SEA tool, as well as thestructure of this study. These tenets, togetherwith the experiences gained through the CAS-

CADE-SEA study, have produced understand-ing that may be useful in structuring futuredevelopmental research activities. They shapethe recommendations in the next section.

Recommendations

Local relevance. Whereas the notion of designingeducational innovation from an implementationperspective is not a new one (cf. van den Akker,1994), determining how to factor in cultural andcontextual realities remains a challenge. Tess-mer and Harris (1990) spoke of a potential con-flict of interest between following soundinstructional design practices and paying atten-tion to real world challenges and limitations.They made a distinction between “doing thingsright” and “doing the right things” in instruc-tional design. Developmental research (particu-larly formative research) addresses thispotential conflict through the combination ofcareful design based on validated models (doingthings right) that is then tested and revised inpractice (increasing the chance for doing theright things). Such successive approximation ofinterventions in interaction with practitionersdistinguishes developmental research fromother research approaches (van den Akker,1999). Evolutionary development (fed by infor-mation from research activities), yields greateropportunity for coping effectively with contex-tual factors that may not have been present, evi-dent, or articulated during earlier phases ofdevelopment. This increases the practical rele-vance of that which is being designed, which, inturn, affords deeper insight into good designpractice for that particular setting.

Collaboration. In a study such as this one, wheresignificant emphasis has been placed on retailor-ing initial design ideas to best fit the context, thenotion of collaboration bears mention. It waspreviously stated that the design and develop-ment activities should (where feasible and func-tional) take place in collaboration with, and notfor, those involved. Although this may add com-plications to the design process, the potentialbenefits of a participatory design process makeit worthwhile. A participatory design approachis distinguished from other, more traditional



approaches by (among others) the assumptionthat users themselves are well equipped todetermine how to improve their work and pro-fessional environment. Further, “it views theusers’ perceptions of technology as being at leastas important to success as fact, and that theirfeelings about technology are at least as impor-tant as what they can do with it,” (Schuler &Namioka, 1993, p. xi). In the case of CASCADE-SEA, user involvement was thought to helpestablish a better fit in terms of system design.That is, the input helped designers to betterunderstand the context in which the programwould be implemented, as well as to sharpentheir understanding of user needs. Secondly,this approach can simultaneously foster owner-ship on behalf of the users, thus increasing thepotential for the innovation to become acceptedand implemented (or at least tried out) in the tar-get setting. Finally, as with the creation of lessonmaterials, participating in the development pro-cess can contribute to professional growth. Asresearchers and respondents jointly reflect onhow curriculum development takes place, andhow that could be supported through the com-puter, both parties learn from the experience (cf.Nieveen, 1997).

Authenticity. Design and development activi-ties should be integrated, where possible, withthe existing endeavors taking place in the targetsetting. In the case of CASCADE-SEA, thismeant that curriculum materials developmentefforts should be integrated into a framework ofinservice education (or, in some cases, pre-service education), which usually occurs atteacher resource centers. Using a concurrentdesign approach, prototypes can be developedtogether with the users, even while doing addi-tional situational analysis (Tessmer & Wedman,1995). This helps to optimize the link betweenteacher development and curriculum develop-ment within the given setting. In turn, it canincrease the authenticity and, thereby, the over-all richness and value of the results.

Mutual benefit. Embedding research activitiesinto an existing framework of teacher profes-sional development requires that the researchersexercise caution as well as creativity in selecting

data collection methods. It must be made clear tothe researchers, as well as those participating inthe study, that the goals of the activities are notrelated only to developing a prototype. Rather,these activities must also serve the professionaldevelopment of the participants. In this study,researchers were careful to maintain this dualfocus throughout the data collection activities,which required imaginative approaches to col-lecting useful information while maintainingcare for the better interest of those involved.

Continuous analysis. One way to maintain adual focus is through the use of prototyping.Prototyping has traditionally been associatedwith engineering fields of study. Yet throughtime, this has grown to include other arenas thatapply a systematic approach to problem solvingand design. Tessmer (1994) noted that theadvent of prototyping allows evaluators toreview functional versions of products at anearly stage, thus making formative evaluationmore a part of the front-end analysis and design.This study involved users in the design processby continuously field-testing working evolu-tionary prototypes. For additional informationon prototyping to reach product quality, pleaserefer to Nieveen, (1999).

Epilogue

The reflections presented earlier ascertained thatthe desired quality aspects of tools such as CAS-CADE-SEA are neither absolute nor completelyobjective. Rather, they are relative to the contextin which the program is used. In addition, theneeds, expectations, and beliefs of individualusers shape perceptions about quality. Further,the user–facilitator agenda determines, in part,the way in which educational technology isused. This study has highlighted the importanceof continuously seeking heightened contextualunderstanding as an integral part of the designand development process. Having been warnedagainst the “wholesale transplantation” of ideas(Ogunniyi, 1996), the CASCADE-SEA study hasgenerated insight into how to research thedesign of interventions that are culturally andsocially portable. Despite its own methodologi-


64 ETR&D, Vol. 53, No. 2

cal challenges, it is our conviction that a devel-opmental research approach has the potential tomake a significant contribution in such efforts.

To date, little research has been published inthe area of computer-based support for curricu-lum development in developing countries. Andwhether or not outputs of this study will be usedto inform future activities remains to be deter-mined. Yet the CASCADE-SEA expedition hasshown that, if carefully embedded into existingactivities where a need and a readiness exists,such an approach can be potentially advanta-geous. It is hoped that this realization will pro-mulgate additional inquiries into this promisingdomain.

Susan McKenney [[email protected]] andJan van den Akker [[email protected]]work in the Department of Curriculum, Faculty ofBehavioral Sciences, University of Twente in theNetherlands. The authors would like to thank our anonymousreviewers and Jim Klein for their constructivecriticism on this article.

REFERENCES

Ball, D., & Cohen, D. (1996). Reform by the book: Whatis—or might be—the role of curriculum materials inteacher learning and instructional reform? Educa-tional Researcher, (25)9, 6–8, 14.

Ben-Peretz, M. (1990). The teacher-curriculum encounter.Albany: State University of New York Press.

Black, P., & Atkin, J. (Eds.), (1996). Changing the subject:Innovations in science, mathematics and technology edu-cation. London: Routledge.

Bracht, G., & Glass, G. (1968). The external validity ofexperiments. American Educational Research Journal,5, 437–474.

Caillods, F., Göttelman-Duret, G., & Lewin, K. (1996).Science education and development: Planning and policyissues at secondary level. Paris: unesco/iiep.

Clandinin, J., & Connelly, M. (1992). Teacher as curric-ulum maker. In P. Jackson (Ed.), Handbook of researchon curriculum (pp. 363–401). New York: Macmillan.

de Feiter, L., Vonk, H., & van den Akker, J. (1995).Towards more effective science teacher development insouthern Africa. Amsterdam: VU University Press.

Doyle, W., & Ponder, G., (1978). The practicality ethicin teacher decision-making. Interchange, 8(3) 1–12.

Eisenhart, M., & Borko, H. (1991). In search of an inter-disciplinary collaborative design for studyingteacher education. Teaching and Teacher Education,7(2), 137–157.

Eisner, E. (1994). The educational imagination (2nd ed.).

New York: Macmillan.Fullan, M. (2001). The new meaning of educational change.

New York: Teachers College Press.Gery, G. (1991). Electronic performance support systems:

How and why to remake the workplace through the stra-tegic application of technology. Boston: Weingarten.

Goodlad, J., (1994) Curriculum as a field of study. In T.Husén & T. Postlethwaite (Eds.), The internationalencyclopedia of education (pp. 1262–1267).

Guskey, T. (2000). Evaluating professional development.Thousand Oaks, CA: Sage Publications.

Lieberman, A. (1986). Collaborative research: Workingwith, not working on . . . . Educational Leadership,43(5), 4–8.

Loucks-Horsley, S., Hewson, P., Love, N., & Stiles, K.(1998). Designing professional development for teachersof science and mathematics. Thousand Oaks, CA:Corwin Press.

Marsh, C., & Willis, G. (1995). Curriculum: Alternativeapproaches, ongoing issues. Englewood Cliffs, NJ:Merrill.

McKenney, S. (2001a). Computer-based support for sci-ence education materials developers in Africa: Exploringpotentials. Doctoral dissertation. Enschede: Univer-sity of Twente.

McKenney, S. (2001b). The CASCADE-SEA programWebsite [Online]. Available: http://pro-jects.edte.utwente.nl/cascade/seasite/.

McKenney, S. (2001c). The CASCADE-SEA researchWebsite [Online]. Available: http://pro-jects.edte.utwente.nl/cascade/seastudy/.

Miller, B. (1997). What is in the realm of possibilities maynot be possible [Online]. Available: http://www.pcd-innovations.com/design_ideas.htm.

Nieveen, N. (1997). Computer-based support for curricu-lum developers: A study on the potential of computer sup-port in the domain of formative curriculum evaluation.Doctoral dissertation. Enschede: University ofTwente.

Nieveen, N. (1999). Prototyping to reach product qual-ity. In J. van den Akker, R. Branch, K. Gustafson, N.Nieveen, & T. Plomp (Eds.), Design approaches andtools in education and training (pp. 125–136).Dordrecht: Kluwer Academic Publishers.

Nieveen, N., & Gustafson, K. (1999). Characteristics ofcomputer-based tools for education and trainingdevelopment: An introduction. In J. van den Akker,R. Branch, K. Gustafson, N. Nieveen & T. Plomp(Eds.) Design approaches and tools in education andtraining (pp. 155–174). Dordrecht: Kluwer AcademicPublishers.

Nieveen, N., & van den Akker, J. (1996). Computersupported curriculum development. In T. Plomp &D. Ely (Eds.), International encyclopedia of educationaltechnology (pp. 153–158). Oxford: Pergamon.

Noddings, N. (1986). Fidelity in teaching, teacher edu-cation and research for teaching. Harvard EducationalReview, 56(4), 496–510.

Ogunniyi, M. (1996). Science, technology and mathe-matics: The problem of developing critical human



capital in Africa. International Journal of Science Edu-cation, 18(3), 267–284.

Raybould, B. (1990). Solving human performanceproblems with computers. Performance and Instruc-tion 29(10), 4–14.

Reeves, T. (2000). Socially responsible educationaltechnology research. Educational Technology 40(6),19–28.

Richey, R., & Nelson, W. (1996). Developmentalresearch. In D. Jonassen (Ed.), Handbook of research foreducational communications and technology (pp. 1213–1245). London: Macmillan.

Sanders, M. (1999). Implementing outcomes basededucation in South Africa: What lessons can scienceeducators learn from classroom practitioners inNew Zealand? In Proceedings of the Seventh AnnualMeeting of the Southern African Association for Researchin Mathematics and Science Education (pp. 382–405).Harare: University of Harare.

Schuler, D., & Namioka, A. (Eds.) (1993). Participatorydesign: Principles and practices. Hillsdale, New Jersey:Lawrence Erlbaum Associates.

Seels, B., & Richey, R. (1994). Instructional technology:The definition and domains of the field. Washington DC:Association for Educational Communications andTechnology.

Smith, N. (1990). Flexibility in the evaluation of emer-gent programs. Studies in Educational Evaluation, 16,209–229.

Stevens, G., & Stevens, E. (1995). Designing electronicperformance support tools: Improving workplace perfor-mance with hypertext, hypermedia and multimedia.Engelwood Cliffs, NJ: Educational Technology Pub-lications.

Taba, H. (1962). Curriculum development: Theory andpractice. New York: Harcourt, Brace & World.

Taylor, N. (2000). Sustainable change for educational sys-tems in transition. Keynote address at the DesigningEducation for the Learning Society Internationalconference, Enschede. Available:Http://192.87.215.20/bslo/AnythingNick.doc.

Tessmer, M. (1994). Formative evaluation alternatives.Performance Improvement Quarterly, 7(1), 3–18.

Tessmer, M., & Harris, D. (1990). Beyond instructionaleffectiveness: Key environmental decisions forinstructional designers as change agents. EducationalTechnology, 30(7), 16–20.

Tessmer, M., & Wedman, J. (1995). Context-sensitiveinstructional design models: A response to designresearch, studies and criticism. Performance Improve-ment Quarterly, 8(3), 38–54.

Thijs, A. (1999). Supporting science curriculum reformin Botswana: The potential of peer coaching. Doc-toral dissertation. Enschede: University of Twente.

van den Akker, J. (1994). Designing innovations froman implementation perspective. In T. Husén & T.Postlethwaite (Eds.), The international encyclopediaof education (pp. 1491–1494). Oxford: PergamonPress.

van den Akker, J. (1996). Het Studiehuis: Ook eenleeromgeving voor docenten? [The study house: Also alearning environment for teachers?] (Inauguraladdress). Amsterdam: Vrije Universiteit Amster-dam.

van den Akker, J. (1999). Principles and methods ofdevelopment research. In J. van den Akker, R.Branch, K. Gustafson, N. Nieveen, & T. Plomp(Eds.), Design approaches and tools in education andtraining (pp. 45–58). Dordrecht: Kluwer AcademicPublishers.

Walker, D. (1990). Fundamentals of curriculum. SanDiego: Harcourt, Brace Jovanovich.

Wilson, B., & Jonassen, D. (1991). Automated instruc-tional systems design: A review of prototype sys-tems. Journal of Artificial Intelligence in Education,2(2), 17–30.

Yager, R., (1994). Science teacher education. In T.Husén & T. Postlethwaite (Eds.), The internationalencyclopedia of education (pp. 5342–5345). Oxford:Pergamon Press.

Zumwalt, K. (1988). Are we improving or undermin-ing teaching? In L. Tanner (Ed.), Critical issues in cur-riculum (87th yearbook of the National Society forthe Study of Education) (pp. 148–174). Chicago: Uni-versity of Chicago Press.


66 ETR&D, Vol. 53, No. 2

Documents

Computer-Based Support for Curriculum Designers: A Case of