Teaching and Learning as Multimedia Authoring:The Classroom 2000 Project

Gregory D. Abowdl’2, Christopher G. Atkeson2, Ami Feinstein4, Cindy Hmeto31

1’2 Nitin ‘(Nick” Sawhneyl (?4Mikiya Tanil’5Rob Koope6’2J Sue Long ,

lGVU Center, 2(lollege of Computing,

3EduTech Institute & 4Office of lnformat ion Technology

Georgia Institute of Technology, Atlanta, GA, USA

5NEC Kansai C&C Research Laboratory, Osaka, JAPAN

ABSTRACT

We view college classroom teaching and learning as amultimedia authoring activity. The classroom providesa rich setting in which a number of different forms ofcommunication co-exist, such as speech, writing andprojected images. Much of the information in a lec-ture is poorly recorded or lost currently. Our hypoth-esis is that tools to aid in the capture and subsequentaccess of classroom information will enhance both thelearning and teaching experience. To test that hypothe-sis, we initiated the Classroom 2000 project at GeorgiaTech. The purpose of the project is to apply ubiquitouscomputing technology to facilitate automatic capture,integration and access of multimedia information in theeducational setting of the university classroom. In thispaper, we discuss various prototype tools we have cre-ated and used in a variety of courses and provide aninitial evaluation of the acceptance and effectiveness ofthe technology. We also share some lessons learned inapplying ubiquitous computing technology in a real set-ting.

KEYWORDS: Educational technology, ubiquitous com-puting, pen-based computing, audio/video capture, me-dia integration.

1 INTRODUCTION

One way to view classroom teaching and learning is asa group multimedia authoring activity. Before class,teachers prepare outlines, slides, or notes and studentsread text books or other assigned readings. During thelecture, the words and actions of the teacher and stu-dents expound and clarify the lessons underlying theprepared materials. It is common practice to anno-

Permission to make digitalkwd copies of all or pati of this material forpersonal or claasroom use is granted without fee provided that the copiesa~e not made or distributed for profit or commercial advantsge, the copy-right notice, the title of the publication and its date appear, and notice isgiven that copyright is by permission of the ACM, Inc. To copy otherwise,to republish, to post on servers or to redistribute to lists, requires specific

permission and/or fee.ACM Multimedia 96, Boston MA USA01996 ACM 0-89791-871-1/96/11 ..$3.50

tate the prepared material duringcreate new material as notes on a

the lecture and towhiteboard or in a

student notebook. These different forms of material —printed, written and spoken — are all related to thelearning experience that defines a particular course, andyet there are virtually no facilities provided to auto-matically record and preserve the relationships betweenthem. Applying computing technology in the classroomsetting to support the classroom’s group multimedia au-thoring and review experience should leisd to an en-hanced teaching and learning experience.

To test this hypothesis, we initiated the Classroom 2000project at Georgia Tech. The Classroom 2000 projectis applying a variety of ubiquitous computing technolo-gies — electronic whiteboards, personal pen-based inter-faces, and the World- Wide Web — together with sc)ft-ware to facilitate automatic capture and content-basedaccess of multimedia information in the educational set-ting of the university classroom. The goal of the projectis to evaluate and understand the effect of ubiquitouscomputing on the educational experience, both in termsof how it improves current practice and suggests n[ewforms of education. This paper reports on our initialexperiences developing and using a number of classroom

prototypes.

Successful uses of ubiquitous technology will fundament-ally alter some forms of education, but in ways thatare hard to predict. To begin to explore the effects ofsuch technology, our approach is to introduce novel tech-nologies gradually into the traditional classroom lecturesetting and see what happens. We do not intend to re-place the traditional lecture-based style of pedagogy, atleast not initially. However, much of the information ina lecture is inefficiently recorded or lost, amd we can use

simple capturing techniques to improve the situation.

The result is a more complete record of what occurredduring the lecture in a form that can be reviewed moreeasily.

In Section 2. we movide a brief overview of related re-. .search in classroom technology, ubiquitous computing,and automated capture to support access and recall. InSection 3, we outline four different teaching and learningstyles that different prototypes of Classroom 2000 aredesigned to support. Section 4 describes the commonarchitect ural framework used to develop various Class-room 2000 prototypes, with several examples from theactual prototypes provided. In Section 5, we summarizesome initial studies of student reactions to the use ofthis technology in the classroom and we provide someinsights on lessons we have learned in applying ubiqui-tous technology to a real life situation. We conclude inSection 6 with a discussion of how we have progressed to-ward our goal of introducing ubiquitous computing intothe classroom and observing its effect on the educationalexperience.

2 RELATED WORK

Shneiderman et al. [13] discuss the effects of introducingtechnology into the classroom in terms of the paradigmshifts that result. All of the existing systems they dis-cuss, and all oft he attempts we know of, have one com-mon feat ure that we are trying to avoid. Technologyin the hands of the student usually translates into aworkstation at each desk. This approach is fine, even

necessary, for classes that involve computer-based ac-tivities (such as programming). We want to investigatethe usefulness of alternative interfaces that are less in-trusive and allow natural handwritten note-taking, suchas pen-based laptops, PDAs, tablets, or palmtop PCS.

Our work has been greatly influenced by the work atXerox PARC in ubiquitous computing [18, 19] and toolsto support electronic capture and access of collabora-tive activities [9, 10]. We want to capture informa-tion movided bv the- teacher durimz a lecture, so elec-troni; whitebo&d capabilities pro;ided by the XeroxLiveWorks LiveBoard [4] are inviting.1 We also wantedto provide the students with an electronic notebook withthe capability to take notes during the class that couldbe the basis for review after class. The Marquee note-taking prototype developed at PARC [17] and the Filochatprototype developed at Hewlett-Packard Labs [20] bothcame close to what we wanted to have in the hands ofthe students. Marquee provided a simple mechanism forproducing notes with a pen-based interface that also cre-ated automatic indexing into a video stream. Filochatused a pen-based PC to capture electronic annotationsthat served as indices into a dirzital audio stream. Wehave also investigated paper-b&ed solutions to note-taking, similar to the work done by Stifelman [14]. Theimplicit connection between the note-taking device andalternate information streams (audio and/or video) is acommon theme that has also been explored at MIT’sMedia Lab [7] and at Apple [3].

1We have not made use so far of the ability to connect the

LiveBoard to other remote information sources (or even the stu-dent notebooks in the same class), as is done in the DOLPHIN

project [15] and other similar efforts.

With the availability of ubiquitous information tech-nolo~ies. such as the World-Wide Web, most universities-.are able to provide students with access to vast repos-itories of educational materials. It is quickly becomingthe norm for individual courses at manv universities tohave their own Web page that serves as” a central clear-ing house for all course documentation.2 While this useof the Web has some obvious advantages for both in-structor and student, it does not take an active role inassisting learning and teaching. We wanted to view thewhole classroom experience as a multimedia Web au-thoring task and provide ways to capture and relate in-formation before, during and after an actual classroomsession. A serious design decision for the Classroom2000 prototypes, a decision that prevented the use ofsome existing solutions for our prototypes, was to makeall information accessible via the Web. The hardshipsincurred by this decision were far outweighed by the easeof cross-platform distribution. a must for our studentpopulation. This more active use of the Web infrastruc-ture is in tune with some recent applications of WWWtechnology in education [8, 12, 5]. Our major contribu-tion beyond this existing work is the concentration onin-class capture of information that is to be augmentedthrough coordination with other classroom informationvia the Web.

Several of the research prototypes cited above have beensubjected to some form of evaluation to determine bothusability and usefulness. The two most substantial eval-uation studies have been conducted at PARC and Hew-lett-Packard. For a two-year period at PARC, a suite oftools for capture, automated indexing and integration,and access [9] was used to support a process of intel-lectual property management [10]. At Hewlett-Packard,the Filochat system was evaluated in field and labora-tory studies, generating both qualitative information onthe reaction of users to the technolo~v and also auan-

“.

titative information comparing the accuracy, eficiencyand confidence of Filochat with paper-based and Dicta-phone systems [20]. The initial evaluation we report inthis paper is b&ed on a 10-week experiment in a liveclassroom and provides both quantitative and qualita-tive evaluation of student reaction to the technolom.This evaluation is more of a feasibility study than--aproper usability or utility study as was done at PARC.

Many researchers investigate the effect of technology ineducation. There is an important distinction for re-search in this area, based on whether the research isfocused on education or on technology. We have takena technology focus in our work so far, as evidenced bythe way we describe our work and evaluate its impact.Before we can honestly assess the educational impact ofubiquitous computing technology, we must first developrobust, though not necessarily perfect, prototypes thathave been tested in real environments. This is a seri-ous challenge, especially when dealing with off-the-shelfubiquitous computing technology that is anything but

‘See, for example, the World Lecture Hall Web page, URL

http: //uuE .utexas .9du/uorld/lecture

l%%

robust. Our technology-driven approach to educationaltechnology contrasts with a more education-driven fo-cus, as demonstrated by Wan and Johnson [16] or byGuzdial et aL [6], in which the purpose of the researchis to understand and inform theories on learning. Inthe wider arena of educational technology, there mustbe both forms of research.

3 SUPPORTING MULTIPLE TEACHING AND LEARNINGSTYLES

A long-range Classroom 2000 goal is to be able to pro-vide augmented classroom support for all courses at auniversity such as Georgia Tech. So we must be able tosupport many different teaching and learning styles.

3.1 Teaching stylesWe have examined several common teaching styles withrespect to the challenges they present for multimediacapture and access. One of the main distinguishing fea-tures of a teaching style is the form of materials, if any,that are made available to students, either before or af-ter the lecture. If material is made public, then we canaugment it (using audio or video links, for example) withinformation captured during the lecture. If no materialis made public, then we can only augment informationthat is produced as part of the lecture itself. The styleswe have so far identified are:

Presentation The teacher prepares a set of slides (thepresentation) before class, and the lecture proceeds sim-ilar to a prepared talk at a conference. The slides aredisplayed during the lecture, and copies of the slides areavailable to students before or after the lecture. Duringthe lecture, the teacher may make annotations on theslides to emphasize or clarify certain points.

Public notes The teacher prepares the content of thelecture before class, but in the form of a paper or setof organized notes. These notes are available to thestudents before or after the lecture. The teacher lecturesto the class, using the notes as a guide, and may alsouse a whiteboard to write down certain points. It is thedifference in the format between the discrete slides ofthe presentation style and the continuous notes in thisstyle that impacts the capture and access problem.

Private notes The teacher prepares only a private setof notes as a means to prompt the lecture, but this ma-terial is not made available to the students at any point,

Discussion The three previous styles emphasized a di-dactic approach to the lecture in which the teacher is theprincipal speaker, interrupted occasionally by questionsor comments from the students. In this style, the class-room session is more of a discussion in which all partic-ipants contribute more or less equally to the speaking.There may be a publicly available agenda for the classdiscussion that serves to highlight the topics that willbe discussed.

We do not suggest that this is a complete categoriza-tion of teaching styles, and we also recognize that some

teachers may choose to combine teaching styles within acourse or even within a single class session. Attemptingto provide support for each of these teaching styles insimultaneously developed prototypes allows us the clp-

portunity to identify general features of an ideal systemthat can support all classes.

3.2 Learning stylesJust as teachers have different styles for tea,ching, so toostudents have different styles for learning. Rather thanaddress these different general learning styles, we fo-cused on one of the primary student activities — record-ing information. We identified several different record-ing, or note-taking, styles that a student could employ,each distinguished by the amount of recording that gcleson.

Verbatim The student acts like a court stenographer,busily writing down aa much of what they experiencefrom the class as possible.

Highlighting The student writes down only key partsof what is said in class.

None The student writes nothing, relying on memoryor provided materials as the only written record of theclassroom experience.

Different teaching styles provide more or less support forthe various recording styles. For example, the highlight-ing student probably receives good support in a presen-tation style lecture in which they can annotate their owncopy of the slides during the lecture.

4 A COMMON FRAMEWORKWith this crude understanding of the expected popu-lation of users for Classroom 2000 prototypes, we at-tempted to construct prototypes of systems to supportdifferent teaching styles for classes that we were cur-rently teaching. To date, we have supported three dif-ferent courses within the College of Computing at Geor-gia Tech. Details on each of these classes is summarizedin Table 1.

We built three separate prototypes to suit different teach-ing styles and to allow us to experiment with differenttechnology in the hands of the students and teacher.One prototype used Apple MessagePads3 for note-takingand another used pen-based PCS. One class used anelectronic whiteboard (the LiveBoard) while others sim-ply used a projector attached to a workstation that theinstructor used to display slides or notes.

To control the engineering problem of (designing andmaintaining distinct prototypes, we devised an overallcommon architecture or organization that each proto-type would obey. The inspiration for this common ar-chitecture came from the movie industry, in which the

3MessagePad is a registered trademark of Apple Computer,Inc.

189

teaching style

enrollment

pre-production

live recording

teacher

students

classroom

post-production

HCI AI FCE

Human-Computer Interaction Artificial Intelligence Future Computing Env.

present ation public notes discussion

25 grad students 60 undergrad CS majors 15 grad students

PostScript converted to IYI)jX notes converted to FrameMaker discussion out-image files; keywords HTML line converted to Newtonextracted from slides Book

ClassPad on LiveBoard LCD projector to display LCD projector to displaycaptured navigation and Web notes; no capture outline and Web pages; no

annotation (Fig. 1) capture

ClassPad on pen-based PC paper notes; no capture outline annotator on Mes-

captured navigation and sagePad to capture outline

annotation (Fig. 1) entry notes (right side ofFig. 1)

single digital audio stream Isingle analog audio-video Isingle analog audio-videorecording stream recording stream recording Ilog file, annotated slides and audio and video links added PERL script t ran.sformskeyword text used by PERL to HTML notes manually Newton data into audio-script to create (Fig. 3); video digitized to enhanced outline with notesaudio-enhanced, searchable QuickTime packets. (Fig. 3)Web notes (Fig. 2)

Table 1: Summary of technology used by phase (pre-production, live recording and post-production) in three Classroom2000 prototypes. In all cases, the products of post-production were a collection of Web pages with various external helperapplications to hear audio and play video.

development of a single movie is divided into three dis-tinct phases — pre-production, live recording and post-production. Table 1 summarizes the main differencesbetween the prototypes by the activities and technologyused in the various phases of production. We will nowdescribe what each of these phases means in the contextof our development.

4.1 Pre-production phase

We assume that the teacher does some preparation foreach lecture. The purpose of the pre-production phaseis to transform this prepared material into the desiredform for use during the lecture. Any prepared materi-als that the teacher wishes to display during the lectureor make available to students in clam must be trans-formed into a format that can reside on the availabletechnology (e.g., the electronic whiteboard or the stu-dent’s electronic notebook).

We currently support only the presentation of statzc in-

formation within the lecture. That means we supportlectures that include writing on a whiteboard, usingoverhead transparencies or slides, We do not supportthe presentation of videos or other dynamzc information,such as a computer simulation. Support for dynamic in-formation is a future consideration.

Teachers are already overworked, so were very concernedwith minimizing the impact the technology had on thepreparation of lecture materials. Most teachers alreadyhave some form of lecture material that they will wantto reuse. The more we required a lecturer to recreatethat information, the less likely they were going to wantto adopt the Classroom 2000 technology. ‘To minimizepreparation effort, we adopted several strategies.

For one class, Human-Computer Interaction (HCI), apresentation-style class, we adopted PostScript as theuniversal representation for material prepared by theteacher. We used public domain filtering programs totransform the PostScript file into whatever image for-mat was necessary for use in the class. In another class,an introduction to Artificial Intelligence (AI), a public-notes-style class, the notes were in the form of a LA’document and were to be presented in class and madeavailable to students via a Web browser. We used ex-isting filters to convert the LAW source to HTML. Thethird class was a seminar on Future Computing Envi-ronments (FCE). Since it was a discussion-style class, atemplate file was provided for discussion leaders to pre-pare an agenda for the class. The completed templatewas then automatically converted into a format (New-ton Book) for our own Apple MessagePad note-taking

190

Cooporatlvo framework

Cooperative work — >1 person

""""""""""""""""""""'""""""""""""""""""""``"""`"$Yls/9&""""""""""lnteBIlgent Agents

Two iiYsX@m!5 tt”t~t prCWid@“lfltF#li9@7t

Merits”● Letizla (Henry LieBerman)e WebWatcher (.Mmstronql, et al)

+& eckgmt+e

L$f rwki-11

Figure 1: Two versions of the student note-taking interface. On the left side is the interface for CiassPad, a Visual Basicnfie-taking prototype. This same interface served as the interface for the teacher% electronic whiteboard. C)n the rightside is the interface for a student-only outline-oriented note-taking application that runs on the Apple MessagePad.

program.

4.2 Live recording phaseThe live recording phase consists of the actual classroomlecture. The most important task to support in thisphase is the capture of all relevant activity that will beuseful for later review. We initially identified severalrelevant streams of activity in the classroom:

● what is heard in class (audio);● what is seen in cla.sk (video);● what is being written down by the teacher (public an-notation); and● what is being written down by individual students

(private annotation);

To make the classroom an easy-to-use multimedia au-thoring environment, it must be equipped to capture

these different streams of information without imposingextraneous work on the part of the participants — thatis, the capture of information must result as a naturalby-product of normal classroom interaction.

We used cameras and microphones placed at various lo-cations around the room to capture one or more videoand audio streams, all of which were eventually digi-tized. This recording required no extra effort on thepart of the participants.

4.2.1 Too/s for the teacher Most classrooms provide ablackboard, whiteboard or overhead projector for publicviewing of information provided by the teacher. Replac-ing these presentation technologies with a computerizeddisplay makes capture relatively simple, We used a highquality Xerox Liveworks LiveBoard, a PC with a pen-sensitive, 67-inch diagonal screen. There are other less

costly solutions, such as a pen-based computer attachedto a LCD projector, or even projecting onto an uprightdigitizing tablet surface. Once computerized, the publicdisplay can be instrumented via software to log the timeat which significant interactions occur.

None of the existing applications we had available forthe LiveBoard allowed us to easily log pen events andconvert the resulting annotated slides into a form (e.g.,

GIF) that was easily displayed on all VVeb browsers.Other similar commercial products suffered the samelimitation. As a result, we had to write our own elect-ronic whiteboard application, a Visual BiAc prototypecalled ClassPad, whose interface is shown on the leftside of Figure 1, We used ClassPad in the presentation-style HCI course to present prepared slides and allow forpublic annotation by the teacher. ClassPad preserves allannotations made to a series of prepared slides. In acldi-tion, ClassPad creates a time-stamped log of when theuser navigates between slides and when each slide ‘wasannotated with the pen (defined as a pen-down followedby a pen-up sequence). This captured information isused in the post-production phase described in the n,extsection.

We developed ClassPad for a presentation teaching style,but it is also appropriate for private notes or discussion-style classes in which the teacher simply wanted to h~avea blank surface upon which to write. In fact, severallectures in the HCI class used the private notes style.However, ClassPad was not appropriate for the publicnotes teaching style of the AI class, because the noteswere displayed as Web pages. There is a serious regis-tration problem that must be solved by the developer

191

Slide5

K~ords

Audio

mwl

Cooperative framework

Coopwatkn work - > 1person

Communication andundsrstandhw “

Figure 2: Frame-based Web presentation of lecture notes with annotations and audio links. This figure shows the notesmade by the teacher for an actual lecture in the HCI class. Student notes are similar in appearance. The top frame isa sequence of thumbnail images of all slides for that lecture. The user selects one thumbnail image, and the full-sizedimage is shown in the lower right frame. The lower left frame contains a list of keywords associated with the slide (noneshown here), the automatically-generated audio links representing each time the slide was entered during the lecture (onetime in this example), and a link to a form that allows keyword searching across all slides for the entire course.

of the capture tool in order to provide persistent penannotations for a markup language such as HTML. Therendered image (including the HTML text and any as-sociated pen annotations) depends on characteristics ofthe bounding window, so it is much more difficult toregister the pen annotation with the underlying text.Because of this difficulty, we did not capture pen anno-tations in the AI class. The only captured data in thatclass were the audio and video streams.

4.2.2 Tools for the student Support for the capture ofstudent notes is also important. We made a consciousdecision in all prototypes not to provide a keyboard in-terface for note-taking. Despite the advantages of key-board input, such as faster input rates, increased leg-ibilit y and easier search capabilities, we did not wantthe distraction in class of a cacophony of keyclicks. Allnote-t aking, therefore, was done either with traditionalpen and paper or with pen-based computing technology.

We also used ClassPad on pen-based PCs, resulting inthe electronic student notebook. This version of theelectronic notebook was suitable for both the verbatimand highlighting modes of note-taking, but it was de-signed with the highlighting student note-t aker in mind.The student would see the same information that theteacher put up on the electronic whiteboard and couldannotate it with personal comments to make certainpoints clearer, as shown on the left side of Figure 1.Students could flip though the class notes the same waythe lecturer did (though the units were not synchronizedso the student was free to browse the slides at their own

pace) and write whatever notes they wanted on top ofthe slides. The navigation between slides and studentannotations were logged by ClassPad.

We also investigated the use of smaller PDA-style elec-tronic notebooks, such as the Apple MessagePad. TheMessagePad’s resolution made it infeasible to use thesame philosophy of note-taking used in ClassPad. Theprepared slide images would not have been legible, andthere would have been little space on the screen for tak-ing notes, Instead, the MessagePad note-taking appli-cation, shown on the right side of Figure 1, provides anoutline of the lecture. A time-stamped note (called a“slide” in the actual interface in Figure 1), is associatedwith each entry in the outline. Touching the entry withthe pen causes a note to appear (there is one note avail-able per outline entry) and the student then writes inthe note. This outline note-taker logs the first time eachnote was revealed.

4.3 Post-production phaseOnce the lecture is completed, we enter the post-pro-duction phase. The purpose of post-production is tosupport the student and teacher in reviewing materialacross all lectures for a given course. Our goal for large-scale content generation is to automate the creation oraugment ation of content. Audio and video records ofwhat happened in the classroom should be automati-cally linked to the appropriate points in the preparedmaterial. Public annotations by the teacher and pri-vate notes of the student should also be automaticallyintegrated with the other streams of information to fa-

lYL

Figure 3: Two further examples of audio- ancf/or video-enhanced Web pages. On the left side is an example ofautomatically-generated audio links for a discussion-style class using the Apple MessagePad outline annotating ap-plication. On the right side is an example of manually-generated audio and video links for a public-notes-style lecture.

cilit ate later review. All of this integration of variousmedia streams should be enabled by the captured infor-mation from the live recording phase.

Recall that the ClassPad application generates a log ofwhen the teacher or student advances from one slide to

the next and when an annotation was made. When re-constructing the annotated views for later review, theselogged events are used to tie the static information (pre-pared slides with student/teacher annotations) to theaudio or video stream associated with that class. Fig-ure 2 shows an example of an automatically-generatedWeb presentation from a single lecture in the HCI classwith audio-enhanced links. The top frame shows thumbnail sketches of all slides from the lecture. The selectedthumbnail image is magnified in the lower right frame.The lower left frame is divided into three main sections:a keywords sections shows words associated with the fileto facilitate a content-based search; an audio section listsautomatically-generated audio links indicating times inthe lecture when that slide was visited; and a searchlink provides access to a search form for simple keywordsearch across all lecture notes. When an audio link isselected, an audio client is launched and begins play-ing the recorded lecture from that point in the lecture.We built our own streaming, indexable audio server andclient players for this purpose.

The static nature of slides in the presentation teach-ing style makes it easy to automatically generate au-dio links. For other teaching styles, it is not always asimple matter to attach the audio links to parts of theprepared material (see the discussion of the registrationproblem in Section 4.2). On the left side of Figure 3

is another example of an automatically generated Webpage cent aining audio links, generated from output inthe discussion-style FCE seminar using the Apple Mes-sagePad as the note-taking device. The Web-accessiblenotes show the prepared outline augmented with notesinserted at the right location. Selecting a note launchesthe audio player at the point in the discussion in whichthe note was initially generated. It is possible to hideand reveal these annotations, so that the original dis-cussion outline can be seen alone, if desirecl.

We did not have tools to automatically generate audio-or video-enhanced review material for the public notes-style AI course. Instead, audio and video links weregenerated manually from the videotaped lecture and theanalog video was digitized into a single audio file andsegments of QuickTime video. On the right side of Fi,g-ure 3 is an example of a lecture with audio (marked withan “A) and video (marked with a “V”) links manu-ally added. It is an interesting research question to askhow recorded information from the lecture (e.g., ges-tures gleaned from the video recording, segmenting theaudio) can be processed to determine when audio linksshould be created and how they can meaningfully beattached to the material [2].

5 EXPERIENCE AND INITIAL EVALUATION

One of the major contributions of this wcmk is its ap-plication in a large-scale educational setting. Table 1summarizes the variety of experiences we have had sofar in applying Classroom 2000 technology. Up to thispoint in the paper, we have focused exclusively on whattechnology we introduced and how we built the variousprototypes. The rest of the paper now focuses on an

193

evaluation of our experience with using this technology.

5.1 Objective evaluationWe have had the most experience operating the Class-

room 2000 prototype that was used in a graduate HCIcourse, and the objective and qualitative results of thenext two subsections refer to that class. The 10-weekclass met twice a week for 90 minute lectures [1]. Therewere 25 graduate students in the class representing awide variety of disciplines across the Georgia Tech cam-pus. We were unable to supply enough units to pro-vide each student with a pen-based electronic notebook.The number of units varied throughout the course froma minimum of 6 to a maximum of 12 working units.The technology in the class was phased in incremen-tally, beginning with the LiveBoard, followed by audiorecording and, finally, student electronic notebooks. Bythe third week of class, students were taking electronicnotes. Four students were selected (from among 8 vol-unteers) to take notes using ClassPad on the electronicnotebook for the remainder of the quarter, which con-sisted of 10 lectures. Four students chose not to takenotes electronically the entire class. The remaining stu-dents used the other units on a first-come, first-servedbasis, averaging 2.9 times each.

Students kept a journal of their notes and reactions tothe technology throughout the course. At the end of thecourse, 24 of the 25 students filled out a questionnairethat investigated their reactions to the use of the tech-nology in the class. The objective questions asked aboutoverall impressions of the use of technology in the classand then specifically about the ClassPad note-taking ap-plication and the use of the LiveBoard with Web-basedreview notes. These quest ions were rated on the follow-ing scale: 1 (strongly disagree); 2 (disagree); 3 (neutral);4 (agree); and 5 (strongly agree). We asked questionsabout overall impression, ease of use, whether the tech-nology made aspects of the class more effective, how thetechnology affected class participation and whether thetechnology contributed to learning the particular sub-ject matter of the course (in this case HCI). Table 2summarizes the results of the objective portion of thequestionnaire.

The results show an overall positive reaction to the pro-totype. The strongest positive reaction is in how theprototype was perceived to contribute to learning theparticular subject matter, and this is not surprising.The course was on HCI and the students were them-selves experiencing a new interface in the classroom.

In addition, the project work was based on develop-ing and evaluating ideaa for new Classroom 2000 pro-tot ypes, and the students appreciated the authenticity yof redesigning a system they were currently using.

One of the initial goals of Classroom 2000 was to exam-ine the effect of personal interfaces in the classroom.Our initial observations show that the students weremost negative toward the personal electronic notebooks(see the next section for qualitative justification). TheLiveBoard and Web notes together comprised the most

I Topic (# of questions) I Avg.(a)I

0~Increased effectiveness of clrxs.s(9) . .

II Improved class participation (2) / 3.40 (.88) /

Contributed to learning subject (2) 3.94 (.86)

N Was desirable technology (1) 3.13 (103)

Was easy to use (3) 3.13 (1.14)

Increased effectiveness of class (1) 2.88 (.90)

Helped me take fewer notes (2) 2.85 (.87)

L Was desirable technology (2) 3.87 (.82)

Was easy to use (3) 3.68 (1.09)

Increased effectiveness of class (1) 3.29 (1.04)

Helped me take fewer notes (2) 2.88 (1.00)

Table 2: Reaction of students to overall technology(0), electronic notebook (N) and LiveBoard with Webnotes (L).

desirable technology from the students’ perspective.

5.2 Qualitative evaluation

The post-course questionnaire also provided an opportu-nisty for the students to provide more detailed reactions.Students provided details on their positive and negativereactions to all aspects of the technology.

Use of the LiveBoard for several in-class usability eval-uation exercises and the group presentations at the endof the class were both very popular. Both of these ac-tivities involved more than just the teacher interactingwith the LiveBoard. But many students did not feel theLiveBoard was any better than an overhead or chalk-board when used exclusively by the teacher in a lecturemode.

The majority of the electronic notebooks used in theclass were palmtop PCs (specifically, Dauphin DTR-lswith a 7-inch diagonal screen) and while the studentsfound them good for drawing pictures, in general thescreens were too small. In addition, the response timeof the units was slow relative to the LiveBoard. Thismade it difficult for students to navigate between slidesas easily as the teacher. Also, a number of the palm-tops were unreliable machines that would crash duringlectures.

Students found the Web-based review notes interestingin their novelty and useful for examining their own notesand the teacher’s notes. Several students found thenotes very useful on the occasions when they missedclass or did not pay close enough attention to some

194

point during class. Despite the relative ubiquity of Webbrowsers on campus and in student rooms, several stu-dents still desired to have a printed copy of their notesbecause then they would be easier to carry around andeasier to review. The Web notes were not always quicklyaccessible (especially over telephone lines) and some-times hard to read.

We were unable to keep logs of use of the audio server,so we cannot give a quantitative indication of its use,but we have determined that audio annotations werenot used very much. Only 4 of the students in the classnoted in their journals that they had made consistentuse of the audio features in more than just full playbackmode. There were two reasons for the overall lack of useoft he audio. First, students did not have regular accessto the correct platform for listening to the audio andwe were unable at that time to provide a cross-platform

audio player. Second, the set-up of the audio servicewas too difficult for somb students to bear, so they did

not bother. Despite this minimal usage of the audiofeatures, several students who did manage to use theaudio found it particularly useful to clarify their ownnotes.

Of particular interest is how electronic notebooks pro-mote different and possibly more effective note-takingstrategies. A overview of the electronic notes takenby students reveals that initially most students wouldwrite quite a bit on the electronic slide, even if whatwas written was exactly what the teacher was writingon the LiveBoard. When questioned about this after-wards, several students who used the elect ronic note-

book throughout the class noted that they felt theirnote-taking became more economical as the course pro-gressed. This is in spite of an apparent lack of use of theaudio features. Upon further investigation, these stu-dents revealed that even without audio services, merelyhaving the teacher’s notes available after class savedthem from the sometimes mundane task of copying. Onestudent in the class, however, stated a preference forwriting down everything himself, even if what he wrotewas identical to the teacher’s notes that he could obtainlater. This again points out the importance of recogniz-ing different learning styles and supporting as many aspossible. We are in the process of completing more sig-nificant quantitative analysis of the student notes andwill report on those findings later.

All Web pages produced for this class (shown in Fig-ure 2) were publicly viewable, a conscious choice of thestudents in the class. Students were unaware that theycould edit their own Web notes, and we had not pro-vided any easy way to do the editing. Several studentscommented on the shortcoming of these supposed staticreview notes, remarking that it was their habit to reviseand rewrite their notes. This represents both a technicaland social failure of the prototype to support long-term

use of class notes. We are now concentrating on provid-ing a better interface to revise notes.

From the instructor’s perspective, there were several ad-

vantages and disadvantages. The Class Pad applicationrunning on the LiveBoard was easy to use i~nd was re-sponsive enough to allow for a natural level of inter-action. All of the lecture material for this class wasavailable from a previous section of the course, but asthe quarter progressed, it was judged necessary to mod-ify the format of the slides. At the request of severalpeople, the slides were redone to increase the amountof whitespace available for making annotations, TheClassPad logging was effective, even though we had in-tended to provide per-annotation audio links instead ofper-slide links. One drawback of our system, however,was the requirement that the teacher load all slides tobe visited during one lecture prior to the beginning ofthe lecture. This seemingly simple requirement caused aproblem twice in the course when the teacher wanted torefer to a slide from a previous lecture but was unable todo so because the capture semantics of ClassPad wouldnot have correctly logged the remainder of the class.

One pleasant surprise came the first time the Class.Pad application running on the LiveBoard misbehavedin class and would not load the slides for the lecture.The lecture proceeded more in the private notes style.In time, we began to appreciate ClassPad as being well--suited to the private notes style of teaching and we planto take advantage of that in the future to attract otherteachers into our experiments.

5.3 Evaluation of Al and FCE classesThe prototype used in the HCI class was relatively sta-ble before that course began. This was not the case forthe other two courses (AI and FCE). Consequently, boththe AI and FCE prototypes changed quite a bit duringthe quarter and we were unable to collect much quanti-tative evaluation information. In the AI class, the onlycapture that occurred during the live recording phasewas the video and audio recording. The manual post-production activity of augmenting the HTML notes withaudio and video links was so time consuming that afterthe fourth lecture we were no longer able to devote theresources to continue the service. We see now the ad-vantage of having tools that use natural actions of the

teacher to automate the audio and video augmentaticmof Web pages, as described in [2, 9]. Such actions couldinclude long pauses in their speech or mouse movementsto draw attention to some area on the page.

In the FCE seminar, we both videotaped and used theMessagePad outline-annotator. We provided a templatefile to prepare the outline of the class discussion; thiswas a useful service for the presenters. The studentnote-takers found it easy to understand how the out-liner application worked, but did not find it all thatuseful to attach notes to the outline. The main prob-lem was that discussion in the class did not follow theoutline very closely. When the note-t aker wanted to jotdown a thought, it was hard to determine which entryin the outline to choose for annotation. Sometimes, thechoice of entry was entirely arbitrary, and the resultingenhanced Web page looked somewhat confusing. As the

195

quarter progressed, we altered the application by remov-ing the outline and allowing the user to bring up blank,time-stamped note pages for writing whatever came tomind. This simpler interface, similar in spirit to Stifel-man’s audio notebook [14] was much less confusing tothe user, but was only used in the class a few times.

5.4 Developer insightsOur year of experience developing a number of Class-room 2000 prototypes has resulted in many valuablelessons and insights into ubiquitous computing and itsapplication in an educational setting. We share some ofthe more significant ones here.

5.4.1 Go live As we mentioned in the introduction, thebest way to understand the effect of ubiquitous technol-ogy in our everyday lives is to experience it. This issimilar to the Moran et al. notion of evolutionary en-gagement, in which the evolution of a tool is informedby its early adoption in a real-life-context [10]. Our con-stant drive to go live with various prototypes was thesingle greatest challenge we faced, and it comes withsome risk given our educational focus. It is not aneasy decision to experiment with education, as failurecan have dire consequences. We are a relatively low-budget project, which forced us to purchase affordablepen-based technology that was neither ideal nor robust.The development of pre-production, live recording andpost-production tools was not difficult work when com-pared to the maintenance and everyday operation of thestudent notebooks. With the advent of a second genera-tion of pen-based computing, we hope this situation willimprove without significant cost increases.

One great advantage of our prototyping approach hasbeen suggestions from the users. We received many sug-gestions from students for possible extensions of Class-room 2000 that we did not initially consider. Some ofthe more promising suggestions are:

● Provide the instructor with real-time anonymous feed-back on student reactions to the lecture.● Allow students to “tune in” to the teacher’s notes dur-ing class to save time writing.● Allow students to anonymously log questions duringclass without interrupting the lecturer. After class, theinstructor can then access the questions and still under-stand the context in which the question was asked.

5.4.2 /mportance of architecture It might appear thatwe are trying to take on too much in this project inour attempts to support such a wide variety of teachingand note-taking styles, but critical architectural deci-sions made very early on in the project have allowedus to experiment more broadly. The simple divisioninto phases of pre-production, live recording and post-production has been critical to our ability to field sev-eral very different prototypes. This has allowed us tosupport a variety of preparation tools used by teach-ers (ranging from LA&jK to more WYSIWYG documentprocessing tools), different in-class presentation tools(Web browsers, PostScript previewers, and the Class-

Pad application) and different post-production tools toautomatically or manually generate audio- and vide~

enhanced notes. Developing tools that served activitiesin different phases of the project enabled concurrent de-velopment and is now allowing us to modify and enhancecertain features of the system with minimal impact else-where.

This architectural division is not ideal, however. Onedrawback has been a limited interpretation of what oc-curs in post-production. Up to now, post-productionhas simply meant the generation of media-integratednotes based on multiple streams of information captured

during the live recording phase. It has not included sup-port for the user in accessing and modifying those notes.Work at Xerox PARC has identified tools to support thisseparate access phase [9] and we would be wise in thefuture to focus more effort there as well.

5.4.3 Access is critical In order to evaluate the effec-tiveness of Classroom 2000 technology, we have to havesome guarantee that it is being used. Students andteachers all need to be able to access the products ofpost-production. Since students have access to a varietyof platforms both in university labs, offices and homes,cross-platform support is critical. This cross-platformissue has not been addressed by other researchers work-ing in this area because either they have been able tocontrol the computing environment of the users or theusers had a limited computing platform. The emergenceof the Web has made it possible to realize a limitedcross-platform distribution for some media types. Stu-dents noted that access to the Web notes was easy, theonly problem being some delays over traditional phoneconnections. Unfortunately, at the time we were begin-ning development of our prototypes this cross-platformsupport did not include streaming, indexable audio. Webuilt our own UNIX-based audio service, including aclient player and central server. This worked extremelywell on laboratory workstations, but was virtually inac-cessible to students working at home using non-UNIXplatforms. As a result, we had relatively limited use ofthe audio. Again, we are fortunate that cross-platformsupport for indexed streaming audio is now commer-cially available; we are transitioning away from our owntools for that service.

5.4.4 Note-taking as image annotation is limited In manyclassroom settings, teachers provide prepared slides ornotes to the students before a lecture, and many stu-dents consider this to be an advantage. In addition,many of the other tasks we perform in our daily livesare annotation tasks. Drafts of a paper are frequentlyannotated by reviewers, and a teacher corrects a st u-dent’s report by annotating it. Support for annotationis useful for the classroom and other settings.

Annotation is simplified when the underlying image —such as a slide used the presentation-style lecture — doesnot change. This approach does not work for all lecturestyles. It would be better to have prepared material forpresentations that differed in form or content from the

196

material used for review. The material for present ationmust be readable when projected and fairly terse, asreading lots of text off a wall display is not effective. Thematerial for review should be more like a user-modifiabletextbook, suitable for a personal display and containing

more explanations.

5.4.5 Using more than audio We believe that video pro-vides added value in addition to audio. Our presentersuse pronouns and point to the class a lot, saying “you”do this and “they” do that, for example. The audiotrack alone is difficult to interpret if the gestures are notvisible. It is also much easier during review to follow theflow of the lecture (understand that the presenter is sud-denly responding to a question, for example) if a viewof the teacher is present.

We are aiming to be able to replay the entire lecture ex-perience, including multiple video views and all studentinteractions with their computers in class. We need tohandle richer media sources at a finer level of granu-larity. For example, the student should be able to askduring review, “What was the lecturer saying when Iwrote this?” while pointing to some arbitrary annota-tion, as in [10, 14, 20]. Or the student might want to findthe notes associated with a live demonstration that oc-curred at some point in the class. The solution we haveproduced for indexing and reviewing an audio streamfor the class is immediately transferable to video, key-board and mouse events, and pen strokes. A constraintat the moment is efficient storage and delivery of thericher media types.

5.4.6 The value of pen interfaces We have debated themerits of pen interfaces, and this paper haa focused onthe use of pen-based interfaces to the exclusion of otherinput mechanisms. There are advantages to using paperand pen or even a keyboard for student notes instead anelectronic device. Paper is familiar, cheaper, more ro-bust, and has much higher resolution than the currentgeneration of pen-based computers. Conventions taughtto note-takers can support time-stamping, but not with-out introducing an added burden and the chance forerrors in the capture. This shortcoming could be re-moved through vision technology, such as demonstratedby the DigitalDesk [11], or by anchoring the paper on adigitizing tablet, similar to the prototype developed byStifelman [14].

Some students noted that they can type faster than theycan write and that the typed-in information is imme-diately available for content-based search mechanisms.We stayed away from keyboards because we felt the con-stant tapping of the keys would be a distraction in theclass. We also feel that the purpose of Classroom 2000is not to enable a student to take more notes, but ratherto be more efficient note-takers.

The use of an electronic whiteboard was universally fa-vored in the classroom, and the LiveBoard provided

an excellent, albeit expensive, solution. It is still too

small, both in terms of physical size and screen resolu-

t ion (VGA), to consider replacing existing whiteboards.It is roughly the size of a whiteboard in tan office orsmall meeting room and about a third the size in rea~lestate of even the smallest classroom whiteboards. Ourexperience shows that the computational capability c)fthe LiveBoard is useful for both encouraging group ex-ercises in a class and also creating an accurate recordof some class activity. We recommend that researchersinvestigate ways to provide larger scale interactive sur-faces, both in terms of display size and resolution.

6 CONCLUSIONSIn this paper, we described the initial work in the Clsss-room 2000 project. We are exploring how ubiquitouscomputing technology can influence the way we teachand learn by empowering both students and teachers.The main theme of Classroom 2000 is to instrument theclassroom and put new technology in the hands of stu-dents and teachers so that we can capture as much ofthe rich exchange of information as possible. The con-tribution of our work so far has been the development ofthree sepa~ate prototypes to exercise the ideaa of Class-roo~ 2000 in support of different teaching i~nd learningstyles.

Though each Classroom 2000 prototype system is dif-ferent in terms of technology provided, information cap-tured and media-integrated review materials produced,they all follow a common organizational theme. Weseparate the functionality of the system into three dis-tinct phases. Pre-production activity prepares all infor-mation leading up to the classroom interaction. Liverecording captures various streams of information andactions during a lecture. Post-production activity gene-rates multimedia-enhanced Web pages for a summaryof the classroom activity.

Along with developing a variety of prototypes to supportdifferent teaching and learning styles, we have had theopportunity in one case to conduct an extended eval-

uation of the effect of the technology on the teachingand learning experience. Though we are nc)t yet able toprovide an assessment of how Classroom 2000 enhanceslearning, our preliminary evaluation does reveal a favctr-able student impression. Most encouraging was the re-sponse toward the use of the electronic whiteboard andWeb notes. Least encouraging was the response towardthe personalized electronic notebooks. We understanda lot of the misgivings with our initial prototype note-books. They were too small and too slow and left thestudents feeling that their notes were unavailable for ]re-vision after class. Armed with these insights, we willcontinue to explore this valuable avenue of research infuture computing environments for education.

ACKNOWLEDGEMENTSClassroom 2000 haa been a group development effort.The authors would like to acknowledge the support ofthe several members of the Future Computing Environ-ments Group, College of Computing and th~eOffice of In-formation Technology at Georgia Tech. Specifically, wethank Savita Chandran, Yusuf Goolarnabbas, Dietmar

Aust, Peter Freeman, and Ron Hutchins. Abowd andAtkeson would like to thank the wonderfully patient andinsightful students in their respective courses for theirhelp in providing an honest evaluation of the Classroom2000 concept. Finally, the authors would like to thankthe referees, particularly Polle Zellweger, for many con-structive comments that have resulted in a much betterpaper.

REFERENCES

1.

2.

3.

4.

5

f

7.

G. D. Abowd. CS 6751: Human–computer in-teraction. Home page for College of Computing,Georgia Tech introductory level graduate course.URL http://www.cc.gatech .edu/computing-

/classes/cs6751 .96.winter, 1996.

B. Arons. SpeechSkimmer: Interactively skim-ming recorded speech. In Proceedings of the ACMUIST’93 Symposium, pages 187-196, 1993.

L. Degen, R. Mander, and G. Salomon, Workingwith audio: Integrating personal tape recorders anddesktop computers. In Proceedings of ACM CHI’92Conference, pages 413-418, May 1992.

S. Elrod, R. Bruce, R. Gold, D. Goldberg, F. Ha-lasz, W. Janssen, D. Lee, K. McCall, E. Pedersen,K. Pier, J. Tang, and B. Welch. Liveboard: A largeinteractive display supporting group meetings, pre-sent ations and remote collaborate ion. In Proceedingsof ACM CHI’92 Conference, pages 599–607, May1992.

J. Fowler, D. Baker, R. Dargahi, V. Kourama-jian, H. Gilson, K. Brook Long, C. Petermann, andG. Gerry. Experience with the virtual notebooksystem: Abstract ion in hypertext, In Proceedings ofACM CSC W’94 Conference, pages 133-143, 1994.

M. Guzdial, J. Kolodner, C. Hmelo, H. Narayanan,J. Carlson, N. Rappin, R. Husbscher, and J. Turns.

Jomputer support for learning through complexproblem solving. Communications of the ACM,39(4):43-45, April 1996.

D. Hindus and C. Schmandt. U-biquitous audio:Capturing spontaneous collaboration. In Proceed-ings of A CM CSC W’92 Conference, pages 210-217,1992.

8. M.-C. Lai, B.-H. Chen, and S.-M. Yuan. Towarda new educational environment. In Proceedings ofWWW’4 International World Wide Web Confer-ence, December 1995. URL http://www.w3.0rg-/pub/Con ferences/WWW4/Papers/238.

9. S. Minneman, S. Harrison, W. Janssen, G. Kurten-bach, T. Moran, I. Smith, and W. van Melle. Aconfederation of tools for capturing and accessingcollaborative activity. In Proceedings of the ACMMultimedia ’95 Conference, pages 523–534, Novem-

ber 1995.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19,

20.

T. Moran, P. Chiu, S. Harrison, G. Kurtenbach,S. Minneman, and W. van Melle. Evolutionary en-

gagement in an ongoing collaborative work process:A case study. In Proceedings of ACM CSC W’96Conference, 1996.

W. Newman and P. Wellner. A desk supportingcomputer-based interaction with paper documents.In Proceedings of A CM CHI’92 Conference, pages

587-592, May 1992.

U. Schroeder, B. Tritsch, and A. Knierriem-Jasnoch. A modular training system for educa-tion in the WWW environment. In Proceedingsof WWW’4 International World Wide Web Con-ference, December 1995. URL http://www.w3.0rg-/pub/Con ferences/WWW4/Papers/306.

B. Shneiderman, M. Alavi, K. Norman, and E. Y.Borkowski. Windows of opportunities in elec-tronic classrooms. Communications of the ACM,

38(11):19-24, November 1995.

L. J. Stifelman. Augmenting real-world objects:A paper-based audio notebook. In Proceedingsof A CM CHI’96 Conference, pages 199–200, April1996. Short paper.

N. Streitz, J. Geissler, J. Haake, and J. Hoi. DOL-PHIN: Integrated meeting support across local andremote desktop environments and LiveBoards. InProceedings of ACM CSC W’94 Conference, pages345-358, 1994.

D. Wan and P. Johnson. Computer supported col-laborative learning using CLARE: the approachand experimental findings. In Proceedings of A CMCSCW’94 Conference, pages 187-198, 1994.

K. Weber and A. Peon. A tool for real-time videologging. In Proceedings of ACM CHI ’94 Confer-ence, pages 58–64, April 1994.

M. Weiser. The computer of the 21st century. Sci-entific American, 265(3) :66–75, September 1991.

M. Weiser. Some computer science issues in ubiq-uitous computing. Communications of the ACM,36(7):75-84, July 1993.

S. Whittaker, P. Hyland, and M. Wiley. Filochat:Handwritten notes provide access to recorded con-versations. In Proceedings of A CM CHI ’94 Confer-ence, pages 271–277, April 1994.

198