Interactive video: A bridge between motion and math

ANDREW BOYD* and ANDEE RUBIN

INTERACTIVE VIDEO: A BRIDGE BETWEEN MOTION AND MATH

ABSTRACT. This paper examines the characteristics of interactive digitized video as a medium in which motion is presented to students learning graphical representations. We situate graphs of motion as early topics in learning calculus, the bugaboo of many math students. In comparing video to both everyday perceptions and mathematical representations, we construct a conceptual framework that compares these three contexts along several dimensions: object extent, scale, time, and space. We then examine one student's experience constructing graphs of her own design from a video image and describe her work in the context of the our conceptual framework. To further specify the unique characteristics of video, we compare it as a medium with that of computer simulations of motion, in particular as studied by diSessa et al. (1991).

1. INTRODUCTION

Students often view calculus not as the king or queen of mathematics, but as the ogre who guards the second bridge down the pike (the first one being algebra) and who exacts a gargantuan toll from those who dare to cross. A successful crossing can require the traveler to abandon the connections she has constructed between mathematics and the world around her and to adopt a belief in a mathematical entity that is unimaginably small, yet not 0. To many students attracted by the solidity and predictability of their early experience of mathematics, calculus is the beginning of the end, the start of a drift into a vague space filled with complex models in search of comprehensible examples. (And both authors admit having experienced this sensation themselves.) Yet calculus originated from and is inexorably tied to studies of everyday actions in the everyday world, although this connection is seldom exploited in education. Chet Raymo, a physicist turned science writer, chronicles his recognition 40 years after college that calculus is about the same beauties of nature that inspire and illuminate his writing:

As I watched that heap of shine and feathers hop, flap and sail itself up into the pine, I realized that the crow was a beautiful physical embodiment of those abstract differential

* The authors are listed in alphabetical order.

International Journal of Computers for Mathematical Learning 1; 57-93, 1996. (g) 1996 Kluwer Aeademic Publishers. Printed in the Netherlands.

5 8 ANDREW BOYD AND ANDEE RUBIN

equations I studied long ago. Calculus was invented as a language for describing continuous change in nature (Raymo, 1995).

Fortunately, there is now a growing tradition in mathematics education that might help more students see the truth and beauty of Raymo's realization. This approach uses students' experiences of motion, among other things, as a basis for their introduction to calculus-like thinking (Nemirovsky, 1993, 1994; Thornton & Sokolow, 1990; Barnes, 1991; Bresnahan, Ducas & Rubin, 1994; Zollman, 1994; Thompson, 1991; Trowbridge and McDermott, 1980). The focus of this approach is on making connections between motion as students perceive and/or experience it and motion as represented mathematically, in graphs and tables. These experiences fit into a redefined calculus curriculum that starts in middle or even upper elementary school with experiences that are not frequently found in current math curricula (Kaput, 1991; Kaput & Nemirovsky, 1995; FelTini-Mundy & Guether, 1991). The ideas of limits and delta/epsilon formulations are not the primary concepts in these emerging curricula. Rather, the common learning goals are for students to understand how the shapes of curves correspond to particular characteristics of motion (e.g. going faster and faster, going at a steady pace, then slowing down), how complex curves can often be partitioned into more easily understandable segments, and how position vs. time curves are related to velocity vs. time curves and acceleration vs. time curves. Such concepts are seen to lead to the more standard topics of calculus, such as limits, differentiation, and integration. In the parlance of one group working in this area, this strand of the mathematics curriculum is about change, in its many manifestations (Nemirovsky et al., 1995) The formal study of calculus is only one stone on the path - and a late one, at that, following many other related learning experiences.

2. FOCUS AND CONTEXT OF THE STUDY

In this new approach to calculus and change, graphs of change assume a central role. Graphs, rather than equations and tables, are viewed as the primary communicative devices, both as a starting point and as constant companions on the road toward infinitesimals, differentiation, and integration. Some graphs are simple to interpret and can form a solid foundation for ongoing study. Other graphs present serious challenges; mastering them requires and supports the understanding of conceptually difficult concepts such as limits. Within this focus on graphs, there has been a growing interest in capturing and integrating students' intuitions and everyday experiences in building up an understanding of change.

INTERACTIVE VIDEO 59

TABLE I

Two dimensions of students' experience in graphing motion

Graphing "Rich Phenomena" Motion Disappears Motion Replayable

Students Construct Graphs

Machine Constructs Graphs

Nemirovsky; Tierney

MBL (Tinker, Thornton)

DiSessa; this paper

CamMotion and

similar products

Students are increasingly regarded as capable of constructing their own understandings from appropriate contexts, building on both school-based and "extracurricular" knowledge they bring to bear.

This paper draws on both of these interests: both the structure of graphs and the effect of motion experiences on learning about change over time. It examines how students create their own graphs in situations where digitized video helps them to revisit and reflect on an object's motion. The central question we ask is the following: What are the effects o f video as a medium when students are creating graphical representations o f motion presented on videotape?

2.1. Related Work

To clarify our approach and its relationship to previous work in mathematics education, we describe previous studies in which students interpret and construct graphs of motion situations presented to them other than strictly in words. In this section, we first describe several such situations, and consider whether in each the student or the computer is responsible for constructing the graph. In order to more completely situate our work, we then discuss a further distinction between these situations: whether or not the motion is available for inspection after it is over. We present video as one of the few settings in which motion is "replayable." Table I illustrates the interaction between these two dimensions. As shown in the table, we discuss the distinction between situations in which students create graphs and those in which they interpret graphs created by technology. We then consider the orthogonal distinction between motion presented in an ephemeral way and motion presented so that it is replayable. The work reported in this paper fits in the upper right cell of Table I.

All the work described in his paper deals with students' experiences of what we have dubbed "rich phenomena" of motion, that is, phenomena that are presented to students other than strictly in words. A rich phenomenon may be one in which the student participates, actually carrying out the motion, or one that the student watches, either as a real event, as a computer

60 ANDREW BOYD AND ANDEE RUBIN

simulation, or as a videotaped event. In the past decade, several curricula and technological advances have expanded the role of rich phenomena in math education. Microcomputer Based Labs (MBL), for example, (Mokros & Tinker, 1987; Thornton & Sokolow, 1990), are probes that act as on-line data acquisition devices, allowing students to capture graphs of motion (as well as temperature change, light change etc.) as they occur. Students then attempt to interpret the graph produced by the computer, predict what other graphs will look like, or produce particular graphs by moving themselves in relation to the motion detector. In addition, students can predict and the software can produce the related velocity and acceleration graphs.

In a less technological approach to graphing rich phenomena, the Inves- tigations elementary mathematics curriculum (Russell et al., 1994--1996) includes activities in which students create a trace of their motion by drop- ping beanbags at equal time intervals while walking along a path. The row of beanbags left on the floor can be seen as a graph of the motion that can be read as specifying both position and speed. When the motion is slower, the bean bags are closer together; when the motion is faster, they are further apart (Tierney et al., 1995a).

Other work in both curriculum and research has focused on how students construct their own graphs of rich phenomena, with few or no specifications as to how the representations should be structured. In a series of teaching experiments, for example, Nemirovsky and Rubin (1991) asked students to draw a velocity graph for the motion of a small car after the distance graph had been produced by the motion detector and computer. Rubin (1994) had students make graphs of motion that fellow students acted out. In each of these cases, it has been possible to learn about how students think of structuring information about motion without being limited to the "standard" representations.

The above examples share a common characteristic that may affect students' ability to draw strong connections between graphs and motion, The problem is that after the event of interest takes place, it is over. Far from being merely tautological, that fact presents students with a difficult situation. In an MBL situation, for example, all students have to connect to their graph is their memory of the original motion, which is likely to be at least somewhat inaccurate. To deal with this situation, we need tools to essentially stop the world - to let students spend time studying and analyzing the details of brief phenomena and to reflect on their relationship to graphical representations. Luckily, there is a tool that has the power to alter time in this way, at least to a first approximation: video. 1 Video has

1 In a formal sense, computer programs can also manipulate time in this way; they can produce motions of objects on the computer screen that can be replayed at varying speeds.


a unique ability to capture the world of motion so that it can be played back slowly or in discrete chunks, making the phenomenon available to reflection. A few educators have recognized this potential already and have been using video to record and play back motion experiences, primarily in university physics classes (e.g. Ducas, 1993). In general, they have used VCRs that can play video one frame at a time, stopping the motion at each frame so that students can record the position of objects. 2 In this mode, objects' positions can be recorded on acetate placed over the screen, measured and graphed, creating an experience from which students can make connections between specific parts of the motion and particular pieces of the graph.

This frame-by-frame manual data recording method works for a while, but it is tedious and prone to measurement and graphing errors. In the last 5 or so years, the availability of digitized video 3 on computers has made it possible to automate the process of taking data from a video, graphing it and even calculating velocities and accelerations from positions. As a result, several groups of researchers and developers in mathematics and science education have constructed video-analysis environments that sup- port students' understanding of the connections between movement and graphs (CamMotion, 1995; Measurement in Motion, 1994; HIP Physics, 1994; GraphAction, 1995; VideoGraph, 1994; Zollman, 1994). Each of these systems allows students to track multiple objects in the video over time and generate the corresponding graph. Some of the systems have tools that can 1) play a graph and video simultaneously, making the correspon- dence visually clear; 2) measure distances and angles, as well as positions; 3) calculate velocities, accelerations, and other derived values of measurements; 4) change origin and scale of measurements; 5) measure relative motion.

CamMotion and the other software listed above generally do the job of constructing graphs for the student - that is one of their advertised strengths. In our research, however, we have been interested in situations in which students construct their own graphs. We note, as does diSessa et

In addition, the code itself is both cause of the motion and a formal representation for it. However, our focus in this paper is on real-life motion, as it plays out in everyday phenomena, as we believe it has unique possibilities for learning. A more complete discussion of "time-altering" technologies would include comparisons between video and computer animations.

: Video is actually made up of 30 "snapshots" (called frames) per second. 3 Digitized video is video whose normally analog signal (the way we receive it in our

homes) has been converted to digital format, the way all information is represented in computers. This process allows the video to be treated as a computational object that can be manipulated (e.g. stretched or contracted) and measured by the computer.


al. (1991), "that it is rare to find instruction that trusts children to create their own representations" (p. 150), especially where technology that can automatically do so is part of the picture. But we believe that giving children the opportunity to construct graphs and observing them as they do so has advantages from both pedagogical and research perspectives. From a pedagogical point of view, students need to use their prior knowledge and mathematical sense-making skills to create their own representations. They have the opportunity to develop meta-representational skills (diSessa et al., 1991) that they can use to understand any new representation which they encounter. From a research perspective, we can learn much more about what students bring to the task, how they cope with unusual circumstances (like the video medium) and what parts of the task are most challenging.

The work reported in this paper, then, can be described by a choice on each of these two dimensions 1) between interpreting and constructing graphs and 2) between ephemeral and replayable motions. In brief, we studied situations in which students constructed graphs from replayable motions. The students also used several video-based pieces of software to create "video overlay graphs" and to automatically collect and graph motion data. The work was done as part of the VIEW (Video for Exploring the World) project at TERC, against a backdrop of developing sophisti- cated digitized video tools for students, developing appropriate curriculum activities, trying them out in classrooms, and helping teachers use the tools in their classrooms. Our focus is on the effect of video as the motion's medium - and on the mathematical issues it brings into relief.

The most similar research to ours along these two dimensions- students making their own graphs of rich, replayable phenomena- is diSessa et al.'s (1991) study of students' "inventing graphing." In that work, students wrote Boxer programs to simulate various "real-life" motions, such as a book shoved across a desk. Then, in the course of several days, they invented a variety of graphical representations for a particular motion they called "the desert motion:" "A motorist is speeding across the desert, and he's very thirsty. When he sees a cactus, he stops short to get a drink from it. Then he gets back in his car and drives slowly away" (p. 128). The paper tells the story of the representations students created and how, as a group, their ideas matured. Our work is similar to diSessa's in its focus on students creating their own graphs; it is different most noticeably in terms of the medium in which the motion was presented. In the last section of the paper, we compare our observations with his, focusing on the different implications of each medium.

In the following parts of the paper, we first introduce a conceptual framework that guides our analysis of video as a bridging medium between


motion and mathematical representation. A description of our interview methodology follows. Next, we describe and analyze three segments of a session with Erica, a 6th grade girl constructing graphs to represent video motion. We end with a discussion of the issues that arose in the transcripts using the perspective of the conceptual framework and a consideration of our observations as compared with diSessa's related research.

2.2. A Conceptual Framework

To better understand the complex sets of issues that we explored in our experiments, we developed a conceptual framework, diagrammed in Table II. The chart includes three different contexts in which motion phenomena can be experienced and ideas about them constructed: everyday perceptions, video, and mathematical representations.

We think of the realm of everyday perceptions as the "raw" world of continuous time and 3D space in which we experience motion in a tangible yet fleeting way. The realm of video includes standard analog video, as well as digital video and the tools which students may use to view or manipulate either. For the purposes of this study, the realm of mathematical representations includes: tables, graphs of position, velocity and acceleration; and students' verbal and written expressions of their conceptual understandings.

One of the characteristics of video that will turn out to be critical in our analysis is the fact that while it looks continuous when we view it at normal speed, it is actually made up of a series of stills. Standard video captures a "snapshot" of the world 30 times per second; this snapshot unit is called a frame. If we look at a video on a VCR that can move forward by single frames, we see 30 separate pictures before we view one second of the action. The action looks continuous only because our brains fill in the missing pieces of the motion.

Imagine, for example, a video of a bouncing ball. If we watched the video one frame at a time, we would see the ball at different positions, separated by the distance it moved in each 1/30 of a second. If the ball were moving quickly, there would be a large distance between the positions; if it were moving more slowly, there would be only a small gap. Because of these gaps, critical moments in the bali's motion may not be captured in ANY frame. For example, the moment that the ball hits the floor may be between the images captured on the video and therefore never actually visible on frame-by-frame replay.

The rows in Table II correspond to four different characteristics of situations that confronted students in our experiments: time, space, object extent and scale. As described in the chart, each characteristic interacts


T A B L E II

Conceptua l f ramework: Video as a br idge be tween everyday percept ions and ma themat i ca l

representat ions

EVERYDAY VIDEO (and VIDEO MATHEMATICAL

PERCEPTIONS BASED TOOLS) REPRESENTATIONS

Time • Time is continu- • Time is structural- • Time can be repre- ous, unidirectional and ly segmented and dis- sented discretely unstoppable continuous but when and/or continuously

"played" can be experienced as continuous

• Time is controllable: • Time can be rep- can be stopped, repeat- resented explicitly (as ed, played backwards, in a standard position slowed down and sped or velocity graph) or up implicitly (as in a y/x

trace graph)

Space • 3D space is repre- • Representations sented in 2 dimensions are constructed in 2D

space

• Objects have extent; • Objects may or may they take up space in not have extent; when 2 dimensions; in a sin- the location of an gle frame, the image of object is represented in an object will extend an iconic fashion, as a across a portion of the picture of itself, it has screen extent; when an object

is represented by a point, it has no extent, takes up no space

Scale • A scale may or may

• Space is 3D

Object extent

Other Considerations

• Objects have extent; they take up space in 3 dimensions

• We unconsciously con- • We can construct a scale between not be provided for the scene captured on us; in formal quanti- video and the original tative graphs a scale scene if we can find is commonly explicit an object in the video and required; in less with characteristic size formal and/or qualita-

tive graphs, a scale is often absent

• Video is familiar and • Learners are influ- appealing to adoles- enced by school-based cents; in some ways a notions of what con- medium they "own" stitutes a valid mathe-

matical representation

struct and alter scales all the time, for example, when we make judgments about the size of a far away object or change our spatial relation to an object

• Everyday experiences of speed are often more dramatic and more memorable, than those of location or distance

• Everyday artifacts of speed, such as mph, sometimes provide a starting point for an understanding of speed as a rate

• Video is often con- sumed passively, without the aid of tools to explore or create • Traditional school practices stress certain graphing conventions

pay

70K

30K

INTERACTIVE VIDEO

V

65

t

Figure 1. Pay vs. time; speed vs. time.

with the three contexts (everyday perceptions, video representations, and mathematical representations), producing three different sets of possible manifestations. Let us look at each in turn.

Time is probably the most pervasive, if at times elusive, concept in the mathematics of motion and change. In everyday perceptions, the most salient features of time are its continuity and its unstoppability. In general, we experience time, not as granular, but as smooth and unidirectional, at least on the order of hours, minutes, and seconds. (Many cultures have different beliefs about the long-term shape of time and Alan Lightman, in Einstein's Dreams, postulates many other possible structures for time.) By preserving, yet distorting, time, video adds a new twist to "natural" time. This feature brings with it unique opportunities as well as unique complications for students.

Video turns time from a continuous stream into a punctuated series of images. As described above, the structure of time in video is segmented and discontinuous. However, because of the multiple modes provided by video tools, time can be experienced continuously (if played through at normal speed) as well as discontinuously (if stepped through frame by frame). In the former mode, we experience time as continuous, even though it actually has significant gaps on the video tape. Our eyes and our brain fill in the blanks.

In mathematical graphical representations, time has both continuous and discrete aspects. Time is most often graphed on the x-axis of a graph and considered the independent variable. (But see Rubin & Boyd, 1994, for an example of a student who consistently thought of time as the dependent variable.) Position or velocity curves vs. time are sometimes smooth, qualitative representations and at other times points, where a dependent variable is measured at individual points in time. In this case,

66 ANDP.EW BOYD AND ANDEE RUBIN

Y

00 O•

Figure 2. Generic trace graph.

X

our underlying belief in the continuity of time can be suggested by a curve connecting the points. So time can be represented in both ways, often on the same graph. Clearly discontinuous events are even sometimes represented continuously, as with the time vs. pay graph in Figure 1. Pay almost never (that we are aware of) increases continuously; more often, increases happen every year, half-year or perhaps month. Yet we accept a graph such as Figure 1 as representing a trend in pay, even if it does not represent its time course accurately. Time can also be implicit in a graph of movement, as in the x-y trace of a thrown ball in Figure 2. Here each point represents an individual time yet time is not on either of the axes.

Space also varies across the three contexts, however, in less complex ways than time. In the world of our everyday perceptions, space is three dimensional and continuous. We share the same space as the objects and motions we investigate. With video, 3-D space is flattened into two dimensions. It still appears 3D because of its life-like image and motion, but we cannot see into or move around in the space. Graphs and other mathematical representations are almost universally presented in the two dimensional space of the page or screen.

The third row in the table is object extent. In our everyday perceptions, objects occupy 3D space; they have extent in three dimensions. In video, objects (the image of the ball, for example) have extent in two dimensions. In most graphical representations, however, the location of an object at a given time is represented by a poin t, theoretically a "dimensionless" mark and, in reality, a very small point that occupies almost no area.


Scale exhibits some interesting variations across the three contexts. In the context of everyday perceptions, we make judgments about distance and the sizes of objects using input such as depth perception and comparisons between objects. In the context of mathematical representations, such as graphs, size is indicated by explicit scales on axes. Video sits between these two contexts. There is no explicit indication of size in video, as there may be in graphs. On the other hand, we cannot figure out the size of objects in video in the same way as we do in our everyday lives, since we do not share the same three-dimensional space. Any comparisons we might make among objects in the video scene can only tell us about their size relative to the video.

In order to establish the size of the original objects in the video, we need to find a recognizable object captured on the video whose size we can use as a reference. Using this object, we can build a scale that maps distances in the video scene to distances in the original scene.

A fifth row in the table describes "other considerations" that affect our interpretation of representations in each of the three contexts. Such considerations include: common informal experiences, cultural artifacts, and conventional practices. For example, while trying to work through motion problems, students often referred to their own everyday experiences of motion such as accelerating in a car, or slowing down on a skate board. Likewise, while trying to grapple with the concept of speed as a rate, students often looked first to familiar measures of speed, such as miles per hour.

Finally, it is important to note that the three contexts are not as sharply divided or as monolithic as Table II might imply. In fact, each context actually contains a spectrum of situations. For example, in the realm of everyday perceptions, there is motion that one bodily participates in and experiences more directly, such as interaction with MBL's distance probe, and there is motion which one indirectly observes, such as a ball moving on a ramp. Likewise, in the domain of video, there is a normal play mode, which feels much closer to everyday perception, and there is a frame by frame analysis mode (perhaps linked with a software tool), which is much closer to the mathematical domain. And finally, among mathematical representations, there are those which are more akin to video, such as trace graphs (see Figure 2), and those which are less akin to video, such as standard 2-coordinate position or speed graphs.


3. METHODOLOGY

Over the course of 10 months we conducted a series of video-taped interviews with 8 students to explore their conceptions and mathematical representations of motion, time, and speed. With the exception of one session, all the interviews were with one student at a time. Students were given various presentations of motion phenomena and asked to make mathematical sense out of them by designing and building their own graphs. The various presentations included multiple exposure photographs, interactive software based around digital video movies, and video on a frame-advance VCR with which marks could be made on a piece of acetate placed over the TV monitor.

These sessions were conducted as clinical interviews, in the tradition of Piaget (1970); Duckworth (1987); and Steffe (1983). The interviewer presented students with materials and a context and set an initial problem, but then interacted with them in an open-ended manner. While the interviewer attempted to "guide the child as the child guided him" (Ackerman, 1991), he at times might intervene deliberately to push the interview in a certain direction or to provide some key information (such as how video frames were structured or how an x-y coordinate space was set up).

Over the course of the project, we honed our research method and design and therefore chose to focus our analysis on our last two students. The final design involved three interviews, each 60 to 90 minutes. We worked separately with each of the two students, Erica, a 6th grade girl and Tom, a 6th grade boy. Neither of the students had had much exposure to conventional graphs. But both students came from family backgrounds which encouraged independent thinking and creative problem-solving, in mathematics as well as other subjects.

In the first session, we worked with TraceBuilder, a piece of software developed at TERC. Here students observed a QuickTime movie of a ball rolling up and then down a softly inclined ramp. They were then asked to represent that motion by making a "trace graph," a series of images of the ball on the ramp at equally-spaced points in time. The students built this graph by dragging images of the ball onto a snapshot of the movie (see Figure 3). Once they were satisfied with their trace graph, the TraceBuilder software would "animate" it, effectively playing the graph like an animation on top of and in synch with the movie. This allowed the students to observe the ways in which their representation did and did not fit the original video. Based on any discrepancies, the student could modify the trace by adding or removing ball images or adjusting their location. The students would then reanimate the graph and again make adjustments.


Figure 3. Erica's last trace builder graph.

After 5 or 6 cycles of this process, both students were able to construct a trace graph that matched the video well.

In the second session, we presented the student with a mission: make a graph (or a number of graphs) that effectively describes the motion of the ball in the movie to a friend who knows nothing about the ball except what they can learn from your graph. The students could play the QuickTime movie of the ball and refer to the final trace graph they had created in the previous session. They were granted a wide degree of latitude to create the kinds of graphs they deemed appropriate. Both Erica and Tom created several different graphs in this session.

In the third session, students used CamMotion software, also developed at TERC, to take data directly off of the digitized movie and automatically build time series tables and graphs. Using this tool the student verified the graphs they created by hand in session 2 and explored additional mathematical aspects of the ball's motion.

For the final research analysis we choose to focus on the middle session, in which each student did her or his most creative, independent work. It was here, outside of the constraints of a software program or tightly scripted activity, that the students were most inclined to experiment with new ideas and work through the knots in their thinking. Of the two students, we choose to focus primarily on Erica and refer to Tom for contrast and comparison.

It is a testament to the power of the open-ended clinical interview that some months later Erica remarked that her time with Andrew had been one of her favorite learning experiences of her life. She had found it particularly satisfying that she had been able to invent something (i.e. several change over time graphs) that she had never seen before.


In ana lyz ing be low Er ica ' s a t tempts to cons t ruc t a ma themat i ca l rep-

resenta t ion o f the m o v i e o f the ball on a ramp, we focus on the effect o f

v ideo ' s character is t ics on her unders tand ing and representa t ion o f the bal i ' s

mot ion. We present first three segments f rom her transcripts, then focus our

d iscuss ion on the issues o f scale and objec t exterxt, t ime, and the sal ience

o f intervals. Finally, we c o m p a r e some o f our observa t ions to those o f

d iSessa et al. (1991), w h o s e students worked on a related g raph-cons t ruc t ion problem.

4. I N T E R V I E W T R A N S C R I P T S

4.1. Segment l : What is the World? What is a Graph?

Erica and A n d r e w begin the interview. O n the c o m p u t e r screen is the final vers ion o f the t race g raph Er ica crea ted in the prev ious sess ion using TraceBui lder sof tware (see Figure 3).

2:05 A: Do you remember what happened last time? E: Yeah, ball was rolling up, each frame we marked with balls where

it stopped. A: Do you remember how you changed the pattern over a couple of tries?

2:20 E: Yeah, it went faster and then got slower as it reached the top. A: And that (the trace graph) is the one you finally thought was right, huh? E: Yeah. A: First thing we are going to do today is ask you to graph that.

3:20 A: Let's say you needed to communicate what the ball did to somebody who could only see the graph.'They knew it was a ball doing something but that's it. Your goal was to create a graph that would tell them about the ball but all they could look at was the graph. So what would you do?

4:00 E: I don't know, something that like shows distance or like length. I don't know, o f how far it moved each time (inaudible) what kinds of graphs I could do to show that. 'Cause most of the graphs I know are like counting . . . like you know how many people or numbers.., hmmmm .. . what kind would work to show i t . . . ?

A: What are your choices, what are you thinking of?. E: I don't know any of the choices... I know what I should show to show

how it moves but not how. A: That makes sense . . . you know what to show but not how. E: Yeah. A: So you know what. So tell me. E: To show how it slowed down, how it was going faster but got slower

each time A: OK. OK. Right. E: 'Cause I couldn't really draw a ramp because that would be just the

same thing but . . . A: Right. Right. But you were saying something earlier about something

5:00


like [drawing?] for distance or something like that... ? 5:45 E: Yeah, just like how much more it moves each time. Like, I could

somehow show that. 'Cause (inaudible) like I did that (pointing at the trace) and I could do just x's or something to show how far it moved ... I would be just drawing it back .. . If I just did like x's to show how far it moved...

6:25 A: OK, what's the problem, here? 6:30 E: It seems like I would just be drawing that... (pointing at the trace).

A: OK. . . you mean you think it's kind of like cheating . . . ? Is that what you mean, sort of?.

6:45 E: I just like think.., like you said .. . I think it's like doing it like that, and see what's happening, somebody could see it as I was drawing it for them ... [inaudible]

7:15 A: We can make more than one graph too. . . It's easier for me to understand what you mean by actually doing it and probably easier for you to see if it works without going against the rules or cheating or whatever. So why don't you take a try at it?

Erica begins this session with the representation she created at the end of the previous session: a trace of the motion of a ball rolling up a ramp at 1/6 second intervals (see Figure 3). She is asked to create a graph by hand that describes the behavior of the ball to an unknown third party. She struggles to imagine an appropriate k ind of graph.

A couple of things are notable about the basic setup of Erica's task. First, she begins, not only with the ball movie (everyday motion documented by video), but with a representation she has previously created (her trace graph). So, she has already done a great deal of thinking and experimenting with this motion. Second, she is allowed a great measure of f reedom to create her own kind of graph. She is given little direction as to what kind of graph to create: no labeled axes on which to build her graph, nor a guiding definition about what would constitute an appropriate graph, although she has her own working meaning for the word "graph."

Throughout this segment, as Erica seeks out what is for her the key aspect of the ball's behavior, she focuses on intervals. She wants to show not so much where the ball is but how the ball "got slower each t ime" (lines 26-27) and "how much more it moves each t ime" (line 33). This tendency to attend to intervals reappeared again and again, not just with Erica, but with every student in our study.

Erica seems to know what aspect of the motion she wants to communicate in her graph - "to show how it slowed down, how it was going faster but got slower each t ime" (line 26-27), but is uncertain precisely how to go about representing it . . . . As she struggles to decide on the form and content of her graph, ("I don ' t know any of the choices . . . I know what I should show to show how it moves but not how" (lines 21-22).) it seems


, / f l

A ball on a ramp

1

Figure 4. Erica's graph 1.

that Erica's understanding of the bali's motion is closely tied to her finding a way to represent it.

While she has never made the kinds of line graphs commonly used to depict such content, she notes that she has experience with discrete quantity graphs and bar graphs ("counting," as she calls them). However, she does not feel that these will help her here. Thus, she finds herself in a situation, unusual in school settings, of being asked to do something without being shown how to do it.

She responds to the challenge by returning to the data she has and building from there. But it is not easy, as she considers and rejects her first few ideas. "I couldn't really draw a ramp because that would be just the same thing . . . . " she says (lines 29-30). Likewise, she cannot just draw the trace graph again (line 39). Although it is the interviewer who introduces the word "cheating" into the conversation, it seems here that Erica is operating with a notion that a proper graph, or any kind of authentic representational act, must re-present its source data in an altered or different way. Thus, she is reluctant to make a graph that feels like a simple copy - either of a series of video images (the ramp) or of a previous representation (the trace) she herself made.

Beginning just after the end of the transcript segment, Erica eventually builds graph 1 (see Figure 4). Graph 1 is similar to her earlier trace graph in that both graphs show the location of the ball at 1/6 second intervals and neither uses a time axis.

However, the two graphs are different enough in a number of significant respects - X's instead of ball images to mark ball locations, a left to right movement of the ball instead of right to left, a grid background - to satisfy her criteria for an authentic representation.

Erica makes Graph 1 on a large piece of graph paper. She places her first X in an arbitrary grid square and then decides where to place each subsequent X by a coarse visual estimation of the size of the interval between adjacent balls in the trace, saying "I 'm just going to make it


a l i t t le shor t e r e ach t ime ." E r i c a ' s g r a p h 1 is b a s e d on r o u g h qua l i t a t ive

c o m p a r i s o n s and i n c l u d e s no uni ts , n u m b e r s o r p r e c i s e m e a s u r e m e n t s .

A f t e r m a k i n g the g raph , she is no t sa t is f ied , j u d g i n g that i t does no t c o n v e y a

c l ea r p i c tu re o f w h a t the ba l l is do ing . She c o n s i d e r s l abe l ing the h o r i z o n t a l

l ine at the b o t t o m o f he r g r a p h and pu t t ing n u m b e r s on it " to show h o w

m u c h [the bal l ] is s l owing down . " W e p i c k up the t r ansc r ip t aga in f rom

there .

4.2. Segment2: How Do I Get Units From the Everyday World?

16:20 E: I know what I could label it but I don't know what the numbers are on i t . . . I can est imate. . , on the computer but not in real l i f e . . , can't measure it.

A: OK. why's that? 16:40 E: It could be really huge or it could be that (points to the monitor) size in

real life . . . well, actually it couldn't be that much bigger because of the hand, the hand would be a certain s ize . . .

17:00 A: OK. great, so you can say . . . So you're not exactly sure how . . . like in a photograph you would have the same problem.. , is that what you are saying?

E: 'Cause you didn't see it in real l i f e . . . A: To get numbers we could roughly figure out like what you said, like

figure out how big our hand is . . . 17:25 E: Maybe it is a ping-pong ball or something . . . then it is probably a couple

of inches or maybe an inch 17:30 A: We can figure it ou t . . , if you want to take numbers off of tha t . . , we

can come up with something that is roughly correct . . . E: Can I do that? Is there any way I can like put in more of these [put

more ball images in between the ball locations on the trace] and take them out again? Put in more ba l l s . . .

A: I 'm not sure what you mean? E: Like, to see how many would fit in the space between them (indicating

X's on graph) A: O h ! . . . yeah, I see what you are saying. . , that's interesting . . .

sure . . . tell me why you would do tha t . . . 18:20 E: Because if I guess how big the ball is I can figure out how much the

distance between them was . . . by putting them in the space. [points to balls in trace image]

A: OK, yeah. (A sets up computer) (E moves ball images into spaces in her trace image; see Figure 6) 19:10 E: So two balls would fit between them [between the first and second balls

in her trace] and that would help because I should put the second one here (pointing to 4th grid square) like this since two other ones will fit in there (indicating 2nd and 3rd grid squares) . . . so . . .

A: Do you want to do that? Should we get another piece of paper? (E proceeds to make second graph)


. . . . . . . . . i n s i d e

cen te r to cen te r m e a s u r e . . . . . . . . . . . . .

to i n s i d e m e a s u r e

Figure 5. Bal ls inser ted in trace image.

Erica discovers that two balls fit between the first and second ball in her trace. She uses this information to revise her previous graph. In that graph, she had placed the first X in grid square 1 and the second in grid square 5. She now decides that the second X should be in grid square 4 instead, since there is room for only two ball images between the first and second balls in her trace.

Here again, as in the first transcript segment, Erica is working out a tension between the everyday world and her representations, this time centered around a number of conflicting demands concerning scale, units, and measurement. On the one hand she wants numbers and units that have a basis in the movie. On the other hand she also wants the numbers and units to be convenient: to assist her in making a neat, clean translation from the movie to her constructed graph. She must honor the world but also service its representation.

She can measure "on the computer but not in real life," she says (lines 1-2). On the computer she has a movie of a real life motion which she can measure with a physical ruler but this would tell her little about the actual distances in the everyday world captured by the video. She has slammed headfirst into the concept of scale. Luckily, however, the everyday world also comes to her rescue: there is a hand in the movie and this gives her an approximate scale - the ball in the movie cannot be a basketball; it is closer in size to a ping-pong ball.

Erica then uses the image of the ball itself as a unit to measure the distance the ball has traveled. She does this in a quite literal way by actually moving copies of the ball image into the spaces between the ball positions on the computer screen. She measures the size of these intervals by seeing how many ball images fit into a given space.

Here she is not only straddling two worlds, the everyday world as captured by video and the world of mathematical representations, but also two modes of measurement, the relative and the absolute. She is using the



image of the ball, an object from the everyday world, as her mathematical unit of measure and is at least partly aware that this is sufficient to give her a relative measure internal to the movie, allowing her to measure anything in the movie in "ball-widths." Beyond this, however, she also feels the need to assign the width of the ball a numerical value in inches, even if this means guessing its approximate size.

By deciding to consider the ball to be one inch wide, Erica settles on a measurement that is both convenient for graph making and falls within the window of reasonable sizes for the ball when roughly compared to the size of the hand. Thus, she finds a fortuitous yet reasonable way to both stay true to the everyday world and cater to the needs of her representation.

From here, Erica spends almost half an hour making graph 2 (see Figure 7). She places her first two X's with two blank grid squares between them, and then decides where to place each subsequent X by how far it is from the previous ball. This time, however, she uses a more precise measurement based on how many ball images she can fit in the intervals between balls in the trace on the computer.

When the graph is finished, the interviewer steers Erica in a new direction, asking her where she can find information about time in her graph. Erica already knows that the video frames take place every 1/6 of a second, a fact that she learned while working with Trace Builder.

4.3.

46:08 46:15

Segment 3: "It Takes Longer to Move the Same Amount o f Time" - Time, Distance, and Speed

A: Now, I've got a question about this graph. Which is, where is time? E: It isn't really time. I don't think. Like. I don't know. Well, it actually

takes about.., like ... a second. (Counting X's on graph) One, two, three, four, five, six, seven, eight, nine. All right, so there's nine frames and


47:23 E: A:

E points to E:

A: 47:45 E:

A: 48:16 E:

49:05 A: E:

A: E: A: E:

50:02 E:

A: E:

50:31 A: E:

there are six in a second, s o . . . (Pointing after sixth X) this would be like a second . . . and this (gestures as if to underline last three X's) would be half a s econd . . , so it's like one and a half seconds. So if I could put like a thing that said, like a mark that said like, one seconds (points after sixth X) and said like one and a half seconds (points after and below last X ) . . . one and a half seconds and I could write when it read . . . like what time . . . . I could start a s topwatch. . , what time it reached a certain point of the ramp. (inaudible) Well, I could say a sixth of a second There. Okay. Where?

the first X Well, here. Because this is . . . (inaudible) (pointing to each X in order) (inaudible) (putting down marker and reaching for a different one) You can use another c o l o r . . , other colors if you want. You know. (Writes "one second" on graph) (This?) is one second. I ' l l probably just be right at this thing. OK B u t . . . and t h e n . . , so since it was moving slower right here (points at right end of graph), it took like longer for it to just move, for it to just go like that. (points with two fingers, pointing at 8th and 9th X, then moves them to right so finger that was pointing to 8th X is now pointing to the 9th X.) so it's like moving s lower . . , since, like, this is half a second (moves finger under first 3 X's, then puts hands up to show boundaries) and it's like one, two, three, four, five, six, seven (counting squares on the graph) that's like seven squares and (counting further to the right on the graph) well this is like half a second (one, t w o ) . . , it 's like four squares. So they're still moving, but it's moving slower, so it takes it longer to move the same amount of time? Why don't you write those in? Wait. I 'm not exactly sure if it would be like right in the middle or like on the end. (referring to where to put line for second and a h a l f - at end of square or in middle of square.) Where'd you put the first one? You put it at the end? I put it at the end. So why don't we just stay with that. Does that seem OK? Mmm . . . So I ' l l write like a half a second. (writes "half a second" on graph) Do you think it would be like . . . well, I think it must he like an even thing? Why? Well, actually I would cause . . . w e l l . . , if they were moving the same speed (gestures with pen in front of her, moving smoothly from left to right), it would be even. but since like since these (points to the first few X's) are moving faster, it doesn't take as long to do a half second as these (moves pen along last few X's) since they're moving, since they're moving like really slow, so they can take up more time. So you're saying they would be even? No. Well, if they 're, well, it would be even if they were all going the same speed, like if was rolling along a f l a t . . .


A: OK. So can you tell what the speed is? E: Well, you mean how long it takes to do the whole thing?

In these four minutes, Erica has to grapple with several complexities about the representation of time and relationships between distance and time. The graph she has already constructed - patterned, as it is, after the trace representation she encountered in Trace Builder, is about motion over time, but has no dimension on which time is explicitly represented. Her first remarks are quite clear; she realizes that the action has gone by at six frames/second, that nine snapshots of the ball have taken a second and a half, that the sixth snapshot happened after a second. Again she struggles with the ambiguities of her representation: should she write "one second" at the center of the sixth X or at its right edge? The interviewer keeps her from re-entering this quandary by suggesting consistency with her decision for the first X, putting it at the end of the interval.

Before considering further the graph Erica finally drew, let us review for contrast how a "standard" graph of distance vs. time looks. Standardly, we see time on the X axis and distance on the Y axis. 4 Conventional rules for forming graphs require that mappings from parameters to axes be consistent, most often linear or logarithmic, and when students first encounter graphs, this mapping is almost always linear. Thus, identical differences in the value of parameters should be represented by equal length intervals on the axes. For example, the interval from 1 and 2 seconds should be the same length on the axis as the interval from 2 to 3 seconds. This last requirement is often emphasized and used as an assessment criterion by mathematics teachers. In fact, in most math classrooms, the conventional format for distance/time graphs is taught as if it is the only way to express the relationship between these two variables.

In the light of this pedagogical insistence on a particular graph structure, it is interesting to note that Erica's graph deviates significantly from the conventional structure, yet still manages to tell the story clearly. The most obvious difference is that the graph has a single axis, rather than the conventional two. For Erica, this was not an explicit decision, but a natural outgrowth of the process by which she created the graph. The strongest influence on her representation was her experience with Trace Builder, which provided her with a picture on which she based her graph. In essence, she took the picture she had from Trace Builder, tumed it into X's and labeled the time that corresponded to each distance. It never occurred to her to create a second axis to indicate how time related to distance.

4 We suspect this assignment of parameters to axes is standard in order to guarantee a functional relationship; putting distance on the X axis may result in a non-function, with the same distance occurring at two different times.


As a result of labeling the ball's locations with the associated times, Erica unwittingly violates the linear mapping convention of the standard graph axes. On her graph, the same amount of time (half a second) is represented by different distances at different places along the X axis. This happens because the X's are already placed on the horizontal axis according to how far the ball moved in each time interval. In order to include time on this same axis, Erica must write the appropriate time at the position of each X. Because the ball slowed down, it moved less far in 1/6 second at the end of the motion than it had at the beginning, so the distance between 1 second and 1 1/2 seconds is smaller than the distance between 1/2 second and I second. This is not a problem for Erica. For one thing, she likely does not know the "rule" she is bending. For another, she probably regards what she is doing as "labeling," rather than as placing values along an axis.

While the differences between Erica's graph and a standard graph are not problematic for her, they are interesting to us as possible influences on the design of classroom activities. Erica's graph is actually similar to a representation some teachers use for stories about distance, rate and time and call "journey lines." (Lampert, as described in Hall & Rubin, in press) Journey lines represent distance and time on two parallel lines and proportional relationships are used to answer questions such as, "How long will it take to go 55 miles?" Like Erica's graph, these"joumey lines" display both time and distance along the same dimension. They are commonly used, however, only to represent journeys of constant velocity, so that congruent segments on either line always represent equal distances or times. On Erica's, congruent segments always represent equal distances, but do not always represent equal times.

While the design of Erica's graphs appears to develop without much constraint of convention, there is also evidence that what she knows about graphs does influence her construction. Erica uses both conventional tick marks and the conventional direction for the horizontal axis. She reverses the motion of the ball from TraceBuilder, so that the graph "runs" from left to right, as conventional graphs and texts do. (Another possibility is that this reversal is part of her attempt, described above in Segment 1, to make her graph different enough from the video that it counts as a genuine representation.)

It is easy to claim that Erica's decision to begin with distance on the X-axis was determined by the structure of the Trace Builder representation. But in many of our teaching interviews in which students did not use Trace Builder, we found a similar tendency to put distance on the X axis. For example, in Rubin and Boyd (1994), we reported that students often put


distance on the X-axis and time on the Y-axis, even in drawing two-variable graphs. This arrangement of variables on axes is related, we think, to the kind of questions that students associate with time/distance relationships. Our conventional graphs with time on the X-axis are best suited to answer questions that specify the time and question the distance, e.g. "Where will the ball be after 2 seconds?" Graphs with distance on the X-axis are better structured to answer questions that specify distance and ask about time, e.g., "When will the ball get to the top?" Interestingly, most of the time/distance questions we ask and answer outside of math class are the latter type. "When will we get to the restaurant?", but not "Where will we be in ten minutes?" In drawing their graphs, the students we worked with often talked about speed in terms of time per distance, as they graphed distance on the horizontal axis - sometimes before they even drew the vertical axis.

We also note that in this segment, Erica's descriptions of the ball slowing down sometimes substitute time for distance. For example, "So they're still moving [X's in the second half of her graph], but it's moving slower, so it takes it longer to move the same amount of time" (lines 31-32) and "since these [X's at the beginning of her graph] are moving faster, it doesn't take as long to do a half second' (lines 47-48) (emphases added). In both cases, Erica uses a time description to refer to a distance, producing sentences that are, on the surface, nonsensical. But we can easily make these sentences correspond to a standard view of the graph by substituting "distance" for "time" in the first case and an appropriate distance description for "a half second" in the second. This consistent error in speaking suggests that Erica has a particular meaning in mind, but makes mistakes in expressing it. Her statements seem to reveal an understanding of the relationship between distance and time obscured by her difficulty constructing sentences to explain about it. It is possibl e that the design of her graph, in which she has "merged" distance and time on the same axis, may contribute to this mis-speaking.

The final lines in this segment of Erica's transcript are the beginning of her transition to talking about speed. All along she has been using words like "slow," "faster," "longer to move," and focusing on the intervals between images of the ball - all of which are in the domain of speed. But the first time she actually uses the word ("if they were all going the same speed ~.." (line 53)), the interviewer pounces, determined to focus on speed more directly. To his question, "So can you tell what the speed is?" (line 54), Erica answers, in the last line of the transcript included here, "Well, you mean how long it takes to do the whole thing?" (line 55). This restatement of the question about speed shows an interpretation of speed


Left M 1

1/6 i

2 3

Ball on a ramp

r'xt NI i[gtl 1>,1 ~41-5 6 -7 i 8i 9" 1~ 11 T2 113 ~4 15"i6,i7, 18. '/611 ~t~ I " '- 5/' I 7 I68/6f'16!

I 1,2 ohd .,/~ 9n,, se,'.ond 11/; 6/6


Right

as time per distance. In contrast, an interpretation of speed as distance per time would be evidenced with a question such as, "You mean how far it goes in a second?" Erica's way of understanding the question echoes the way she constructed her graph.

All of these differences between Erica's graph and a conventional distance/time graph bring up pedagogical questions whose answers might profitably influence our thoughts about the best ways for students to learn about the conventional graphs seen as the "goal" of formal education. What mathematical insights does Erica exhibit in creating her graph that might become the basis for learning other graph structures? What ideas underlying the design of her graph might interfere with her using other graphical structures? How might the dual definition of speed as distance per time and time per distance influence pedagogical directions? What learning settings would be most useful to build on this experience?

After the end of Segment 3, Erica spends another half hour constructing her third graph, shown in Figure 8. Directly after the above conversation, the interviewer suggests some work with speed by saying, "Let's say the person reading the graph wanted to know what the speed was at any point . . . [italics ours]" Erica first frames the question as "How long it took for . . , like in seconds of speed, because you gotta do like how many miles per hour it's going," sticking with her idea of speed as time per distance. But in the next breath, she plans to "write the fraction of 1 8 inches that would come in a second," making a sudden switch to distance per time. She follows this general plan with a plan more specific to her data, " . . . and then like I'll try to figure out what the frames and the pictures of the ball in a 1/2 second is." She sticks to this choice of variables and progresses to a graph with time on the horizontal axis, and speed on the vertical - a conventional two-dimensional graph. Her movement to a two-dimensional representation seems to be at least partly due to her move away from the trace representation. In her speed graph, she needs to start "from


scratch," since she is representing a different quantity from what the trace shows.

5. REVISITING THE CONCEPTUAL FRAMEWORK

In the following sections, we recap our analyses of Erica's graph-making activities, seen through the perspective of the conceptual framework developed in Section 2, attempting to identify the effect of the video medium on salient characteristics of her graph.

5.1. Scale and Object Extent

In the analysis of the first and second transcript segments, we see Erica negotiating issues of scale and object extent. By considering these from the perspective of the conceptual framework we described earlier in Table II, we can better understand Erica's struggles and begin to generalize from them.

Let us look at Erica's work with scale. For the graph she has chosen to build, Erica decides she needs to know the approximate size of the original objects in the video scene. In our earlier discussion of scale, we noted that in order to establish the size of the original objects in a video scene, it is necessary to find a recognizable object whose size can be used as a reference. Erica does exactly this with the image of the hand. While the ball and ramp are too non-specific to indicate their original sizes (it is not clearly a golf ball or a tennis ball . . . etc.), the hand has an identifiable approximate size. Using the hand, Erica is able to build a scale that maps distances in the video scene to distances in the original scene. Using this scale she is then able to estimate the size of the ball. Here video nicely foregrounds the problem of scale. If Erica had been working with an actual ramp and ball she could have measured them directly. Instead she has a video representation of the world which requires her to identify and solve the problem of scale.

Erica's encounter with the issue of object extent can also be understood through the lens of the conceptual framework, which allows us to see her first two graphs (graphs 1 and 2 of Figures 4 and 5), as hybrids: intermediate representations that occupy a curious middle ground between the video and mathematical domains. The symbols she uses in these graphs owe something to both the 2-dimensional video images she is working from and the more "mathematical" representations she is trying to create. While they are no longer simply pictures of the motion, they are still somewhat iconic. The X and/or box, which Erica uses as a place marker, is not the


M o v e m e n t

in inches

7

6

5

4

2

1

Speed and M o v e m e n t of

a ball on a r amp

%

Seconds

Figure 8. TWo ways to measure.

kind of abstract point that we usually see on a graph of motion. Like the image of the ball on which it is based, it has width and takes up real space on the graph. Thus, Erica is forced to deal with the space it takes up in constructing the graph. In the process of making both of her graphs, she often becomes uncertain as to how to measure the distance between adjacent objects: should she measure between the middle of adjacent X's or between the end of one image and the beginning of the next? Figure 5 shows these two possibilities.

Part-way through the construction of graph 2, Erica decides to put a box around each X to make it clear that her representation of the ball took up space. She then continues to make her graph using a scale in which one square on the paper is equivalent to one ball-width on the computer screen. Later, however, she is perplexed for a time by the last two balls which overlap in her trace. Because she has chosen to use squares instead of points to represent the location of the ball on her graph, she now has to draw two overlapping squares on her graph, a choice which is consistent with her scale, but not with her idea of a graph.

Here as before, video serves Erica as a bridging medium. It captures the feeling and information of everyday motion as well as forcing particular representational issues to the fore. Erica must consider exactly where the


ball is located; how to measure its location in a consistent, mathematical way; whether or not to represent it with extent and what the consequences of this decision are. Her first two graphs stick close to the video in terms of how she represents the bali's position while she attempts to build sparser and more powerfully abstract representations. These graphs represent the location of the ball with an icon that still retains the bali's two-dimensional extent.

We suggest that the conflicts that emerge from this choice provoke Erica into dealing with representational issues that would otherwise not occur. Without the video, she may not have used the boxed X representation that mimics the ball's image on the video. The availability of the bali's image on video throughout the construction process - rather than the memory of it, which is all that is usually available - may be what pushed Erica to work with symbols that took up space. Eventually, with graph 3, a graph of speed, Erica makes a transition to a representation that uses points to mark values on the graph. Was this partly a result of representing speed, a property not immediately present in the video, rather than the position of the ball, a property that is immediately present in the video?

5.2. Conceptions o f Time in Video-Based Contexts

In the analysis of the third transcript segment, we see Erica struggling to locate time on her graph and to articulate a coherent relationship between time and distance given that she has represented both on the same axis. During the course of our experiments we saw every student attempt in similar profound and creative ways to build conceptions and representations of time. "Where's the time at?", some said, as they tried to locate time on a y/x trace position graph. "There are two kinds of time," another student remarked as he tried to distinguish between the time over which an event occurs and the time embedded in a rate. A number of students were torn about whether to graph time against distance or distance against time. And every student, at least once - and more often on numerous occasions and in diverse ways - conflated terms for time and distance and struggled with whether frames were a kind of time, a kind of distance or a third kind of thing entirely.

Once again, video serves as a medium by which students work on the relationship between the everyday and represented worlds. Working with video, they can lay their hands on time and motion and begin to disembed time, in its various guises, from the motion itself. However, as we have seen before, video is also a particular distortion of the everyday world and brings with it its own constraints and affordances.


Erica's work on graph 2 reveals a range of these issues. In video, time is not explicit but it is very present. The representations that Erica first builds (the trace image and graphs 1 & 2) reflect this aspect of video. They are based around measures of distance and do not show time explicitly. In her attempt to locate and identify time in graph 2, she places time on the same axis she has already unitized for distance. Representing both time and distance explicitly moves her closer to a clear understanding of their relationship to motion but the conflation of the two on the same axis and in her language still gives her problems. This leads her to graph 3, in which she separates time and speed on to two separate axes and sets up speed as dependent on time.

Another issue about time that is raised by video images is continuity. Video frames represent action (and, therefore, time) as points 1/30 of a second apart. Yet most people believe that time is, at least to our perception, continuous, without gaps. How do these two beliefs manifest themselves in Erica's graphs? Her first two graphs are made of unconnected symbols, representing only those images that are explicitly shown on the video. Graph 3, however, includes a line that joins the points. We do not know for sure why her representational strategy changes here, but at least one possibility suggests itself: The position of the ball is isomorphically derived from the video, so the gaps between images remain in the graph. Speed requires more interpretation and computation on numbers derived from the video, so the graph is freer to depart from the video's structure.

5.3. The Salience of Intervals and Speed

Throughout the transcript segments, we see Erica working with intervals as a basic structural element of both the ball's motion and her representation of it. In describing what she wants to graph, she says, "how much more [the ball] moves each time." In quantifying her trace graph, she chooses to measure "the space between" the ball images. In placing marks on graphs 1 and 2, she counts not from a common origin but from the previous mark.

Erica's focus on intervals was not unique. With uncommon regularity, every student in our study focused on intervals, attending to relative measures over absolutes, differentials over accumulations, the space in between images (or marks or points) over the distance from a fixed starting location. In consequence, speed, a differential measure, came to the fore over distance, an accumulated measure.

For example, we asked three 7th grade girls to describe and graph the motion of a diver captured by Harold Edgerton in a multiple exposure photograph (Figure 9). They quickly fell to comparing and measuring the distances between sequential images of the diver. For these learners, as


Figure 9. Harold Edgerton multiple exposure photograph of diver.

for Erica, intervals appeared to be the basic structural unit of the motion under investigation. In addition, one of the interval-related issues that arose for Erica arose also for these girls. Their most spirited debate concerned exactly how to measure the interval between sequential images of the diver: from the forehead of one diver image to the back of the head of the next (the "empty space" in between two sequential images) or from the forehead of


one image to the forehead of the next (the distance between the same part of two sequential images). In the end, one of the girls managed to convince the others to go with the latter approach by arguing that in one time segment the diver's forehead "passes through all the points in between" the forehead of the first image and the forehead of the subsequent image (including the back of the subsequent divers head) and thus more properly measures the actual distance traveled by the diver.

Other students we worked with exhibited a similar focus on intervals while investigating varied presentations of motion phenomena. For example, a number of students analyzed the video of an accelerating toy car. They stepped one frame at a time through the video and marked off the car's sequential locations on a piece of acetate laid over the video monitor. When they examined the trace graph created through this process, every student zeroed in on the spaces between the marks. In their first remarks they tried to explain the car's acceleration in terms of the spaces between the marks and the manner in which the spaces grew larger and larger. They then went on to use these intervals as the basic building blocks for further representations.

Whether our students were recording successive locations as they stepped through a video sequence frame by frame, or analyzing a trace image or measuring a multiple-exposure photograph, they found a raw visual salience to the space between sequential images that drew their attention, inviting them to describe the structure of the motion in terms of these gaps. We observed this attraction to intervals in every session. We attribute it to two related influences: the "choppy" nature of video and the trace representation, which show successive images (or locations) simultaneously. The first of these attracts students' attention to discrete changes; the second represents them in such a way that the intervals are easily readable on a single page.

Video-based media are structured as repeating images with intervening gaps in time. When our students stepped through a video frame by frame they saw a ball or toy car "jerk" forward. Their attention was drawn to what had changed, to what was different from one frame to the next. They easily noted discrete changes in position that occurred in discrete amounts of time. But what brought the students to focus on intervals as a computational tool was the trace representation. Measuring and comparing intervals by looking at a sequence of images is possible, but difficult. Putting all the locations (and thus all the intervals) on a single page brings the intervals into plain view.

A further consequence of the salience of intervals is that speed too becomes salient. Intervals are, after all, the differences between the dis-


tances an object has traveled in (equally spaced, if we arrange it so) moments in time. Thus, they inherently measure speed. Naturally, students do not recognize the relationship between intervals and speed immediately but they do develop some ideas about this relationship as they work to mathematically understand the video image or trace and move towards a representation of it.

To some degree, acceleration becomes salient for similar reasons. Sub- tracting adjacent intervals (and then dividing by time) is the critical step in measuring acceleration. A rough visual comparison of intervals (as is available in a trace) can reveal the basic shape of an acceleration pattern. For some students, it is the acceleration pattern that seizes their interest and becomes the aspect they most want to understand and represent. In the case of Tom, this happened to an unusual degree.

Like Erica, Tom was largely unfamiliar with x-y coordinate space and had never before made a time series graph. Yet, largely on his own initiative, he made both a speed and an acceleration graph. Tom based his speed graph on the intervals in the trace image and his acceleration graph on the intervals in his speed graph. Later on, as he struggled to better understand what these two graphs actually meant, he applied an interval-like method to the sequence of speeds in the speed graph and realized that in this manner, he could derive values to describe the speed of the ball slowing down: "This is the speed of the ball (indicates speed graph) and you can see it slowing down so you could take the slowing down information and then put it into that graph (indicates acceleration graph) because here (indicates early portion of speed graph) it is slowing down s lower . . . "

Although Tom's ability to talk about the relationships among his trace, velocity and acceleration graphs was not fully developed, it is notable that he chose to explore this direction before making a distance graph. He was not only able to invent for himself a "rate of rate" language to speak about acceleration (e.g. "slowing down slower"), but also to turn his velocity graph, previously an "output" from an earlier translation, into an "input" to a new translation. This example, in good company with earlier ones, inevitably raises the question, "What characteristics of video lead students to explore representations of velocity and acceleration over those of distance?"

6. EFFECTS OF VIDEO AND SIMILAR MEDIA ON STUDENTS' REPRESENTATIONS

Our observations of students using digitized video as a source of data have led us to several conjectures about video's effect on the way students Vans-


late object motion into graphs. These conjectures connect to the following characteristics of video:

- Video is a replayable medium, in which a motion can be experienced multiple times and at different speeds°

- Video is made up of discrete "snapshots," separated by short intervals of time.

- Objects on video are not necessarily the same size as they are in reality, and we cannot judge their size by moving among them.

- V i d e o is a two-dimensional representation of three-dimensional scenes.

Video shares these characteristics to a large extent with computer simulations of motion, such as those diSessa et al. (1991) used in their study of students' graphical representations of speed. Yet, the representational issues students in our study grappled with are somewhat distinctive from those in diSessa's work. Below, as we explain the connections we see between the video medium and Erica's graph, we also compare our results to those diSessa describes. We hope in this way to specify more exactly the representational influences of video and similar media, as well as the interactions between media and task descriptions.

1. When a graph is the result of a student's manipulation of video frames, the appearance of the object in the video may affect its representation on the graph. Because Erica spent her first session using TraceBuilder, she had already made a representation that contained video images. Her first attempts at representing the motion of the ball in a graph were influenced by this experience, and she began by using X's to represent the ball's position. Her work with determining an appropriate method for locating the X's is described in detail above.

The issue of object extent does not appear in diSessa's paper. Students offer many representations that use symbols other than points (horizontal lines, vertical lines, slashes), but their relationship to the actual appearance of the moving car is not part of the discussion. The only "real-life" objects that appear in their graphs are those in the imagined scene - e.g. roads, a cactus. (DiSessa categorizes these appearances as "figural influence.") This is probably due to at least two factors: 1) Their graph-making work was preceded by programming, in which, it seems, the car was represented by a Logo-like turtle. (Actually, diSessa's paper is unclear on this point - it may be that they used a turtle roughly shaped like a car.) Thus, they had already departed from the appearance of a real car when they were creating and tracking the motion. 2) The students all programmed the simulation so that the turtle left a trace graph behind - a trail of dots, spaced farther apart


as the turtle moved more quickly. This representation is different from that with which Erica first worked, in which the objects in the trace graph were the ball itself, rather than a dot.

2. I f a graph o f a video scenario is going to be labeled in terms o f "real distance," a scale must be determined using some video object whose real size is known. Erica spent considerable effort constructing a scale to translate distances in the video to distances in her graph, using the real size of the hand to determine the scale. It is interesting that Erica felt the necessity to label her graph axes with distances in the real world, rather than with, for example, distances on the video. This was certainly due at least in part to the fact that the objects in the video were representations of real objects, whose sizes were accessible through everyday knowledge and estimation.

Students in diSessa's work did not in general feel the need to label their axes with numbers; only toward the end of their work did they add some numbers - and these were indications of speed, not distance. This must have been due at least in part to the words used for by the teacher: "draw a picture that shows the five words: fast, abrupt stop, slow, fast." A "picture" does not necessarily suggest a representation with numbers. But it is also likely that the computer simulations led less clearly to a numerical scale because they have no obvious reference in everyday perception. One of the unique characteristics of video may be that it is almost indistinguishable from the world of everyday perceptions, and the sizes of objects are thus salient and more emphatically demand representation.

3. Graphs derived from video may predispose students to representing distance on the horizontal axis. As described above, both Erica and Ivan produced graphs with distance on the horizontal axis; Erica later created a graph with time on the horizontal axis, but Ivan stuck with his original decision to put distance on the horizontal axis. Erica's earliest graph included both distance and time on the horizontal axis.

In diSessa's paper, we also see students first using the horizontal axis ambiguously for either time or distance. Later they decide as a group to use it for time, a decision which diSessa notes as one of the major suc- cesses of the discussion. More interesting to us than the final decision of diSessa's students is the fact that placing distance on the horizontal axis is a common tendency. Two possible influences are the following: 1) As we have argued above and in Rubin and Boyd (1994), it seems that some students create distance-time graphs with distance on the horizontal axis because they relate to questions of the form, "How long will it take to


go this distance?" 2) It is also likely that the media used in these experiences supported graphs in which distance (or both distance and time) appeared on the x-axis because that motion was pictured - either in video or in a computer simulation - as a horizontal linear motion along a "path." Motion along this horizontal path may have suggested to students using the horizontal axis of the graph for distance. Future research may profitably observe the effects of vertical motion on students' graphs.

4. Video and similar media make intervals - and therefore speed- salient to students. DiSessa remarks that speed was quite accessible for his students, as evidenced by their consistency in using the length of lines in their graphs to represent speed, rather than distance. (This was certainly also influenced by the teacher's request to represent "fast, abrupt stop, slow, fast.") We found a similar tendency in our students to focus on speed - or at least on the intervals between the ball's positions, i.e. the distance the ball moved in one unit of time. As described above, much of work Erica did on her graph focused on the intervals between the ball's positions, and the girls who worked with the Edgerton diver also focused on the intervals between the diver's positions. We attribute some of this focus to parallels between the computer simulation medium and frame-by-frame video andto the ways in which they were used. In both cases, it is possible to view the motion over and over, at different speeds, and in frame-by- frame fashion. In both cases, a trace graph, in which the positions of the ball in individual frames were all represented in a single view, was an important intermediate representation. It is the trace graph that makes the intervals so visually salient (and a multi-exposure photo or a TraceBuilder image are the same as trace graphs, with photographic images substituted for dots). It seems that the trace graph and its natural derivation from the original motion in these media has a significant influence on students' work in graph construction tasks. Research on a possible role for these trace graphs in elementary and middle school, as students are first encountering representations of motion, could provide valuable directions for curriculum reform with an eye toward calculus.

As it becomes clear that different media have quite different affordances, other research suggests itself. What have been the effects of the strobe pictures physics teachers have often used to illustrate classic experiments? Does it matter if the series of images are shown sequentially or simultaneously, as a multiple exposure? What about the old PSSC movies? How similar were these to modem videos in their impact? Although students in school are seldom allowed to construct their own graphs, each of these media may affect differently the ease with which students can


create conventional graphs from it, if it leads naturally only to certain representations.

To return to our ogre, traditional calculus instruction waiting for unsus- pecting students wending down the mathematics path, we consider how the work these students did might relate to calculus. While the concept of a limit is undeniably central to calculus, so are the concepts of differences (e.g. the differences between consecutive positions of an object), the relationship between graphs of a measure (e.g. distance) and its differences (e.g. velocity), and the idea that the operation of taking differences that leads from distance to velocity can be applied over and over. If students can arrive easily at these proto-calculus representations and concepts using video and similar media, how can we use their experiences to smooth the road and tame the ogre?

ACKNOWLEDGMENTS

This research was supported by a grant from the National Science Founda- tion, Application of Advanced Technology, MDR-9154070. All opinions and especially errors are our responsibility, not that of the Foundation. We thank Ricardo Nemirovsky, Tracey Wright, Cornelia Tierney, David Carraher, Jan Mokros, Judit Moschkovich, and three awesomely diligent anonymous reviewers for their wisdom in improving this paper. Our special thanks to the students who let us watch them at hard play: Eri- ca, Tom, and Ivan.

REFERENCES

Ackerman, E. (1991). Personal communication. Barnes, M. (1991). Investigating Change: An Introduction to Calculus for Australian

Schools. Carlton South, Victoria, Australia: Curriculum Corporation. Bresnahan, S., Ducas, T. and Rubin, A. (1994). Cartwheeling in CamMotion. Hands On/

(Vol. 17, No. 2). CamMotion. (1995). Cambridge, MA: VIEW Project, TERC. Confrey, J. (1994). How compatible are radical constructivism, social-cultural approaches

and social constructivism? In L. Steffe (Ed.), Constructivism in Education. Hillsdale, N J: Lawrence Erlbaum Associates.

diSessa, A., Hammer, D., Sherin, B. and Kolpakowski, T. (1991). Inventing graphing: Meta- representational expertise in children. Journal of Mathematical Behavior 10:117-160.

Ducas, T. (1993). Active video: the promise of AVID learning. Journal for College Science Teaching. 23(3): (Dec. 1993/Jan 1994) 166-172.

Duckworth, E. (1987). The Having of Wonderful Ideas. New York: Teachers College Press. Edgerton, H. (1987). Stopping Time: The Photographs of HaroM Edgerton. New York:

Harry N. Abrams Inc.


Ferrini-Mundy, J. and Guether, K. (1991) An overview of the calculus curriculum reform effort: Issues for learning, teaching, and curriculum development. The American Mathe- matical Monthly 98(7): 62%635.

GraphAction. (1995). Watertown, MA: Tom Snyder Productions, Inc. Hall, R. and Rubin, A. (in press) . . . . there's five little notches in here: Dilemmas in Teaching

and learning the conventional structure of rate. In J. Greeno and S.G. Goldman (Eds.), Thinking Practices. Hillsdale, NJ: Lawrence Erlbaum and Associates.

HIP Physics. (1994). Watertown, MA: Tom Snyder Productions, Inc. Kaput, J. (1991). Democratizing Access to Calculus: New Routes to Old Roots. Dartmouth,

MA: University of Massachusetts. Kaput, J. and Nemirovsky, R. (1995) Moving to the next level: A mathematics of change

theme throughout the K-16 curriculum. UME Trends 6(6): 20-21. Lightman, Alan E (1992). Einstein's Dreams. London: Bloomsbury. Measurement in Motion. (1994). Santa Cruz, CA: Learning in Motion. Mokros, J.R. and Tinker, R.E (1987). The impact of microcomputer-based labs on children's

ability to interpret graphs. Journal of Research in Science Teaching 24: 369-383. Nemirovsky, R. (1993). Rethinking calculus education. Hands On 16(1). Nemirovsky, R. (1994) On ways of symbolizing: The case of Laura and velocity sign. The

Journal of Mathematical Behavior 13: 389-422. Nemirovsky, R. and Rubin, A. (1991), "It makes sense if you think about how the graphs

work. But in reality... "In F. Furinghetti (Ed.), Proceedings of the 15th Annual Meeting, North American Chapter of the International Group for the Psychology of Mathematics Education 3: 5%64.

Piaget, J. (1970). Genetic Epistemology. New York: Norton & Norton. Raymo, C. (1995). "The ways of a crow - That's calculus in motion". The Boston Globe,

February 20, 1995, 48. Rubin, A. (1994) Annual Report on the VIEW Project. Unpublished document, Cambridge:

TERC. 1994. Rubin, A. and Boyd, A. (1994). Perspectives on Learning the Mathematics of Motion:

Videos and Discussion. Paper presented at the conference of the American Educational Research Association, New Orleans, LA.

Russell, S.J., Mokros, J., Tierney, C. and others (1994-1996). Investigations in Number, Data, and Space. A K-5 Mathematics Curriculum. Palo Alto: Dale Seymour Publications.

Steffe, L. and Cobb, E (1983). The constructivist researcher as teacher and model builder. Journal for Research in Mathematics Education 14(2): 83-94.

Thompson, R (1991). The Development of the concept of speed and its relationship to concepts of rate. G. Harel and J. Confrey (Eds), The Development of Multiplicative Reasoning in the Learning of Mathematics New York: SUNY Press.

Thornton, R. and Sokolow, D. (1990). Learning motion concepts using real time microcomputer based tools. American Journal of Physics 58(9): 858-866.

Tierney, C., Nemirovsky, C. and Noble, T. (1995a). Investigations in Number, Data and Space: Patterns of Change. Palo Alto, CA: Dale Seymour Publications.

Tiemey, C., Nemirovsky, C. and Weinberg, A, (1995b). Investigations inNumber, Data and Space: Changes over Time. Palo Alto, CA: Dale Seymour Publications.

Trowbridge, D.E. and McDermott, L.C. (1980). Investigation of student understanding of the concept of velocity in one dimension. American Journal of Physics 48(12): 1020.

VideoGraph. (1994). Raleigh, NC: Robert Beichner, North Carolina State University. von Glasersfeld, E. (1984). An introduction to radical constructivism. In P. Watzlawick

(Ed.), The Invented Reality (pp. 17-40). New York: W. W. Norton.


Zollman, D. (1994). Digital Video Interactive: A Case Study in Physics. Lawrence, KA: Kansas State University.

TERC, 2067 Massachusetts Ave., Cambridge, MA 02140, USA

Documents

Interactive video: A bridge between motion and math