Programmable Motion Effects

Johannes SchmidDisney Research Zurich

ETH Zurich

Robert W. SumnerDisney Research Zurich

Huw BowlesDisney Research Zurich

Markus GrossDisney Research Zurich

ETH Zurich

Figure 1: The trails behind this spinning ball are generated by a programmable motion effect, a new rendering mechanism for depictingmotion in computer-generated animation. Customized styles can be achieved in a flexible manner, similar to the way programmable surfaceshaders are used to generate customized surface appearances.

Abstract

Although animation is one of the most compelling aspects of com-puter graphics, the possibilities for depicting the movement thatmake dynamic scenes so exciting remain limited for both still im-ages and animations. In our work, we experiment with motiondepiction as a first-class entity within the rendering process. Weextend the concept of a surface shader, which is evaluated on aninfinitesimal portion of an object’s surface at one instant in time,to that of a programmable motion effect, which is evaluated withglobal knowledge about all portions of an object’s surface that passin front of a pixel during an arbitrary long sequence of time. Withthis added information, our programmable motion effects can de-cide to color pixels long after (or long before) an object has passedin front of them. In order to compute the input required by themotion effects, we propose a 4D data structure that aggregates anobject’s movement into a single geometric representation by sam-pling an object’s position at different time instances and connectingcorresponding edges in two adjacent samples with a bilinear patch.We present example motion effects for various styles of speed lines,multiple stroboscopic images, temporal offsetting, and photorealis-tic and stylized blurring on both simple and production examples.

1 Introduction

A huge variety of animation techniques, ranging from keyframingto dynamic simulation to motion capture, populates the animationtoolbox and can be used to create dynamic and compelling worldsfull of action and life. During the rendering process, production-quality renderers employ motion blur as a form of temporal an-tialiasing. While motion blur effectively removes aliasing artifacts

such as strobing, it does so at the cost of image clarity. Fast-movingobjects may be blurred beyond recognition in order to properlyremove high-frequency signal content. Although correct from asignal-processing point of view, this blurring may conflict with theanimator’s concept of how motion should be treated based on his orher creative vision for the scene. Since the appreciation of motionis a perceptual issue, the animator may wish to stylize its depictionin order to stimulate the brain in a certain manner, in analogy to theway an impressionist painter may stylize a painting in order to elicitsome aesthetic response. To this end, comic book artists, whose en-tire medium of expression is based on summarizing action in stilldrawings, have pioneered a variety of techniques for depicting mo-tion. Similar methods are employed in 2D hand-drawn animationto emphasize, accent, and exaggerate the motion of fast-movingobjects, including speed lines, multiple stroboscopic copies, streak-ing, stretching, and stylized blurring. Although these effects haveplayed an important role in traditional illustration and animationfor the past century, computer-graphics animation cannot accom-modate them in a general and flexible manner because productionrenderers are hard-wired to deliver realistic motion blur. As such,stylized motion effects are relegated to one-off treatment and infre-quently used.

In our research, we experiment with motion effects as first-classentities within the rendering process. Rather than attempting to re-produce any particular style, we aim at creating a general-purposerendering mechanism that can accommodate a variety of visualiza-tion styles, analogous to the way surface shaders can implementdifferent surface appearances. Many effects from traditional medi-ums are directly related to an object’s movement through a regionof space and take the form of transient, ephemeral visuals left be-hind. This observation motivates a simple yet powerful change inthe rendering process. We extend the concept of a surface shader,which is evaluated on an infinitesimal portion of an object’s sur-face at one instant in time, to that of a programmable motion effect,which is evaluated with global knowledge about all portions of anobject’s surface that pass in front of a pixel during an arbitrary longsequence of time. With this added information, our programmablemotion effects can decide to color pixels long after (or long be-fore) an object has passed in front of them, enabling speed lines,stroboscopic copies, streaking, and stylized blurring (Figure 1). Byrendering different portions of an object at different times, our ef-fects also encompass stretching and bending. Other effects that ex-tend beyond the object’s position in space, such as clouds of dust

or smoke, flashes of color that fill the frame, and textual annota-tions (e.g., “BANG” or “POW”) are not addressed by our frame-work. Traditional motion blur is a special case within our system,implemented as a motion effect program that averages the relevantsurface contributions during a specified shutter time. In general,however, our method dissolves the classic notion of a scene-wideshutter time and allows each motion effect program to indepen-dently specify its operating time range, so that a single renderedframe may compose information from different periods of time.

From a technical standpoint, the most challenging aspect of our sys-tem is efficiently computing global information about an object’smovement. We make this computation in the context of a ray tracer,where a single ray cast is modified to find every part of an objectthat has moved past a given pixel throughout an arbitrary long timerange. Since the motion of an object may be complex, an analyticalsolution to the problem is not feasible. Instead, we construct a newgeometric object called a time aggregate object (TAO) that aggre-gates an object’s movement into a single geometric representation.This representation is augmented with additional information thatenables the reconstruction of a set of points representing the pathalong the surface of the object that is visible to the pixel as theobject moves through the scene, along with the associated times.Using this global information, one can develop shading algorithmsthat utilize information about an object’s movement through an ar-bitrarily large window of time.

Our primary contribution is an approach to motion depiction forthree-dimensional computer animation that fits naturally into cur-rent rendering paradigms and offers the same generality and flexi-bility as programmable surface shading. We also make the technicalcontribution of the TAO data structure and present several examplesof programmable motion effects.

2 Background

James Cutting [2002] presents a detailed treatise about motion de-piction in static images that examines parallels in art, science, andpopular culture. He identifies five categories that encompass thevast majority of motion depiction techniques used to-date: photo-graphic blur, speed lines, multiple stroboscopic images, shearing,and dynamic balance.

Of these categories, photographic blur has received by far the mostattention within computer graphics, as an answer to temporal alias-ing problems in computer animation. Since a rendered anima-tion represents a sampled view of continuous movement, disturbingaliasing artifacts such as strobing can occur if the temporal signalis sampled naively [Potmesil and Chakravarty 1983]. Consequen-tially, temporal antialiasing is a core component of all production-quality renderers, and the research community has developed manysophisticated algorithms for this purpose. These methods can beseen as convolution with a low-pass filter to remove high-frequencydetails, yielding a blurry image. Or, in analogy to traditional pho-tographic processes, they can be interpreted as opening a virtualshutter for a finite period of time during which fast-moving objectsdistribute their luminance over regions of the film, creating blurredmotion.

Sung, Pearce, and Wang [2002] present a taxonomy of motionblur approaches, with associated references, and reformulated thesepublished methods in a consistent mathematical framework. Twofeatures of this framework help distinguish existing motion-bluralgorithms from our presented work. First, existing methods canbe interpreted as scene-wide integration over a fixed shutter time.In our work, we abandon the notion of a fixed shutter time andempower individual motion effect programs to determine the timerange over which to operate. In this way, a single rendered imagemay combine information from a variety of different time ranges.

This added flexibility incurs a more complicated compositing situ-ation (Section 4.6) when multiple motion effects overlap in depth.Second, existing methods employ a spatio-temporal reconstructionfilter responsible for averaging contributions from different times.In our method, we generalize this filtering concept to an arbitraryprogram that allows a variety of different looks to be expressed.The averaging operation used in traditional motion blur becomes aspecial case within this framework.

The value of stylized motion depiction is evident from its treat-ment in traditional artistic mediums, including the “Futurism” artmovement of the 1900’s [Hulten 1986] and techniques taught astools-of-the-trade to comic book artists [McCloud 1993] and an-imators [Goldberg 2008; Whitaker and Halas 2002]. Perceptualstudies even provide direct evidence that speed lines influence low-level motion processing in the human visual system [Burr and Ross2002]. Due to the importance of stylized motion depiction, manyresearchers have explored ways to incorporate it into computer-generated imagery, animation, photographs, and video.

Masuch and colleagues [1999] describe the use of speed lines, stro-boscopic copies, and arrows in 3D graphics as a post-processingoperation, Lake and colleagues [2000] present a similar method forspeed line generation, and Haller and colleagues [2004] present asystem for generating these effects in computer games. These meth-ods are specialized for the targeted effects and may not generalizeeasily to different visual styles. A framework for generating vi-sual cues based on object motion is introduced by Nienhaus andDollner [2005]. This method allows one to define rules for the de-piction of certain events or sequences based on a scene graph rep-resentation of geometry and a behavior graph representation of an-imation. The authors do not address the issue of how to implementthe depictions in a generalized fashion. Researchers also presentalgorithms to add stylized motion cues to 2D animation [Hsu andLee 1994; Kawagishi et al. 2003] and video [Bennett and McMil-lan 2007; Collomosse et al. 2005], to filter motion in images [Liuet al. 2005] or 3D animation [Wang et al. 2006; Noble and Tang2007; Chenney et al. 2002] in order to create a magnified or car-toony effect, to create stylized storyboards from video [Goldmanet al. 2006], and to summarize the action in motion-capture data[Assa et al. 2005; Bouvier-Zappa et al. 2007] or sequences of pho-tographs [Agarwala et al. 2004].

Taken all together, these methods cover the five categories of mo-tion depiction techniques that Cutting [2002] proposes. However,existing algorithms target specific looks and must explicitly pa-rameterize stylistic deviations (e.g., Haller and colleagues’ sys-tem [2004] represents speed lines as connected linear segments witha parameter to control line thickness over time). The central thrustand distinguishing characteristic of our contribution is an open-ended system for authoring motion effects as part of the renderingprocess. We extend the concept of programmable surface shading[Hanrahan and Lawson 1990] to take the temporal domain into ac-count for pixel coloring. While our motion effect programs definehow this extra dimension should be treated for a certain effect, theycan still call upon conventional surface shaders for the computationof surface luminance at a given instant in time. We show examplesin four of the five categories proposed by Cutting (stylized blurring,speed lines, multiple stroboscopic images, and shearing). And, no-tably, within these categories, different styles can be achieved withthe same flexibility afforded by programmable surface shaders.

3 Method Principles

Many stylized motion effects from traditional mediums summarizean object’s movement over a continuous range of time with tran-sient, ephemeral visuals that are left behind. Motivated by this ob-servation, we propose an alternative rendering strategy that operates

on the scene configuration during an arbitrarily long time range T .In this section, we introduce the concept of motion effect programs,our time aggregate object data structure, and the renderer’s com-positing system. Section 4 discusses more specific implementationdetails.

3.1 Motion Effect Programs

In analogy to a state-of-the-art renderer that relies on surfaceshaders to determine the color contributions of visible objects toeach pixel, we delegate the computation of a moving object’s colorcontribution within the time range T to motion effect programs. Amotion effect program needs to know which portions of all sur-faces have been “seen” through a pixel during T . In general, thisarea is the intersection of the pyramid extending from the eye lo-cation through the pixel corners with the objects in the scene overtime. Although Catmull [1978] presents an analytic solution to thispixel coverage problem for static scenes, extending it to the spatio-temporal domain is non-trivial. As such, we follow the approach ofKorein and Badler [1983] and collapse the pyramid to a single linethrough the pixel center before computing analytic coverage. Thesurface area seen by the pixel for a particular object then becomesa line along the surface, which we call a trace. We address the spa-tial aliasing incurred by this simplification via supersampling in thespatial domain.

A motion effect program calculates a trace’s contribution to the finalpixel color. In doing so, it utilizes both positional information (thelocation of a trace on the object’s surface) and temporal information(the time a given position was seen) associated with the trace. Itcan evaluate the object’s surface shaders as needed and draw uponadditional scene information, such as the object’s mesh data (vertexpositions, normals, texture coordinates, etc.), auxiliary textures, thecamera view vector, and vertex velocity vectors.

3.2 Time Aggregate Objects

Computing a trace is a four-dimensional problem in space and time,where intersecting the 4D representation of a moving object withthe plane defined by the view ray yields the exact trace. Unfor-tunately, since the influence of the underlying animation mechan-ics on an object’s geometry can be arbitrarily complex, a closed-form analytic solution is infeasible. Monte Carlo sampling [Cooket al. 1984] could be considered as an alternative, since it is used byprominent production renderers [Sung et al. 2002] to produce high-quality motion blur. However, it is also not effective in the presentscenario since the time period associated with a trace may be veryshort in comparison to the time range over which the motion effectis active. For example, a ball may shoot past a pixel in a fraction ofa second but leave a trailing effect that persists for several seconds.A huge number of samples distributed in time would be required inorder to effectively sample the short moment during which the ballpasses.

Consequentially, we propose a new geometric data structure thatallows our system to reconstruct a linear approximation of the fulltrace from a single ray cast. Our data structure is inspired by the 4Dpolyhedra used in Grant’s temporal anti-aliasing method [1985] andaggregates an object’s geometry sampled at a set of times ti (Fig-ure 2) into a single geometric primitive. In addition, correspondingedges of adjacent samples are connected by a bilinear patch, whichis the surface ruled by the edge as its vertices are interpolated lin-early between ti and ti+1. We call the union of the sampled objectgeometry and swept edges a time aggregate object (TAO).

The intersection of a view ray with a bilinear patch of the TAO rep-resents a time and location where the ray, and thus also a trace, hascrossed an edge of the mesh. By computing all such intersections

(not just the closest one) and connecting the associated edge cross-ings with line segments, we obtain a linear approximation of thetrace. Intersections with the sampled geometry represent additionaltime and space coordinates, which improve the accuracy of the traceapproximation. The accuracy of the approximated traces thus de-pends on the number of TAO intersections, which scales with boththe geometric complexity of the object’s geometry and the numberof samples used for aggregation. In practice, we can use finely sam-pled TAOs without a prohibitive computation time, since the com-plexity of ray-intersection tests scales sub-linearly with the numberof primitives when appropriate spatial acceleration structures areused [Foris et al. 1996].

3.3 Compositing

A motion effect program acts on a trace as a whole, which mayspan a range of depths and times. If objects and contributions fromtheir associated programmable motion effects are well separated indepth, our renderer can composite each effect’s total contributionto the image independently according to depth ordering. The com-positing algorithm used can be defined by the effect itself, draw-ing upon standard techniques described by Porter and Duff [1984].Compositing becomes more complex when multiple traces overlap,since there may be no unequivocal ordering in depth or time. Addi-tionally, different motion effect programs may operate over differ-ent, but overlapping, time domains since a single scene-wide shuttertime is not enforced.

We resolve this ambiguity by introducing additional structure to theway in which motion effects operate. All traces are resampled at afixed scene-wide resolution. Each motion effect program processesits traces’ samples individually, and outputs a color and coveragevalue if that sample should contribute luminance to the renderedpixel. Our compositing system processes the output samples infront-to-back order, accumulating pixel color according to the cov-erage values until 100% coverage is reached.

4 Implementation

We implement our method as a plug-in to Autodesk Maya 2010[Aut 2010]. Our system is divided into two parts: a module to cre-ate and encapsulate TAOs from animated 3D objects, and a newrendering engine, which generates images with motion effects. Therendering engine computes a color for each pixel independently bycomputing all intersections of the pixel’s view ray with the TAOs,and connecting them to form a set of traces. Then it calls the mo-tion effect programs for each object, which compute that object’sand effect’s contribution to the pixel color using the object’s traces.Finally, a compositing step computes the resulting pixel color.

4.1 TAO Creation

We implement the TAO concept as a custom data structure withinMaya. We assume that animated objects are represented by trianglemeshes with static topology. Our system builds a TAO by samplingan object’s per-vertex time dependent data (e.g., vertex positions,normals, and texture coordinates) at a set of times ti, aggregat-ing those samples into a single geometric primitive, and connectingadjacent edges with bilinear patches (Figure 2). For a mesh withNE edges and NF triangles, Nt object samples yield a TAO withNt ·NF triangles and (Nt − 1) ·NE bilinear patches.

The density and placement of the object sample times ti deter-mine how well the motion of an object is approximated by ourTAO data structure. Since the approximation is linear by nature,it perfectly captures linear motion. For rotation and non-rigid de-formation, however, proper sampling of the motion is necessary

(a) (b) (c) (d) (e)

t1

t2

t3

t4

Figure 2: The time aggregate object (TAO) data structure encodes the motion of an object (a) using copies of the object sampled at differenttimes t1 . . . t4 (b) and bilinear patches that connect corresponding edges in adjacent samples edges. These patches are shown in (c) for oneedge and in (d) for the whole mesh. This data structure is not just the convex hull of the moving object, but has a complex inner structure, asseen in the cutaway image (e).

Figure 3: Undersampled (left) versus properly sampled (right) mo-tion. The trace sampling rate is 10 times larger in the right image.

(Figure 3). Our systems supports both a uniform sampling and anadaptive sampling strategy that starts with a uniform temporal sam-pling and repeatedly inserts or deletes sample positions based on themaximum non-linearity αi between the vertex positions of adjacentsamples:

αi = maxjangle(vj(ti−1),vj(ti),vj(ti+1)), (1)

where angle(A,B,C) is the angle between the line segments ABand BC and vj(ti) is the position of vertex j in object samplei. First, the adaptive sampling strategy iteratively removes sampletimes whenever αi is smaller than a given coarsening threshold. Ina second step, it insert two new samples at ti−ti−1

2and ti+1−ti

2if

αi exceeds a refinement threshold. In order to keep the maximumnumber of object samples under control, the refinement criterion isapplied iteratively on the sample with the largest αi until the max-imum number of samples is reached. In practice, we assume a cer-tain amount of coherence in the motion of nearby vertices, and oursystem approximates the optimal sample placement by consideringonly a subset of the mesh vertices in Equation 1.

4.2 TAO Intersection

Our renderer generates a view ray for each pixel according to thecamera transform. Each ray is intersected with the primitives ofthe TAO (mesh faces and bilinear patches) to compute all intersec-tions along the ray, not just the one which is closest to the camera.Normals, texture coordinates, and other surface properties are in-terpolated linearly over the primitives to the intersection point andstored.

For the intersection with bilinear patches, we use the algorithm de-scribed by Ramsey and colleagues [2004]. If an edge connects vj

and vk, then for each 0 < i < Ni − 1 a bilinear patch is defined as

pijk(r, s) = (1− r)(1− s) · vj(ti) + (1− r) · s · vk(ti)

+ r · (1− s) · vj(ti+1) + r · s · vk(ti+1).(2)

Solving for a ray intersection yields a set of patch parameters (r, s)that corresponds to the intersection point. With parameter r, we cancompute the time at which the ray has crossed the edge:

t = ti + r · (ti+1 − ti). (3)

The parameter s represents the position along the edge at which thecrossing has happened. It can be used to interpolate surface proper-ties stored at the vertices to the intersection point, and to computethe position of the intersection point in a reference configuration ofthe object. Mesh face primitives in the TAO are intersected accord-ing to standard intersection algorithms. A parametric representationof the intersection points with respect to the primitives is necessaryfor interpolating the surface properties.

To accelerate the intersection computation, we partition the im-age plane into tiles and create a list of all primitives for which thebounding box intersects a given tile. Each view ray only needs tobe intersected with the primitive list of the tile containing the ray.

4.3 Trace Generation

The set of all intersection points of a ray with the TAO can be usedto reconstruct the locus of points traced by the ray on the mov-ing object. Consider the interpretation of the ray-TAO intersectionpoints on the input mesh. As an individual triangle passes fully bythe ray, the ray crosses the edges of the triangle an even number oftimes, assuming that the ray was not intersecting at the beginning orat the end of the observed time. Each edge crossing corresponds toan intersection of the ray with a bilinear patch of the TAO, and viceversa. Since our system is restricted to a piecewise linear approxi-mation of motion, we assume that the trace forms a linear segmentin between two edge crossings. The task of constructing the traceis thus equivalent to connecting the intersection points in the properorder.

To facilitate this discussion, we refer to the volume covered by a tri-angle swept between two object samples within the TAO as a TimeVolume Element (TVE). This volume is delimited by two corre-sponding triangles that belong to adjacent object samples and thethree bilinear patches formed by the triangle’s swept edges (Fig-ure 4). Since there is an unambiguous notion of inside and outside,a ray originating outside of a TVE will always intersect the TVEan even number of times, unless it exactly hits the TVE’s boundary.A pair of consecutive entry and exit points corresponds to a seg-ment of the trace on the triangle. Therefore, by sorting all ray-TVEintersection points according to their distance from the viewer andpairing them sequentially, we can construct all trace segments that

A

A

B

B

C

C

D

D

t1

t2

t3

Figure 4: This figure shows two triangles at 3 consecutive objectsamples, t1, t2 and t3, which results in four Time Volume Elements(TVEs). A ray intersects the TVEs 4 times. At intersection A, the rayenters the blue triangle though an edge. Point B is an intersectionwith a sampled mesh triangle, indicating that the ray moves fromone time sample to the next. In C, the ray leaves the blue and entersthe green triangle, which it finally exits at D. The reconstructedtrace visualized on the two triangles is shown in the inset figure.

cross the corresponding triangle within the time range spanned bythat individual TVE.

Next, we consider trace connectivity. TVEs that share a commonTAO primitive (triangle or bilinear patch) are adjacent in space. Ifthe shared primitive is a triangle, the TVEs were formed by thesame triangle of the input mesh in adjacent object samples. If it isa bilinear patch, the triangles that formed the TVEs are adjacent toone another on the input mesh, separated by the edge that formedthe patch. This combined adjacency information determines theconnectivity of trace segments associated with neighboring TVEs.Our system processes TVEs in turn to reconstruct the individualtrace segments, and then uses this connectivity information to con-nect them together into different connected components.

4.4 Trace Resampling

At this point, we have the traces that the current pixel’s view rayleaves on moving objects in the form of connected sequences ofintersections with the TAO. Each intersection point corresponds toan animation time and a position on an object, and, in conjunctionwith the trace segments that connect to it, we can determine and in-terpolate surface properties such as attached shaders, normals, andtexture coordinates.

In order to facilitate compositing, our system imposes a consistenttemporal sampling on all traces by dividing them into individualtrace fragments of fixed spacing in time. This resampling shouldbe dense enough so that depth conflicts among different traces canbe resolved adequately. The visibility of individual trace fragmentsduring each sampling interval is also determined at this stage. Oursystem does not delete occluded or back facing trace fragments,however. Instead, it is left up to the motion effect program to decidewhether obstructed segments should contribute to pixel coverage.

4.5 Motion Effect Programming

A motion effect program operates on the resampled representationof a trace. It is called once per object and processes all trace frag-ments associated with that object. The motion effect program hastwo options when processing an individual trace fragment. It cansimply discard the fragment, in which case no contribution to thefinal pixel color will be made. Or, it can assign a color, depth, andpixel coverage value and output the fragment for compositing. Thecoverage value determines the amount of influence a fragment hason the final pixel color in the compositing step. When making thisdecision, the motion effect program can query an object’s surfaceshader, evaluate auxiliary textures (e.g., noise textures, painted tex-ture maps, etc.), or use interpolated object information.

The effect program can also subdivide the trace even further if theeffect requires a denser sampling of the object’s surface. For ex-ample, further subdivision could be the needed if the effect mustintegrate a surface texture of high frequency. Decreasing the traceresampling distance (Section 4.4) has a similar effect, although itaffects all motion effects in the scene.

4.6 Compositing

By emitting trace fragments, each motion effect program specifiesits desired contribution to the pixel color. The compositing enginecombines all of the emitted fragments to determine the final pixelcolor by processing fragments in depth order (front to back) usinga clamped additive model based on the coverage values. The cov-erage values of the fragments, which range between 0 and 1, areaccumulated until a full coverage of 1 is reached or until all frag-ments have been processed. If the coverage value exceeds 1, thelast processed fragment’s coverage is adjusted so that the limit isreached exactly. The final pixel color is computed by summing theprocessed fragment colors weighted by the corresponding coveragevalues. The accumulated coverage value is used as the alpha valuefor the final pixel color.

5 Results

In this section, we show a number of results obtained with our pro-grammable motion effect renderer and describe how the effect pro-grams were set up to achieve these results.

5.1 Motion Effects

To better explain the mechanism of programmable motion effects,we supply pseudo-code algorithms for five basic effects used inour examples. These algorithms show the framework of each ef-fect using a common notation. A trace segment ts consists of twoend points, denoted ts.left and ts.right. Each of the end points car-ries information about the animation time at which this surface lo-cation has been intersected by the view ray and linearly interpo-lated surface properties. They may be interpolated further with theINTERPOLATE function, which takes a trace segment and a time atwhich it should be interpolated as parameters. By ts.center, we des-ignate the interpolation of the end points’ properties to the center ofthe segment. The SHADE function evaluates the object’s surfaceshader with the given surface parameters and returns its color. Theanimation time of the current frame being rendered is denoted as“current time.” The output of a motion effect program is a numberof fragments, each with a color and coverage value, that representthe effect’s contribution to the pixel color. Passing a fragment to thecompositing engine is designated by the keyword emit.

Instant Render The most basic program renders an object at asingle instant in time. The program loops through all trace seg-

Test Scene Maya (10 sec) Mental Ray (10 sec) Ours (10 sec) Ours (44 sec)

Figure 5: A comparison of our motion blur implementation with production renderers. The test scene, rendered without blur, consists of atranslating checkerboard square with a stationary spotlight. The three middle images were created with equal render time. The rightmostimage shows a high-quality render obtained with our system with a denser trace sampling.

ments and checks whether the desired time lies within a segment.If so, it interpolates the surface parameters to that point, evaluatesthe surface shader accordingly, and emits just one fragment to thecompositing stage with a coverage value of 1. Any surfaces behindthat fragment will be hidden, as is expected from an opaque surface.

Algorithm 1 INSTANT RENDER(time)

for all trace segments ts doif ts.left.time <= time < ts.right.time then

eval at := INTERPOLATE(ts, time)color := SHADE(eval at)coverage := 1emit fragment(color, coverage)

Weighted Motion Blur Photorealistic motion blur integrates thesurface luminance within a given shutter time. We extend this pro-cess with an arbitrary weighting function w(t) that can be usedboth for realistic and stylized blurring. For photorealistic blur, theweighting function is the temporal sampling reconstruction filterfunction. A flat curve corresponds to the commonly used box filter.In general, the weighting function need not be normalized, whichmeans that luminance energy is not necessarily preserved. Thisflexibility increases the possibilities for artistic stylization. For ex-ample, a flat curve with a spike at t = 0 results in motion blur thathas a clearly defined image of the object at the current frame time.

Algorithm 2 MOTION BLUR(w(t))

for all trace segments ts doif ts time interval overlaps with {t | w(t) > 0} then

clip ts against {t | w(t) > 0}color := SHADE(ts.center)δt := ts.right.time - ts.left.timecoverage := δt · w(ts.center.time)emit fragment(color, coverage)

The effect program in Algorithm 2 describes weighted motion blurin its most basic form. A more advanced implementation couldtake more than one surface shading sample within a segment anduse a higher order method for integrating the luminance. With thegiven effect program, however, the fidelity of the image can be im-proved by increasing the global trace sampling rate (Section 4.4). Acomparison of motion blur from our implementation with that fromproduction renderers is shown in Figure 5.

Speed Lines Speed lines are produced by seed points on an ob-ject that leave streaks in the space through which they travel. They

can be distributed automatically or placed manually. The programcomputes the shortest distance between trace segments and seedpoints. If the distance is smaller than a threshold, the seed point haspassed under or close to the pixel, and the pixel should be shadedaccordingly. A falloff function based on distance or time can beused to give the speed line a soft edge.

Algorithm 3 SPEED LINES(seed vertices, width, length)

for all trace segments ts dofor all seed vertices v do

if DISTANCE(ts, v) < widthand current time − ts.center.time < length then

color := SHADE(ts.center)compute coverage using a falloff functionemit fragment(color, coverage)

Stroboscopic Images This effect places multiple instant rendersof the object at previous locations. It imitates the appearance of amoving object photographed with stroboscopic light. The intensityof the stroboscopic images is attenuated over time to make themappear as if they are washing away. We keep the locations of thestroboscopic images fixed throughout the animation, but they couldalso be made to move along with the object. The falloff function canbe composed of factors considering the time of the stroboscopic im-age, geometric properties of the mesh (e.g., the angle between themotion vector and the normal at a given point), or an auxiliary mod-ulation texture to shape the appearance of the stroboscopic images.

Algorithm 4 STROBOSCOPIC IMAGES(spacing, length)

for all trace segments ts doif ts.left.time < current time − length then

t1 mod := ts.left.time modulo spacingt2 mod := ts.right.time modulo spacingif t1 mod <= 0 < t2 mod then

color := SHADE(ts.center)compute coverage using a falloff functionemit fragment(color, coverage)

Algorithm 5 TIME SHIFT(ts, shift magnitude)

time shift := |ts.center.vmotion| · shift magnitudeshift time values in trace segment ts according to time shiftreturn ts

Time Shifting The time shifting program modulates the instanta-neous time selected from the trace. We use it in conjunction with

Figure 6: Different speed-line styles.

the INSTANT RENDER so that each pixel in the image may repre-sent a different moment in time. If fast-moving parts of an objectare shifted back in time proportional to the magnitude of their mo-tion, these parts appear to be lagging behind. They “catch up” withthe rest of the object when it comes to a stop or changes direction.This effect is only visible if an object’s motion is not uniform acrossthe surface, as in the case of rotation. In this algorithm, vmotion des-ignates the motion vector of the corresponding part of the surface.

5.2 Examples

Translating and Spinning Ball The results in Figure 6 demon-strate speed lines in different visual styles. The first two images usethe SPEED LINES effect (Algorithm 3) with a falloff function thatfades in linearly at both ends of the speed lines and modulates thewidth of the speed line over time. For the last image, the distance ofthe seed point to the current pixel is used to manipulate the normalpassed to the surface shader, giving a tubular appearance.

Bouncing Ball The example in Figure 7 shows a bouncing toyball, rendered with a modified version of the MOTION BLUR effect(Algorithm 2). The effect program computes a reference time forthe input trace, and uses the difference between this reference timeand the current time to determine the amount of blur, or the shutteropening time in conventional terms. As a result, the blur increasestoward the end of the trail.

Spinning Rod This result shows a rod spinning about an axisnear its lower end. Figure 8 (a) combines a MOTION BLUR effect(Algorithm 2) that uses a slightly ramped weighting function withan INSTANT RENDER effect (Algorithm 1) to render a crisp copyof the rod. Figure 8 (b) additionally uses the TIME SHIFT func-tion (Algorithm 5) with a negative shift magnitude so that quicklymoving surface parts lag behind. As shown in the accompany-ing video, these parts “catch up” when the rod stops moving orchanges direction. In Figure 8 (c), MOTION BLUR is replaced witha STROBOSCOPIC IMAGES effect (Algorithm 4). The falloff func-tion fades the stroboscopic images out as time passes and addition-ally uses the angle between the motion vector and the surface nor-mal to make the rod fade toward its trailing edge.

Toy UFO In Figure 9, we have attached speed lines to a UFOmodel to accentuate and increase the sensation of its speed. To keepthe effect convincing when the camera is moving with the UFO,the length and opacity of the speedlines are animated over time.An animated noise texture is sampled using the texture coordinatesfrom the seed point of each speed line, which gives the attenuationvalue for that speed line.

Figure 7: A bouncing ball rendered with non-uniform blur.

(a) (b)

(c)

Figure 8: A spinning rod showing (a) weighted motion blur, (b)weighted motion blur and time shift, and (c) stroboscopic imageswith time shift.

Pinocchio’s Peckish Pest The last set of examples show our re-sults on a production-quality scene. Figure 10 uses a combinationof speed lines and weighted motion blur. The motion blur’s weight-ing curve has a spike around the current time, so that Pinocchiois shown clearly in every frame, even when he is moving quickly.In Figure 11 a comparison between conventional motion blur (alsorendered by our system) and our weighted motion blur is made.

Statistics about all examples are shown in Table 1. The values aretaken from a representative frame for each animation, where thefull TAO is within the visible screen area. Render time dependsgreatly on the motion and the amount of screen space covered bythe objects. The numbers shown resulted from rendering an imagewith 1280x720 pixels on an 8-core 2.8 GHz machine.

6 Conclusion

In this paper, we have presented a novel approach to depict mo-tion in computer-generated animation. Our method fits naturallyinto current rendering paradigms and offers the same generality andflexibility as programmable surface shading. Our results demon-strate that it is a powerful platform for experimenting with differentdepiction styles.

Limitations of our work motivate a number of rich future researchpossibilities. In our current implementation, the screen-space re-

Figure 9: Speed lines applied to a flying UFO animation.

Figure 10: Results from applying speed line and motion blur effects(bottom), and the stroboscopic image effect (top) to an animationof Pinocchio and a woodpecker.

Figure 11: A comparison of conventional motion blur (left) withour weighted motion blur including speed lines (right).

gion of the motion effect is limited to the convex hull of the object’smotion, since the motion effect programs operate only on pixels thatintersect the TAO. One future research avenue would be to considerinstalling the motion depiction framework further upstream in therendering process as a geometry shader, allowing new geometry tobe created and expanding the range of possible depiction styles.

We experiment with the core idea of programmable motion effectsand provide a proof-of-concept that they can be used to expressa variety of motion depiction styles. However, our renderer im-

Example # Tri TAO Trace Mem TimeSpinning Ball 1200 160 0.1 12 2.6Rod 80 100 0.1 0.5 0.25UFO (shot 2) 3400 20 0.5 4.5 2Pinocchio (shot 2) 37k 60 0.005 130 4.3Pinocchio (shot 3) 122k 60 0.005 300 65

Table 1: Example statistics for a representative frame. # Tris:number of source mesh triangles; TAO: number of object samples(Section 4.1); Trace: trace sampling distance (Section 4.4); Mem:memory required to store the TAO in MiB; Time: render time inminutes.

plements only the most basic functionality and does not considerglobal illumination, reflection, refraction, caustics, participatingmedia, or other effects that are standard in production renderers.One immediate avenue of future work is applying the core princi-ple of recursive ray tracing to our framework by casting secondaryrays for shadowing, reflection, and refraction. Naturally, computa-tion time may become an issue if many secondary rays are used.

The performance of our system is reasonable for the examplesshown, with most frames requiring only a few minutes to render.The slowest aspect of the system lies in the way it interfaces withMaya. Creating the TAO structure requires sampling the animationsystem at tens or hundreds of sample positions to cache time vary-ing mesh data. Additionally, many of our motion effect programsevaluate an object’s surface shader or other auxiliary textures. Ifthe parameters of these shaders and textures are themselves ani-mated, then each and every evaluation must call back to the ani-mation software in order to work in the proper temporal context.Due to Maya’s system design and assumptions about the distinctseparation of animation and rendering, such out-of-context shadernetwork evaluations are prohibitively expensive. This observationspeaks to a future animation and rendering design that does notdraw a hard line between the two, but rather couples both as tightlyas possible.

Our current TAO data structure requires the mesh connectivity to beconstant throughout the animation. This limitation prohibits the useof our system in scenarios where the connectivity changes, such asin the presence of level of detail or certain physical simulation tech-niques. A related issue comes with the requirement for the motionto be sampled consistently within one TAO, even if some parts ofthe input object move with a different complexity than other parts.Sampling each primitive’s motion at its optimal rate would improveperformance and flexibility.

The implementation presented in this paper uses a ray tracing ap-proach for rendering, but we believe that the general concept canbe adapted to other rendering paradigms. It would be interesting toinvestigate how programmable motion effects can be implementedon GPUs and in the Reyes architecture [Cook et al. 1987]. An al-

ternative approach would be to replace the TAO data structure withan image sequence that stores additional per pixel data such as facecorrespondence and surface parameters. Motion effect programscould combine a number of these images to create one frame. Thisapproach can be seen as an extension of deferred shading for mo-tion depiction.

Acknowledgements

We would like to thank Martin Senn, Dawn Rivera-Ernester, AndyHendrickson, and the WDAS summer interns for their help withthe Pinocchio animation. We are also grateful to David Spuhlerand Matthias Buhlmann for their work on the project, and to MarkPauly, Robert Cook, Wojciech Jarosz, Jovan Popovic, Fredo Du-rand, and William Freeman for discussions about the project. Fig-ures 3, 7, 10, 11 c© Disney Enterprises.

References

AGARWALA, A., DONTCHEVA, M., AGRAWALA, M., DRUCKER,S., COLBURN, A., CURLESS, B., SALESIN, D., AND COHEN,M. 2004. Interactive digital photomontage. ACM Trans. Graph.23, 3.

ASSA, J., CASPI, Y., AND COHEN-OR, D. 2005. Action synopsis:pose selection and illustration. ACM Trans. Graph. 24, 3.

AUTODESK. 2010. Autodesk Maya 2010.http://www.autodesk.com/maya.

BENNETT, E. P., AND MCMILLAN, L. 2007. Computational time-lapse video. ACM Trans. Graph. 26, 3.

BOUVIER-ZAPPA, S., OSTROMOUKHOV, V., AND POULIN, P.2007. Motion cues for illustration of skeletal motion capturedata. In Non-Photorealistic Animation and Rendering 2007.

BURR, D. C., AND ROSS, J. 2002. Direct evidence that “speed-lines” influence motion mechanisms. Journal of Neuroscience22, 19.

CATMULL, E. 1978. A hidden-surface algorithm with anti-aliasing.SIGGRAPH Comput. Graph. 12, 3.

CHENNEY, S., PINGEL, M., IVERSON, R., AND SZYMANSKI,M. 2002. Simulating cartoon style animation. In NPAR’02: Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering.

COLLOMOSSE, J. P., ROWNTREE, D., AND HALL, P. M. 2005.Rendering cartoon-style motion cues in post-production video.Graph. Models 67, 6.

COOK, R. L., PORTER, T., AND CARPENTER, L. 1984. Dis-tributed ray tracing. SIGGRAPH Comput. Graph. 18, 3.

COOK, R. L., CARPENTER, L., AND CATMULL, E. 1987. Thereyes image rendering architecture. SIGGRAPH Comput. Graph.21, 4.

CUTTING, J. E. 2002. Representing motion in a static image: con-straints and parallels in art, science, and popular culture. Percep-tion 31, 10.

FORIS, T., MARTON, G., AND SZIRMAY-KALOS, L. 1996. Rayshooting in logarithmic time. In WSCG’96 - 4-th InternationalConference in Central Europe on Computer Graphics and Visu-alization’96.

GOLDBERG, E. 2008. Character Animation Crash Course!Silman-James Press.

GOLDMAN, D. B., CURLESS, B., SALESIN, D., AND SEITZ,

S. M. 2006. Schematic storyboarding for video visualizationand editing. ACM Trans. Graph. 25, 3.

GRANT, C. W. 1985. Integrated analytic spatial and temporal anti-aliasing for polyhedra in 4-space. SIGGRAPH Comput. Graph.19, 3.

HALLER, M., HANL, C., AND DIEPHUIS, J. 2004. Non-photorealistic rendering techniques for motion in computergames. Comput. Entertain. 2, 4.

HANRAHAN, P., AND LAWSON, J. 1990. A language for shadingand lighting calculations. SIGGRAPH Comput. Graph. 24, 4.

HSU, S. C., AND LEE, I. H. H. 1994. Drawing and animationusing skeletal strokes. Proceedings of SIGGRAPH 94.

HULTEN, P., Ed. 1986. Futurism & Futurisms. Abbeville Press.

KAWAGISHI, Y., HATSUYAMA, K., AND KONDO, K. 2003. Car-toon blur: Non-photorealistic motion blur. Computer GraphicsInternational Conference.

KOREIN, J., AND BADLER, N. 1983. Temporal anti-aliasing incomputer generated animation. SIGGRAPH Comput. Graph. 17,3.

LAKE, A., MARSHALL, C., HARRIS, M., AND BLACKSTEIN, M.2000. Stylized rendering techniques for scalable real-time 3danimation. In NPAR ’00: Proceedings of the 1st internationalsymposium on Non-photorealistic animation and rendering.

LIU, C., TORRALBA, A., FREEMAN, W. T., DURAND, F., ANDADELSON, E. H. 2005. Motion magnification. ACM Trans.Graph. 24, 3.

MASUCH, M., SCHLECHTWEG, S., AND SCHULZ, R. 1999.Speedlines: depicting motion in motionless pictures. In ACMSIGGRAPH 99 Conference abstracts and applications.

MCCLOUD, S. 1993. Understanding Comics. Kitchen Sink Press.

NIENHAUS, M., AND DOLLNER, J. 2005. Depicting dynamicsusing principles of visual art and narrations. IEEE ComputerGraphics and Applications 25, 3.

NOBLE, P., AND TANG, W. 2007. Automatic expressive deforma-tions for implying and stylizing motion. The Visual Computer23, 7.

PORTER, T., AND DUFF, T. 1984. Compositing digital images.SIGGRAPH Comput. Graph. 18, 3.

POTMESIL, M., AND CHAKRAVARTY, I. 1983. Modeling mo-tion blur in computer-generated images. SIGGRAPH Comput.Graph. 17, 3.

RAMSEY, S. D., POTTER, K., AND HANSEN, C. 2004. Ray bi-linear patch intersections. Journal of Graphics, GPU, and GameTools 9, 3.

SUNG, K., PEARCE, A., AND WANG, C. 2002. Spatial-temporalantialiasing. IEEE Transactions on Visualization and ComputerGraphics 8, 2.

WANG, J., DRUCKER, S. M., AGRAWALA, M., AND COHEN,M. F. 2006. The cartoon animation filter. ACM Trans. Graph.25, 3.

WHITAKER, H., AND HALAS, J. 2002. Timing for Animation.Focal Press.