Projections Volume 4 Issue 2 Winter 2010 19ndash40 copy Berghahn Journalsdoi 103167proj2010040203 ISSN 1934-9688 (Print) ISSN 1934-9696 (Online)

Through the Looking GlassPhilosophical Toys andDigital Visual EffectsStephen Prince

Abstract Digital visual effects bridge art and science in ways that have ex-panded the expressive tools available to filmmakers Digital imaging also hasenlarged a domain for realism in cinema based on indexical and perceptualfactors Examining these factors the article questions the visual skepticismthat often surrounds discussion of visual effects in film studies A conjunctionof art and science has characterized cinema throughout its history especiallyin the era of ldquophilosophical toysrdquo from which the medium originated The ar-ticle examines that era in light of what it suggests about digital imaging to-day and the aesthetic forms that it enables

Keywords digital effects indexical image perceptual realism philosophicaltoy special effects

Digital visual effects come to us by way of the phenakistoscope Nothing everhappens for the first time in film history and we can learn about contempo-rary modes of image making by taking a detour into the past to the begin-nings of cinema in an era when science and entertainment wereconnected by a well-traveled bridge The bridge between art and sci-ence that gave birth to movies is relevant to our understanding ofhow digital imaging tools function in cinema today Art and sciencecoexist in a domain where fantastic worlds are built with a physicallyaccurate rendition of the behavior of solids liquids gases light movementand sound This area of contemporary culture is the domain of digital visualeffects in cinema

To enhance the credibility of their visual effects filmmakers collaboratewith scientists utilizing software programs that simulate environments andobjects whose behavior is rendered according to known laws of physics theproperties of light and of fluid and particle dynamics The rendering of fluiddynamics for example has come a long way on screen in a short time Digital

Digital visual effects come to us by way of the phenakistoscope

simulation of water proved very difficult for effects artists to master The gi-ant tidal wave at the end of The Abyss (1989) is unconvincing in part becauseit lacks texturingmdashfroth spray bubblesmdashand is insufficiently translucent Itwas a first digital go at doing water In 2006 for Poseidon Industrial Light andMagic (ILM) collaborated with Stanford University to incorporate current re-search on fluid dynamics Stanfordrsquos researchers worked on a phenomenonthey called ldquovorticityrdquo which is the swirling effect seen as water crashesaround solid objects and ILM built this information into the film helping tomake the rendering of water far more convincing than in years previous

Although visual effects may draw on science they are works of aestheticimagination and moviemakers often cheat reality for dramatic purposes Or-ange firelight would have made the blue-skinned Navi in Avatar look gray anundesirable outcome so the filmmakers decided to implement a policy theycalled ldquospectral compensationrdquo in effect ignoring the interaction of orangeand blue light in this context (Duncan 2010 130 Figure 1)

Perceptual RealismVisual effects manifest a dialectic of art and science and as such they negoti-ate terrain for a realist aesthetic in digital cinema Claiming a basis for realismin digital visual effects may seem a counter-intuitive move but several no-table film scholars have indeed made such a move Warren Buckland (1999185) for example found the dynamic staging in depth that digital effects en-abled Spielberg to achieve in Jurassic Park to be compatible with Bazinian no-tions of deep focus Tom Gunning (2006b 347) has argued forcefully that weneed to move beyond the familiar dichotomies of theory ldquoI believe we distortour experience of films if we try to assign the effect of realismmdashor even thesensation of physical presencemdashexclusively to the photographic or confinethe artificial to lsquospecial effectsrsquordquo Indeed there are very good reasons for insist-ing on a critical perspective that is amenable to integrating computer graph-ics capabilities with aesthetic properties of realism in cinema In an earlieressay (Prince 1996) I identified a digital basis for realism in cinema in terms ofwhat I called ldquoperceptual realismrdquo which was the replication via digital meansof contextual cues designating a three-dimensional world These cues includeinformation sources about the size and positioning of objects in space theirtexturing and apparent density of detail the behavior of light as it interactswith the physical world principles of motion and anatomy and the physics in-volved in dynamic systems such as water clouds and fire Digital tools givefilmmakers an unprecedented ability to replicate and emphasize these cuesas a means for anchoring the scene in a perceptual reality that the viewerfinds credible because it follows the same observable laws of physics as theworld they inhabit The referential status of the representation does not mat-ter in this conception of realism

2 0 P R O J E C T I O N S

Dinosaurs are not living beings in the age of cinema They cannot be pho-tographed as sentient creatures Thus their logical status in Jurassic Park is asobjects that are referentially false They correspond to no reality that the filmrsquosviewer could inhabit Yet as depicted in the film they are perceptually realis-tic They interact in relatively convincing ways with the live actors in dynami-cally staged scenes that bond the domains of live action and digitalanimation And because they are perceptually realistic they are able to com-pel belief in the fictional world of the film in ways that traditional special ef-fects could not accomplish The creation of perceptual realism is a major goalof special effects artists Visual effects seek to persuade viewers that effectsare real within the referential terms of the story Therefore the more compre-hensive is a scene in evoking perceptual realism the likelier it is to compel thespectatorrsquos belief No one watching Jurassic Park was fooled into thinking thatdinosaurs were actually alive but because digital tools established perceptualrealism with new levels of sensory detail viewers could be sensually per-suaded to believe in the fiction and to participate in the pleasures it offeredHad the film employed established and traditional effects tools this sensorypersuasion would have been far less remarkable

That is because traditional effects tools have been quite limited in theirability to create perceptual realism The compositing of live action mattepaintings and miniatures in The Lost World (1925) King Kong (1933) The Valleyof Gwangi (1969) and other comparable creature movies was visibly falsecompromised by overt matte lines between the elements and by the planarrendition of space that prevented the matted creature from interacting with

T H R O U G H T H E L O O K I N G G L A S S 2 1

Figure 1 ThroughoutAvatar the Nrsquoaviretain the blue oftheir skin in firelightan impossible colorcombination but onepermitted by thefilmmakers fordramatic purposes

the live actors The composited elements exhibit a perceptual disparitymdashlim-ited interaction between the domains contradictory manifestations of mo-tion blurmdashworking against the emergence of an organic unity of action Underthese terms the dramatic space of the screen action becomes perceptuallysuspect Earlier generations of special effects images manifested similar prob-lems and elicited degrees of skepticism in viewers Digital effects have helpedcinematic imagery overcome problems triggering a reality check by viewersthat undermined the fictional enterprise

Orthodox Assumptions about Visual EffectsPerceptual realism then is central to understanding special effects in cinemathe goal of effects artists and the credibility that the effects image seeks toelicit among viewers Moreover it has important implications for two ortho-dox assumptions in film studies 1) that visual effects equate with spectacleand that spectacle is anti-narrative and 2) that visionmdashthe act of seeingmdashisprimarily a culturally coded activity and therefore relative to social formationsin a given period About the first appearance of the digital dinosaurs in Juras-sic Park Michelle Pierson (2002 120) writes that the filmrsquos narrative stopsdead so that the digital effects can be showcased at length ldquoThe narrative allbut comes to a halt the music gradually builds and shots of characters react-ing to the appearance of the dinosaur with wonder and amazement are inter-spersed with long takes displaying the computer-generated brachiosaurcenter screenrdquo She argues that during the ldquowonder yearsrdquo of the early 1990sdigital effects broke the narrative action by being showcased in sequencesthat dwelled on visual spectacle for its own sake Scott Bukatman (2003 113)writes that ldquowhat is evoked by special effects sequences is often a hallucina-tory excess as narrative yields to kinetic spectatorial experiencerdquo (It shouldalso be said however that Bukatmanrsquos work on special effects acknowledgesthe intersection of art and science and does not approach them with suspi-cion) Annette Kuhn (1999 5) echoes this idea finding that ldquowhen such [spe-cial effects] displays become a prominent attraction in their own right theytend to eclipse narrative plot and character The story becomes the display andthe display becomes the storyrdquo Andrew Darley (2000 104) writes that specta-cle is ldquothe antithesis of narrative Spectacle effectively halts motivated move-ment In its purer state it exists for itself consisting of images whose maindrive is to dazzle and stimulate the eye (and by extension the other senses)rdquo

In distinction with these views Shilo T McClean (2007) has shown clearlyand persuasively the manifold ways in which digital effects serve the art ofstorytelling in contemporary film and she argues that poorly motivated ef-fects are better understood as reflecting deficiencies of storytelling than anylogic or teleology that is inherent in the cinematic application of digital tech-nology The view that digital effects equal spectacle and that spectacle is anti-

2 2 P R O J E C T I O N S

T H R O U G H T H E L O O K I N G G L A S S 2 3

Figure 2 A cinema of attractionsmdashthecrash that concludesAeroplane Flightand Wreck (1910)

narrative has entailed that the field often exhibits a marked skepticism to-ward digital imaging and its aesthetic potential This suspicion has severalroots and one of the most important is the legacy of a school of highly influ-ential scholarship arguing that early cinema was a ldquocinema of attractionsrdquoless interested in narrative than in offering viewers startling visual displaysTom Gunningrsquos work on the history of early film helped to establish the cin-ema of attractions as a core idea in the field After studying features of earlysilent cinema that he took as being radically different from the style of narra-tive continuity that Hollywood helped establish he penned an influential essay ldquoThe Cinema of Attractionrdquo Objecting to what he described as thehegemony of narrative in the study of cinema he argued in favor of a viewthat stressed cinema as a medium of spectacle rather than narrative Earlyfilms emphasized spectacular views ndash attractions ndash rather than stories (Gun-ning 2000) He claimed that this function ruled cinema until around 1907when narrative became more dominant In a related essay Gunning devel-oped the notion of an ldquoaesthetic of astonishmentrdquo as the expressive outcomeof the cinema of attractions Visual tricks created by editing or in-cameramattes shots of locomotives rushing toward the camera and other abrupt orsurprising views caused viewers to vacillate ldquobetween belief and incredulityrdquo(2006a 119) He suggested that the cinema of attractions never fully disap-peared even after narrative became the dominant mode of popular cinema Instressing that it ldquoremains an essential part of popular filmmakingrdquo he con-nected it with contemporary special effects ldquoClearly in some sense recentspectacle cinema has reaffirmed its roots in stimulus and carnival rides inwhat might be called the Spielberg-Lucas-Coppola cinema of effectsrdquo (Gun-ning 2000 234 see Figure 2)

The skepticism among film scholars toward digital effects is partly a man-ifestation of the legacy of an anti-narrative spectacle-based conception ofearly cinema and its proliferation in the field as a persuasive template for con-ceptual analysis It was compatible with an abiding suspicion in film theoryabout the truthfulness of images and about the cameras and the projectorsthat produced them an outlook reinforced by the digital turn taken by con-temporary film Especially in its ideological and psychoanalytic componentspost-classical film theory regarded photographic images as a repressive legacyof Renaissance perspective fooling viewers with illusions of the real In ldquoMa-chines of the Visiblerdquo for example Jean-Louis Comolli (1980 124) decried theeffects of photographic realism which he attributed to an ideological forma-tion that he termed ldquothe ideology of the visiblerdquo a belief system linked to therationalism of Western culture under which the visible world is perceived asbeing centered by and organized through the gaze of the observer Realism incinema is a code he maintained a cultural practice a deceptive one that is accepted by spectators who wish to be fooled and who therefore oscillate between belief and disbelief ldquoanalogy in the cinema is a deception a lie a fiction that must be straddledmdashin disavowing knowing but not wanting toknowrdquo (126)

Comollirsquos approach manifests what Malcolm Turvey (2008) has identifiedas the lure of visual skepticism Turvey points out that ldquoa distrust of human vision has played a foundational role in film theoryrdquo (99) He elaborates ldquoIt isa general systematic doubt about normal human vision a distrust of every-day sight It is a belief that the standard exercise of the visual faculty is not tobe trusted in some significant respect because it possesses one or moreflawsrdquo (101) From this standpoint observersmdashpeople viewing paintings filmsor other objects of visual art and designmdashare socially constructed and thephysics of light the optics of lenses and the perceptual mechanisms of hu-man vision are less important than the discursive practices of a social forma-tion As Jonathan Crary (1992 6) writes ldquoIf it can be said there is an observerspecific to the nineteenth century or to any period it is only as an effect of anirreducibly heterogeneous system of discursive social technological and in-stitutional relations There is no observing subject prior to this continuallyshifting fieldrdquo According to this view there is no science of vision no physiol-ogy of sight indeed no scientific basis for representational pictures apartfrom their status as cultural constructions Crary writes ldquoWhether perceptionor vision actually change is irrelevant for they have no autonomous historyWhat changes are the plural forces and rules composing the field in whichperception occursrdquo (6) The consequence of such a view has been a hermeneu-tics of suspicion (to borrow Paul Riceourrsquos terminology for the analytic tradi-tion that Freud developed) applied to traditions of image-making and theinstruments that produce them a mistrust of vision and of the arts that play

2 4 P R O J E C T I O N S

to its characteristics Mary Ann Doane (2002 72) writes that discussions of thepersistence of vision in relation to moving images tend to invoke ldquoan insistentvocabulary of deception and failurerdquo

The Eye as an Optical InstrumentDoane is among a group of film scholars who have emphasized the bridge be-tween art and science in early cinema Scott Curtis (2004) has written elo-quently about the relationship between the modes of vision instantiated incinematic viewing devices and in practices of medical imaging John DurhamPeters (2004) has emphasized the importance of Hermann von Helmholtzrsquosphysiological studies for the history of sound film These and other scholarshave shown that todayrsquos digital culture is as Lauren Rabinovitz and AbrahamGeil (2004 13) point out a phenomenon ldquolarger and older than the informa-tion agerdquo Indeed cameras and other optical devices were important compo-nents used in the scientific study of vision by artists and natural philosophersand subsequently by experimental psychologists As Lisa Cartwright (1995 3)notes ldquomany of the techniques and instruments that contributed to theemergence of cinema were designed and used by scientistsrdquo But she pointsout in film studies ldquothe historical narrative quickly shifts however from sci-ence to popular culturerdquo Without an emphasis on the continuing interplaybetween science and popular culture however the operation of digital effectsin cinema today cannot be grasped as much beyond spectacle or display notas the occasion for the emergence of a new aesthetic of realism Digital toolsemulate properties of human vision as well as the camerarsquos customary way ofseeing things In this regard the application of digital tools continues a cen-turies-old tradition of analogizing camera and eye In 1868 Helmholtz wroteldquoRegarded as an optical instrument the eye is a camera obscurardquo ([1868] 196297) Natural theology in the period extolled the perfection of human vision asan instance of divine intervention into the world evidence of Godrsquos plan forhumankind (Silverman 1993) By contrast Helmholtz explicated the numerousflaws in the eyersquos instrumentationmdashchromatic and spherical aberration lackof clarity and optical uniformity in the crystalline lens the blind spot andother gaps in the retina He impishly observed ldquoNow it is not too much to saythat if an optician wanted to sell me an instrument which had all these de-fects I should think myself quite justified in blaming his carelessness in thestrongest terms and giving him back his instrumentrdquo (110) But he went on topoint out that binocular vision enables each eye to compensate for the defi-ciencies in the other providing a means of rectifying these flaws and that theeyersquos speed was superior to that of a camera

As Nicholas Wade and Stanley Finger (2001 1158) write ldquoThe overarchinganalogy that has been applied to the eye is that of the cameramdashboth devicesbeing capable of focusing on objects at variable distancesrdquo The principles of

T H R O U G H T H E L O O K I N G G L A S S 2 5

the camera obscuramdasha dark chamber admitting a smallamount of light to produce an upside down and reversed im-age of the scene on a flat surface or object outside the cham-ber (the image is upside down and reversed because the lightrays cross as they pass through the hole)mdashwere known in theeleventh century to the Islamic philosopher and scientist Ibnal-Haytham and were subsequently widely studied by artistswho used it as a device for tracing images and by scientistsseeking to understand optics and the eye (Figure 3) Leonardoda Vinci compared the eye with a camera obscura and exper-imented with the device proposing the use of a translucentscreen for tracing that would correct image reversal and elim-inate the problem of the observerrsquos head being in the path ofthe light Da Vinci though could not reconcile the upsidedown image captured by the camera obscura with the phe-nomenally correct perspectives supplied by human visionThe Venetian patrician Daniele Barbaro in 1568 used a convexlens and varying aperture sizes in a camera obscura to pro-duce sharpened images on a sheet of paper

Lenses were rapidly applied to the camera obscura andindeed mirrors and lenses provided vital aids to the scientificstudy of vision and assisted in the development of what Mar-tin Kemp has termed the science of art The inventor of linearperspective Filippo Brunelleschi used a peep-hole and mirrordevice to heighten the illusion of depth in a painting he madeof the Baptistry of St John The viewer looked through a holein a wooden panel at a mirror opposite that reflected the im-age of the painting from the other side of the panel By elim-

inating the problem that retinal disparity introduces into the perception ofdepth on a two-dimensional surface Brunelleschirsquos peep-show device height-ened the illusion of depth perspective in the image He even used burnishedsilver on a part of the mirror to make sky and clouds more luminescent (Kemp1990 13) By analyzing the perspective geometry in Vermeerrsquos paintingsPhillip Steadman (2002) has conclusively demonstrated that Vermeer used acamera obscura to produce a series of portraits set in the same room of ahouse or studio Artist David Hockney (2001 12) has argued that ldquofrom theearly fifteenth century many Western artists used opticsmdashby which I meanmirrors and lenses (or a combination of the two)mdashto create living projectionsrdquoas tools for producing paintings and drawings

The astronomers Tycho Brahe and Johannes Kepler and Jesuit scholarChristopher Scheiner used camera obscuras to make solar observations Kep-ler stated that the camera obscura provided a safe way to view a solar eclipse

2 6 P R O J E C T I O N S

Figure 3 Diagram of camera obscuraappearing in Denis Diderot and Jean leRond DrsquoAlembertrsquos Encyclopeacutedie (1751)

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

simulation of water proved very difficult for effects artists to master The gi-ant tidal wave at the end of The Abyss (1989) is unconvincing in part becauseit lacks texturingmdashfroth spray bubblesmdashand is insufficiently translucent Itwas a first digital go at doing water In 2006 for Poseidon Industrial Light andMagic (ILM) collaborated with Stanford University to incorporate current re-search on fluid dynamics Stanfordrsquos researchers worked on a phenomenonthey called ldquovorticityrdquo which is the swirling effect seen as water crashesaround solid objects and ILM built this information into the film helping tomake the rendering of water far more convincing than in years previous

Although visual effects may draw on science they are works of aestheticimagination and moviemakers often cheat reality for dramatic purposes Or-ange firelight would have made the blue-skinned Navi in Avatar look gray anundesirable outcome so the filmmakers decided to implement a policy theycalled ldquospectral compensationrdquo in effect ignoring the interaction of orangeand blue light in this context (Duncan 2010 130 Figure 1)

Perceptual RealismVisual effects manifest a dialectic of art and science and as such they negoti-ate terrain for a realist aesthetic in digital cinema Claiming a basis for realismin digital visual effects may seem a counter-intuitive move but several no-table film scholars have indeed made such a move Warren Buckland (1999185) for example found the dynamic staging in depth that digital effects en-abled Spielberg to achieve in Jurassic Park to be compatible with Bazinian no-tions of deep focus Tom Gunning (2006b 347) has argued forcefully that weneed to move beyond the familiar dichotomies of theory ldquoI believe we distortour experience of films if we try to assign the effect of realismmdashor even thesensation of physical presencemdashexclusively to the photographic or confinethe artificial to lsquospecial effectsrsquordquo Indeed there are very good reasons for insist-ing on a critical perspective that is amenable to integrating computer graph-ics capabilities with aesthetic properties of realism in cinema In an earlieressay (Prince 1996) I identified a digital basis for realism in cinema in terms ofwhat I called ldquoperceptual realismrdquo which was the replication via digital meansof contextual cues designating a three-dimensional world These cues includeinformation sources about the size and positioning of objects in space theirtexturing and apparent density of detail the behavior of light as it interactswith the physical world principles of motion and anatomy and the physics in-volved in dynamic systems such as water clouds and fire Digital tools givefilmmakers an unprecedented ability to replicate and emphasize these cuesas a means for anchoring the scene in a perceptual reality that the viewerfinds credible because it follows the same observable laws of physics as theworld they inhabit The referential status of the representation does not mat-ter in this conception of realism

2 0 P R O J E C T I O N S

Dinosaurs are not living beings in the age of cinema They cannot be pho-tographed as sentient creatures Thus their logical status in Jurassic Park is asobjects that are referentially false They correspond to no reality that the filmrsquosviewer could inhabit Yet as depicted in the film they are perceptually realis-tic They interact in relatively convincing ways with the live actors in dynami-cally staged scenes that bond the domains of live action and digitalanimation And because they are perceptually realistic they are able to com-pel belief in the fictional world of the film in ways that traditional special ef-fects could not accomplish The creation of perceptual realism is a major goalof special effects artists Visual effects seek to persuade viewers that effectsare real within the referential terms of the story Therefore the more compre-hensive is a scene in evoking perceptual realism the likelier it is to compel thespectatorrsquos belief No one watching Jurassic Park was fooled into thinking thatdinosaurs were actually alive but because digital tools established perceptualrealism with new levels of sensory detail viewers could be sensually per-suaded to believe in the fiction and to participate in the pleasures it offeredHad the film employed established and traditional effects tools this sensorypersuasion would have been far less remarkable

That is because traditional effects tools have been quite limited in theirability to create perceptual realism The compositing of live action mattepaintings and miniatures in The Lost World (1925) King Kong (1933) The Valleyof Gwangi (1969) and other comparable creature movies was visibly falsecompromised by overt matte lines between the elements and by the planarrendition of space that prevented the matted creature from interacting with

T H R O U G H T H E L O O K I N G G L A S S 2 1

Figure 1 ThroughoutAvatar the Nrsquoaviretain the blue oftheir skin in firelightan impossible colorcombination but onepermitted by thefilmmakers fordramatic purposes

the live actors The composited elements exhibit a perceptual disparitymdashlim-ited interaction between the domains contradictory manifestations of mo-tion blurmdashworking against the emergence of an organic unity of action Underthese terms the dramatic space of the screen action becomes perceptuallysuspect Earlier generations of special effects images manifested similar prob-lems and elicited degrees of skepticism in viewers Digital effects have helpedcinematic imagery overcome problems triggering a reality check by viewersthat undermined the fictional enterprise

Orthodox Assumptions about Visual EffectsPerceptual realism then is central to understanding special effects in cinemathe goal of effects artists and the credibility that the effects image seeks toelicit among viewers Moreover it has important implications for two ortho-dox assumptions in film studies 1) that visual effects equate with spectacleand that spectacle is anti-narrative and 2) that visionmdashthe act of seeingmdashisprimarily a culturally coded activity and therefore relative to social formationsin a given period About the first appearance of the digital dinosaurs in Juras-sic Park Michelle Pierson (2002 120) writes that the filmrsquos narrative stopsdead so that the digital effects can be showcased at length ldquoThe narrative allbut comes to a halt the music gradually builds and shots of characters react-ing to the appearance of the dinosaur with wonder and amazement are inter-spersed with long takes displaying the computer-generated brachiosaurcenter screenrdquo She argues that during the ldquowonder yearsrdquo of the early 1990sdigital effects broke the narrative action by being showcased in sequencesthat dwelled on visual spectacle for its own sake Scott Bukatman (2003 113)writes that ldquowhat is evoked by special effects sequences is often a hallucina-tory excess as narrative yields to kinetic spectatorial experiencerdquo (It shouldalso be said however that Bukatmanrsquos work on special effects acknowledgesthe intersection of art and science and does not approach them with suspi-cion) Annette Kuhn (1999 5) echoes this idea finding that ldquowhen such [spe-cial effects] displays become a prominent attraction in their own right theytend to eclipse narrative plot and character The story becomes the display andthe display becomes the storyrdquo Andrew Darley (2000 104) writes that specta-cle is ldquothe antithesis of narrative Spectacle effectively halts motivated move-ment In its purer state it exists for itself consisting of images whose maindrive is to dazzle and stimulate the eye (and by extension the other senses)rdquo

In distinction with these views Shilo T McClean (2007) has shown clearlyand persuasively the manifold ways in which digital effects serve the art ofstorytelling in contemporary film and she argues that poorly motivated ef-fects are better understood as reflecting deficiencies of storytelling than anylogic or teleology that is inherent in the cinematic application of digital tech-nology The view that digital effects equal spectacle and that spectacle is anti-

2 2 P R O J E C T I O N S

T H R O U G H T H E L O O K I N G G L A S S 2 3

Figure 2 A cinema of attractionsmdashthecrash that concludesAeroplane Flightand Wreck (1910)

narrative has entailed that the field often exhibits a marked skepticism to-ward digital imaging and its aesthetic potential This suspicion has severalroots and one of the most important is the legacy of a school of highly influ-ential scholarship arguing that early cinema was a ldquocinema of attractionsrdquoless interested in narrative than in offering viewers startling visual displaysTom Gunningrsquos work on the history of early film helped to establish the cin-ema of attractions as a core idea in the field After studying features of earlysilent cinema that he took as being radically different from the style of narra-tive continuity that Hollywood helped establish he penned an influential essay ldquoThe Cinema of Attractionrdquo Objecting to what he described as thehegemony of narrative in the study of cinema he argued in favor of a viewthat stressed cinema as a medium of spectacle rather than narrative Earlyfilms emphasized spectacular views ndash attractions ndash rather than stories (Gun-ning 2000) He claimed that this function ruled cinema until around 1907when narrative became more dominant In a related essay Gunning devel-oped the notion of an ldquoaesthetic of astonishmentrdquo as the expressive outcomeof the cinema of attractions Visual tricks created by editing or in-cameramattes shots of locomotives rushing toward the camera and other abrupt orsurprising views caused viewers to vacillate ldquobetween belief and incredulityrdquo(2006a 119) He suggested that the cinema of attractions never fully disap-peared even after narrative became the dominant mode of popular cinema Instressing that it ldquoremains an essential part of popular filmmakingrdquo he con-nected it with contemporary special effects ldquoClearly in some sense recentspectacle cinema has reaffirmed its roots in stimulus and carnival rides inwhat might be called the Spielberg-Lucas-Coppola cinema of effectsrdquo (Gun-ning 2000 234 see Figure 2)

The skepticism among film scholars toward digital effects is partly a man-ifestation of the legacy of an anti-narrative spectacle-based conception ofearly cinema and its proliferation in the field as a persuasive template for con-ceptual analysis It was compatible with an abiding suspicion in film theoryabout the truthfulness of images and about the cameras and the projectorsthat produced them an outlook reinforced by the digital turn taken by con-temporary film Especially in its ideological and psychoanalytic componentspost-classical film theory regarded photographic images as a repressive legacyof Renaissance perspective fooling viewers with illusions of the real In ldquoMa-chines of the Visiblerdquo for example Jean-Louis Comolli (1980 124) decried theeffects of photographic realism which he attributed to an ideological forma-tion that he termed ldquothe ideology of the visiblerdquo a belief system linked to therationalism of Western culture under which the visible world is perceived asbeing centered by and organized through the gaze of the observer Realism incinema is a code he maintained a cultural practice a deceptive one that is accepted by spectators who wish to be fooled and who therefore oscillate between belief and disbelief ldquoanalogy in the cinema is a deception a lie a fiction that must be straddledmdashin disavowing knowing but not wanting toknowrdquo (126)

Comollirsquos approach manifests what Malcolm Turvey (2008) has identifiedas the lure of visual skepticism Turvey points out that ldquoa distrust of human vision has played a foundational role in film theoryrdquo (99) He elaborates ldquoIt isa general systematic doubt about normal human vision a distrust of every-day sight It is a belief that the standard exercise of the visual faculty is not tobe trusted in some significant respect because it possesses one or moreflawsrdquo (101) From this standpoint observersmdashpeople viewing paintings filmsor other objects of visual art and designmdashare socially constructed and thephysics of light the optics of lenses and the perceptual mechanisms of hu-man vision are less important than the discursive practices of a social forma-tion As Jonathan Crary (1992 6) writes ldquoIf it can be said there is an observerspecific to the nineteenth century or to any period it is only as an effect of anirreducibly heterogeneous system of discursive social technological and in-stitutional relations There is no observing subject prior to this continuallyshifting fieldrdquo According to this view there is no science of vision no physiol-ogy of sight indeed no scientific basis for representational pictures apartfrom their status as cultural constructions Crary writes ldquoWhether perceptionor vision actually change is irrelevant for they have no autonomous historyWhat changes are the plural forces and rules composing the field in whichperception occursrdquo (6) The consequence of such a view has been a hermeneu-tics of suspicion (to borrow Paul Riceourrsquos terminology for the analytic tradi-tion that Freud developed) applied to traditions of image-making and theinstruments that produce them a mistrust of vision and of the arts that play

2 4 P R O J E C T I O N S

to its characteristics Mary Ann Doane (2002 72) writes that discussions of thepersistence of vision in relation to moving images tend to invoke ldquoan insistentvocabulary of deception and failurerdquo

The Eye as an Optical InstrumentDoane is among a group of film scholars who have emphasized the bridge be-tween art and science in early cinema Scott Curtis (2004) has written elo-quently about the relationship between the modes of vision instantiated incinematic viewing devices and in practices of medical imaging John DurhamPeters (2004) has emphasized the importance of Hermann von Helmholtzrsquosphysiological studies for the history of sound film These and other scholarshave shown that todayrsquos digital culture is as Lauren Rabinovitz and AbrahamGeil (2004 13) point out a phenomenon ldquolarger and older than the informa-tion agerdquo Indeed cameras and other optical devices were important compo-nents used in the scientific study of vision by artists and natural philosophersand subsequently by experimental psychologists As Lisa Cartwright (1995 3)notes ldquomany of the techniques and instruments that contributed to theemergence of cinema were designed and used by scientistsrdquo But she pointsout in film studies ldquothe historical narrative quickly shifts however from sci-ence to popular culturerdquo Without an emphasis on the continuing interplaybetween science and popular culture however the operation of digital effectsin cinema today cannot be grasped as much beyond spectacle or display notas the occasion for the emergence of a new aesthetic of realism Digital toolsemulate properties of human vision as well as the camerarsquos customary way ofseeing things In this regard the application of digital tools continues a cen-turies-old tradition of analogizing camera and eye In 1868 Helmholtz wroteldquoRegarded as an optical instrument the eye is a camera obscurardquo ([1868] 196297) Natural theology in the period extolled the perfection of human vision asan instance of divine intervention into the world evidence of Godrsquos plan forhumankind (Silverman 1993) By contrast Helmholtz explicated the numerousflaws in the eyersquos instrumentationmdashchromatic and spherical aberration lackof clarity and optical uniformity in the crystalline lens the blind spot andother gaps in the retina He impishly observed ldquoNow it is not too much to saythat if an optician wanted to sell me an instrument which had all these de-fects I should think myself quite justified in blaming his carelessness in thestrongest terms and giving him back his instrumentrdquo (110) But he went on topoint out that binocular vision enables each eye to compensate for the defi-ciencies in the other providing a means of rectifying these flaws and that theeyersquos speed was superior to that of a camera

As Nicholas Wade and Stanley Finger (2001 1158) write ldquoThe overarchinganalogy that has been applied to the eye is that of the cameramdashboth devicesbeing capable of focusing on objects at variable distancesrdquo The principles of

T H R O U G H T H E L O O K I N G G L A S S 2 5

the camera obscuramdasha dark chamber admitting a smallamount of light to produce an upside down and reversed im-age of the scene on a flat surface or object outside the cham-ber (the image is upside down and reversed because the lightrays cross as they pass through the hole)mdashwere known in theeleventh century to the Islamic philosopher and scientist Ibnal-Haytham and were subsequently widely studied by artistswho used it as a device for tracing images and by scientistsseeking to understand optics and the eye (Figure 3) Leonardoda Vinci compared the eye with a camera obscura and exper-imented with the device proposing the use of a translucentscreen for tracing that would correct image reversal and elim-inate the problem of the observerrsquos head being in the path ofthe light Da Vinci though could not reconcile the upsidedown image captured by the camera obscura with the phe-nomenally correct perspectives supplied by human visionThe Venetian patrician Daniele Barbaro in 1568 used a convexlens and varying aperture sizes in a camera obscura to pro-duce sharpened images on a sheet of paper

Lenses were rapidly applied to the camera obscura andindeed mirrors and lenses provided vital aids to the scientificstudy of vision and assisted in the development of what Mar-tin Kemp has termed the science of art The inventor of linearperspective Filippo Brunelleschi used a peep-hole and mirrordevice to heighten the illusion of depth in a painting he madeof the Baptistry of St John The viewer looked through a holein a wooden panel at a mirror opposite that reflected the im-age of the painting from the other side of the panel By elim-

inating the problem that retinal disparity introduces into the perception ofdepth on a two-dimensional surface Brunelleschirsquos peep-show device height-ened the illusion of depth perspective in the image He even used burnishedsilver on a part of the mirror to make sky and clouds more luminescent (Kemp1990 13) By analyzing the perspective geometry in Vermeerrsquos paintingsPhillip Steadman (2002) has conclusively demonstrated that Vermeer used acamera obscura to produce a series of portraits set in the same room of ahouse or studio Artist David Hockney (2001 12) has argued that ldquofrom theearly fifteenth century many Western artists used opticsmdashby which I meanmirrors and lenses (or a combination of the two)mdashto create living projectionsrdquoas tools for producing paintings and drawings

The astronomers Tycho Brahe and Johannes Kepler and Jesuit scholarChristopher Scheiner used camera obscuras to make solar observations Kep-ler stated that the camera obscura provided a safe way to view a solar eclipse

2 6 P R O J E C T I O N S

Figure 3 Diagram of camera obscuraappearing in Denis Diderot and Jean leRond DrsquoAlembertrsquos Encyclopeacutedie (1751)

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

Dinosaurs are not living beings in the age of cinema They cannot be pho-tographed as sentient creatures Thus their logical status in Jurassic Park is asobjects that are referentially false They correspond to no reality that the filmrsquosviewer could inhabit Yet as depicted in the film they are perceptually realis-tic They interact in relatively convincing ways with the live actors in dynami-cally staged scenes that bond the domains of live action and digitalanimation And because they are perceptually realistic they are able to com-pel belief in the fictional world of the film in ways that traditional special ef-fects could not accomplish The creation of perceptual realism is a major goalof special effects artists Visual effects seek to persuade viewers that effectsare real within the referential terms of the story Therefore the more compre-hensive is a scene in evoking perceptual realism the likelier it is to compel thespectatorrsquos belief No one watching Jurassic Park was fooled into thinking thatdinosaurs were actually alive but because digital tools established perceptualrealism with new levels of sensory detail viewers could be sensually per-suaded to believe in the fiction and to participate in the pleasures it offeredHad the film employed established and traditional effects tools this sensorypersuasion would have been far less remarkable

That is because traditional effects tools have been quite limited in theirability to create perceptual realism The compositing of live action mattepaintings and miniatures in The Lost World (1925) King Kong (1933) The Valleyof Gwangi (1969) and other comparable creature movies was visibly falsecompromised by overt matte lines between the elements and by the planarrendition of space that prevented the matted creature from interacting with

T H R O U G H T H E L O O K I N G G L A S S 2 1

Figure 1 ThroughoutAvatar the Nrsquoaviretain the blue oftheir skin in firelightan impossible colorcombination but onepermitted by thefilmmakers fordramatic purposes

the live actors The composited elements exhibit a perceptual disparitymdashlim-ited interaction between the domains contradictory manifestations of mo-tion blurmdashworking against the emergence of an organic unity of action Underthese terms the dramatic space of the screen action becomes perceptuallysuspect Earlier generations of special effects images manifested similar prob-lems and elicited degrees of skepticism in viewers Digital effects have helpedcinematic imagery overcome problems triggering a reality check by viewersthat undermined the fictional enterprise

Orthodox Assumptions about Visual EffectsPerceptual realism then is central to understanding special effects in cinemathe goal of effects artists and the credibility that the effects image seeks toelicit among viewers Moreover it has important implications for two ortho-dox assumptions in film studies 1) that visual effects equate with spectacleand that spectacle is anti-narrative and 2) that visionmdashthe act of seeingmdashisprimarily a culturally coded activity and therefore relative to social formationsin a given period About the first appearance of the digital dinosaurs in Juras-sic Park Michelle Pierson (2002 120) writes that the filmrsquos narrative stopsdead so that the digital effects can be showcased at length ldquoThe narrative allbut comes to a halt the music gradually builds and shots of characters react-ing to the appearance of the dinosaur with wonder and amazement are inter-spersed with long takes displaying the computer-generated brachiosaurcenter screenrdquo She argues that during the ldquowonder yearsrdquo of the early 1990sdigital effects broke the narrative action by being showcased in sequencesthat dwelled on visual spectacle for its own sake Scott Bukatman (2003 113)writes that ldquowhat is evoked by special effects sequences is often a hallucina-tory excess as narrative yields to kinetic spectatorial experiencerdquo (It shouldalso be said however that Bukatmanrsquos work on special effects acknowledgesthe intersection of art and science and does not approach them with suspi-cion) Annette Kuhn (1999 5) echoes this idea finding that ldquowhen such [spe-cial effects] displays become a prominent attraction in their own right theytend to eclipse narrative plot and character The story becomes the display andthe display becomes the storyrdquo Andrew Darley (2000 104) writes that specta-cle is ldquothe antithesis of narrative Spectacle effectively halts motivated move-ment In its purer state it exists for itself consisting of images whose maindrive is to dazzle and stimulate the eye (and by extension the other senses)rdquo

In distinction with these views Shilo T McClean (2007) has shown clearlyand persuasively the manifold ways in which digital effects serve the art ofstorytelling in contemporary film and she argues that poorly motivated ef-fects are better understood as reflecting deficiencies of storytelling than anylogic or teleology that is inherent in the cinematic application of digital tech-nology The view that digital effects equal spectacle and that spectacle is anti-

2 2 P R O J E C T I O N S

T H R O U G H T H E L O O K I N G G L A S S 2 3

Figure 2 A cinema of attractionsmdashthecrash that concludesAeroplane Flightand Wreck (1910)

narrative has entailed that the field often exhibits a marked skepticism to-ward digital imaging and its aesthetic potential This suspicion has severalroots and one of the most important is the legacy of a school of highly influ-ential scholarship arguing that early cinema was a ldquocinema of attractionsrdquoless interested in narrative than in offering viewers startling visual displaysTom Gunningrsquos work on the history of early film helped to establish the cin-ema of attractions as a core idea in the field After studying features of earlysilent cinema that he took as being radically different from the style of narra-tive continuity that Hollywood helped establish he penned an influential essay ldquoThe Cinema of Attractionrdquo Objecting to what he described as thehegemony of narrative in the study of cinema he argued in favor of a viewthat stressed cinema as a medium of spectacle rather than narrative Earlyfilms emphasized spectacular views ndash attractions ndash rather than stories (Gun-ning 2000) He claimed that this function ruled cinema until around 1907when narrative became more dominant In a related essay Gunning devel-oped the notion of an ldquoaesthetic of astonishmentrdquo as the expressive outcomeof the cinema of attractions Visual tricks created by editing or in-cameramattes shots of locomotives rushing toward the camera and other abrupt orsurprising views caused viewers to vacillate ldquobetween belief and incredulityrdquo(2006a 119) He suggested that the cinema of attractions never fully disap-peared even after narrative became the dominant mode of popular cinema Instressing that it ldquoremains an essential part of popular filmmakingrdquo he con-nected it with contemporary special effects ldquoClearly in some sense recentspectacle cinema has reaffirmed its roots in stimulus and carnival rides inwhat might be called the Spielberg-Lucas-Coppola cinema of effectsrdquo (Gun-ning 2000 234 see Figure 2)

The skepticism among film scholars toward digital effects is partly a man-ifestation of the legacy of an anti-narrative spectacle-based conception ofearly cinema and its proliferation in the field as a persuasive template for con-ceptual analysis It was compatible with an abiding suspicion in film theoryabout the truthfulness of images and about the cameras and the projectorsthat produced them an outlook reinforced by the digital turn taken by con-temporary film Especially in its ideological and psychoanalytic componentspost-classical film theory regarded photographic images as a repressive legacyof Renaissance perspective fooling viewers with illusions of the real In ldquoMa-chines of the Visiblerdquo for example Jean-Louis Comolli (1980 124) decried theeffects of photographic realism which he attributed to an ideological forma-tion that he termed ldquothe ideology of the visiblerdquo a belief system linked to therationalism of Western culture under which the visible world is perceived asbeing centered by and organized through the gaze of the observer Realism incinema is a code he maintained a cultural practice a deceptive one that is accepted by spectators who wish to be fooled and who therefore oscillate between belief and disbelief ldquoanalogy in the cinema is a deception a lie a fiction that must be straddledmdashin disavowing knowing but not wanting toknowrdquo (126)

Comollirsquos approach manifests what Malcolm Turvey (2008) has identifiedas the lure of visual skepticism Turvey points out that ldquoa distrust of human vision has played a foundational role in film theoryrdquo (99) He elaborates ldquoIt isa general systematic doubt about normal human vision a distrust of every-day sight It is a belief that the standard exercise of the visual faculty is not tobe trusted in some significant respect because it possesses one or moreflawsrdquo (101) From this standpoint observersmdashpeople viewing paintings filmsor other objects of visual art and designmdashare socially constructed and thephysics of light the optics of lenses and the perceptual mechanisms of hu-man vision are less important than the discursive practices of a social forma-tion As Jonathan Crary (1992 6) writes ldquoIf it can be said there is an observerspecific to the nineteenth century or to any period it is only as an effect of anirreducibly heterogeneous system of discursive social technological and in-stitutional relations There is no observing subject prior to this continuallyshifting fieldrdquo According to this view there is no science of vision no physiol-ogy of sight indeed no scientific basis for representational pictures apartfrom their status as cultural constructions Crary writes ldquoWhether perceptionor vision actually change is irrelevant for they have no autonomous historyWhat changes are the plural forces and rules composing the field in whichperception occursrdquo (6) The consequence of such a view has been a hermeneu-tics of suspicion (to borrow Paul Riceourrsquos terminology for the analytic tradi-tion that Freud developed) applied to traditions of image-making and theinstruments that produce them a mistrust of vision and of the arts that play

2 4 P R O J E C T I O N S

to its characteristics Mary Ann Doane (2002 72) writes that discussions of thepersistence of vision in relation to moving images tend to invoke ldquoan insistentvocabulary of deception and failurerdquo

The Eye as an Optical InstrumentDoane is among a group of film scholars who have emphasized the bridge be-tween art and science in early cinema Scott Curtis (2004) has written elo-quently about the relationship between the modes of vision instantiated incinematic viewing devices and in practices of medical imaging John DurhamPeters (2004) has emphasized the importance of Hermann von Helmholtzrsquosphysiological studies for the history of sound film These and other scholarshave shown that todayrsquos digital culture is as Lauren Rabinovitz and AbrahamGeil (2004 13) point out a phenomenon ldquolarger and older than the informa-tion agerdquo Indeed cameras and other optical devices were important compo-nents used in the scientific study of vision by artists and natural philosophersand subsequently by experimental psychologists As Lisa Cartwright (1995 3)notes ldquomany of the techniques and instruments that contributed to theemergence of cinema were designed and used by scientistsrdquo But she pointsout in film studies ldquothe historical narrative quickly shifts however from sci-ence to popular culturerdquo Without an emphasis on the continuing interplaybetween science and popular culture however the operation of digital effectsin cinema today cannot be grasped as much beyond spectacle or display notas the occasion for the emergence of a new aesthetic of realism Digital toolsemulate properties of human vision as well as the camerarsquos customary way ofseeing things In this regard the application of digital tools continues a cen-turies-old tradition of analogizing camera and eye In 1868 Helmholtz wroteldquoRegarded as an optical instrument the eye is a camera obscurardquo ([1868] 196297) Natural theology in the period extolled the perfection of human vision asan instance of divine intervention into the world evidence of Godrsquos plan forhumankind (Silverman 1993) By contrast Helmholtz explicated the numerousflaws in the eyersquos instrumentationmdashchromatic and spherical aberration lackof clarity and optical uniformity in the crystalline lens the blind spot andother gaps in the retina He impishly observed ldquoNow it is not too much to saythat if an optician wanted to sell me an instrument which had all these de-fects I should think myself quite justified in blaming his carelessness in thestrongest terms and giving him back his instrumentrdquo (110) But he went on topoint out that binocular vision enables each eye to compensate for the defi-ciencies in the other providing a means of rectifying these flaws and that theeyersquos speed was superior to that of a camera

As Nicholas Wade and Stanley Finger (2001 1158) write ldquoThe overarchinganalogy that has been applied to the eye is that of the cameramdashboth devicesbeing capable of focusing on objects at variable distancesrdquo The principles of

T H R O U G H T H E L O O K I N G G L A S S 2 5

the camera obscuramdasha dark chamber admitting a smallamount of light to produce an upside down and reversed im-age of the scene on a flat surface or object outside the cham-ber (the image is upside down and reversed because the lightrays cross as they pass through the hole)mdashwere known in theeleventh century to the Islamic philosopher and scientist Ibnal-Haytham and were subsequently widely studied by artistswho used it as a device for tracing images and by scientistsseeking to understand optics and the eye (Figure 3) Leonardoda Vinci compared the eye with a camera obscura and exper-imented with the device proposing the use of a translucentscreen for tracing that would correct image reversal and elim-inate the problem of the observerrsquos head being in the path ofthe light Da Vinci though could not reconcile the upsidedown image captured by the camera obscura with the phe-nomenally correct perspectives supplied by human visionThe Venetian patrician Daniele Barbaro in 1568 used a convexlens and varying aperture sizes in a camera obscura to pro-duce sharpened images on a sheet of paper

Lenses were rapidly applied to the camera obscura andindeed mirrors and lenses provided vital aids to the scientificstudy of vision and assisted in the development of what Mar-tin Kemp has termed the science of art The inventor of linearperspective Filippo Brunelleschi used a peep-hole and mirrordevice to heighten the illusion of depth in a painting he madeof the Baptistry of St John The viewer looked through a holein a wooden panel at a mirror opposite that reflected the im-age of the painting from the other side of the panel By elim-

inating the problem that retinal disparity introduces into the perception ofdepth on a two-dimensional surface Brunelleschirsquos peep-show device height-ened the illusion of depth perspective in the image He even used burnishedsilver on a part of the mirror to make sky and clouds more luminescent (Kemp1990 13) By analyzing the perspective geometry in Vermeerrsquos paintingsPhillip Steadman (2002) has conclusively demonstrated that Vermeer used acamera obscura to produce a series of portraits set in the same room of ahouse or studio Artist David Hockney (2001 12) has argued that ldquofrom theearly fifteenth century many Western artists used opticsmdashby which I meanmirrors and lenses (or a combination of the two)mdashto create living projectionsrdquoas tools for producing paintings and drawings

The astronomers Tycho Brahe and Johannes Kepler and Jesuit scholarChristopher Scheiner used camera obscuras to make solar observations Kep-ler stated that the camera obscura provided a safe way to view a solar eclipse

2 6 P R O J E C T I O N S

Figure 3 Diagram of camera obscuraappearing in Denis Diderot and Jean leRond DrsquoAlembertrsquos Encyclopeacutedie (1751)

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

the live actors The composited elements exhibit a perceptual disparitymdashlim-ited interaction between the domains contradictory manifestations of mo-tion blurmdashworking against the emergence of an organic unity of action Underthese terms the dramatic space of the screen action becomes perceptuallysuspect Earlier generations of special effects images manifested similar prob-lems and elicited degrees of skepticism in viewers Digital effects have helpedcinematic imagery overcome problems triggering a reality check by viewersthat undermined the fictional enterprise

Orthodox Assumptions about Visual EffectsPerceptual realism then is central to understanding special effects in cinemathe goal of effects artists and the credibility that the effects image seeks toelicit among viewers Moreover it has important implications for two ortho-dox assumptions in film studies 1) that visual effects equate with spectacleand that spectacle is anti-narrative and 2) that visionmdashthe act of seeingmdashisprimarily a culturally coded activity and therefore relative to social formationsin a given period About the first appearance of the digital dinosaurs in Juras-sic Park Michelle Pierson (2002 120) writes that the filmrsquos narrative stopsdead so that the digital effects can be showcased at length ldquoThe narrative allbut comes to a halt the music gradually builds and shots of characters react-ing to the appearance of the dinosaur with wonder and amazement are inter-spersed with long takes displaying the computer-generated brachiosaurcenter screenrdquo She argues that during the ldquowonder yearsrdquo of the early 1990sdigital effects broke the narrative action by being showcased in sequencesthat dwelled on visual spectacle for its own sake Scott Bukatman (2003 113)writes that ldquowhat is evoked by special effects sequences is often a hallucina-tory excess as narrative yields to kinetic spectatorial experiencerdquo (It shouldalso be said however that Bukatmanrsquos work on special effects acknowledgesthe intersection of art and science and does not approach them with suspi-cion) Annette Kuhn (1999 5) echoes this idea finding that ldquowhen such [spe-cial effects] displays become a prominent attraction in their own right theytend to eclipse narrative plot and character The story becomes the display andthe display becomes the storyrdquo Andrew Darley (2000 104) writes that specta-cle is ldquothe antithesis of narrative Spectacle effectively halts motivated move-ment In its purer state it exists for itself consisting of images whose maindrive is to dazzle and stimulate the eye (and by extension the other senses)rdquo

In distinction with these views Shilo T McClean (2007) has shown clearlyand persuasively the manifold ways in which digital effects serve the art ofstorytelling in contemporary film and she argues that poorly motivated ef-fects are better understood as reflecting deficiencies of storytelling than anylogic or teleology that is inherent in the cinematic application of digital tech-nology The view that digital effects equal spectacle and that spectacle is anti-

2 2 P R O J E C T I O N S

T H R O U G H T H E L O O K I N G G L A S S 2 3

Figure 2 A cinema of attractionsmdashthecrash that concludesAeroplane Flightand Wreck (1910)

narrative has entailed that the field often exhibits a marked skepticism to-ward digital imaging and its aesthetic potential This suspicion has severalroots and one of the most important is the legacy of a school of highly influ-ential scholarship arguing that early cinema was a ldquocinema of attractionsrdquoless interested in narrative than in offering viewers startling visual displaysTom Gunningrsquos work on the history of early film helped to establish the cin-ema of attractions as a core idea in the field After studying features of earlysilent cinema that he took as being radically different from the style of narra-tive continuity that Hollywood helped establish he penned an influential essay ldquoThe Cinema of Attractionrdquo Objecting to what he described as thehegemony of narrative in the study of cinema he argued in favor of a viewthat stressed cinema as a medium of spectacle rather than narrative Earlyfilms emphasized spectacular views ndash attractions ndash rather than stories (Gun-ning 2000) He claimed that this function ruled cinema until around 1907when narrative became more dominant In a related essay Gunning devel-oped the notion of an ldquoaesthetic of astonishmentrdquo as the expressive outcomeof the cinema of attractions Visual tricks created by editing or in-cameramattes shots of locomotives rushing toward the camera and other abrupt orsurprising views caused viewers to vacillate ldquobetween belief and incredulityrdquo(2006a 119) He suggested that the cinema of attractions never fully disap-peared even after narrative became the dominant mode of popular cinema Instressing that it ldquoremains an essential part of popular filmmakingrdquo he con-nected it with contemporary special effects ldquoClearly in some sense recentspectacle cinema has reaffirmed its roots in stimulus and carnival rides inwhat might be called the Spielberg-Lucas-Coppola cinema of effectsrdquo (Gun-ning 2000 234 see Figure 2)

The skepticism among film scholars toward digital effects is partly a man-ifestation of the legacy of an anti-narrative spectacle-based conception ofearly cinema and its proliferation in the field as a persuasive template for con-ceptual analysis It was compatible with an abiding suspicion in film theoryabout the truthfulness of images and about the cameras and the projectorsthat produced them an outlook reinforced by the digital turn taken by con-temporary film Especially in its ideological and psychoanalytic componentspost-classical film theory regarded photographic images as a repressive legacyof Renaissance perspective fooling viewers with illusions of the real In ldquoMa-chines of the Visiblerdquo for example Jean-Louis Comolli (1980 124) decried theeffects of photographic realism which he attributed to an ideological forma-tion that he termed ldquothe ideology of the visiblerdquo a belief system linked to therationalism of Western culture under which the visible world is perceived asbeing centered by and organized through the gaze of the observer Realism incinema is a code he maintained a cultural practice a deceptive one that is accepted by spectators who wish to be fooled and who therefore oscillate between belief and disbelief ldquoanalogy in the cinema is a deception a lie a fiction that must be straddledmdashin disavowing knowing but not wanting toknowrdquo (126)

Comollirsquos approach manifests what Malcolm Turvey (2008) has identifiedas the lure of visual skepticism Turvey points out that ldquoa distrust of human vision has played a foundational role in film theoryrdquo (99) He elaborates ldquoIt isa general systematic doubt about normal human vision a distrust of every-day sight It is a belief that the standard exercise of the visual faculty is not tobe trusted in some significant respect because it possesses one or moreflawsrdquo (101) From this standpoint observersmdashpeople viewing paintings filmsor other objects of visual art and designmdashare socially constructed and thephysics of light the optics of lenses and the perceptual mechanisms of hu-man vision are less important than the discursive practices of a social forma-tion As Jonathan Crary (1992 6) writes ldquoIf it can be said there is an observerspecific to the nineteenth century or to any period it is only as an effect of anirreducibly heterogeneous system of discursive social technological and in-stitutional relations There is no observing subject prior to this continuallyshifting fieldrdquo According to this view there is no science of vision no physiol-ogy of sight indeed no scientific basis for representational pictures apartfrom their status as cultural constructions Crary writes ldquoWhether perceptionor vision actually change is irrelevant for they have no autonomous historyWhat changes are the plural forces and rules composing the field in whichperception occursrdquo (6) The consequence of such a view has been a hermeneu-tics of suspicion (to borrow Paul Riceourrsquos terminology for the analytic tradi-tion that Freud developed) applied to traditions of image-making and theinstruments that produce them a mistrust of vision and of the arts that play

2 4 P R O J E C T I O N S

to its characteristics Mary Ann Doane (2002 72) writes that discussions of thepersistence of vision in relation to moving images tend to invoke ldquoan insistentvocabulary of deception and failurerdquo

The Eye as an Optical InstrumentDoane is among a group of film scholars who have emphasized the bridge be-tween art and science in early cinema Scott Curtis (2004) has written elo-quently about the relationship between the modes of vision instantiated incinematic viewing devices and in practices of medical imaging John DurhamPeters (2004) has emphasized the importance of Hermann von Helmholtzrsquosphysiological studies for the history of sound film These and other scholarshave shown that todayrsquos digital culture is as Lauren Rabinovitz and AbrahamGeil (2004 13) point out a phenomenon ldquolarger and older than the informa-tion agerdquo Indeed cameras and other optical devices were important compo-nents used in the scientific study of vision by artists and natural philosophersand subsequently by experimental psychologists As Lisa Cartwright (1995 3)notes ldquomany of the techniques and instruments that contributed to theemergence of cinema were designed and used by scientistsrdquo But she pointsout in film studies ldquothe historical narrative quickly shifts however from sci-ence to popular culturerdquo Without an emphasis on the continuing interplaybetween science and popular culture however the operation of digital effectsin cinema today cannot be grasped as much beyond spectacle or display notas the occasion for the emergence of a new aesthetic of realism Digital toolsemulate properties of human vision as well as the camerarsquos customary way ofseeing things In this regard the application of digital tools continues a cen-turies-old tradition of analogizing camera and eye In 1868 Helmholtz wroteldquoRegarded as an optical instrument the eye is a camera obscurardquo ([1868] 196297) Natural theology in the period extolled the perfection of human vision asan instance of divine intervention into the world evidence of Godrsquos plan forhumankind (Silverman 1993) By contrast Helmholtz explicated the numerousflaws in the eyersquos instrumentationmdashchromatic and spherical aberration lackof clarity and optical uniformity in the crystalline lens the blind spot andother gaps in the retina He impishly observed ldquoNow it is not too much to saythat if an optician wanted to sell me an instrument which had all these de-fects I should think myself quite justified in blaming his carelessness in thestrongest terms and giving him back his instrumentrdquo (110) But he went on topoint out that binocular vision enables each eye to compensate for the defi-ciencies in the other providing a means of rectifying these flaws and that theeyersquos speed was superior to that of a camera

As Nicholas Wade and Stanley Finger (2001 1158) write ldquoThe overarchinganalogy that has been applied to the eye is that of the cameramdashboth devicesbeing capable of focusing on objects at variable distancesrdquo The principles of

T H R O U G H T H E L O O K I N G G L A S S 2 5

the camera obscuramdasha dark chamber admitting a smallamount of light to produce an upside down and reversed im-age of the scene on a flat surface or object outside the cham-ber (the image is upside down and reversed because the lightrays cross as they pass through the hole)mdashwere known in theeleventh century to the Islamic philosopher and scientist Ibnal-Haytham and were subsequently widely studied by artistswho used it as a device for tracing images and by scientistsseeking to understand optics and the eye (Figure 3) Leonardoda Vinci compared the eye with a camera obscura and exper-imented with the device proposing the use of a translucentscreen for tracing that would correct image reversal and elim-inate the problem of the observerrsquos head being in the path ofthe light Da Vinci though could not reconcile the upsidedown image captured by the camera obscura with the phe-nomenally correct perspectives supplied by human visionThe Venetian patrician Daniele Barbaro in 1568 used a convexlens and varying aperture sizes in a camera obscura to pro-duce sharpened images on a sheet of paper

Lenses were rapidly applied to the camera obscura andindeed mirrors and lenses provided vital aids to the scientificstudy of vision and assisted in the development of what Mar-tin Kemp has termed the science of art The inventor of linearperspective Filippo Brunelleschi used a peep-hole and mirrordevice to heighten the illusion of depth in a painting he madeof the Baptistry of St John The viewer looked through a holein a wooden panel at a mirror opposite that reflected the im-age of the painting from the other side of the panel By elim-

inating the problem that retinal disparity introduces into the perception ofdepth on a two-dimensional surface Brunelleschirsquos peep-show device height-ened the illusion of depth perspective in the image He even used burnishedsilver on a part of the mirror to make sky and clouds more luminescent (Kemp1990 13) By analyzing the perspective geometry in Vermeerrsquos paintingsPhillip Steadman (2002) has conclusively demonstrated that Vermeer used acamera obscura to produce a series of portraits set in the same room of ahouse or studio Artist David Hockney (2001 12) has argued that ldquofrom theearly fifteenth century many Western artists used opticsmdashby which I meanmirrors and lenses (or a combination of the two)mdashto create living projectionsrdquoas tools for producing paintings and drawings

The astronomers Tycho Brahe and Johannes Kepler and Jesuit scholarChristopher Scheiner used camera obscuras to make solar observations Kep-ler stated that the camera obscura provided a safe way to view a solar eclipse

2 6 P R O J E C T I O N S

Figure 3 Diagram of camera obscuraappearing in Denis Diderot and Jean leRond DrsquoAlembertrsquos Encyclopeacutedie (1751)

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

T H R O U G H T H E L O O K I N G G L A S S 2 3

Figure 2 A cinema of attractionsmdashthecrash that concludesAeroplane Flightand Wreck (1910)

narrative has entailed that the field often exhibits a marked skepticism to-ward digital imaging and its aesthetic potential This suspicion has severalroots and one of the most important is the legacy of a school of highly influ-ential scholarship arguing that early cinema was a ldquocinema of attractionsrdquoless interested in narrative than in offering viewers startling visual displaysTom Gunningrsquos work on the history of early film helped to establish the cin-ema of attractions as a core idea in the field After studying features of earlysilent cinema that he took as being radically different from the style of narra-tive continuity that Hollywood helped establish he penned an influential essay ldquoThe Cinema of Attractionrdquo Objecting to what he described as thehegemony of narrative in the study of cinema he argued in favor of a viewthat stressed cinema as a medium of spectacle rather than narrative Earlyfilms emphasized spectacular views ndash attractions ndash rather than stories (Gun-ning 2000) He claimed that this function ruled cinema until around 1907when narrative became more dominant In a related essay Gunning devel-oped the notion of an ldquoaesthetic of astonishmentrdquo as the expressive outcomeof the cinema of attractions Visual tricks created by editing or in-cameramattes shots of locomotives rushing toward the camera and other abrupt orsurprising views caused viewers to vacillate ldquobetween belief and incredulityrdquo(2006a 119) He suggested that the cinema of attractions never fully disap-peared even after narrative became the dominant mode of popular cinema Instressing that it ldquoremains an essential part of popular filmmakingrdquo he con-nected it with contemporary special effects ldquoClearly in some sense recentspectacle cinema has reaffirmed its roots in stimulus and carnival rides inwhat might be called the Spielberg-Lucas-Coppola cinema of effectsrdquo (Gun-ning 2000 234 see Figure 2)

The skepticism among film scholars toward digital effects is partly a man-ifestation of the legacy of an anti-narrative spectacle-based conception ofearly cinema and its proliferation in the field as a persuasive template for con-ceptual analysis It was compatible with an abiding suspicion in film theoryabout the truthfulness of images and about the cameras and the projectorsthat produced them an outlook reinforced by the digital turn taken by con-temporary film Especially in its ideological and psychoanalytic componentspost-classical film theory regarded photographic images as a repressive legacyof Renaissance perspective fooling viewers with illusions of the real In ldquoMa-chines of the Visiblerdquo for example Jean-Louis Comolli (1980 124) decried theeffects of photographic realism which he attributed to an ideological forma-tion that he termed ldquothe ideology of the visiblerdquo a belief system linked to therationalism of Western culture under which the visible world is perceived asbeing centered by and organized through the gaze of the observer Realism incinema is a code he maintained a cultural practice a deceptive one that is accepted by spectators who wish to be fooled and who therefore oscillate between belief and disbelief ldquoanalogy in the cinema is a deception a lie a fiction that must be straddledmdashin disavowing knowing but not wanting toknowrdquo (126)

Comollirsquos approach manifests what Malcolm Turvey (2008) has identifiedas the lure of visual skepticism Turvey points out that ldquoa distrust of human vision has played a foundational role in film theoryrdquo (99) He elaborates ldquoIt isa general systematic doubt about normal human vision a distrust of every-day sight It is a belief that the standard exercise of the visual faculty is not tobe trusted in some significant respect because it possesses one or moreflawsrdquo (101) From this standpoint observersmdashpeople viewing paintings filmsor other objects of visual art and designmdashare socially constructed and thephysics of light the optics of lenses and the perceptual mechanisms of hu-man vision are less important than the discursive practices of a social forma-tion As Jonathan Crary (1992 6) writes ldquoIf it can be said there is an observerspecific to the nineteenth century or to any period it is only as an effect of anirreducibly heterogeneous system of discursive social technological and in-stitutional relations There is no observing subject prior to this continuallyshifting fieldrdquo According to this view there is no science of vision no physiol-ogy of sight indeed no scientific basis for representational pictures apartfrom their status as cultural constructions Crary writes ldquoWhether perceptionor vision actually change is irrelevant for they have no autonomous historyWhat changes are the plural forces and rules composing the field in whichperception occursrdquo (6) The consequence of such a view has been a hermeneu-tics of suspicion (to borrow Paul Riceourrsquos terminology for the analytic tradi-tion that Freud developed) applied to traditions of image-making and theinstruments that produce them a mistrust of vision and of the arts that play

2 4 P R O J E C T I O N S

to its characteristics Mary Ann Doane (2002 72) writes that discussions of thepersistence of vision in relation to moving images tend to invoke ldquoan insistentvocabulary of deception and failurerdquo

The Eye as an Optical InstrumentDoane is among a group of film scholars who have emphasized the bridge be-tween art and science in early cinema Scott Curtis (2004) has written elo-quently about the relationship between the modes of vision instantiated incinematic viewing devices and in practices of medical imaging John DurhamPeters (2004) has emphasized the importance of Hermann von Helmholtzrsquosphysiological studies for the history of sound film These and other scholarshave shown that todayrsquos digital culture is as Lauren Rabinovitz and AbrahamGeil (2004 13) point out a phenomenon ldquolarger and older than the informa-tion agerdquo Indeed cameras and other optical devices were important compo-nents used in the scientific study of vision by artists and natural philosophersand subsequently by experimental psychologists As Lisa Cartwright (1995 3)notes ldquomany of the techniques and instruments that contributed to theemergence of cinema were designed and used by scientistsrdquo But she pointsout in film studies ldquothe historical narrative quickly shifts however from sci-ence to popular culturerdquo Without an emphasis on the continuing interplaybetween science and popular culture however the operation of digital effectsin cinema today cannot be grasped as much beyond spectacle or display notas the occasion for the emergence of a new aesthetic of realism Digital toolsemulate properties of human vision as well as the camerarsquos customary way ofseeing things In this regard the application of digital tools continues a cen-turies-old tradition of analogizing camera and eye In 1868 Helmholtz wroteldquoRegarded as an optical instrument the eye is a camera obscurardquo ([1868] 196297) Natural theology in the period extolled the perfection of human vision asan instance of divine intervention into the world evidence of Godrsquos plan forhumankind (Silverman 1993) By contrast Helmholtz explicated the numerousflaws in the eyersquos instrumentationmdashchromatic and spherical aberration lackof clarity and optical uniformity in the crystalline lens the blind spot andother gaps in the retina He impishly observed ldquoNow it is not too much to saythat if an optician wanted to sell me an instrument which had all these de-fects I should think myself quite justified in blaming his carelessness in thestrongest terms and giving him back his instrumentrdquo (110) But he went on topoint out that binocular vision enables each eye to compensate for the defi-ciencies in the other providing a means of rectifying these flaws and that theeyersquos speed was superior to that of a camera

As Nicholas Wade and Stanley Finger (2001 1158) write ldquoThe overarchinganalogy that has been applied to the eye is that of the cameramdashboth devicesbeing capable of focusing on objects at variable distancesrdquo The principles of

T H R O U G H T H E L O O K I N G G L A S S 2 5

the camera obscuramdasha dark chamber admitting a smallamount of light to produce an upside down and reversed im-age of the scene on a flat surface or object outside the cham-ber (the image is upside down and reversed because the lightrays cross as they pass through the hole)mdashwere known in theeleventh century to the Islamic philosopher and scientist Ibnal-Haytham and were subsequently widely studied by artistswho used it as a device for tracing images and by scientistsseeking to understand optics and the eye (Figure 3) Leonardoda Vinci compared the eye with a camera obscura and exper-imented with the device proposing the use of a translucentscreen for tracing that would correct image reversal and elim-inate the problem of the observerrsquos head being in the path ofthe light Da Vinci though could not reconcile the upsidedown image captured by the camera obscura with the phe-nomenally correct perspectives supplied by human visionThe Venetian patrician Daniele Barbaro in 1568 used a convexlens and varying aperture sizes in a camera obscura to pro-duce sharpened images on a sheet of paper

Lenses were rapidly applied to the camera obscura andindeed mirrors and lenses provided vital aids to the scientificstudy of vision and assisted in the development of what Mar-tin Kemp has termed the science of art The inventor of linearperspective Filippo Brunelleschi used a peep-hole and mirrordevice to heighten the illusion of depth in a painting he madeof the Baptistry of St John The viewer looked through a holein a wooden panel at a mirror opposite that reflected the im-age of the painting from the other side of the panel By elim-

inating the problem that retinal disparity introduces into the perception ofdepth on a two-dimensional surface Brunelleschirsquos peep-show device height-ened the illusion of depth perspective in the image He even used burnishedsilver on a part of the mirror to make sky and clouds more luminescent (Kemp1990 13) By analyzing the perspective geometry in Vermeerrsquos paintingsPhillip Steadman (2002) has conclusively demonstrated that Vermeer used acamera obscura to produce a series of portraits set in the same room of ahouse or studio Artist David Hockney (2001 12) has argued that ldquofrom theearly fifteenth century many Western artists used opticsmdashby which I meanmirrors and lenses (or a combination of the two)mdashto create living projectionsrdquoas tools for producing paintings and drawings

The astronomers Tycho Brahe and Johannes Kepler and Jesuit scholarChristopher Scheiner used camera obscuras to make solar observations Kep-ler stated that the camera obscura provided a safe way to view a solar eclipse

2 6 P R O J E C T I O N S

Figure 3 Diagram of camera obscuraappearing in Denis Diderot and Jean leRond DrsquoAlembertrsquos Encyclopeacutedie (1751)

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

The skepticism among film scholars toward digital effects is partly a man-ifestation of the legacy of an anti-narrative spectacle-based conception ofearly cinema and its proliferation in the field as a persuasive template for con-ceptual analysis It was compatible with an abiding suspicion in film theoryabout the truthfulness of images and about the cameras and the projectorsthat produced them an outlook reinforced by the digital turn taken by con-temporary film Especially in its ideological and psychoanalytic componentspost-classical film theory regarded photographic images as a repressive legacyof Renaissance perspective fooling viewers with illusions of the real In ldquoMa-chines of the Visiblerdquo for example Jean-Louis Comolli (1980 124) decried theeffects of photographic realism which he attributed to an ideological forma-tion that he termed ldquothe ideology of the visiblerdquo a belief system linked to therationalism of Western culture under which the visible world is perceived asbeing centered by and organized through the gaze of the observer Realism incinema is a code he maintained a cultural practice a deceptive one that is accepted by spectators who wish to be fooled and who therefore oscillate between belief and disbelief ldquoanalogy in the cinema is a deception a lie a fiction that must be straddledmdashin disavowing knowing but not wanting toknowrdquo (126)

Comollirsquos approach manifests what Malcolm Turvey (2008) has identifiedas the lure of visual skepticism Turvey points out that ldquoa distrust of human vision has played a foundational role in film theoryrdquo (99) He elaborates ldquoIt isa general systematic doubt about normal human vision a distrust of every-day sight It is a belief that the standard exercise of the visual faculty is not tobe trusted in some significant respect because it possesses one or moreflawsrdquo (101) From this standpoint observersmdashpeople viewing paintings filmsor other objects of visual art and designmdashare socially constructed and thephysics of light the optics of lenses and the perceptual mechanisms of hu-man vision are less important than the discursive practices of a social forma-tion As Jonathan Crary (1992 6) writes ldquoIf it can be said there is an observerspecific to the nineteenth century or to any period it is only as an effect of anirreducibly heterogeneous system of discursive social technological and in-stitutional relations There is no observing subject prior to this continuallyshifting fieldrdquo According to this view there is no science of vision no physiol-ogy of sight indeed no scientific basis for representational pictures apartfrom their status as cultural constructions Crary writes ldquoWhether perceptionor vision actually change is irrelevant for they have no autonomous historyWhat changes are the plural forces and rules composing the field in whichperception occursrdquo (6) The consequence of such a view has been a hermeneu-tics of suspicion (to borrow Paul Riceourrsquos terminology for the analytic tradi-tion that Freud developed) applied to traditions of image-making and theinstruments that produce them a mistrust of vision and of the arts that play

2 4 P R O J E C T I O N S

to its characteristics Mary Ann Doane (2002 72) writes that discussions of thepersistence of vision in relation to moving images tend to invoke ldquoan insistentvocabulary of deception and failurerdquo

The Eye as an Optical InstrumentDoane is among a group of film scholars who have emphasized the bridge be-tween art and science in early cinema Scott Curtis (2004) has written elo-quently about the relationship between the modes of vision instantiated incinematic viewing devices and in practices of medical imaging John DurhamPeters (2004) has emphasized the importance of Hermann von Helmholtzrsquosphysiological studies for the history of sound film These and other scholarshave shown that todayrsquos digital culture is as Lauren Rabinovitz and AbrahamGeil (2004 13) point out a phenomenon ldquolarger and older than the informa-tion agerdquo Indeed cameras and other optical devices were important compo-nents used in the scientific study of vision by artists and natural philosophersand subsequently by experimental psychologists As Lisa Cartwright (1995 3)notes ldquomany of the techniques and instruments that contributed to theemergence of cinema were designed and used by scientistsrdquo But she pointsout in film studies ldquothe historical narrative quickly shifts however from sci-ence to popular culturerdquo Without an emphasis on the continuing interplaybetween science and popular culture however the operation of digital effectsin cinema today cannot be grasped as much beyond spectacle or display notas the occasion for the emergence of a new aesthetic of realism Digital toolsemulate properties of human vision as well as the camerarsquos customary way ofseeing things In this regard the application of digital tools continues a cen-turies-old tradition of analogizing camera and eye In 1868 Helmholtz wroteldquoRegarded as an optical instrument the eye is a camera obscurardquo ([1868] 196297) Natural theology in the period extolled the perfection of human vision asan instance of divine intervention into the world evidence of Godrsquos plan forhumankind (Silverman 1993) By contrast Helmholtz explicated the numerousflaws in the eyersquos instrumentationmdashchromatic and spherical aberration lackof clarity and optical uniformity in the crystalline lens the blind spot andother gaps in the retina He impishly observed ldquoNow it is not too much to saythat if an optician wanted to sell me an instrument which had all these de-fects I should think myself quite justified in blaming his carelessness in thestrongest terms and giving him back his instrumentrdquo (110) But he went on topoint out that binocular vision enables each eye to compensate for the defi-ciencies in the other providing a means of rectifying these flaws and that theeyersquos speed was superior to that of a camera

As Nicholas Wade and Stanley Finger (2001 1158) write ldquoThe overarchinganalogy that has been applied to the eye is that of the cameramdashboth devicesbeing capable of focusing on objects at variable distancesrdquo The principles of

T H R O U G H T H E L O O K I N G G L A S S 2 5

the camera obscuramdasha dark chamber admitting a smallamount of light to produce an upside down and reversed im-age of the scene on a flat surface or object outside the cham-ber (the image is upside down and reversed because the lightrays cross as they pass through the hole)mdashwere known in theeleventh century to the Islamic philosopher and scientist Ibnal-Haytham and were subsequently widely studied by artistswho used it as a device for tracing images and by scientistsseeking to understand optics and the eye (Figure 3) Leonardoda Vinci compared the eye with a camera obscura and exper-imented with the device proposing the use of a translucentscreen for tracing that would correct image reversal and elim-inate the problem of the observerrsquos head being in the path ofthe light Da Vinci though could not reconcile the upsidedown image captured by the camera obscura with the phe-nomenally correct perspectives supplied by human visionThe Venetian patrician Daniele Barbaro in 1568 used a convexlens and varying aperture sizes in a camera obscura to pro-duce sharpened images on a sheet of paper

Lenses were rapidly applied to the camera obscura andindeed mirrors and lenses provided vital aids to the scientificstudy of vision and assisted in the development of what Mar-tin Kemp has termed the science of art The inventor of linearperspective Filippo Brunelleschi used a peep-hole and mirrordevice to heighten the illusion of depth in a painting he madeof the Baptistry of St John The viewer looked through a holein a wooden panel at a mirror opposite that reflected the im-age of the painting from the other side of the panel By elim-

inating the problem that retinal disparity introduces into the perception ofdepth on a two-dimensional surface Brunelleschirsquos peep-show device height-ened the illusion of depth perspective in the image He even used burnishedsilver on a part of the mirror to make sky and clouds more luminescent (Kemp1990 13) By analyzing the perspective geometry in Vermeerrsquos paintingsPhillip Steadman (2002) has conclusively demonstrated that Vermeer used acamera obscura to produce a series of portraits set in the same room of ahouse or studio Artist David Hockney (2001 12) has argued that ldquofrom theearly fifteenth century many Western artists used opticsmdashby which I meanmirrors and lenses (or a combination of the two)mdashto create living projectionsrdquoas tools for producing paintings and drawings

The astronomers Tycho Brahe and Johannes Kepler and Jesuit scholarChristopher Scheiner used camera obscuras to make solar observations Kep-ler stated that the camera obscura provided a safe way to view a solar eclipse

2 6 P R O J E C T I O N S

Figure 3 Diagram of camera obscuraappearing in Denis Diderot and Jean leRond DrsquoAlembertrsquos Encyclopeacutedie (1751)

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

to its characteristics Mary Ann Doane (2002 72) writes that discussions of thepersistence of vision in relation to moving images tend to invoke ldquoan insistentvocabulary of deception and failurerdquo

The Eye as an Optical InstrumentDoane is among a group of film scholars who have emphasized the bridge be-tween art and science in early cinema Scott Curtis (2004) has written elo-quently about the relationship between the modes of vision instantiated incinematic viewing devices and in practices of medical imaging John DurhamPeters (2004) has emphasized the importance of Hermann von Helmholtzrsquosphysiological studies for the history of sound film These and other scholarshave shown that todayrsquos digital culture is as Lauren Rabinovitz and AbrahamGeil (2004 13) point out a phenomenon ldquolarger and older than the informa-tion agerdquo Indeed cameras and other optical devices were important compo-nents used in the scientific study of vision by artists and natural philosophersand subsequently by experimental psychologists As Lisa Cartwright (1995 3)notes ldquomany of the techniques and instruments that contributed to theemergence of cinema were designed and used by scientistsrdquo But she pointsout in film studies ldquothe historical narrative quickly shifts however from sci-ence to popular culturerdquo Without an emphasis on the continuing interplaybetween science and popular culture however the operation of digital effectsin cinema today cannot be grasped as much beyond spectacle or display notas the occasion for the emergence of a new aesthetic of realism Digital toolsemulate properties of human vision as well as the camerarsquos customary way ofseeing things In this regard the application of digital tools continues a cen-turies-old tradition of analogizing camera and eye In 1868 Helmholtz wroteldquoRegarded as an optical instrument the eye is a camera obscurardquo ([1868] 196297) Natural theology in the period extolled the perfection of human vision asan instance of divine intervention into the world evidence of Godrsquos plan forhumankind (Silverman 1993) By contrast Helmholtz explicated the numerousflaws in the eyersquos instrumentationmdashchromatic and spherical aberration lackof clarity and optical uniformity in the crystalline lens the blind spot andother gaps in the retina He impishly observed ldquoNow it is not too much to saythat if an optician wanted to sell me an instrument which had all these de-fects I should think myself quite justified in blaming his carelessness in thestrongest terms and giving him back his instrumentrdquo (110) But he went on topoint out that binocular vision enables each eye to compensate for the defi-ciencies in the other providing a means of rectifying these flaws and that theeyersquos speed was superior to that of a camera

As Nicholas Wade and Stanley Finger (2001 1158) write ldquoThe overarchinganalogy that has been applied to the eye is that of the cameramdashboth devicesbeing capable of focusing on objects at variable distancesrdquo The principles of

T H R O U G H T H E L O O K I N G G L A S S 2 5

the camera obscuramdasha dark chamber admitting a smallamount of light to produce an upside down and reversed im-age of the scene on a flat surface or object outside the cham-ber (the image is upside down and reversed because the lightrays cross as they pass through the hole)mdashwere known in theeleventh century to the Islamic philosopher and scientist Ibnal-Haytham and were subsequently widely studied by artistswho used it as a device for tracing images and by scientistsseeking to understand optics and the eye (Figure 3) Leonardoda Vinci compared the eye with a camera obscura and exper-imented with the device proposing the use of a translucentscreen for tracing that would correct image reversal and elim-inate the problem of the observerrsquos head being in the path ofthe light Da Vinci though could not reconcile the upsidedown image captured by the camera obscura with the phe-nomenally correct perspectives supplied by human visionThe Venetian patrician Daniele Barbaro in 1568 used a convexlens and varying aperture sizes in a camera obscura to pro-duce sharpened images on a sheet of paper

Lenses were rapidly applied to the camera obscura andindeed mirrors and lenses provided vital aids to the scientificstudy of vision and assisted in the development of what Mar-tin Kemp has termed the science of art The inventor of linearperspective Filippo Brunelleschi used a peep-hole and mirrordevice to heighten the illusion of depth in a painting he madeof the Baptistry of St John The viewer looked through a holein a wooden panel at a mirror opposite that reflected the im-age of the painting from the other side of the panel By elim-

inating the problem that retinal disparity introduces into the perception ofdepth on a two-dimensional surface Brunelleschirsquos peep-show device height-ened the illusion of depth perspective in the image He even used burnishedsilver on a part of the mirror to make sky and clouds more luminescent (Kemp1990 13) By analyzing the perspective geometry in Vermeerrsquos paintingsPhillip Steadman (2002) has conclusively demonstrated that Vermeer used acamera obscura to produce a series of portraits set in the same room of ahouse or studio Artist David Hockney (2001 12) has argued that ldquofrom theearly fifteenth century many Western artists used opticsmdashby which I meanmirrors and lenses (or a combination of the two)mdashto create living projectionsrdquoas tools for producing paintings and drawings

The astronomers Tycho Brahe and Johannes Kepler and Jesuit scholarChristopher Scheiner used camera obscuras to make solar observations Kep-ler stated that the camera obscura provided a safe way to view a solar eclipse

2 6 P R O J E C T I O N S

Figure 3 Diagram of camera obscuraappearing in Denis Diderot and Jean leRond DrsquoAlembertrsquos Encyclopeacutedie (1751)

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

the camera obscuramdasha dark chamber admitting a smallamount of light to produce an upside down and reversed im-age of the scene on a flat surface or object outside the cham-ber (the image is upside down and reversed because the lightrays cross as they pass through the hole)mdashwere known in theeleventh century to the Islamic philosopher and scientist Ibnal-Haytham and were subsequently widely studied by artistswho used it as a device for tracing images and by scientistsseeking to understand optics and the eye (Figure 3) Leonardoda Vinci compared the eye with a camera obscura and exper-imented with the device proposing the use of a translucentscreen for tracing that would correct image reversal and elim-inate the problem of the observerrsquos head being in the path ofthe light Da Vinci though could not reconcile the upsidedown image captured by the camera obscura with the phe-nomenally correct perspectives supplied by human visionThe Venetian patrician Daniele Barbaro in 1568 used a convexlens and varying aperture sizes in a camera obscura to pro-duce sharpened images on a sheet of paper

Lenses were rapidly applied to the camera obscura andindeed mirrors and lenses provided vital aids to the scientificstudy of vision and assisted in the development of what Mar-tin Kemp has termed the science of art The inventor of linearperspective Filippo Brunelleschi used a peep-hole and mirrordevice to heighten the illusion of depth in a painting he madeof the Baptistry of St John The viewer looked through a holein a wooden panel at a mirror opposite that reflected the im-age of the painting from the other side of the panel By elim-

inating the problem that retinal disparity introduces into the perception ofdepth on a two-dimensional surface Brunelleschirsquos peep-show device height-ened the illusion of depth perspective in the image He even used burnishedsilver on a part of the mirror to make sky and clouds more luminescent (Kemp1990 13) By analyzing the perspective geometry in Vermeerrsquos paintingsPhillip Steadman (2002) has conclusively demonstrated that Vermeer used acamera obscura to produce a series of portraits set in the same room of ahouse or studio Artist David Hockney (2001 12) has argued that ldquofrom theearly fifteenth century many Western artists used opticsmdashby which I meanmirrors and lenses (or a combination of the two)mdashto create living projectionsrdquoas tools for producing paintings and drawings

The astronomers Tycho Brahe and Johannes Kepler and Jesuit scholarChristopher Scheiner used camera obscuras to make solar observations Kep-ler stated that the camera obscura provided a safe way to view a solar eclipse

2 6 P R O J E C T I O N S

Figure 3 Diagram of camera obscuraappearing in Denis Diderot and Jean leRond DrsquoAlembertrsquos Encyclopeacutedie (1751)

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

Kepler proposed a system employing two convex lenses to correct the in-verted image and in 1611 he published Dioptrice a seminal study of optics thatemerged from his use of a telescope (ldquoDioptricsrdquo was the terminology in usefor the study of refraction) He also proposed an account of retinal visionwhereby an image was focused on the retina as if produced by a camera ob-scura on a sheet of paper and was then transmitted to the brain and its visualfaculty Scheiner produced diagrams of the eye and observed upside downretinal images on the excised eyes of animals He created a portable cameraobscura called the Pantograph for making drawings of solar phenomenaKepler too designed a portable camera obscura using a tent for the enclosureThe mathematician linguist and experimental scientist Athanasius Kircheralso used a camera obscura for study of the sun and he designed a picturewheel projection device and a magic lantern projection system to exploit theproperties of image formation in the eye that he had previously studied In de-scribing this history Nicholas Wade (2004 105) wrote that ldquothe photographiccamera enabled artists to capture scenes in perspective with comparativeease whereas scientists could consider the eye as a similar optical instru-mentrdquo Chromatic aberration in telescopes (color separation due to differ-ences in the way a lens refracted light of varying wavelengths) and methodsof correcting it helped to advance astronomy and pointed away from a cor-puscular theory of light and toward a wave theory The astronomer ChristianHuygens an early proponent of a wave theory used the camera obscura andPhilip Steadman speculates that it was Huygensrsquos father Constantijn who in-troduced Vermeer to optics

Lenses and mirrors as well as the optical devices built from them provideda technical foundation upon which the study of vision could proceed Thisconjunction between image-making devices and science gives us a rather dif-ferent inflection to cinemarsquos historical preconditions than the familiar modelthat privileges boulevard amusements and fairground attractions in whichoptical devices like the thaumatrope the phenakistoscope and the zoetropebecome toys diversions offered to a restless public keenly interested in visualentertainments Historian David Cook (2004 1) describes them as ldquosimple op-tical devices used for entertainmentrdquo and Keith Griffiths (2003 16) writes thatthey helped create ldquophantasmagoric illusions and performances (an aestheticof the supernatural) for the visual entertainment of the middle classes Theseparlor room and entertainment hall projections helped create the public ap-petite for the range of entertainment genres that would soon encompassmost of the cinema and television of the futurerdquo

By contrast psychologist Nicholas Wade whose scholarship focuses on thenatural history of vision proposes that these devices be regarded as ldquophilo-sophical toysrdquomdasha term commonly employed in the nineteenth centurymdashbe-cause they served dual interests ldquoPhilosophical instruments like microscopes

T H R O U G H T H E L O O K I N G G L A S S 2 7

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

were used to examine natural phenomena but philosophical toys served thedual function of scientific investigation and popular amusementrdquo (Wade2004 102) Wade suggests that the camera obscura was the first philosophi-cal toy because of its applications to both art and science Scientists and nat-ural philosophers invented these optical devices to aid their inquiries intosuch visual phenomena as persistence of vision stroboscopic motion andbinocular depth perception Sir Charles Wheatstone professor of Experimen-tal Philosophy at Kingrsquos College defined philosophical toys as devices in-tended to illustrate and to popularize the principles of science ldquoTheapplication of the principles of science to ornamental and amusing purposescontributes in a great degree to render them extensively popular for the ex-hibition of striking experiments induces the observer to investigate theircauses with additional interest and enables him more permanently to re-member their effectsrdquo (Wheatstone [1827] 1983 205) The devices originatedat the hands not of carnival barkers but credentialed experimental philoso-phers The optical devices helped advance experimental inquiries into visionAs Wade (2004 102) emphasizes ldquothe development of visual science was asdependent on these devices as biology had been upon the microscoperdquo

Although visual persistence was an optical phenomena that had beennoted for centuries the optical toys sharpened its study and its quantificationand via stroboscopes connected it with the perception of apparent motionHis own investigations and experiments and those carried out by Newtoninto color perception using prisms stimulated David Brewster to invent thekaleidoscope in 1816 He wrote ldquowhen I discovered the development of thecomplementary colours by the successive reflections of polarized light be-tween two [glass] plates of gold and silver the effects of the Kaleidoscope were again forced upon my noticerdquo ([1819] 1983 202) Influenced by Brewsterrsquosdevice and intending to illustrate visual persistence Wheatstone created asonic kaleidoscope in 1827 that he called the kaleidophone He attached silverglass beads to rods which when struck made the beads vibrate and their re-flected light to trace pleasing abstract figures in the air Wheatstone wrotethat his objective was to subject ldquoto ocular demonstration the orbits or pathsdescribed by the points of greatest excursion in vibrating rods The entiretrack of each orbit is rendered simultaneously visible by causing it to be delin-eated by a brilliantly luminous point and the figure being completed in lesstime than the duration of the visual impression the whole orbit appears as acontinuous line of lightrdquo ([1827)] 1983 206) That same year John Paris a physi-cian devised the thaumatrope a disk with drawings on each side whichwhen whirled caused the drawings to be seen as one (eg a rat inside a cage)Paris intended that the device serve a teaching function illustrating the clas-sics Descriptions published in the early 1820s of stroboscopic illusions pro-duced by counter-rotating cogwheels or by the spoke wheels of a carriage

2 8 P R O J E C T I O N S

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

when viewed through a Venetian blind led to papers by physician Peter MarkRoget and chemist and physicist Michael Farraday analyzing the phenomenaquantitatively and to the invention of several varieties of stroboscopic disksFarradaymdashWheatstonersquos friend who was influenced by the latterrsquos interest invisual persistencemdashpublished a paper in 1831 which led to design applicationsIn 1833 after reading Farradayrsquos paper the Belgian scientist Joseph Plateau in-vented the phenakistoscope and Simon Stampfer a professor of geometry inVienna created the stroboscopic disk (Figure 4) Similar devices the disks helda series of drawings on one side separated by slits When the disk was held be-fore a mirror and rotated and when viewed through the slits the drawingsappeared to move Roget claimed that he had invented a similar device a fewyears earlier In 1834 in the daedaleum (akathe zoetrope) William Horner placed draw-ings on a horizontal wheel rather than a ver-tical one making it possible for severalpeople to view the illusion at once Scientistsand natural philosophers studied the opticalphenomena produced by the disks andnoted the velocity and degree of light thatwere needed to produce the illusion

In addition to visual persistence Wheat-stone was intrigued by more general ques-tions about space and depth perception Heinvented the stereoscope in 1832 after notingthat retinal disparities increase as the eyesconverge to focus on an object very near athand (Brewster invented a lenticular stereo-scope a few years later) Wheatstone won-dered if a similar experience of depth percep-tion could be produced using plane images instead of three-dimensional ob-jects ldquoWhat would be the visual effect of simultaneously presenting to eacheye instead of the object itself its projection on a plane surface as it appearsto that eyerdquo ([1819] 1983 67) He constructed the mirrored stereoscope in or-der to pursue a series of experiments into binocular vision that established forthe first time its role in depth perception (Figure 5) Visual scientists beforeWheatstone had noted the existence of retinal disparity but it had not beenconnected with depth perception The stereoscope enabled Wheatstone to in-vestigate and demonstrate this connection His device used mirrors to reflectpaired line drawings of geometric forms Using the line drawings eliminatedthe presence of monocular depth cues which could have confounded the results He mounted the mirrors onto adjustable arms that enabled him to introduce variations into retinal size and retinal disparity and degrees of

T H R O U G H T H E L O O K I N G G L A S S 2 9

Figure 4 Phenakistoscope circa 1833

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

convergence and accommodation as elicited by the drawings He thus wasable to study these responses as separate variables After William Fox Talbotrsquosnegative-to-positive photographic process was invented Wheatstone hadstereoscopic daguerreotypes made for the device

The PseudoscopeThe stereoscope also pointed toward a new and strange optical domain tak-ing its observer through a looking glass into a disorienting world Wheatstonevaried the device to create what he called the pseudoscope which producedconversions of relief and depth If the pictures in the stereoscope were trans-posed from one eye to the other reversing the manner in which they weremeant to be viewed or inverted in other ways an impossible world appearedThe interior of a teacup becomes a solid convex body A globe of the earth be-comes a concave hemisphere ldquoA bust regarded in front becomes a deep hol-low mask A framed picture hanging against a wall appears as if imbeddedin a cavity made in the wallrdquo A flowering shrub in front of a hedge appears be-hind it ldquoA tree standing outside a window may be brought visible within theroom in which the observer is standingrdquo These strange perceptions were as ofanother world operating according to different physical laws Wheatstonewrote ldquoWith the pseudoscope we have a glance as it were into another visi-ble world in which external objects and our internal perceptions have nolonger their habitual relation with each otherrdquo (1838 67) This fascination withnovel visual experiences held a major appeal for the computer scientists whowould write the algorithms that produced digital simulations of the phenom-

3 0 P R O J E C T I O N S

Figure 5 Charles Wheatstonersquos mirrored stereoscope

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

enal world and the new vistas offered to audiences by such films as JurassicPark Coraline and Avatar furnish much of their appeal Indeed as Anne Fried-berg (2006 60) notes this ldquofascination with virtualityrdquo with visual approxi-mations of the real has been exerted by optical devices from the cameraobscura on as a fundamental allure the pleasures derived from extending vi-sion in novel ways

Whereas the pseudoscope pointed toward new aesthetic experiences thestereoscope placed the scientific investigation of vision onto firm groundWade (2004 116) notes that the stereoscope ldquoperhaps more than any other in-strument ushered in the era of experimentation in visionrdquo It pointed towardthe cognitive components that operate in visual perception and also to thedifferences between eye and camera Accommodationmdashthe eyersquos ability toshift focus between near and farmdashis possible because the curvature of its lenschanges becoming more extreme with nearer objects but a camera lens doesnot change its shape Wheatstonersquos stereoscope could evoke accommodationresponses from viewers according to changes in the positioning of its mir-rored arms and this offered one challenge to the camera-eye analogyHelmholtz who used an ophthalmometer to study more precise changes inaccommodation remarked on the differences ldquoA photographic camera cannever show near and distant objects clearly at once nor can the eye but theeye shows them so rapidly one after another that most people who have notthought of how they see do not know that there is any change at allrdquo (cited inWade and Finger 2001 1172) Mechanical devices such as the camera or earlierthe camera obscura could change focus on different parts of a scene butwithout the eyersquos swiftness or suppleness

Wheatstonersquos stereoscope made important contributions to the empiricaltheory of vision associated with Helmholtz in distinction to nativist ap-proaches which held that visual skills such as depth perception are innateand are not subject to learning Helmholtz studied the images produced byWheatstonersquos stereoscope and used them to argue forcefully for the role ofmind in vision People do not see their retinal images he maintained They donot perceive a world that is upside down as are retinal images Moreover howare the different retinal images combined to produce a single visual field seenin depth He maintained that depth perception is a psychological rather thana physiological process that vision involves an interpretive act rather than astrictly physical one ldquoThe combination of these two sensations into the singlepicture of the external world of which we are conscious in ordinary vision isnot produced by any anatomical mechanism of sensation but by a mentalactrdquo (Helmholtz [1868] 1962 173) This approach construing vision as an inter-pretive cognitively active process contrasts sharply with the rather moremechanistic views found in the tradition of visual skepticism enunciated infilm and cultural studies The Helmholtzian viewer cognitively constructs the

T H R O U G H T H E L O O K I N G G L A S S 3 1

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

visual field the viewer implicated in a ldquovisual regimerdquo is positioned emplacedand entrapped by visual sensations over which she exerts little conscious con-trol Wheatstonersquos stereoscope enabled Helmholtz and others to deepen theirunderstanding of the perceptual processes involved in vision The relatively in-timate circles through which the art and science of philosophical toys andtheir associated inquires were pursued is illustrated by Helmholtzrsquos atten-dance at an 1881 demonstration by Eadweard Muybridge of his zoopraxiscopea projecting phenakistoscope that he used to show his photographs of horsesin motion (Braun 1992 52) The event was held at the home of Etienne-JulesMarey a physiologist who constructed numerous instruments includingcameras and projectors for measuring animal and human motion (Figure 6)Marey had invited Muybridge to show his device to the leading scientists ofthe time

Realism and Digital Visual EffectsThe upsurge of vision research in the nineteenth century was an essentialcondition for the invention of cinema the boulevard amusements and fair-

ground attractions that often are described as themediumrsquos roots should be ranked alongside the develop-ing science of visual perception But as Lisa Cartwrightpoints out ldquoThe prehistory of the cinema is conventionallytold as a tale of early scientific experimentation marked by

a break with science around 1895 with the emergence of a popular film cul-ture and industryrdquo (1995 3) No such breach has occurred Art and science

3 2 P R O J E C T I O N S

Figure 6 Frenchphysiologist Etienne-Jules Mareyconstructed this 3D zoetrope usingsculptures of birds in flight rather than pictures

The upsurge of vision researchin the nineteenth century was

an essential condition for theinvention of cinema

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

commingled in the invention of cinema as they have continued to do in thedecades since most obviously in the digital turn that the medium has takenThe new toolbox available to filmmakers obviously enhances their abilities tocreate artificial realms evident in such films as Lord of the Rings Speed Racerand Avatar But it also provides new methods for establishing perceptual andindexical modes of realism Consider David Fincherrsquos Zodiac and The CuriousCase of Benjamin Button

Zodiac portrays efforts by police and city journalists to discover the identityof the Zodiac killer who terrorized San Francisco and surrounding eras with astring of serial killings that began in 1968 The film uses digital methods tocreate a naturalistic almost documentary-like style Fincher shot the murderscenes on many of the actual sites but these and other city locations had tobe altered significantly in order to show San Francisco as it appeared at thetime An early establishing shot for example introduces the city with an aer-ial view that sweeps from the waters of the bay to the Port Authority terminaland surrounding buildings (Figure 7) The terminal is now a shopping com-plex The area today looks more upscale than it did in the period and manybuildings have disappeared or undergone renovation Thus Fincher could notgo on location and shoot what he found if his goal was to depict San Franciscoin historically accurate terms Although the location exists it would need tobe filmed as a built environment rather than as a found environment Accord-ingly the flyover of the Port Authority terminal is an all-CG sequence consist-ing of a geometrical model of the site treated as a 3D matte painting andgiven animated effects which include CG cars traveling along a street Theshot was built from architecturally accurate information derived from cityblueprints from the period and photographs taken by a U-2 spy plane Pho-togrammetric analysis enabled the effects artists to assemble the scenersquos 3Dgeometry Photogrammetry is a method of extracting the 3D structure of ascene from 2D images in this case photographs of the Port Authority area

T H R O U G H T H E L O O K I N G G L A S S 3 3

Figure 7 Thisestablishing shotin Zodiac of SanFrancisco is an all-CG environment

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

showing what it looked like at the time Lines of sight by the cameras in thephotographs are triangulated to recover the underlying dimensional struc-ture Lighting information can be recovered as well enabling digital modelersto simulate highlights and shadowing on the buildings in ways consistentwith how light would be distributed in the actual location Paul Debevec andassociates at the University of California at Berkeley had developed algo-rithms permitting the derivation of a computational stereopsis uncoveringscene structure from only a few widely spaced photographs as well as amethod of view-dependent texturing enabling the projection of detailed 2Dinformation from the images onto a scenersquos 3D geometry (Debevec Taylorand Malik 1996) This approach and others like it furnished methods of scenebuilding that have now become quite standard in contemporary film

The photogrammetric methods the wireframe geometry built from themand the digital painting and CG animation laid atop the geometry create aphotographically convincing helicopter shot of the port area one that per-forms the traditional narrative task of establishing a location It is an auda-cious shot because everything in it is CG and because the action takes place inbright sunshine lighting that would reveal flaws rather than concealingthem But very few of the filmrsquos viewers unless they knew these productiondetails were aware they were watching a fabricated environment on screenIt is however an indexical environment in that the CG imagery was built fromarchival sources and retains their trace (Figure 8)

Other environments in the film are all CG such as a high angle shot fromatop the Golden Gate Bridge or the digital matte painting introducing Vallejoand the Napa River on the Fourth of July Many others are augmentations ofCG and live action such as the murder scene at Washington and Cherrystreets where Zodiac shoots a taxi driver and the subsequent police investi-gation at the site Fincher shot the scenes there but the intersection did notlook as it did then so he recreated much of it as CG with the actors on blue-

3 4 P R O J E C T I O N S

Figure 8 DetectiveDave Toschi (MarkRufallo) investigatesthe murder of a taxidriver at theintersection ofWashington andCherry streets Theenvironmentdepicted on screen isa composite of liveaction and CGconstructed fromphotographs andarchival documents

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

screen sets When Mark Ruffalo as Detective Dave Toschi walks along thestreet tracing Zodiacrsquos path he was filmed with a mobile bluescreen permit-ting the actor to be composited into a digital environment The intersectionwas recreated from photogrammetry based on photographs from the periodand the buildings appearing in the scene are alternately CG objects and mattepaintings Live action camera moves were tracked onto the CG environment sothat as the camera travels with Toschi along the street the correspondingmotion perspective is replicated in the surrounding digital environment

These methods enabled Fincher to visualize San Francisco in historicallyaccurate terms something that could not be done with this degree of percep-tual realism using a traditional photo-chemical approach Under that ap-proach scenes could be dressed with props and period vehicles to suggest theera but the large-scale flyovers of the port or Vallejo or the exacting visualrecreation of the environments where the killings took place would be beyondthese methods Moreover Zodiacrsquos approach exemplifies the goals of locationshooting in that the locations are reproduced reliably as constructions ratherthan as found environments The digital backlot in this case permits the film-maker to replicate historical locations from a now-distant era according to pa-rameters of indexical realism Far from undermining such realism Zodiacrsquosdigital design establishes a realist aesthetic measured from and authenti-cated by the photographic record of place in the period

The performance challenges of Benjamin Button about a character whoages backward were too challenging to be handled by the traditional solu-tions (Figure 9) As Jody Duncan (2009 72) points out a conventional ap-proach to showing a character aging across a span of story time is to have thecharacter played by different actors But no matter how skillfully executed theperformances may be or how closely the actors resemble one another view-ers feel the deception and know they are seeing different players In The Note-book (2004) for example Ryan Gosling and Rachel McAdams play young

T H R O U G H T H E L O O K I N G G L A S S 3 5

Figure 9 In sceneswhere Benjaminbecomes an old(young) man BradPitt was ldquoyouthenedrdquodigitally These shotswere not digitalhead replacementsbut alterations ofthe actorrsquos actualappearance so thathe would lookdecades younger

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

lovers who marry spend their lives together and then in their later years turnnone too convincingly into James Garner and Gena Rowlands as the charac-ters Another traditional approach portrays extreme changes in a characterrsquosage by using prosthetics and make-up as Dick Smith did when aging DustinHoffman to a centenarian in Little Big Man (1970) or Murray Abraham from ayouthful to an aged and decrepit Salieri in Amadeus (1984) No matter howbrilliant the make-upmdashand these instances are supremely accomplishedmdashviewers feel quite rightly that the actors are wearing things on their faces Vi-sual effects supervisor Eric Barba clarified why this approach does not workwell ldquoThe problem with old-age makeup is that it is additive whereas the ag-ing process is reductive You have thinner skin less musculature everything isreceding There is no way to do that 100 percent convincingly by adding pros-theticsrdquo (Duncan 2009 72)

Brad Pitt who plays Benjamin wanted to do the film only if he could playthe lifespan of the character rather than doing one or two age intervals andthen handing the character off to other actors The trick was to age Benjaminbackward from his eighties all the way to an infantrsquos appearance while still re-taining the viewerrsquos conviction that the characterrsquos many forms remain BradPitt that the characterrsquos transformations are anchored by a single actorrsquos per-formance During the filmrsquos first hour Pitt appears as a digital head replace-ment on the bodies of three different actors who are performing Benjaminrsquosphysical movements in the scenes where he is aging from his eighties to hissixties The key creative challenge was placing Pittrsquos facial performance con-vincingly in the scenes The filmmakers captured Pittrsquos facial data in severalways 3D sculpted plaster models were built from a life scan of Pittrsquos faceThese depicted Benjamin in his sixties seventies and eighties by remappingPittrsquos features to imagine how he might appear at these ages The sculpturesthen were scanned to create digital models for animating based on a large ref-erence library of Pittrsquos facial expressions This had been created using non-markerndashbased motion capture a facial contour system developed by MOVAthat employed phosphorescent make-up worn by the performer and triangu-lated in 3D space by an array of mocap cameras The makeup provided moredata points than a marker-based system In the reflective makeup Pitt mod-eled a series of micro-expressions that were broken down according to theunits specified by researcher Paul Ekman in his Facial Action Coding System(FACS)

Keyframe animation shaped and modulated the characterrsquos responses mo-ment to moment But to bring off the illusion required perfect rendering ofBenjaminrsquos skin and eyes the lighting on his face to match the environmentsthat had been lit on set and flawless motion tracking of the head replace-ment to the characterrsquos body as supplied by the three actor stand-ins Bumpspores blemishes age spots and tissue thinning were rendered using dis-

3 6 P R O J E C T I O N S

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

placement maps (a form of texture mapping that alters the shape of themodel) ray-tracing and subsurface scattering

Because eyes had always been a big problem in previous efforts to renderphotorealistic human beings a special animator was tasked solely with visual-izing Benjaminrsquos eyes Barba said ldquoWe knew if we didnrsquot get the eyes right itwouldnrsquot matter how good the rest of it looked Without the eyes it wouldnrsquotbe Brad Pitt and it wouldnrsquot be Benjamin Buttonrdquo (Duncan 2009 88) Multi-pass rendering isolated essential characteristics of life-like but aging eyesldquoEvery elementmdashthe amount of water in the eyes the different layers of theskin the red in the conjunctiva of the eyemdashwas rendered out separately forcontrol and then the compositors layered those things together again shotby shotrdquo (88)

Executing the filmrsquos 329 digital head replacement shots required precisemotion tracking to ensure that the head replacements aligned properly withthe characterrsquos spine and exactly matching the light effects to those createdon set during filming When Benjamin walks through a dimly lit corridor in aNew Orleans brothel for example the movement and the lighting of headand body had to look identical The filmmakers accomplished this by survey-ing every light source on set and every bounce card and then replicated thisinformation in a CG environment in order to light Benjaminrsquos digital face in-teractively with his environment Hard light sources on set visibly reflect offthe digital head replacement and characters moving off-camera cast shad-ows over Benjaminrsquos features The matching of specular highlights ambientocclusive shadows global illumination characteristics and color tone be-tween on-set lighting and digital animation create the perceptually convinc-ing bridges between Benjaminrsquos head and body and the acting performancesthat unite them

Benjaminrsquos face at sixty or seventy is Brad Pitt (Figure 10) The evidence ispersuasive and indexical He is an animated computer model not a photo-graphic image but this model was derived from a Lidar scan of Pittrsquos face Themodel was constructed based on that data and altering it selectively enabledthe filmmakers to change the characterrsquos age So although Brad Pitt does notphysically appear as an actor in these sequences or as a photographically de-rived image the computer-based image that we see is indexical It persuadesus that it is an age-altered version of Pitt because it carrieshis trace And if one accepts the translation of physical spaceinto binary data then one must accept that the Pitt head re-placement is physically connected to the actor as an indexThere is little difference between this example wherebyPittrsquos face is quantified and then regenerated as an image and a conventionalanalog photograph in which light furnishes the medium of translation fromthe object to the image

T H R O U G H T H E L O O K I N G G L A S S 3 7

the Pitt head replacementis physically connected to the actor as an index

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

These examples demonstrate that as Timothy Binkley (1997) has pointedout numbers invigorate pictures He stresses that digital imaging builds onthe past rather than breaking with it ldquoDespite its novelty the digital revolu-tion builds upon long-standing if sometimes misunderstood traditions inthe artsrdquo (108) He notes ldquodigital media augment rather than undermine theiranalog forbearsrdquo (112) Digital imaging also bridges art and science in waysthat have continued to nourish cinema and on which the medium has de-pended for its existence and its aesthetics When C P Snow talked about ldquothetwo cultures and the scientific revolutionrdquo in a lecture at Cambridge in 1959he expressed dismay at the failure of scientists and humanities scholars to no-tice much about what the other group did When Snow asked a scientist whatbooks he had recently read the scientist replied ldquoWell I tried Dickens oncerdquoWhen he asked literary scholars if they could describe the Second Law of Ther-modynamics he received looks of cold disdain Snow was conversant in bothworlds having trained as a scientist and spending much of his time writingnovels Were Snow to survey todayrsquos cultural landscape he might happily dis-cover precisely the convergence he wished for The conjunction of art and sci-ence that nourished cinemarsquos roots continues to enrich the medium today

Stephen Prince is a professor of cinema at Virginia Tech and the author of nu-merous books and essays on film history and criticism His most recent bookis Firestorm American Film in the Age of Terrorism (2009) He is currently com-pleting a book titled Digital Visual Effects in Cinema The Seduction of Reality

3 8 P R O J E C T I O N S

Figure 10 Brad Pittas a digital headreplacement Motiontracking andexacting digitalrecreation of on-setlighting helped toestablish thenecessary spatial and environmentalrealism

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

T H R O U G H T H E L O O K I N G G L A S S 3 9

ReferencesBinkley Timothy 1997 ldquoThe Vitality of Digital Creationrdquo Journal of Aesthetics and Art Criti-

cism 55 (2) 107ndash116Braun Marta 1992 Picturing Time The Work of Etienne-Jules Marey (1830ndash1904) Chicago

University of Chicago PressBrewster David [1819] 1983 ldquoCircumstances which Led to the Invention of the Kaleido-

scoperdquo Pp 202ndash204 in Brewster and Wheatstone on Vision ed Nicholas Wade New YorkAcademic Press

Buckland Warren 1999 ldquoBetween Science Fact and Science Fiction Spielbergrsquos Digital Di-nosaurs Possible Worlds and the New Aesthetic Realismrdquo Screen 40 (2) 177ndash192

Bukatman Scott 2003 Matters of Gravity Special Effects and Supermen in the 20th CenturyDurham Duke University Press

Cartwright Lisa 1995 Screening the Body Tracing Medicinersquos Visual Culture MinneapolisUniversity of Minnesota Press

Comolli Jean-Louis 1980 ldquoMachines of the Visiblerdquo Pp 121ndash133 in The Cinematic Apparatused Teresa de Lauretis and Stephen Heath London Macmillan

Cook David A 2004 A History of Narrative Film New York NortonCrary Jonathan 1992 Techniques of the Observer On Vision and Modernity in the Nineteenth

Century Cambridge MIT PressCurtis Scott 2004 ldquoStillMoving Digital Imaging and Medical Hermeneuticsrdquo Pp 218ndash254

in Memory Bytes History Technology and Digital Culture eds Lauren Rabinovitz andAbraham Geil Durham NC Duke University Press

Darley Andrew 2000 Visual Digital Culture Surface Play and Spectacle in New Media GamesLondon Routledge

Debevec Paul E Camillo J Taylor and Jitendra Malik 1996 ldquoModeling and Rendering Archi-tecture from Photographs A Hybrid Geometry-and-Image-Based Approachrdquo Pp 11ndash20 inProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Tech-niques New York ACM

Doane Mary Ann 2002 The Emergence of Cinematic Time Cambridge Harvard UniversityPress

Duncan Jody 2009 ldquoThe Unusual Birth of Benjamin Buttonrdquo Cinefex 116 (1) 70ndash99 118mdashmdashmdash 2010 ldquoThe Seduction of Realityrdquo Cinefex 120 (1) 68ndash146Friedberg Ann 2006 The Virtual Window Cambridge MIT PressGriffiths Keith Griffiths 2003 ldquoThe Manipulated Imagerdquo Convergence 9 (4) 12ndash26Gunning Tom 2000 ldquoThe Cinema of Attraction Early Film Its Spectator and the Avant-

Garderdquo Pp 229ndash235 in Film Theory An Introduction ed Robert Stam and Toby MillerNew York John Wiley and Sons

mdashmdashmdash 2006a ldquoAn Aesthetic of Astonishment Early Film and the (In)Credulous SpectatorrdquoPp 114ndash133 in The Cinema of Attractions Reloaded ed Wanda Strauven Amsterdam Am-sterdam University Press

mdashmdashmdash 2006b ldquoGollum and Golem Special Effects and the Technology of Artificial BodiesrdquoPp 319ndash350 in From Hobbits to Hollywood Essays on Peter Jacksonrsquos Lord of the Rings edErnest Mathijs and Murray Pomerance New York Rodopi

Helmholtz Hermann von [1868] 1962 ldquoThe Recent Progress of the Theory of Visionrdquo TransPhilip H Pye-Smith Pp 93ndash185 in Popular Scientific Lectures ed Morris Kline New YorkDover

Hockney David 2001 Secret Knowledge Rediscovering the Lost Techniques of the Old MastersLondon Thames and Hudson

4 0 P R O J E C T I O N S

Kemp Martin 1990 The Science of Art Optical Themes in Western Art from Brunelleschi toSeurat New Haven Yale University Press

Kuhn Annette 1999 Alien Zone II The Spaces of Science Fiction Cinema London VersoMcClean Shilo T 2007 Digital Storytelling The Narrative Power of Visual Effects in Film Cam-

bridge MIT PressPeters John Durham 2004 ldquoHelmholtz Edison and Sound Historyrdquo Pp 177ndash198 in Memory

Bytes History Technology and Digital Culture ed Lauren Rabinovitz and Abraham GeilDurham NC Duke University Press

Pierson Michelle 2002 Special Effects Still in Search of Wonder New York Columbia Univer-sity Press

Prince Stephen 1996 ldquoTrue Lies Perceptual Realism Digital Images and Film Theoryrdquo FilmQuarterly 49 (2) 27ndash37

Rabinovitz Lauren and Abraham Geil 2004 ldquoIntroductionrdquo Pp 1ndash19 in Memory Bytes His-tory Technology and Digital Culture ed Lauren Rabinovitz and Abraham Geil DurhamNC Duke University Press

Silverman Robert J 1993 ldquoThe Stereoscope and Photographic Depiction in the NineteenthCenturyrdquo Technology and Culture 34 (4) 729ndash756

Steadman Philip 2002 Vermeerrsquos Camera Uncovering the Truth behind the MasterpiecesNew York Oxford University Press

Turvey Malcolm 2008 Doubting Vision Film and the Revelationist Tradition New York Ox-ford University Press

Wade Nicholas J 2004 ldquoPhilosophical Instruments and Toys Optical Devices Extending theArt of Seeingrdquo Journal of the History of the Neurosciences 13 (1) 102ndash124

Wade Nicholas J and Stanley Finger 2001 ldquoThe Eye as an Optical Instrument From Cam-era Obscura to Helmholtzrsquos Perspectiverdquo Perception 30 1157ndash1177

Wheatstone Charles [1838] 1983 ldquoContribution to the Physiology of VisionmdashPart the FirstrdquoPp 65ndash92 in Brewster and Wheatstone on Vision ed Nicholas Wade New York AcademicPress

mdashmdashmdash [1827] 1983 ldquoDescription of the Kaleidophone or Phonic Kaleidoscoperdquo Pp 205ndash211in Brewster and Wheatstone on Vision ed Nicholas Wade New York Academic Press

FilmographyCameron James 1989 The Abyss USAmdashmdashmdash 2010 Avatar USACassavetes Nick 2004 The Notebook USACooper Merian C and Ernest B Shoedsack 1933 King Kong USAFincher David 2007 Zodiac USAmdashmdashmdash 2008 The Curious Case of Benjamin Button USAForman Milos 1984 Amadeus USAHoyt Harry O 1925 The Lost World USAJackson Peter 2001ndash2003 The Lord of the Rings USANo director 1910 Aeroplane Flight and WreckOrsquoConnolly Jim 1969 Valley of Gwangi USAPenn Arthur 1970 Little Big Man USAPeterson Wolfgang 2006 Poseidon USASelick Henry 2009 Coraline USASpielberg Steven 1993 Jurassic Park USAWachowski Andy and Larry Wachowski 2008 Speed Racer USA