Thinking Images

Thinking ImagesAuthor(s): Paul Kaiser, Marc Downie and Johannes BirringerSource: PAJ: A Journal of Performance and Art, Vol. 30, No. 2 (May, 2008), pp. 17-37Published by: The MIT Press on behalf of Performing Arts Journal, IncStable URL: http://www.jstor.org/stable/30133338Accessed: 17-04-2016 04:28 UTC

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at

http://about.jstor.org/terms

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted

digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about

JSTOR, please contact [email protected].

Performing Arts Journal, Inc, The MIT Press are collaborating with JSTOR to digitize, preserve andextend access to PAJ: A Journal of Performance and Art

This content downloaded from 200.130.19.156 on Sun, 17 Apr 2016 04:28:42 UTCAll use subject to http://about.jstor.org/terms

THINKING IMAGES

Paul Kaiser and Marc Downie in conversation with Johannes Birringer

Paul Kaiser and Marc Downie are two of the most prominent artists cur- rently working in the field of digital creation. Together with Shelley Eshkar,

they formed a collaborative team in 2001, operating under the name the OpenEnded Group. Though the three largely work together, they sometimes work in pairs, create solo artworks, and pursue collaborative projects with others, includ-

ing key collaborators from a range of arts and science fields (architects, composers,

electrical engineers, programmers).

Paul Kaiser's background is in experimental filmmaking; throughout the 1980s he taught students with severe learning disabilities, with whom he collaborated on making multimedia depictions of their own minds. From this work, he derived the key ideas-mental space and drawing as performance-which became points of departure for the digital artworks he has been making since the mid-90s, including

his path-breaking motion-captured performance collaborations with Merce Cun- ningham (BIPED), Bill T. Jones (Ghostcatching), and Trisha Brown (how long does the subject linger on the edge of the volume . . . ). Visionary of Theater was created

as a multimedia documentary on the early theatre work of Robert Wilson, further

elaborating "drawing as performance" and offering a video primer of movement on the stage. More recently, a series of public art works and installations (Pedestrian,

Trace) shifted attention to everyday movements of pedestrians and of children by projecting trompe-l'oeil figures and miniature urban landscapes directly onto city sidewalks. Recovered Light was a massive "virtual X-ray" projection created for York

Minster, UK, while Enlightenment and Breath, commissioned by Lincoln Center for

the Mostly Mozart Festivals (2006, 2007). Enlightenment is considered to be the highest-resolution live digital artworks ever created. They investigate, visualize, and

reconstruct the deeper musical structures of Mozart by means of artificial intelligence

and real-time graphics.

Marc Downie brings a scientific background to the OpenEnded Group, with an MSci in physics from the University of Cambridge and a PhD from MIT's Media Lab based on artificial intelligence research. His complex algorithmic systems are inspired by natural systems and a critique of prevalent digital tools and techniques.

His interactive installations, compositions, and projections have advanced the fields

PAJ 89 (2008), pp. 17-37. 17 O 2008 Johannes Birringer


of interactive music, machine learning, and computer graphics. At the MIT Media Lab, he collaborated extensively with engineers on the development of projects such

as (void *), presented at SIGGRAPH in 2000, AlphaWolf (ars electronica, 2002), Dobie (SIGGRAPH 2002), and Jeux Deux (2006). His solo works include the series Musical Creatures (2000-3), which has been exhibited internationally.

This conversation was conducted via Internet in the spring of 2007.

BIRRINGER: Paul, you participated in the 2006 Monaco Dance Forum where

you and your collaborators recreated how long does the subject linger on the edge of

the volume . .. (2005) with Trisha Brown's dance company. At the festival, you also showed 22, a collaboration with Bill T. Jones that had emerged from the same artistic research begun several years earlier. I take it, from having seen some of your

previous work, that you think of yourself as an image maker. Do you see how long

does the subject linger on the edge of the volume . . . and 22 as live performance works

that present a new kind of image art?

KAISER: Yes, we had the strong sense of opening a new door, especially with how long ... In that particular piece, we were able to push the door open pretty wide. So how is this a new kind of image? Well, to begin with, the art work doesn't consist

of a skein of pre-made pictures that are triggered "interactively" in the course of the performance. No, it works very differently from that. It has its own autonomy,

thanks to the artificial intelligence that Marc Downie has endowed it with. Its imag-

ery comes as it pictures things to itself, trying to make sense of what it sees onstage

in real-time as the dance unfolds. Of course, it doesn't proceed from a completely blank slate any more than a newborn baby does. Instead it draws upon a series of structures and intentions that we've given to it, which in combination sometimes

bring to mind simple living "creatures," to use Marc's term.

For example, the piece opens up with a triangle creature, whose intention is to

move from stage right to stage left. It does so by hitching rides on points in the

motion-captured dancers' bodies, guessing which ones are moving in the right direc-

tion. Thus it extends a line out to a likely point, and is then tugged that way if it's

guessed correctly. Of course, sometimes its hunch is wrong, and it has to relinquish

its grip on that point and await the next opportunity. In such a case, that line is left as a trace, and thus the whole image as it progresses is simultaneously a history

of its attempts.

This touches on a new element of picturing-making: memory. Not only does our artwork work with a sense of present and of future-by perceiving in real-time what

is occurring on stage and guessing what might happen next-but it also works with its memory of the past. Often its images come from its attempting to work out correspondences between past and present configurations on stage. Of course, this

is all very well and good on technical grounds, as computer science if you will. But

our feeling is that the work should stand or fall on artistic grounds. It's up to the

18 PAJ 89


audience whether the quality of our line is exquisite or not, or whether their experi-

ence of time as it unfolds in the performance is somehow new and unexpected.

BIRRINGER: The images generated during the performance, in real-time, have a memory as part of their behavior, as well as the power of anticipation. These are not properties we generally associate with the photographic or filmic notion of the

image. How are we to understand the idea of such "creatures" in a theatre work? Are you suggesting that there is a new dimension of "artificial life" acting upon the performance?

KAISER: Yes, I think this work does push our notion of what imagery can be. Even

"live" imagery of the kind normally seen on stage or in installations reacts only to

the present moment (unless of course it's actually being driven by preset external cues). But Marc's creation of live "creatures" does give our imagery the ability both

to remember and to form expectations. And it's crucial to bear in mind that each such creature's memories and expectations are its own-that is, unlike each other's

and, more to the point, unlike our own. But the question to ask is: why are artificial

memory and anticipation so important? To answer this, let's see why our own abilities

to remember to form expectations are so crucial to our experience of the world.

As we observe any event, our perception of it derives not simply from the present

moment itself, but rather how it stands in contrast to what we think led up to it and where we suspect it might be headed. In a complex event, like that of a great performance, such memories and expectations are never fixed, but are set in constant

play as we continually readjust our perceptions and understanding. That's precisely what we want our imagery to do. Each creature works within its own limits to make

sense of the world in just this way. And as I said before it pictures its understanding,

and it's this process of picturing that becomes the live image.

BIRRINGER: Do you see the creatures as abstract figurations or as having a narrative dimension?

KAISER: The creatures are capable of being either abstract or figurative, and also of having an explicit narrative dimension. To my mind, we humans tend to interpret almost any sequence of events as cause and effect and therefore as narrative,

so even a fairly abstract creature automatically suggests a story in much the way an

inkblot does. The triangle creature I described above, for example, has a clear goal (to move across the stage)-which we immediately interpret as its desire. And we soon make out its strategy (to link its points to those of the dancers moving in the

same direction), which we interpret as some sort of narrative interconnection. One

crucial aspect of this is that we see the virtual creature's effort-its mistakes and its

misunderstandings, as well as its eventual success-which again adds to its being curiously life-like.

BIRRINGER: Are the creatures' behaviors generated in real-time from the actions on stage, or are they picturing something different, something of their own?

BIRRINGER / Thinking Images 19


Top: how long does the subject linger at the edge of the volume . .. Monaco Dance Forum,

2006. Photo: Courtesy MDF; Bottom: Pre-analysis visualization for Enlightenment (2006). Photo: Courtesy the OpenEnded Group.

20 PAJ 89


KAISER: The software always looks directly at what is happening on stage and then

creates its imagery in response to its understanding of what it perceives. Of course,

the picture it forms is idiosyncratic-in part because a given creature attends to just

one kind of pattern on stage, rather than to the aggregate of patterns that we in the

audience are following. But because it attends so closely to that pattern, it often reveals relationships that we don't notice.

BIRRINGER: How do you and Marc understand the relationship of computing and artificial intelligence to the theatre, to choreography and movement, since here the

"images" are neither film nor visualization or illustration. The creatures do have a geometric form, in my mind, animated forms, shapes, architectures. They are non-

anthropomorphic, but as you suggest the audience might "read" a narrative into their

behavior, humanize it. On the other hand, what intelligence do the creatures have?

Do they observe motion, do they respond to the motor sensory-action observa- tion, as the neurophysiologists might call it-or can they respond to choreographic

content? Emotions, affect? What is the relationship between movement animation and consciousness/cognition?

DOWNIE: "Artificial intelligence" informs our work in a number of ways, ranging from the mundane to the exquisite, from the general to the specific. To see this you

can begin with an extremely broad definition of what "AI" is. Most simply put, this

is the field dedicated to getting computers to do "the right thing," by themselves, in

situations where programmatic descriptions, of the kind that computers generally require, of what that "right thing" is are inaccessible to us. Stated this way, AI clearly

ought to be central to almost any digital artist practice-as soon as, that is, they go

beyond the commercially conventional ways of making images and music. What it is that we are wanting out of the computer-the images, the relationships-cannot be

programmed in a forward or a forethought-out way. Firstly, many of the "algorithms"

are in fact just too complex. Too many values to tune, too many layered decisions, renders conventional engineering impossible. Secondly, and more importantly, our "specifications" for the computer programs-what it is we want the images and relationships to do-will only be discovered as we begin to glimpse them. Thus we

seek solutions that allow us to work with computers rather than merely on them.

AI, as described this way, is still incredibly broad (and from both a programming and an artistic standpoint rather uselessly broad) but this should at least suggest that

AI is not an arbitrary posture, that we introduce it not for its own sake, but rather

because the field offers potential solutions to problems that many artists face.

That said, we are focused on a particular class of AI problems and solutions-described

variously as "agent-based," "creature-like" or even "reactive"-that emphasizes the computer as an embodied agent. That is, one that is deeply coupled to its environment

such that its actions on its environment-mediated by the physical constraints of some "body"-must be carefully produced and its perceptions of its environment- mediated by its all too limited sensory apparatus-must be carefully maintained. Such AI is also often where the most interesting integrative machine learning work

occurs, and there learning takes the place of extensive human level domain-specific

BIRRINGER / Thinking Images U 21


knowledge (think: birds rather than chess players). And it also makes perfect sense

given our preference for images that move between abstraction and figuration to be

near a field with a concern for virtual animated bodies. Finally, this "style" ofAI is

usually practiced in a frankly more interactive way by its community. These prefer-

ences dominate our choice of techniques-not any need for our creatures to share a neurophysiological basis with the dancers, which they resolutely do not, nor do we have any goal for "human level" choreographic intelligence.

Thus, while this explanation should clarify much of the architecture of our imagery

for how long. . .-it is this thinking that our "thinking images" are about-but it's important to realize that our creatures are not robots. Their bodies and their physics

are purely imaginary; their sensory apparatus are not sensing the "real world" but sensing the motion-capture data and other agents in view. Their motivation and affect systems are for their regulation not for our narrative. And the problems that

they are forced to solve are not given by the natural world: they are also designed

by us as we see opportunities in Trisha's choreography.

BIRRINGER: I am not sure how close theatre and dance are to games, let's say, and the "virtual animated bodies" in game environments, if that is what you meant.

DOWNIE: I don't feel qualified to talk about how close theatre or dance is to games,

in general, either in the classical sense or in the sense of computer games. However

there is a game-like aspect to our forms in two overlapping senses. Firstly, algorithms

are rules-nothing more. Our agents describe a set of rules that in application will produce the images seen as the artwork. While it's we who come up with the rules

initially, once set we tend to quite rigorously follow them to see where they might

lead-we play out the game without arguing over the rules. Coming up with a new

set of rules is generally preferable to trying to break our previous set. Secondly, our agents often end up participating in games: the opening Triangle agent for how long

... has to get from stage right to stage left by hitching a ride on the dancers at key

moments. Clearly, neither of these two aspects are in any way unique to our work;

there's precedent in every corner of art history, including, of course, the choreogra-

phers we have worked with.

There is also a relationship, but not a terribly interesting one, between our work

at a technical level, and computer games. Like everybody else in the field we are parasitic to some extent on the consumer-level graphics hardware created for the game industry. Although recently it has become much easier to avoid the com- mon aesthetics of computer games, this has come at the price of it being much harder to get anything on the screen at all. Finally, much of my code for making "abstract" bodies was tested much more rigorously than you'd expect while at the Media Lab-and it still sees use in drawing and controlling much more computer game-like characters and frighteningly "concrete" robots. I can testify that playing

between the abstract and the figurative is technically much easier once you have the

figurative worked out on a robot.

22 PAJ 89


BIRRINGER: But you speak of more fully interactive practices in the AI community, while I understood Paul to imply that interactivity on stage is not necessarily

your goal.

DOWNIE: It is correct to say that "interactivity on stage" was not our goal for how

long ... . Ultimately the choreography is fixed, and the dancers are accurate from evening to evening (to my eye, supernaturally so). There was no sense while we

were making the work that the dancers would ultimately interact with the imagery

during the performance. Rather we were seeking to interact with Trisha Brown's dance, and we took advantage of the three years that we worked on the project to build a set of tools and ideas to allow us to do just that. This difference is crucial and is present throughout our work-our recent work Enlightenment is a cluster of

ten computers that interact with themselves. And at present Paul and I are making

a series of prints: interactively-but obviously they are not interactive prints. It's

not that we are uninterested in the problem of interacting with "the public," we just

haven't had any good ideas yet.

BIRRINGER: You said that you designed the creatures' "thinking" behavior in response to the choreography, is that correct to say? In terms of such thinking

or learning, is sensing the kinematic data a mathematical/geometrical process of regulation (based on the computer's analysis of the tracked points in space), and an "improvisatory" transformation of a particular body model?

DOWNIE: The "thinking" in how long.., certainly extends down to the lowest (and most tedious) technical levels of the piece. One of the lessons we quickly learned when we started to work with real-time motion-capture was that we would have to

reconstruct most of the motion-capture pipeline ourselves, on our terms. The priori-

ties of the hardware manufacturers are deeply misaligned with ours, emphasizing the quality of data over the quantity of it. Frustrating and time-consuming as this

engineering was, it worked out happily: much of the piece evolved from "staging" these increasingly sophisticated algorithms of machine perception, and in particu-

lar staging their repeated failures and attempts to compensate. The piece would have been impossible to execute had we simply accepted the manufacturers' "black

boxes" as closed to us; and any AI would have found itself in a much-impoverished environment. However, an unwillingness or an inability to understand, reconstruct

or even reject the pre-made pieces of technology that populate the digital art world remains the norm.

BIRRINGER: Paul, has the AI approach to physical performance privileged a certain kind of abstract movement or body model and a non-representational creation of geometry? How do you see such computational sensibilities, digital graphics and animated geometries relating to other forms of music or dance theatre, spoken word

theatre or installations, in terms of audience participation? Do you foresee a growing

use of such graphical environments in performance?



KAISER: Well, if you take a close look at the particular choreographers we're working

with here-Merce Cunningham and Trisha Brown-you'll see that that their reli- ance on abstract diagrams predates ours by several decades! Both Merce and Trisha choreograph by means of such diagrams, creating amazing graphical depictions of

such key dance elements as stage geometry, movement trajectories, temporal repeti-

tions, body kinesphere, and so on. Look, for example, at even so early a work as Merce's Suite by Chance of 1953, and you'll see him charting the spatial and temporal

axes of the work, with no recourse to stage pictures. Trisha makes similar use of drawing, for example in creating her amazing 1975 solo Locus, in which she draws a simple cube and a string of letters, which she then uses to set her choreographic

requirements. The cube she imagines as being roughly the size of her kinesphere (the

space surrounding her body that she can potentially reach). By labeling each vertex

of the cube (actually, cuboid) with a letter, she determines the order in which she must hit each point of space with any given part of her body. The resulting solo is

of course a well-known tour-de-force, and Trisha's method led directly, I'd say, to Bill Forsythe's more elaborate invisible geometries.

Let me also point out that Trisha elevated this kind of drawing to a formal art practice, exhibiting the works in galleries and museums, so it's not as if she wanted

to keep her diagrammatic thinking hidden from view-behind the scenes, so to speak. To the contrary, which is why she took so quickly and enthusiastically to the

diagrammatic creatures of our how long. . . projections. Parenthetically, I'd like to say something that I hope Marc will take up further, which is that in many ways Merce and Trisha were exploring ideas and processes in a manner very much paral-

lel to what was going on simultaneously in computer science. I'm not suggesting

that von Neumann and Turing were ever aware of Cunningham and Brown, or vice

versa, but I will say this: what a shame! And it's not too late to cross-pollinate their

ideas, even if they never had a chance to do so themselves.

However, I don't want to paint us into too tight a corner here. Your question implies

that we committed ourselves exclusively to a style of diagrammatic abstraction and

so on, which is far from the case. We've worked with equal interest and intensity with representation. Indeed my first encounter with the stage was through Robert

Wilson's drawings, the vast majority of which are simple but incredibly effective series of stage scene thumbnails.

More to the point, we were committed to an AI approach to representation in

another collaboration done in parallel with how long . . . . This was with Bill T. Jones, in a work entitled 22. It entailed an identical technical set-up to Trisha's piece, with twenty-four infra-red cameras capturing Bill's movement in real-time,

and our software's responding simultaneously not through abstract creatures but through life-like figures (a young boy and a man) that provided a thematic and visual counterpoint to Bill's solo stage presence. I didn't use this example earlier, for the final piece fell well short of our aspirations for it, for complicated reasons

having nothing to do with us. From our standpoint, however, the virtual figures of

24 PAJ 89


Top: Robert Wilson sketching scenes of Einstein on the Beach. Still frames from

Visionary of Theater (1996). Photo: Courtesy Paul Kaiser and The Byrd Hoffman

Watermill Foundation; Bottom: "Airport as dollhouse" storyboard for Horizon (2005). Photo: Courtesy the OpenEnded Group.



22 could work just as well as the abstract creatures of how long.., in fulfilling this

dream of "thinking pictures."

DOWNIE: I could mention that one of the "complicated reasons having nothing to do with us" was that in 22 there was a need for the images to interact as much with the narrative that was being told in performance by Bill T. Jones as with his

movement. If the getting a computer to "understand choreography" live is research,

getting a computer to "understand narrative" live is simply foolhardy. So foolhardy

in fact that we didn't attempt it in our contribution to this piece which, as it was,

seemed to begin to sink under the weight of its rather traditional stage-managed cuing. Such short fallings, however, often indicate future opportunities and each of our

more recent pieces have increasingly engaged language, and computational responses

to it. This has led us, again, far away from the terrain of how long... or 22.

BIRRINGER: It's interesting that you mention the choreographers' and Robert Wilson's drawings. I tend to think that Wilson's scenography and directing have a computational, almost mathematical-abstract quality, but of course he also works with the figure. Perhaps the terms figurative and non-representational are no longer

helpful in this context of complex systems, digital or AI. But given your interest in

animation, graphic visual art, and the kind of complexities afforded by an agent- based aesthetics as well as your fascination with disequilibrium, could you speak a little about your work not directly intended for the theatre but for galleries and

public spaces? What are the aesthetic differences between the public projections, the

gallery works and the collaborative performance works?

KAISER: I'm afraid I don't agree with you on the first point, Johannes. I'd say, actually, that Robert Wilson's scenography and directing are more mechanical than

computational. It's certainly no secret that much of his staging is a revival of the sophisticated proscenium mechanics of the nineteenth century, repurposed for avant-

garde spectacle. In any case, from what I've seen, he mostly works by drawing scenes

from a stage perspective rather than diagramming relationships and movements more

abstractly the way Merce, Trisha, and Bill Forsythe tend to do.

But to address the main part of your question, it's mainly an issue of context, which

of course has aesthetic implications. The great thing about the stage is the way it frames not only the space of the artwork, but also the time of it. In our hectic day

and age, it's quite a wonderful throwback to an earlier age where people more often

gave things their undivided attention. Certainly that's rarely the case for museum exhibits, much less for public art, so the pieces you make for those settings need to

allow for random access, as it were-people come and go much more randomly in such settings, and the artwork has to withstand, address, or overcome that somehow.

For public artworks, you also have to imagine some of the viewers encountering the piece repeatedly over the days of its installation, which again is a very different

kind of viewing experience. We try to make works that can disclose themselves in

different ways depending on the depth and the frequency of their exposure.

26 PAJ 89


DOWNIE: The parallels between Merce, Trisha, Bill Forsythe's, and others' practices and digital art, at least as it could be practiced, are quite striking. If you pardon the

language: these choreographers are producing algorithms that are executed on their

dancers, observing the unforeseeable results of the computation, making adjustments,

and iterating; culling the potential of their algorithmic ideas; carefully pruning the

possibilities they create; constantly negotiating the constraints of human performance,

perception and their interfaces with the abstract-computational. Of course, there are

often wonderfully talented dancers that perform these computations, and we just have our keyboards and our stupid computers-but those problems are separate,

they are just technical. After enough work, enough engineering, you strive to find

yourself in a position very similar to that of a choreographer and their dancers. I've

complained for a long time now that digital art, as a field that refuses to grow up and understand its history, or that there is a history, has generally ignored the pos-

sibility that there may be a useful prehistory in the performing arts.

BIRRINGER: Paul, do the projected animations you installed in public places reflect an aspect of image work that you have always been interested in? What has

happened in your thinking and practice since your earlier collaborations with Bill T. Jones and Merce Cunningham?

KAISER: The simplest answer to your question is that I started out as a filmmaker and I never gave up my desire for the kind of cinematic experience that large pro-

jections allow. It was obvious that I'd never sneak any of my work into the local multiplex cinema, so I found different spaces-a huge scrim at the State Theater or a

broad expanse of sidewalk at Rockefeller Center. But there are important differences

between the public works and the dance works. The first has to do with framing and

audience. You realize when you step into the world of dance how distant it is from

the everyday world, even despite various valiant efforts to break down that divide.

But I came to realize, as I've said many times before, that the only people who go see

modern dance are the people who go see modern dance. Very few outsiders stumble

into it-"stumbling in" becoming exactly what public art allows.

Most of the audience encountering Pedestrian on the street in Harlem or in a bus station in Seoul or on a market square in Bruges do so completely by accident, unex-

pectedly having to make sense of an artwork whose context does not automatically supply all the answers as a gallery or museum or stage context invariably does. But

the other impetus behind some of the public artworks was a desire to get away from

dance movement itself, which has never fascinated me so much as the movements I

see through my window on the street below or from a bench in a playground when

I take my daughters there. To me, children playing tag on a field or pedestrians maneuvering through a busy intersection in midtown are supreme feats of choreog-

raphy, but choreography without a choreographer. Absolutely fascinating. Which is

not to say that my perceptions of such everyday movements and patterns were not

sharpened tremendously by our work with Merce and Bill T. and Trisha.



BIRRINGER: With your interest in more mundane, pedestrian, and non-expert

movement, are you going through your Judson phase?

KAISER: Well, it's an interesting parallel, and I sometimes wish we could have worked

with Trisha when she herself was still in that phase. But the answer still is no, for

two reasons. First, when a self-conscious adult performer does ordinary movement

on stage, well, the movement is no longer ordinary. Certainly it doesn't feel so to any audience member, which for us is the crucial part. So Judson movements are really not the same either physically or psychologically or spiritually as playground

movements, and the latter interest us much more.

But second, when I was about twenty years old-back around 1979-I was already

fascinated by pedestrian movement, and made an experimental film in Super 8 called

Colorblind that took as its subject people crossing a single intersection on West End

Avenue. In those days, of course, I knew nothing about postmodern dance. In any case I couldn't advance that interest any further for two decades, not only because

the right tools didn't exist, but also because I didn't have the right collaborators (after

all, Shelley was nine years old at the time, and Marc was two!).

BIRRINGER: How has OpenEnded Group evolved: are you three now a production and research company?

KAISER. The OpenEnded Group is more of a collective than a company-there is

no leader, for example, and we make all our decisions unanimously. Research is part

of art-making, and art-making part of research. The secret to great collaboration, I

learned, is to work with people much smarter than I am. I met Shelley Eshkar in the

mid-90s through his former teacher, Robert Breer, one of my filmmaking heroes, and

it was immediately apparent that he was incredibly gifted. It was he who solved the problem of making a convincing "hand-drawn space" in 3-D, and he who designed

the hand-drawn bodies that made Ghostcatching and BIPED what they were. We've

been working together now for almost twelve years and are essentially telepathic.

Marc Downie came in an arranged marriage, really, for a piece that the MIT Media Lab commissioned from us-our abstract portrait of Merce Cunningham, Loops,

which was based on our motion-capture of Merce's hand dances. When we first sat down with Marc over coffee, he nodded quietly at every preposterous goal we stated, and a couple of weeks later, had us shaping the piece in real-time in front of

his screen at MIT. You have no idea how revolutionary that was! What had taken us overnight to render before, Marc was now drawing to the screen instantaneously.

Independently, he had been pursuing some of the same hand-drawn, non-photoreal-

istic goals that we had, most notably in a series called Musical Creatures, so it was a

natural fit. I also realized he was smartest person I'd ever met, so I helped steer him

out of academia. The three of us have been working together ever since.

BIRRINGER: In a recent symposium at the 2007 Cyberarts Festival in Boston, I

raised the question why there are so few substantive technological performance works

or innovative dance works which use computer augmentation. The works that have

28 PAJ 89


had an impact we can count on one hand, why do you think this is so? If there are

so few works, perhaps we cannot even speak of impact, in a cultural sense?

KAISER: The most obvious reason is expense. Practically speaking, dance budgets are small, and even set-up times tend to be very short. Talking to lighting design-

ers, for example, I learned that they have far less time to design on stage for dance

than they do for plays, for example-and far less than for operas. But a much more

important reason has to do with method and approach, for the tools commonly used in interactive work (Max/MSP and Jitter, for example) largely lead to terrible

results. They seem to promote the shallowest and the easiest kinds of thinking, and

it's no surprise that no masterpieces seem to have emerged from there! I'm no tools

determinist, and it's certainly conceivable to create a great artwork with poor tools,

but they certainly don't help. But Marc, who devoted part of his thesis to this ques-

tion, can answer it better than I.

DOWNIE: Perhaps I am a "tools determinist": I just take a unusually broad definition of "tool," one that is expanded to include all of the ideas one uses to arrive at the artwork itself. And I believe that while pleasant surprises can happen, you generally end up in a place that bears the marks of how you got there. The separa-

tion of digital tool and formal idea is artificial and convenient only if you are in the unfortunate position of having to purchase your tools from strangers. It's worth

studying the tools used for digital art as forms: not just from the perspective of what they can do (and how fast they can do it) but what ideas they embody and privilege or exclude and prevent. There are then two scandals: firstly, that there aren't

a tremendous number of tools available today; and secondly they are all the same!

Max/MSP/Jitter and its brethren embody nothing more than a set of ideas firmly

worked out in the mid-80s which themselves were constructed to be parallel to the

ideas embodied by the analog synthesizers of the 50s, which in turn were simply inherited from the physics of electrical engineering.

Outside the digital art world, nobody takes these ideas about interaction and programming seriously: not the artworld, not computer science, not philosophy, not

even physics. The third scandal is, of course, that nobody seems to care and digital

art becomes increasingly un-interdisciplinary as it becomes un-serious to these fields.

My response is two-fold: to make sure that I can take responsibility for any digital

tool I use, usually by making it or remaking it; and soon, to make our tools freely available. The goal of our upcoming "Open Source" release is not to skew the hege- mony in our direction, but simply to disrupt it and make artists begin to question

what many have simply "received" up until this point.

BIRRINGER: Do you think interactivity or interaction design in live performance on stage is limiting or artistically uninteresting? What forms of interactive art do you

feel has had a significant artistic impact or corresponds to the increasing interactive

design in the culture at large, as in communications, games, etc.?



DOWNIE: It's impossible to rule anything out, but I suspect that on stage "interaction design" is a bad place to start from and a bad place to end up. And as for the influence of interactive art I find that it has had almost no effect on computers

let alone culture at large. Digital artists continue to have no effect on Apple's next operating system, Adobe's next image editing tool, Microsoft's next word processor,

Google's next start up purchase, W3C's next Web standard, etc. (This makes the idea

that artists are content to simply wait for better tools all the more appalling).

BIRRINGER: What is computer augmentation of performance? What did the designers of the "intelligent stage" at Arizona State University have in mind with the

architectures of interactive, networked spaces? If universities are investing in such

laboratories, why are arts organizations not following suit? BAM stopped supporting

media arts, Dance Theatre Workshop discontinued its digital fellowship program. Is

the support for digital art mostly going to be found in visual arts organizations?

KAISER: If I were more politic, I would sidestep this question, but what the hell-as you know, I tend to speak my mind. And in this case I do speak from experience, since I was one of the three artists chosen for the short-lived BAM digital installation

program, and since we have worked at Arizona State and in several other university

projects as well. You'll remember, Johannes, that you and I first met when you were

teaching at Ohio State University, where the dance department was atempting to collaborate with the Computer Graphics department in a motion-capture studio. As

is inevitably the case, the problem comes down to money. Universities and corpora-

tions have it; artists and art organizations do not. Let's look at universities first. Big

universities are not only committed to advanced research, but many also have very

large performing arts centers that are a crucial mainstay for contemporary American

dance. I'm thinking of the Zellerbach stage in Berkeley, the Krannert Center in Illi-

nois, the Hancher stage in Iowa, and quite a few others. These places are not only venues for dance, they also commission new works: a fantastic thing.

On paper, then, it makes sense that universities should go even further by supporting

advances in digital art and performance, not only because they can tap into many sources of funding that are (unfairly!) out of the reach of artists, but also-and more

appropriately-because they have the physical facilities (equipment, labs, stages) to host the development and the staging of complex projects. Universities even have, potentially, many scientific and artistic collaborators who could contribute to such

work.

Why then has our experience been largely one of failure and exasperation? And, even worse, why do we harbor suspicions of fraudulence? The sad story is that universities tend to make lofty pronouncements, especially to funders, but fall ter-

ribly short in execution. Even worse, when they fail, they don't acknowledge their

failure, but simply explain it away as the necessary consequence of experimentation

and research, part of what they inevitably call the "process"-a word I've come to detest. I think we need to get back to good old-fashioned "results" instead. The embarrassing fact is that such failures often come in areas of basic competence, not

30 PAJ 89


advanced research. University engineers, for example, simply won't know how to operate their expensive optical motion-capture equipment. When we were in long, repeated residence with Trisha Brown at Arizona State, they kept failing to capture

any data whatsoever-heartbreakingly, at times. We persuaded Trisha to don the motion-capture costume and to perform her Locus solo for posterity, only to learn

much later on, and with no accompanying apology, that the data had never been acquired. With little skill in matters of simple competence, was it any surprise that

the loftier research goals were rarely met? Of course, you'd never know this from the

press releases and status reports streaming forth; and meanwhile those hundreds of

thousands of dollars had already dried up in the academic sands.

Well, enough of this. If I sound bitter, that's because I am, but I'll spare you the excruciating details. Despite this terrible experience, I would like to believe that universities could support real research in the arts, but only if they really confront

their previous failures in the field. And I would add that it was an art school, Cooper

Union, that gave Shelley and me crucial support in working with Bill T. Jones on Ghostcatching very early on.

As for corporate support, that is more unabashedly about money or at least value: the value to companies being promotional. Indeed, funds usually come directly from

marketing departments, not from any charitable corporate offshoot. Sometimes such

support is very simple: in the mid- to late-90s, for example, Compaq supported our

work because we were using what were then their advanced NT workstations, and they could boast about that at the National Broadcasters convention and at SIG- GRAPH. The most pervasive, and the most pernicious, support for the performing arts has long come from Phillip Morris, the tobacco company-blood money, really,

with politically correct artists and art groups putting themselves through all kinds

of odd contortions to justify that backing. My opposition to Phillip Morris got me in trouble at the Whitney, where a group show I was in found much-needed but tainted support from them. I protested publicly, and also paid the actual costs of installation myself.

BIRRINGER: I think it's remarkable that you have worked on some of the most crucial artistic performance collaborations over the past decade, especially if your initial background was in film. You mentioned BAM's digital installation program: why did BAM not encourage more of the kind of work you were producing?

KAISER: Again, I think it comes down to simple economics. The Brooklyn Academy

initiative was with Lucent, which owns Bell Labs. Both BAM and Lucent wanted

to associate themselves with the glory days of the 1960s, of the EAT collaborations

(Rauschenberg, Breer, Trisha Brown, et al., working with engineers like Billy Kliiver).

But there was a key difference: originally, all that was done informally rather than

institutionally, with the artists and engineers pairing up on their own. Bottom-up

versus top-down. In the later case, Lucent provided money and some equipment, and artists and engineers met in a kind of rapid dating game to see who could mate

successfully. The actual results were pretty good-for example, Ben Rubin and Mark



Hansen worked together seamlessly to create Listening Post, a wonderful installation

that did indeed draw on Bell Labs research meaningfully.

For my part, perhaps in reaction to the top-down institutional nature of the whole

process-and perhaps perversely-I created an intensely personal work entitled

Trace. Several research projects I saw at Bell Labs were about new technologies of

surveillance, which took me back to my childhood as a diplomat's son in Eastern Europe, where we were constantly shadowed and bugged. Trace was about the "self-surveillance" that sort of self-consciousness led me to. But the upshot was that

Lucent's share prices tanked shortly thereafter, and when that support went away, so

did BAM's. In any case, it was always clear that BAM's heart was in performance, understandably enough. And they still had Phillip Morris to fund it.

DOWNIE: To tie these last two questions together I think it might be useful to ponder the overlap and differences between artists, engineers, and scientists-speak-

ing as an artist who has pretended to be a scientist and an engineer at times. The EAT collaborations are in some ways the canonical bottom-up meetings of artists with engineers. Despite having often irreconcilably different sets of tastes, engineers

and artists share a dedication to actually making things, a pride in the things that

they make and an understanding that they judge, and are judged on, the things that

they end up making. This alignment of perspectives can be tremendously exciting when it happens, and it's possible for everybody to ensure that they are getting back something from the collaboration. But collaborations inside a university are often formulated completely as science-art collaborations; certainly our most recent

university experience was ultimately funded by the NSE Academic science has as uneasy a relationship with engineering as it has with art. And since nobody can independently replicate, say, a motion-recognition system built for Trisha Brown's

movement, the junior science faculty and graduate students involved in such collaborations are left scrambling for the limited value that they can extract from the

relationship in ways that are often independent of the resulting artwork. Artists

then respond by defending the integrity of their process and the artifacts that they

are going to be left with. In such pathological collaborations the only uncertainty is just how long it will be before everybody starts bolting for the exits.

Despite an abundance of interdisciplinary ventures by American universities, only a

handful will accept artifacts (and even fewer, performances) as part of the tenure or

advanced degrees processes for their "science" halves. No wonder then that dot coms

were a better bet than the academy. Until this changes, interdisciplinary research is

not being done by institutions except by accident or resistance.

BIRRINGER: Yes, I have similar experiences, but in the UK there is some effort made now to fuse the funding for sciences and humanities in so-called sci-art projects. More interesting is the bottom-up model and the many small research networks that I see growing across the world. Can we go back one step: Paul, what made you work on a project involving theatre director Bob Wilson? What kind of exhibition did you create of Wilson's works and drawings? Were you interested in

32 PAJ 89


his early long-durational or outdoor work, or his particular interest in working with

autistic adolescents?

KAISER: A long story, but I'll shorten it. My start in multimedia was in working with severely learning disabled children, whom I taught for ten years in the 1980s.

Well, "whom I taught"-and who taught me. What I learned from them, primarily, was how to collaborate, for in creative classes I soon learned to give up instruct- ing in favor of open-ended exploring. And when Apple's HyperCard program was released on the Macintosh in the mid-90s, we used its multimedia tools to carry

on those collaborations on this new frontier. That's in fact when I came up with guiding principles like "drawing as performance" that carried through to many later

artworks-and which proved to be one of the keys to my work with Wilson.

Fast forward to 1993 or so, when Wilson's foundation was thinking about a digital archive. A friend told them of my work, thinking of the parallel: Bob Wilson had also started out by working with handicapped and "special needs" kids, two of whom

directly inspired the first two-and to my mind most impressive- phases of his work. And then, too, there was the central practice of drawing, with Bob creating all

of his work not verbally but visually, in the never-ceasing activity of his thumbnail

sketching. So just as with the children I'd taught, my goal was to capture the act

of Bob's drawing, as it happened-to put it back into the time of its performance, which eventually I did. Working with Babette Mangolte, I filmed him drawing and narrating his recollections of such pivotal works as Deafinan Glance, KA MOUN- TAIN, and Einstein on the Beach, and used the resulting video as a guide to dissect the resulting drawings in Photoshop. I could then tie this drawn performance to the video, films, photographs, maquettes, etc., that I found in the archive, which allowed me to reconstruct works otherwise lost to time.

That was just one of several methods I used in making the piece, Visionary of The- ater. In it, I was at pains to put Bob's best foot forward, to make the best possible case for his early work, and in particular to reveal the extraordinarily collaborative

nature of it-true collaboration with the likes of Christopher Knowles, Cindy Lubar,

Andy deGroat, and so many others. Though the piece was exhibited widely, among other things in a show of Bob's drawings that I co-curated with Brooke Hodge of Harvard's Graduate School of Design, it was only this year that it finally reached its

intended form of CD-ROM-and has now-belatedly, and on teetering, outdated technical underpinnings-entered the collection of the Performing Arts Library at Lincoln Center.

So, yes, I put Wilson's work in the best possible light but all the while I was reacting

against it and its kind. To long a philosophical point to go into here, but I can't omit

the fact that I'm deeply troubled by the severing of causality and of history that you

find in Wilson and his forbearers (Cage, for instance) and their many followers who

crowd the field of digital art. This objection relates to our critique of arbitrariness,

the same critique Marc makes of the arbitrary "mapping" techniques embedded in the digital tools like Max MSP. I think that dream logic, random juxtaposition,



Top: Pedestrian at Rockefeller

Center, 2002. Photo: © Peter

Cunningham; Bottom: Still from

the digital work. Photo: Courtesy

the OpenEnded Group.

34 PAJ 89


asynchronicity, and so on, have become easy evasions of hard thinking. The proof,

perhaps, is the degree to which they've been gleefully adopted by mass media-adver-

tising, video games, music videos, and so on. A welter of disconnects.

BIRRINGER: After Pedestrian and, later, Playground, Paul, you have now been commissioned to create public works in England (York Minster). Are you becoming interested in very large scale work or are these projection works logical continuations of your image-making for the theatre?

KAISER: Well, all the world's a stage, right? So, yes, we've been very interested in expanding the possibilities of public art, and in particular of making permanent

works. In the United States at least, "public" is a slightly misleading term, for increas-

ingly our public spaces are in fact "public/private." Bookstores replace libraries; malls

replace public squares; and even civic spaces are often partly private, having been underwritten by companies or supported through advertising. Take Times Square, for example: how public is it, really?

So one question when trying to make vast and permanent public art projects is how

to work in that odd seam between civic and corporate structures. We've had two tantalizingly close prospects, though neither worked out in the end. One, which I

can't discuss in any detail, was to put a single networked artwork in possibly hundreds

of stores throughout the country. That would have been a fascinating encounter! But the other one, entitled Horizon, was actually commissioned and announced, so I can describe it fully. Atlanta Airport was adding a new international terminal, for

which the city's percent-for-the-arts set-aside ensured a pretty fair budget-for our

project, three million dollars. We were going to build a long LED display (280 by 30 feet) with intelligent imagery generated interactively and in real-time.

Shelley created an elaborate storyboard for the piece, one of which illustrates our basic idea best-"airport as dollhouse." Here the idea was to invert the sense of scale,

by creating large virtual children who play with elements of the airport as if they

were parts of a dollhouse. And so there were children's hands propelling airplanes down the runway, and their fingers becoming air bridges for disembarking passen-

gers, passing them from one hand to the next. Here Shelley was inspired by the way

in which children sometimes play by passing ladybugs back and forth-carefully, delicately, but with an immense sense of power.

The idea was to reframe the airport experience, which in our post-9/11 world is a particularly unpleasant one. So for example as a passenger you're always aware of being under constant surveillance, with surveillance cameras constantly monitored to

detect terrorist activities. Horizon would also have such cameras, but for completely

benign means: to look at the weather, to study patterns of people's movement, and

above all to play. So for example Shelley had another storyboard of children playing

in Atlanta's Olympic fountains, which might be called up by the AI when it started

to rain outside-we'd celebrate the wet weather rather than bemoan it. Or the camera

might allow the virtual children to spot people browsing among the displays of a



newsstand, and then play hide-and-seek with them there. Or as dusk came on, it would bring down the night in our virtual world as well.

The key was to make a piece that never repeated, continually engaging not only the

frequent business travelers who pass through that Atlanta hub regularly, but also of

course the airport workers who would have to live with the piece. It's in this context

that the idea of "thinking images" plays so crucial and so practical a role. I hate to harp on money and the lack of it, but again this project was indefinitely suspended

not because of our budget but rather because of enormous cost overruns on the architectural and engineering design of the overall terminal building. A comedy of

errors, which has since devolved into a tangle of legal suits and counter-suits between

the city and its various contractors. We've stayed well clear of that mess.

BIRRINGER: What a shame, your description is very vivid and I'd have loved to visit a playful airport, for a change. How do you see the current artistic culture in

its gradual embrace of new media technologies? What social or aesthetic aspects

do you value, and how do you think about "digital culture"? Why is theatre in the U.S. a relatively minor art form today which has barely developed new ideas or techniques, whereas so much current discourse on performance process, becoming, hybridity and interactive digital media tends to be progressive and also increasingly

politically aware of the challenges we face in the biotechnological era?

KAISER. Hmm, I'm loathe to prognosticate on too broad a scale here. We work within our culture, and so while we definitely respond to the sorts of shifts and concerns you outline in your question, we certainly haven't risen to any elevated, Olympian position from which to assess them as seers. But perhaps I can say a little

bit about digital culture and what we make of it as artists. It's a commonplace that

the Internet, especially as mediated by Google, has changed the context and the reach of information. Search is now a nearly instantaneous procedure, with tremendously

wide scope, and it has become the quintessential mode of thought for our era-the

first recourse for nearly everyone, that first step we all take almost automatically.

And it's not just Internet search that's so important-DNA sequencing, for example,

works essentially the same way.

So we've been interested in applying techniques of search in our own works. Which

is not to say that we make so-called "Web art" which uses-and often tries to sub- vert-the various protocols of the Internet for various postmodernist reasons. Instead,

we bring search algorithms into our work itself. We're fascinated by this newfound

ability to consider millions of possibilities and contingencies that are well beyond what the naked mind can address by itself.

And so, for example, for Enlightenment Marc wrote code that could analyze the twenty-five-second coda to Mozart's last symphony and detect on its own not only the five themes of its invertible counterpoint, but also trace back related thematic

material through the whole of the fourth movement. Similarly for the public artwork,

36 PAJ 89


Breath, we perform other sorts of searches: for example, revealing essential semantic

threads running through the entire Book of Psalms.

Now if search represents an advance in our culture, I must also point to a sore point

of decline. We're often struck by how contemporary culture seems to be heading towards increasingly low resolution-the voice quality on your cell phone is far worse

than on your grandparents' old landline; the image quality on your iPod video or YouTube feed is mud compared to the 35 or 70mm films they enjoyed in cinemas; the language of e-mail and chat is impoverished by contrast to that of their old-

fashioned letter writing. Attention itself is increasingly low res, as I said earlier: the

world seems designed for distraction.

So to this, as artists, we oppose ourselves, rather than just going with the cultural

flow, so to speak. Visually, we insist on high resolution projections and displays: there's no reason why artists should have to accept the fuzzy interlacing of DVDs, for example. For a while now, we've said that we want digital resolution to feel like

that of paper-and as a logical conclusion, two of our current projects will be printed

rather than projected. We're making lines so fine that we can't even see them on our

screens, but only when they emerge from our printer. But this is not some sort of

retreat into tried-and-true old-fashioned art-making (the persistence of painting as

an art form astonishes me): we're printing texts and imagery for projects that you

could never have done by hand. If art advances, it does so unpredictably and not quite in lock-step with the culture at large.

JOHANNES BIRRINGER is a choreographer, media artist and critic. His

latest stage production, Suna no Onna, opened at the Laban Center, London,

in December 2007. He is the founder of Interaktionslabor, an independent media laboratory based in Germany, and directs the Center for Contempo-

rary and Digital Performance at Brunel University, London. His new book,

Performance, Technology, and Science, from which this interview is excerpted,

will be published by PAJ Publications in the spring of 2008.



Documents

Thinking Images