Towards aSupportive Technological Environment

for Digital Art

Greg Turner and Ernest EdmondsCreativity and Cognition Studios, UTS, Sydney

greg@gregturner.org and ernest@ernestedmonds.com

Abstract

This paper presents the case for extendingprogramming languages to support digital artistsengaged in technologicallv-innovativc work. Theanticipated result is an "environment for buildingenvironments ", which will need to satisfy certaintechnological requirements according to the areas inwhich digital artists most need creative support. Areview ofthese areas is undertaken, and a proposal ismade to capture the specific areas in which digitalartists most need technological support.

1. Context

1.1. Artists are Technology Innovators

Artists, as the quintessential creative workers, giveform to their imaginations. The physical world ofartefacts is very different to the conceptual world of theimagination, and artists often find themselves pushingtechnology forward. creating new artefacts either as partof. or in order to construct. their art. These new artefactspresent new ways of using and thinking about otherthings. In other words. artists are excellent catalysts forinvention.

Specifically, digital artists are responsible for manyof the most exciting advances in human-computerinteraction today, precisely because they arc notexclusively technologists, who "arc often taken bysurprise to find that their world can be looked at inunfamiliar terms" (L. Candy & E. A. Edmonds, 2002, p.32). The anthropologist Lucy Suchman cites HeidiTikkas 2002 piece Mother, Child as an example ofwhere the most promising developments in interactionare coming from (Suchrnan, 2002). Stephen Wilsonstates, "We are at an interesting place in history, inwhich it is sometimes difficult to distinguish betweentechno-scientific research and art - a sign that broaderintegrated views of art and research are developing"

(Wilson, 2002, p. 4). In this sense, digital art is mostinteresting to technologists. including the HCIcommunity. when it is pushing technology forward.

1.2. Artists, like Most People, Strugglewith Computers

The main barrier in attempting to achieve such anadvancement of technology is the lack of understanding,control. and consequent perceived power over currenttechnology. This sense of powerlessness overtechnology and the onrush of progress is concomitantwith the dystopian future vision often advocated by thetraditional arts and humanities, of a world governed bytechnology; the abandonment of the body and thus ourhuman qualities. For example. Hubert Dreyfusexpresses his fears about todays Internet, arguing that it"diminishes one's sense of reality and of the meaning inone's life." (Dreyfus, 200 I, p. l02). In contrast.technologists remain more optimistic; in control of theirdestiny. Alan Kay said in 1971, "Don't worry about whatanybody else is going to do ... The best way to predictthe future is to invent it" (\\'\\'\l'.smalltalkorg: Alan Kay.2003). However, thirty years later. he is more neutral.claiming that "the computer revolution hasn't happenedyet" (Steinberg. 2003), that computers have helpedscientists and engineers think in entirely new ways, buthave had much less impact on the thinking in otherfields, blaming the lack of new creative environments inthese fields.

Artists have also written books and articles onprogramming for other artists: John Maeda describes hispersonal perceptions of the relationship betweencomputation and digital form and introduces the DBNprogramming language as a result (Maeda. 1999); GolanLevin et al. present artwork, case studies and code byfour artists known for their experiments 1I1

computational design (Levin. Ward. lia, & meta, 2001).However. these and others are accounts of artists whoare also computer scientists, which is to say that theyhave also had to overcome the current problems withlearning to control technology.

Instead, our goal should be to improve the technologyin order to give artists, and others not primarilyconcerned with technology, control over it. That meansnot just the power to usc technology, but the power toexplore, and hence create technology, and to create withtechnology.

Digital artists need to be provided with a creativitysupport system, an environment in which to create art. Inthe technologically-uninteresting case, one or more off-the-shelf packages are used, unmodified, as theenvironment. In more interesting situations, wheretechnological innovation is taking place, theenvironment is constructed, by a technologist ortechnologists, to fit the individual needs of the artist (L.Candy & E. A. Edmonds, 2002, pp. 2-35). (It is worthnoting that, often, the artist and technologist is the sameperson, but for endeavours beyond the technical skills ofthe artist, the environment is constructed in conjunctionwith a second party.) Such interdisciplinarycollaborations have been classified into three levels: thetechnologist-as-assistant model, the partnership-with-artist-control model, and the full-partnership model(Mamykina, Candy, & Edmonds, 2002); roughlyaccording to the passivity of the technologist'sparticipation. The technologist's role in each of thesemodels is to construct and participate in an environmentwhich turns the expressive ideas of the artist intosomething executable by a computer, and in tum, tomake the processes of the computer intelligible to theartist (see Figure I). The expressivity of the computercomes from both the teehnologist(s) and theenvironment they provide. Hence, the environment, notthe artwork, is the technological creation, so it is thecreation of the environment which needs to betechnologically supported, possibly with a so-called"environment for building environments" (L. Candy &E. Edmonds, 2002).

Figure 1. Technologist (right) constructing andparticipating in the artist's (left) creative

environment.

2. The RouteThe first step towards providing the necessary

support for environment creation is to ask: Where arc wenow" What are artists doing with digital technologynow'? Where are they requiring new digital technology"The converse question: Where are they struggling withexisting digital technology?

2.1. Historical Programming

To illustrate why we need to ask these questions. it isuseful brieflv to consider the development of computerlanguages to date. There are currently about fivegenerations of languages (Campbell, 2003), starting WIththe zeroth generation, machine code and assemblylanguage, which arc single instructions for thecomputer's processor. From working with these singleinstructions. programmers found they were oftenworkinz with mathematical expressions, so the firstgeneration of languages created (FORTRAN I, Algol,c 1954-8) made these common expressions easier towork with. The second generation (COBOL, Lisp,c 1958-62) added higher-level constructs such asfunctions and subroutines. The third generation (Basic,Pascal. c 1962-1970) concentrated on ease ofprogramming, and interpretive languages. It is nocoincidence that the first digital art appeared at aroundthis time.

In short, each successive generation of programminglanauaze has arisen from the need to simplify thecOI~pu~er for the programmer. This is a goal shared withthe HCI community, reinterpreted to encouragecreativity, exploration and PO\\"Cf. rather than to "case"the completion of a task. (c. Bradley Dilger presents awell-considered analysis of the concept of "ease" anddescribes the paradoxical notion that a technology whichsupposedly eases a particular task can actually increasethe complexity of, and skills required for, the completionof that task (Dilger. 2003).)

To simplify programming, and computing in general,difficult operations are made more straightforward, andcommon operations are sped up. Overall, intelligibility isimproved and the power of the programmer is increased.Often, and lamentably, the "hood" is closed on thelower-level technologies, never to be reopened,inhibitinz even interested users from looking at orchanging exactly what is going on, and forcing them toreinvent old technologies or hack and fudge around theexisting ones.

So-called "high-level" languages are appropriatelynamed when compared to assembly language, but weneed to keep going. This power should now be spread toother creative fields, and not continue to be the solepreserve of the programmer. since programmers are nolonzer the sole innovators of computer technology. JohnMa~da wrote "There is no greater need for visual designthan rethinking and redesigning programming itself'(Maeda, 2000, p. 406) - and we should expand this

2

statement to include all forms of digital creativity. Theimplication IS the eventual abandonment ofprogramming languages as we know them altogether,working instead towards generalised creativity supportsystems.

2.2. Building upon What We Know

In finding answers to the questions asked at thebeginning of this section, we arc identifying the aspectsof digital art development which arc currentlyunnecessarily difficult, and the aspects that are tediousand prosaic. These are areas to make quicker and easierin the new generation of creative environments. Wewould also be able to see where the boundaries ofexisting practice lie - where the edges are being pushed.

Currcntlv, it is difficult to find answers to thesequestions. In general, most digital artists do not publiclydiscuss in-depth the technological processes theyunderwent in order to create their work - the finishedpiece is often seen as the only aspect of importance tothe outside world. Since the environment for creating thepiece (rather than the piece itself) is of paramountimportance to the identification of technologicalrequirements, the artistic requirements of suchenvironments must be identified before they can besatisfied. To achieve this, it will be useful to examine thegeneral requirements for environments found so far, andto auzment these with a new database of specifictechn~ogical experiences and requests of artists.Information from analysis of the database will allow thespecification of which tools and techniques to provide toaid the efficient building of new. customisedenvironments. Such a database would includeinformation about:

Which existing technologies were used: This willallow the mapping of current digital art practiceaccording to technological areas, to indicate trends inand relationships between technology uses in the arts.

What custom technologies were created: Wherecommon answers arc given, this will serve to highlightwere existing technology is repeatedly failing artists.Where less common answers are given, this is a guide tothe new technological areas which are the foci of artisticendeavours and 111 which artists require mostcustomisation.

Which hardware and software products wereused: As well as assisting in the above analyses,individual projects can use this data as a problem-avoiding/solution-finding device, where the projectexhibits~similarities to works in the database.

How the artist achieves control of the technology:The path of the high-level concept to technologicalmanipulation and the nature of any collaboration whichtook place, either with the technology directly orindirectly through human collaborators. Analysis of theresults in this category will help to identify the

popularity of techniques that artists employ to imparttheir expressivity to a computer.

The technological barriers encountered: Whattechnological issues or constraints particularly hinderedthe development of work. Of particular interest are HCIissues that can be addressed by technological or designimprovements.

This database may be populated in a number of ways.each with its own advantages. The database should allowfor all of these levels of information, as each type cancontribute to trend-finding and analysis:

1. At the most basic level, the technological featuresexternally apparent in digital artwork can be inferredfrom the description provided by the artist or others. forexample, the usc of animated displays or webinteraction. Such information could be gained merelyfrom a technologist's review of existing artwork. andsome examples are given in section 4. This approach islimiting because it provides no insight into thedevelopmental process. or even the nature of theenvironment used. However, in a publicly-accessibledatabase. such information docs act as a lure for artiststo check over their entry and make additions'

2. A questionnaire completed by the artist wouldprovide more details, such as the exact technologies used(for example, the development environment andmanufacturers of equipment used), the nature of anycollaboration, and some idea of the difficultiesencountered. This would be useful because of the abilityto disseminate the questionnaire and to obtain results ofstatistical siznificance but is limiting in the types ofb ,

response sought.3. The database could also contain the results of

interviews with artists, where open-ended questionscould be asked and clarifications sought, and to whichthe artist could add additional observations.

4. Information could also be used from observationalstudies of artists and technologists during collaborations.as exemplified in the "COST ART' projects of theCreativity & Cognition Studios (L. Candy & E. A.Edmonds, 2002, ; Edmonds & Candy, 2002). Thesestudies contain detailed descriptions of the events whichtook place during each residency, from the perspectivesof artist, technologist, and an observer, to control forsubjectivity, and are a rich source of empirical data.

In order to demonstrate the technological usefulnessof this approach, the next two sections contain a reviewof the technological implications of support for a) thegeneralised features of creativity support systems and b)the specific technological features of digital art whichhave been identified so far.

3. Identifiable Generalised CreativitySupport Features

The results of the proposed database will build uponthe work already done in the area of creativity supporttools. in which some generalised features of good

3

creativity support systems have been identified. Theproposed new work will combine these generalguidelines with specific technological requirements inorder to specify and create actual tools for buildingenvironments.

In earlier research. empirical studies were used toidentify some examples of aspects of creativeexploration: Breaking with convention. immersion in theactivity, holistic view. parallel channels of exploration(Edmonds & Candy, 2002). Ben Shneiderman lists eightspecific operations that should "help more people bemore creative more of the time": Searching (forknowledge and inspiration), Visualising, Consulting.Thinking, Exploring, Composing, Reviewing,Disseminating (Shnciderman, 2002). In the digital artdomain some of these tasks would be highly interrelated(for instance. visualisation. exploration andcomposition). Michael Terry and Elizabeth D. Mynatthighlight the need for support of Schon' s theory ofreflection-in-action: near-term experimentation(previews of the results of actions), longer-termvariations, and evaluation of actions, each demonstratedin the "Side Views" application (Terry & Mynatt, 2002).

These general requirements map well onto desiredtechnological features of creative environments. Someexamples of these follow:

3.1. Visualisation

Obviously, all creative environments arc dynamic,interactive systems - even if the finished piece is non-dynamic and/or non-interactive. Technologically, thiswould mean the standard provision of, for example, adouble-buffered graphical display, on more than onescreen, which is capable of showing the artwork andenvironmental controls and, if relevant, multi-channel,multi-tracked audio.

The artist should be able to visualise aspects of thedevelopment in different ways. For instance, a section ofcode could be viewed as text, a diagram, a plain- orpseudo-English explanation (with the aid of metadata todescribe the semantics of the code) or as an interactiveprocess in which the artist can try inputs and witness theoutputs. The same applies for debugging. For example,adding a "watch" to a variable which is a y-coordinateshould cause a visual indication of the coordinate to beoverlaid on the space to which it applies.

The work of Manfred Mohr, for example, is centrallyabout handling data that can only be visualised with theaid of a computer, because of its multi-dimensionalcomplexity (Mohr, 2002), see Figure 2.

3.2. Exploration and Thinking

Generally, as with any interactive system, theprinciples of good HCI design (most importantly, usertesting) should be followed. Specifically in this context,the results of changes made to the environment shouldbe immediately apparent where possible, either during

execution of the artwork or in preview. Code-modification during runtime is a feature of someinterpreted programming languages such as Max(Edmonds et al., 2003), and so-called "late-binding"virtual machines such as Squeak Small talk (Ingalls,Kaehler, Maloney, Wallace, & Kay, 1997). Manual-override variable modification during runtime should besupported.

Yasunao Tone's work with Creativity and CognitionStudios used Max/MSP as the basis of a very openexploration in which, quite deliberately, nobody newwhat the outcome might be. In this case exploration intothe unknown was at the core of the approach to the art(Edmonds et al., 2003).

3.3. Reviewing

As is generally understood (Lieberman & Fry, 1997).the ability to undo/redo and keep-recover/move previousversions of the program/artwork is useful, as would bethe ability to artificially slow down or step through aprocess in order to understand its behaviour moreclearly.

3.4. Holistic View

The ability to "fold up" complex processes orstructures into simple entities allows the artist to take aholistic view. This is a feature of many visualprogramming languages (Edmonds et al., 2003) anddocument-centric creativity support tools (for example,the "Document Map" feature of MS Word).Additionally, the functionality (particularly that providedby the technologist) should be expressible in a way thatis intelligible to the artist.

3.5. Immersion in the Activity

This IS not only a physical environmentalrequirement. but it is also important not to burden theartist with the minutiae of operating the interface ordealing with peripheral tasks. For example, syntax errorsshould be fixed automatically or semi-automatically, ifpossible. For example, the pedantic requirement forsemicolons at the ends of lines of code should beremoved or reduced to the level of automation to avoidannoying interruptions to the user's process. Asmentioned in 3.2, the delay of the code-compile-executecycle should be minimised.

3.6. Breaking with Convention

The creative environment should allow the breakingof its own convention - its own modification andextension, which, when taken to the extreme, impliesthat the environment should be created in (a subset of)itself: and should allow its own manipulation. Suchrecursion is often seen in Art-Technology collaborationslike the one depicted in Figure I, where the technologist

4

Type of Input

Table 1. Depth of interactivity (input)

Additional Technological implicationsNone (except potentiallv the passage oftime).e,g. Picturesifilm: animation; recorded music

Simple input: rhe user or phvsical environment crosses thresholds ormakes selections, usuall: continuous along one dimension only.e,g. Switches: Hyperlinks: Interactive DVD; Simple Motion Sensing;specifically cubeLifc (Everitt, Turner, 2002)

Complex input: rhe user or physical environment provides input that canrake a large number otvalues. permutations, or dimensions, allowingforgreater exprcssivitve.g. Text; Speech; Gesture; Video; Continuous Biosensors, specificallyIamascopc (Fels & Mase, I999;

Few, bearing in mind thor the artwork may need tobe stopped or restarted.

The actions oithe user, and possibly the actions orrhe program, can be represented in a flow-chan-stvle diagram, and be debugged relatively easilv.Some discrete input must be derived. using simplesignal analvsis, from continuous input (forexample. sensing a heartbeats.There is a needfor signal processing andcalibrasion tools in the environment. For example,

fuzzv logic processing Debugging is more complex~ manually-overriding the input should besupported

Table 2, Depth of interactivity (output)

Type of Output Additional Technological implications.vone Sane, Although conceptual artists mav disagree,

\\'c presume that all an has WI output ofsome sort.Simple output: The art outputs onlv one result, or one ala discrete set oj' The provision of toolsfor pre-calculating,

results (the intricacv ofthese results is irrelevantfor this case). co lice ring and enumerating these states. ifthere aree,g. Static art; interactive D I'D; linearly-animated orr; most "predictable a large number of them, would be useful. Thebehaviour " art (see Tahle 3), operation ofthe artwork might also be described

with flow-charts, or state-eventmodelling.Complex output: The art outputs anv ofan extremelv large ser ofstates. Marhematicol modetling. Al and signal processingee i 'trtual environments: eenerati\'e art tools are all potentially usetul.

places himself in the creative environment which he hasconstructed for the artist. In the software world. this isless common, although several compilers and tools forprogramming arc written in the very languages theysupport.

The environment should also be general, In otherwords, every problem that is solvable by computershould have some solution In the environment(Campbell, 2003),

A number of examples of such behaviour have beensccn to be partly supported by interpreted visualprogramming methods (Edmonds et al., 2003), Theimmediacy of feedback and the holistic view providedseem to be significant factors,

4. Identifiable Creativity SupportFeatures for Digital Art

The authors examined digital artworks created inprevious studies (L. Candy & E. A, Edmonds, 2002) andpublished descriptions of works, c.g. the review byStephen Wilson (Wilson, 2002), By considering theexternal attributes of these works. it was possible tobegin to map out the specific technological requirementsof the environments which gave rise to them. whichaugment the list in the previous section. These externalattributes fall into two general categories - the waydigital artworks are presented, and their apparent

behaviour. (The purpose here is not to "classify" art, norto present an exhaustive review, but to show howconsidering various aspects and dimensions of art canhelp describe future technological rcquircments.)

4.1. Presentation

Digital art has brought with it artworks whichcommunicate, or are perceived, in two directions.Communication takes place on a number of levels, fromthe excitation of photons or electrons (the level at whichhardware and software operates) to the 'obvious',concrete object (the level at which both artists andtechnologists can communicate), to the 'subtle' languageof the work (the level at which artists conceptualise).

The technological communications requirements ofcompleted artworks are straightforward to identify byobserving them. To support the most common paradigm,onscreen drawing of 2D output and mouse-and-keyboard input, would be the trivial case. Commonextensions to simple visual media are more complexforms such as video and 3D. Of these, 3D output isparticularly difficult to satisfy convincingly, because itoften imposes a "look" on the image as a result of theuse of conventional 3D rendering techniques. Ideally,artists would be able to modify the rendering engine tocustomise the depiction of 3D objects.

5

Table 3. Complexity of change

Level of Complexitv Additional Technological implicationsPredictable: the artist, andfamiliarised audience, is able to Bug-finding is relatively easv. Predictable art normally haspredict what should happen in response to certain events. discrete input and output, or no input.e.£; Interactive DVDExploratory: the art produces interesting responses to stimuli, Bug-fixing is more complex, as many permutations of complexwhich mar be entirelv deterministic and comprehensible, but inputs and outputs all need to he evaluated. Aids to testing andthe atfordance of exploration. or new comprehension bv the process-control displavs are paramount.artist or audience is imperative.e,g. Manf;'ed Mohr 1.\10111'. :!OOJ). (see Figure J). lamascope(Fels & Mase, 1999)Emergent-Algorithmic: The computer makes unpredictable Emergent art calls/or extensive experimentation andchanges as a result ofinternal calculations or random debugging before it is exhibited. Additionallv, emergent art isdecisions. the result of complex systems, so tools to inspect, override,e.g. Complex Svstems; simulate or replay the complex interactions would be useful.

Artificial Life: speciticallv Olivine Trees (Miranda. JOO/) The science of complex svstcms is in its infancv so such toolsEmergent-Environmental: The computer makes changes 10 its are not widelv available.behaviour as a result ofunpredictablefactors outside a/irscontrol, namelv humans and natural entities ti.e. the weather)e.g. AI and Expert Svstems

Saved and printed output should also be provided:a difficulty arises if there is a need for high-resolutionoutput. This means that the information for a vector-based description of the onscreen image should either bemaintained in real-time or calculable off-line, perhaps bya high-resolution recreation of the actions which formedthe display.

Audio output, via multi-channel. multi-trackwaveform or MIDI events should be supported.However, real-time digital audio processing (as withdigital video processing) is both an extremely complexand extremely wide-ranging technology, and is likely torequire the ability to communicate with third-partysoftware for anything more than simple (or ere at cd-from-scratch) applications.

Almost all specialised hardware devices communicatewith the serial, or USB protocol. The environmentshould case the interpretation of the messages whichsuch devices use to communicate, for example by aidingthe mapping of high-level commands to sequences ofserial data (and vice-versa) and providing visualisationtools to examine these sequences at a higher level (forexample, video).

A growing number of artworks (and hardwaredevices) exhibit some form of communication over anetwork. Given the limitations of the Intemet Protocol(IP), these artworks most often communicate in UDP(change-centric data; not all data must arrive at itsdestination) and TCP (event-centric data: all data isimportant). Again. aiding the mapping of these messagesto higher-level constructs is useful. In addition. someartworks may require distribution (one-to-many. ormany-to-many) or storage of messages via a centralserver, rather than just communication with one othercomputer (one-to-one). Tools for communicating with acomputer not under the control of an artist (for example,to get the weather) would also be beneficial.

Last but not least, the communication which acomputer has with files is important for storinginformation ready to be used at a later time. The threemost ubiquitous types are text files, binary files (difficultto work with for non-technologists) and SQL databases(again, an understanding of SQL is currently necessary).Fast previews and overviews of such files, again wherepossible, would aid their design and creation by theartist.

4.2. Behaviour

Stroud Comock and Ernest Edmonds classified artaccording to behaviour in 1973 (Comock & Edmonds,1973). The categories given were Static, Dynamic-Passive, Dynamic-Interactive and Dynamic-Interactive(varying). In the light of the present examination of thedigital art climate, we can reconfigure these categories inorder to elicit further technological requirements. Threeaxes of behaviour become apparent: depth ofinteractivity for both input and output. and complexity ofchange. A single artwork may exhibit one or morecharacteristics in each axis.The depth of the interactivity is independent for theinput (see Table I) and output (sec Table 2) media forany given artwork, and consists of three levels. Eachlevel. for both input and output, has certaintechnological implications for a supportive environment,beyond those which may arise from other requirements.

The complexity of change in behaviour of anartwork is an informal ranking of how predictable theartwork IS (see Table 3). This IS a subjectiveclassification, as it depends on the level at which theartwork is analysed (on a state-event level, everyoperation of a computer is inherently predictable.Conversely, it is often easier to execute the instructionthan to predict its result). The classification here

6

illustrates the types of exploration and debuggingactivities at the human level.

Figure 2. P-701/B, Manfred Mohr,enduraChrome/canvas, 1999 (from

http://research.it.uts.edu.au/creative/ccrs/gallery/mmohr/mmohr.htm)

5. ConclusionThe externally-apparent requirements of digital art

and the general requirements for creativity support mapclosely onto technological requirements for a supportiveenvironment for digital art. Digital artworks havefurther, distinctive and unique technologicalrequirements not obvious from an external observer'sperspective: types of behaviour and calculation notdirectly related to output (and simple input); thetechnologies used by the environment; and other, new-technology aspects. The proposed database will assistgreatly in identifying these further requirements, andthus can be used to actually construct an "environmentfor building environments".

(As a further consideration, technical quality shouldnot be ignored in meeting the requirements for such anenvironment. If recognised, open standards andprotocols are used for depiction, communication andinteraction within the environment, the flexibility,extensibility, and hence longevity of the artworks willbenefit. )

Existing programming languages are all examples ofcreative environments - someone with knowledge, skillsand perseverance can create an environment which does

exactly what is required. The problem is thatprogramming languages are hardly any good for anyonewithout these skills. As Alan Kay remarked, "I thoughtwe would be way beyond where we are now ... Theirony is that today it looks pretty good. The result of our[Xerox PARC] work is techniques for doing software inan interesting and more powerful way. That was back inthe seventies. People today aren't doing a lot of work tomove programming to its next phase" (Steinberg, 2003).The issue for the HeI community in particular is to buildexploratory, rather than task-oriented environments,since it is exploration which most encourages creativity.

It is time to broaden the focus of creating new digitaltechnology from new computer tools towards newcreativity support systems, and from the programmer tothe generalised digital creator. In the time to come, thecomputer should be moulded by the ideas of anyone whowishes to create within its possibilities, much as wood orclay has for millennia. It will become a creative mediumin the truest sense. Digital artists are currently the closestfit to the ubiquitous digital creators of the future, and itis their work which is responsible for much of theinnovation in interaction technology today.

6. AcknowledgementsThe authors would like to thank the OZCHI reviewers,Dave Everitt. Keir Smith and Alastair Weakley for theirvaluable perspectives and insights on the issuespresented in this paper.

7. ReferencesCampbell. 1. (2003). Lessons in Object-Oriented

Programming, fromhttp:'·\\\\\\.cs.gub.ac. uk/~J. Campbe llimv\\eb/oop/ooph tmlnode4.html

Candy, L & Edmonds, E. (2002). Interaction in Art andTechnology, Crossings: e.Iournal of Art and Technology(ht/(J.cmssings.lcd. ie), 2( I).

Candy, L., & Edmonds, E. A. (2002). Explorations in Art andTechnology: Springer.

Cornock. S.. & Edmonds, E. (1973). The Creative ProcessWhere The Artist Is Amplified Or Superseded By TheComputer, Leonardo(6), 11-16.

Dilger, C. B. (2003). Ease in Composition Studies. PhDDissertation, University of Florida. Florida.

Dreyfus, H. L. (2001). On The Internet: Thinking in Action.:\ew York: Routledge.

Edmonds. E., & Candy, L. (2002). Creativity. Art Practice andKnowledge. Communications of the A CAl, 45(10),91-95.

Edmonds, E., Candy, L, Fell, M., Knott, R., Pauletto, S., &Weakley, A. (2003). Developing Interactive Art UsingVisual Programming. Paper presented at the Proceedings ofHCI International 2003. Crete,

Everitt, D. (Artist), G. Turner (Collaborator). (2002). cubel.ite:An Internet-Resident Artwork [Interactive Art]:http: \\\\\\.cubclifc,or1!.

Fels, S., & Mase, K. (1999). Iamascope: A Graphical MusicalInstrument. Computers and Graphics, 2(23),277-286.

7

Ingalls, D., Kaehler, T., Maloney, 1., Wallace, S., & Kay, A.( 1997). Back to the Future: The StOlY oj Squeak, APractical Smalltalk Written in Itself. Paper presented at the00PSLA'97 Conference, Atlanta, Georgia.

Levin, G., Ward, A .. lia. & meta. (2001). 4x4 GenerativeDesign' Bevond Photoshop: Friends Of Ed.

Lieberman, H .. & Fry, C. (1997). ZStep 95, A Reversible,Animated Source Code Stepper. In 1. Stasko, 1. Domingue,M. Brown & B. Price (Eds.), Software Visualization:Programming as a Multimedia Experience. Cambridge,MA,: MIT Press.

Maeda. J. (1999). Design B, Sumbers: MIT Press.Macda.T, (2000) .. vtaedo avtedia. Thames & Hudson.Marnykina, L., Candy. L.. & Edmonds, E. (2002).

Collaborative Creativity. Communications ofthe AC\f,45( 10),96-99.

Miranda, E. R. (200 I). On the Origins and Evolution of Musicin Virtual Worlds, In D. W. Come & P. 1. Bentley (Eds.),Creative Evolutionarv Svstems: Morgan Kaufmann.

:'lohr. M. (2002). Generative Art. In L. Candy & E.. -\.Edmonds (Eds.). Explorations in Art and Technologv:Springer.

Shneidennan, B. (2002). Creativity Support Tools:Establishing a framework of activities for creative work.Communications ofthe ACM. 45( 10). 116-120.

Steinberg. D. H. (2003) Daddv. Are We There Yet) ADiscussion with Alan Kav, fromhttp: \\\\\\.opcnp~J1.col11'puba p~p J(H)3 0403 alan k<1\.hIml

Suchman, L. (2002). Replicants and lrrcductions: Affectiveencounters at the interface. European Association for theStudy ofScience and Technology (EASSTj.

Tcrry.M; & Mynatt, E. D. (2002), Recognizing GreativcSeeds in User Inter/ace Design. Paper presented at theCreativity and Cognition Conference 2002. Loughborough.CK.

Wilson, S. (2002). Information Arts: Intersections ofArt,Science and Technology. Cambridge, Massachusetts: TheMIT Press,

"'\I'II.sma!!Tu!k.org. Alan Kay. (2003). fromhttp: \\\\\\ .sma1ltal k ,org alank<1\. hlml

8

In Viller & Wyeth (Ed5.) Proceedings ofOzCHI2003: New directions in interaction, information environments,media and technology. 26-28 November 2003, Brisbane, Australia: Information Environments Program,University of Queensland. Copyright the authors and CHISIG

Towards aSupportive Technological Environment

for Digital Art

Greg Turner and Ernest EdmondsCreativity and Cognition Studios, UTS, Sydney

greg@gregturner.org and ernest@ernestedmonds.com

Abstract

This paper presents the case for extendingprogramming languages to support digital artistsengaged in technologica!!y-innovative work. Theanticipated result is an "environment for buildingenvironments ", which will need to satisfy certaintechnological requirements according to the areas inwhich digital artists most need creative support. Areview of these areas is undertaken, and a proposal ismade to capture the specific areas in which digitalartists most need technological support.

1. Context

1.1. Artists are Technology Innovators

Artists, as the quintessential creative workers, giveform to their imaginations. The physical world ofartefacts is very different to the conceptual world of theimagination, and artists often find themselves pushingtechnology forward, creating new artefacts either as partof, or in order to construct, their art. These new artefactspresent new ways of using and thinking about otherthings. In other words, artists are excellent catalysts forinvention.

Specifically, digital artists are responsible for manyof the most exciting advances in human-computerinteraction today, precisely because they are notexclusively technologists, who "are often taken bysurprise to find that their world can be looked at inunfamiliar terms" (L. Candy & E. A. Edmonds, 2002, p.32). The anthropologist Lucy Suchman cites HeidiTikkas 2002 piece Mother, Child as an example ofwhere the most promising developments in interactionare coming from (Suchrnan, 2002). Stephen Wilsonstates, "We are at an interesting place in history, inwhich it is sometimes difficult to distinguish betweentechno-scientific research and art - a sign that broaderintegrated views of art and research are developing"

(Wilson, 2002, p. 4). In this sense, digital art is mostinteresting to technologists, including the HCIcommunity, when it is pushing technology forward.

1.2. Artists, like Most People, Strugglewith Computers

The main barrier in attempting to achieve such anadvancement of technology is the lack of understanding,control, and consequent perceived power over currenttechnology. This sense of powerlessness overtechnology and the onrush of progress is concomitantwith the dystopian future vision often advocated by thetraditional arts and humanities, of a world governed bytechnology; the abandonment of the body and thus ourhuman qualities. For example, Hubert Dreyfusexpresses his fears about today's Internet, arguing that it"diminishes one's sense of reality and of the meaning inone's life." (Dreyfus, 200 I, p. 102). In contrast,technologists remain more optimistic; in control of theirdestiny. Alan Kay said in 1971, "Don't worry about whatanybody else is going to do ... The best way to predictthe future is to invent it" twwwsmalltalk.org: Alan Kay,2003). However, thirty years later, he is more neutral,claiming that "the computer revolution hasn't happenedyet" (Steinberg, 2003), that computers have helpedscientists and engineers think in entirely new ways, buthave had much less impact on the thinking in otherfields, blaming the lack of new creative environments inthese fields.

Artists have also written books and articles onprogramming for other artists: John Maeda describes hispersonal perceptions of the relationship betweencomputation and digital form and introduces the DBNprogramming language as a result (Maeda, 1999); GolanLevin et al. present artwork, case studies and code byfour artists known for their experiments incomputational design (Levin, Ward, lia, & meta, 2001).However, these and others are accounts of artists whoare also computer scientists, which is to say that theyhave also had to overcome the current problems withleaming to control technology.

Instead, our goal should be to improve the technologyin order to givc artists, and others not primarilyconcerned with technology, control over it. That meansnot just the power to usc technology, but the power toexplore, and hence create technology, and to create withtechnology.

Digital artists need to be provided with a creativitysupport system, an environment in which to create art. Inthe technological1y-unintercsting case, one or more off-the-shelf packages are used, unmodified, as theenvironment, In more interesting situations, wheretechnological innovation IS taking place, theenvironment is constructed, by a technologist ortechnologists, to fit the individual needs of the artist (L.Candy & E. A. Edmonds, 2002, pp. 2-35). (It is worthnoting that, often, the artist and technologist is the sameperson, but for endeavours beyond the technical skil1s ofthe artist, the environment is constructed in conjunctionwith a second party.) Such interdisciplinarycol1aborations have been classified into three levels: thetechnologist-as-assistant model, the partnership-with-artist-control model, and the full-partnership model(Mamykina, Candy, & Edmonds, 2002); roughlyaccording to the passivity of the technologist'sparticipation. The technologist's role in each of thesemodels is to construct and participate in an environmentwhich turns the expressive ideas of the artist intosomething executable by a computer, and in turn, tomake the processes of the computer intel1igible to theartist (see Figure I). The expressivity of the computercomes from both the technologist(s) and theenvironment they provide. Hence, the environment, notthe artwork, is the technological creation, so it is thecreation of the environment which needs to betechnologically supported, possibly with a so-cal1ed"environment for building environments" (L. Candy &E. Edmonds, 2002).

Figure 1. Technologist (right) constructing andparticipating in the artist's (left) creative

environment.

2. The RouteThe first step towards providing the necessary

support for environment creation is to ask: Where are wenow? What are artists doing with digital technologynow? Where are they requiring new digital technology?The converse question: Where are they struggling withexisting digital technology?

2.1. Historical Programming

To il1ustrate why we need to ask these questions, it isuseful briefly to consider the development of computerlanguages to date, There are currently about fivegenerations oflanguagcs (Campbell, 2003), starting withthe zeroth generation, machine code and assemblylanguage, which are single instructions for thecomputer's processor. From working with these singleinstructions, programmers found they were oftenworking with mathematical expressions, so the firstgeneration of languages created (FORTRAN I, Algol,c 1954-8) made these common expressions easier towork with. The second generation (COBOL, Lisp,c 1958-62) added higher-level constructs such asfunctions and subroutines. The third generation (Basic,Pascal, c 1962-1970) concentrated on ease ofprogramming, and interpretive languages. It is nocoincidence that the first digital art appeared at aroundthis time.

In Sh011, each successive generation of programminglanguage has arisen from the need to simplify thecomputer for the programmer. This is a goal shared withthe HCI community, reinterpreted to encouragecreativity, exploration and power, rather than to "ease"the completion of a task. (c. Bradley Dilger presents awell-considered analysis of the concept of "ease" anddescribes the paradoxical notion that a technology whichsupposedly eases a particular task can actually increasethe complexity of, and skills required for, the completionof that task (Dilger, 2003).)

To simplify programming, and computing in general,difficult operations are made more straightforward, andcommon operations are sped up. Overal1, intelligibility isimproved and the power of the programmer is increased.Often, and lamentably, the "hood" is closed on thelower-level technologies, never to be reopened,inhibiting even interested users from looking at orchanging exactly what is going on, and forcing them toreinvent old technologies or hack and fudge around theexisting ones.

So-called "high-level" languages are appropriatelynamed when compared to assembly language, but weneed to keep going. This power should now be spread toother creative fields, and not continue to be the solepreserve of the programmer, since programmers are nolonger the sole innovators of computer technology. JohnMaeda wrote "There is no greater need for visual designthan rethinking and redesigning programming itself'(Maeda, 2000, p. 406) - and wc should expand this

45

statement to include all forms of digital creativity. Theimplication is the eventual abandonment ofprogramming languages as we know them altogether,working instead towards generalised creativity supportsystems.

2.2. Building upon What We Know

In finding answers to the questions asked at thebeginning of this section, we are identifying the aspectsof digital art development which are currentlyunnecessarily difficult, and the aspects that are tediousand prosaic. These are areas to make quicker and easierin the new generation of creative environments. Wewould also be able to see where the boundaries ofexisting practice lie - where the edges are being pushed.

Currently, it is difficult to find answers to thesequestions. In general, most digital artists do not publiclydiscuss in-depth the technological processes theyunderwent in order to create their work - the finishedpiece is often seen as the only aspect of importance tothe outside world. Since the environment for creating thepiece (rather than the piece itself) is of paramountimportance to the identification of technologicalrequirements, the artistic requirements of suchenvironments must be identified before they can besatisfied. To achieve this, it will be useful to examine thegeneral requirements for environments found so far, andto augment these with a new database of specifictechnological experiences and requests of artists.Information from analysis of the database will allow thespecification of which tools and techniques to provide toaid the efficient building of new, customisedenvironments. Such a database would includeinformation about:

Which existing technologies were used: This willallow the mapping of current digital art practiceaccording to technological areas, to indicate trends inand relationships between technology uses in the arts.

What custom technologies were created: Wherecommon answers are given, this will serve to highlightwere existing technology is repeatedly failing artists.Where less common answers are given, this is a guide tothe new technological areas which are the foci of artisticendeavours and in which artists require mostcustomisation.

Which hardware and software products wereused: As well as assisting in the above analyses,individual projects can use this data as a problem-avoiding/solution-finding device, where the projectexhibits similarities to works in the database.

How the artist achieves control of the technology:The path of the high-level concept to technologicalmanipulation and the nature of any collaboration whichtook place, either with the technology directly orindirectly through human collaborators. Analysis of theresults in this category will help to identify the

popularity of techniques that artists employ to imparttheir expressivity to a computer.

The technological barriers encountered: Whattechnological issues or constraints particularly hinderedthe development of work. Of particular interest are HCIissues that can be addressed by technological or designimprovements.

This database may be populated in a number of ways,each with its own advantages. The database should allowfor all of these levels of information, as each type cancontribute to trend-finding and analysis:

1. At the most basic level, the technological featuresexternally apparent in digital artwork can be infenedfrom the description provided by the artist or others, forexample, the use of animated displays or webinteraction. Such information could be gained merelyfrom a technologist's review of existing artwork, andsome examples are given in section 4. This approach islimiting because it provides no insight into thedevelopmental process, or even the nature of theenvironment used. However, in a publicly-accessibledatabase, such information does act as a lure for artiststo check over their entry and make additions'

2. A questionnaire completed by the artist wouldprovide more details, such as the exact technologies used(for example, the development environment andmanufacturers of equipment used), the nature of anycollaboration, and some idea of the difficultiesencountered. This would be useful because of the abilityto disseminate the questionnaire and to obtain results ofstatistical significance, but is limiting in the types ofresponse sought.

3. The database could also contain the results ofinterviews with artists, where open-ended questionscould bc asked and clarifications sought, and to whichthe artist could add additional observations.

4. Information could also be uscd from observationalstudies of artists and technologists during collaborations,as exemplified in the "COST ART" projects of theCreativity & Cognition Studios (L. Candy & E. A.Edmonds, 2002, ; Edmonds & Candy, 2002). Thesestudies contain detailed descriptions of the events whichtook place during each residency, from the perspectivesof artist, technologist, and an observer, to control forsubjectivity, and are a rich source of empirical data.

In order to demonstrate the technological usefulnessof this approach, the next two sections contain a reviewof the technological implications of support for a) thegeneralised features of creativity support systems and b)the specific technological features of digital art whichhave been identified so far.

3. Identifiable Generalised CreativitySupport Features

The results of the proposed database will build uponthe work already done in the area of creativity supporttools, in which some generalised features of good

46

creativity support systems have been identified. Theproposed new work wi II combine these generalguidelines with specific technological requirements inorder to specify and create actual tools for buildingenvironments,

In earlier research, empirical studies were used toidentify some examples of aspects of creativeexploration: Breaking with convention, immersion in theactivity, holistic view, parallel channels of exploration(Edmonds & Candy, 2002). Ben Shneiderman lists eightspecific operations that should "help more people bemore creative more of the time": Searching (forknowledge and inspiration), Visualising, Consulting,Thinking, Exploring, Composing, Reviewing,Disseminating (Shneidcrman, 2002). In the digital artdomain some of these tasks would be highly interrelated(for instance, visualisation, exploration andcomposition). Michael Terry and Elizabeth D. Mynatthighlight the need for support of Schon's theory ofreflection-in-action: ncar-term experimentation(previews of the results of actions), longer-termvariations, and evaluation of actions, each demonstratedin the "Side Views" application (Terry & Mynatt. 2002).

These general requirements map well onto desiredtechnological features of creative environments. Someexamples of these follow:

3.1. Visualisation

Obviously, all creative environments are dynamic,interactive systems - even if the finished piece is non-dynamic and/or non-interactive. Technologically, thiswould mean the standard provision of, for example, adouble-buffered graphical display, on more than onescreen, which is capable of showing the artwork andenvironmental controls and, if relevant, multi-channel,multi-tracked audio.

The artist should be able to visualise aspects of thedevelopment in different ways. For instance, a section ofcode could be viewed as text, a diagram, a plain- orpseudo-English explanation (with the aid of metadata todescribe the semantics of the code) or as an interactiveprocess in which the artist can try inputs and witness theoutputs. The same applies for debugging. For example,adding a "watch" to a variable which is a y-coordinateshould cause a visual indication of the coordinate to beoverlaid on the space to which it applies.

The work of Manfred Mohr, for example, is centrallyabout handling data that can only be visualised with theaid of a computer. because of its multi-dimensionalcomplexity (Mohr, 2002), see Figure 2.

3.2. Exploration and Thinking

Generally, as with any interactive system, theprinciples of good HCI design (most importantly, usertesting) should be followed. Specifically in this context,the results of changes made to the environment shouldbe immediately apparent where possible, either during

execution of the artwork or in preview. Code-modification during runtime IS a feature of someinterpreted programming languages such as Max(Edmonds et al., 2003), and so-called "late-binding"virtual machines such as Squeak Small talk (Ingalls.Kaehler. Maloney, Wallace. & Kay, 1997). Manual-override variable modification during runtime should besupported.

Yasunao Tone's work with Creativity and CognitionStudios used Max/MSP as the basis of a very openexploration in which, quite deliberately, nobody newwhat the outcome might be. In this case exploration intothe unknown was at the core of the approach to the art(Edmonds et al., 2003).

3.3. Reviewing

As is generally understood (Lieberman & Fry, 1997),the ability to undo/redo and keep/recover/move previousversions of the program/artwork is useful, as would bethe ability to artificially slow down or step through aprocess in order to understand its behaviour moreclearly.

3.4. Holistic View

The ability to "fold up" complex processes orstructures into simple entities allows the artist to take aholistic view, This is a feature of many visualprogramming languages (Edmonds et al., 2003) anddocument-centric creativity support tools (for example,the "Document Map" feature of MS Word).Additionally, the functionality (particularly that providedby the technologist) should be expressible in a way thatis intelligible to the artist.

3.5. Immersion in the Activity

This IS not only a physical environmentalrequirement, but it is also important not to burden theartist with the minutiae of operating the interface ordealing with peripheral tasks. For example, syntax errorsshould be fixed automatically or semi-automatically, ifpossible. For example, the pedantic requirement forsemicolons at the ends of lines of code should beremoved or reduced to the level of automation to avoidannoying interruptions to the user's process. Asmentioned in 3.2, the delay of the code-compile-executeeye Ie should be minimised.

3.6. Breaking with Convention

The creative environment should allow the breakingof its own convention - its own modification andextension, which, when taken to the extreme, impliesthat the environment should be created in (a subset of)itself, and should allow its own manipulation. Suchrecursion is often seen in Art-Technology collaborationslike the one depicted in Figure 1, where the technologist

47

Table 1. Depth of interactivity (input)

Tvoe of Inout Additional Technoloeical imolicationsNone (except potentially the passage of time). Few, bearing in mind that the artwork may need toe.g. Pictures; film; animation; recorded music be smppedor resffined

Simple input: the user or physical environment crosses thresholds or The actions of the user, and possibly the actions ofmakes selections, usually continuous along one dimension only. the program, can be represented in a flow-chart-e.g. Switches; Hyperlinks; Interactive DVD; Simple Motion Sensing; style diagram, and be debugged relatively easily.specifically cubeLife (Everitt, Turner, 2002) Some discrete input must be derived, using simple

signal analysis, from continuous input (forexample, sensing a heartbeat).

Complex input: the user or physical environment provides input that can There is a needfor signal processing andtake a large number of values, permutations, or dimensions, allowing for calibration tools in the environment. For example,greater expressivity. fuzzy logic processing. Debugging is more complexe.g. Text; Speech; Gesture; Video; Continuous Biosensors; specifically - manually-overriding the input should beIamascope (Fels & Mase, 1999) supported

Table 2. Depth of interactivity (output)

Type of Output Additional Technological implication»None None. Although conceptual artists may disagree,

we presume that all art has an output of some sort.Simple output: The art outputs only one result, or one of a discrete set of The provision of tools for pre-calculating,results (the intricacy of these results is irrelevant for this case). collecting and enumerating these states, if there aree.g. Static art; interactive DVD; linearly-animated art; most "predictable a large number of them, would be useful. Thebehaviour " art (see Table 3). operation of the artwork might also be described

with flow-charts, or state-event modelling.Complex output: The an outputs any of an extremely large set of states. Mathematical modelling, AI and, signal processinge.g. Virtual environments; generative art tools are all notentiallv useful.

places himself in the creative environment which he hasconstructed for the artist. In the software world, this isless common, although several compilers and tools forprogramming are written in the very languages theysupport.

The environment should also be general. In otherwords, every problem that is solvable by computershould have some solution in the environment(Campbell, 2003).

A number of examples of such behaviour have beenseen to be partly supported by interpreted visualprogramming methods (Edmonds et aI., 2003). Theimmediacy of feedback and the holistic view providedseem to be significant factors.

4. Identifiable Creativity SupportFeatures for Digital Art

The authors examined digital artworks created inprevious studies (L. Candy & E. A. Edmonds, 2002) andpublished descriptions of works, e.g. the review byStephen Wilson (Wilson, 2002). By considering theexternal attributes of these works, it was possible tobegin to map out the specific technological requirementsof the environments which gave rise to them, whichaugment the list in the previous section. These externalattributes fall into two general categories - the waydigital artworks are presented, and their apparent

behaviour. (The purpose here is not to "classify" art, norto present an exhaustive review, but to show howconsidering various aspects and dimensions of art canhelp describe future technological requirements.)

4.1. PresentationDigital art has brought with it artworks which

communicate, or are perceived, in two directions.Communication takes place on a number of levels, fromthe excitation of photons or electrons (the level at whichhardware and software operates) to the 'obvious',concrete object (the level at which both artists andtechnologists can communicate), to the 'subtle' languageof the work (the level at which artists conceptualise).

The technological communications requirements ofcompleted artworks are straightforward to identify byobserving them. To support the most common paradigm,onscreen drawing of 2D output and mouse-and-keyboard input, would be the trivial case. Commonextensions to simple visual media are more complexforms such as video and 3D. Of these, 3D output isparticularly difficult to satisfy convincingly, because itoften imposes a "look" on the image as a result of theuse of conventional 3D rendering techniques. Ideally,artists would be able to modify the rendering engine tocustomise the depiction of 3D objects.

48

Table 3. Complexity of change

Level of Complexity Additional Technolaeical imnlicationsPredictable: the artist, andfamiliarised audience, is able to Bug-jinding is relatively easy. Predictable art normally haspredict what should happen in response to certain events. discrete input and output, or no input.e.z. Interactive D VDExploratory: the art produces interesting responses to stimuli, Bug-fixing is more complex, as many permutations of complexwhich may be entirely deterministic and comprehensible, but inputs and outputs all need to be evaluated. Aids to testing andthe affordance of exploration, or new comprehension by the process-control displays are paramount.artist or audience is imperative.e.g. Manfred Mohr (Mohr, 2002), (see Figure 2); 1amascope(Fels & Mase, 1999)Emergent-Algorithmic: The computer makes unpredictable Emergent art calls for extensive experimentation andchanges as a result of internal calculations or random debugging before it is exhibited. Additionally, emergent art isdecisions. the result of complex systems, so tools to inspect, override,e.g. Complex Systems; simulate or replay the complex interactions would be useful.

Artificial Life; snecificallv Olivine Trees (Miranda, 2001) The science of complex systems is in its infancy so such toolsEmergent-Environmental: The computer makes changes to its are not widely available.behaviour as a result of unpredictable factors outside of itscontrol, namely humans and natural entities (i.e. the weather)e.z, AI and Expert Systems

Saved and printed output should also be provided;a difficulty arises if there is a need for high-resolutionoutput. This means that the information for a vector-based description of the onscreen image should either bemaintained in real-time or calculable off-line, perhaps bya high-resolution recreation of the actions which formedthe display.

Audio output, via multi-channel, multi-trackwaveform or MIDI events should be supported.However, real-time digital audio processing (as withdigital video processing) is both an extremely complexand extremely wide-ranging technology, and is likely torequire the ability to communicate with third-partysoftware for anything more than simple (or created-from-scratch) applications.

Almost all specialised hardware devices communicatewith the serial, or USB protocol. The environmentshould ease the interpretation of the messages whichsuch devices use to communicate, for example by aidingthe mapping of high-level commands to sequences ofserial data (and vice-versa) and providing visualisationtools to examine these sequences at a higher level (forexample, video).

A growing number of artworks (and hardwaredevices) exhibit some form of communication over anetwork. Given the limitations of the Internet Protocol(IP), these artworks most often communicate in UDP(change-centric data; not all data must arrive at itsdestination) and TCP (event-centric data; all data isimportant). Again, aiding the mapping of these messagesto higher-level constructs is useful. In addition, someartworks may require distribution (one-to-many, ormany-to-many) or storage of messages via a centralserver, rather than just communication with one othercomputer (one-to-one). Tools for communicating with acomputer not under the control of an artist (for example,to get the weather) would also be beneficial.

Last but not least, the communication which acomputer has with files is important for storinginformation ready to be used at a later time. The threemost ubiquitous types are text files, binary files (difficultto work with for non-technologists) and SQL databases(again, an understanding of SQL is currently necessary).Fast previews and overviews of such files, again wherepossible, would aid their design and creation by theartist.

4.2. Behaviour

Stroud Cornock and Ernest Edmonds classified artaccording to behaviour in 1973 (Cornock & Edmonds,1973). The categories given were Static, Dynamic-Passive, Dynamic-Interactive and Dynamic-Interactive(varying). In the light of the present examination of thedigital art climate, we can reconfigure these categories inorder to elicit further technological requirements. Threeaxes of behaviour become apparent: depth ofinteractivity for both input and output, and complexity ofchange. A single artwork may exhibit one or morecharacteristics in each axis.The depth of the interactivity is independent for theinput (see Table 1) and output (see Table 2) media forany given artwork, and consists of three levels. Eachlevel, for both input and output, has certaintechnological implications for a supportive environment,beyond those which may arise from other requirements.

The complexity of change in behaviour of anartwork is an informal ranking of how predictable theartwork is (see Table 3). This is a subjectiveclassification, as it depends on the level at which theartwork is analysed (on a state-event level, everyoperation of a computer is inherently predictable.Conversely, it is often easier to execute the instructionthan to predict its result). The classification here

49

illustrates the types of exploration and debuggingactivities at the human level.

Figure 2. P-701/B, Manfred Mohr,enduraChrome/canvas, 1999 (from

http://research.it.uts.edu.au/creative/ccrs/gallery/mmohr/mmohr.htm)

5. ConclusionThe externally-apparent requirements of digital art

and the general requirements for creativity support mapclosely onto technological requirements for a supportiveenvironment for digital art. Digital artworks havefurther, distinctive and unique technologicalrequirements not obvious from an external observer'sperspective: types of behaviour and calculation notdirectly related to output (and simple input); thetechnologies used by the environment; and other, new-technology aspects. The proposed database will assistgreatly in identifying these further requirements, andthus can be used to actually construct an "environmentfor building environments".

(As a further consideration, technical quality shouldnot be ignored in meeting the requirements for such anenvironment. If recognised, open standards andprotocols are used for depiction, communication andinteraction within the environment, the flexibility,extensibility, and hence longevity of the artworks willbenefit.)

Existing programming languages are all examples ofcreative environments - someone with knowledge, skillsand perseverance can create an environment which does

exactly what is required. The problem is thatprogramming languages are hardly any good for anyonewithout these skills. As Alan Kay remarked, "I thoughtwe would be way beyond where we are now... Theirony is that today it looks pretty good. The result of our[Xerox PARC] work is techniques for doing software inan interesting and more powerful way. That was back inthe seventies. People today aren't doing a lot of work tomove programming to its next phase" (Steinberg, 2003).The issue for the HeI community in particular is to buildexploratory, rather than task-oriented environments,since it is exploration which most encourages creativity.

It is time to broaden the focus of creating new digitaltechnology from new computer tools towards newcreativity support systems, and from the programmer tothe generalised digital creator. In the time to come, thecomputer should be moulded by the ideas of anyone whowishes to create within its possibilities, much as wood orclay has for millennia. It will become a creative mediumin the truest sense. Digital artists are currently the closestfit to the ubiquitous digital creators of the future, and itis their work which is responsible for much of theinnovation in interaction technology today.

6. AcknowledgementsThe authors would like to thank the OZCHI reviewers,Dave Everitt, Keir Smith and Alastair Weakley for theirvaluable perspectives and insights on the issuespresented in this paper.

7. ReferencesCampbell, J. (2003). Lessons in Object-Oriented

Programming, fromhttp://www.cs.gub.ac.ukJ-J.Campbell/myweb/oop/oophtml/node4.html

Candy, L., & Edmonds, E. (2002). Interaction in Art andTechnology. Crossings: eJournal of Art and Technology(!lttp://crossings. tcd. iel), 2(1).

Candy, L., & Edmonds, E. A. (2002). Explorations in Art andTechnology: Springer.

Cornock, S., & Edmonds, E. (1973). The Creative ProcessWhere The Artist Is Amplified Or Superseded By TheComputer. Leonardo(6), 11-16.

Dilger, C. B. (2003). Ease in Composition Studies. PhDDissertation, University of Florida, Florida.

Dreyfus, H. L. (2001). On The Internet: Thinking in Action.New York: Routledge.

Edmonds, E., & Candy, L. (2002). Creativity, Art Practice andKnowledge. Communications of the ACM, 45(10),91-95.

Edmonds, E., Candy, L., Fell, M., Knott, R., Pauletto, S., &Weakley, A. (2003). Developing Interactive Art UsingVisual Programming. Paper presented at the Proceedings ofHCI International 2003, Crete.

Everitt, D. (Artist), G. Turner (Collaborator). (2002). cubeLife:An Internet-Resident Artwork [Interactive Art];http://www.cubelife.org.

Fels, S., & Mase, K. (1999). Iamascope: A Graphical MusicalInstrument. Computers and Graphics, 2(23), 277-286.

50

Ingalls, D., Kaehler, T., Maloney, 1., Wallace, S., & Kay, A.(1997). Back to the Future: The Story of Squeak, APractical Smalltalk Written in Itself Paper presented at the00PSLA'97 Conference, Atlanta, Georgia.

Levin, G., Ward, A., lia, & meta. (2001). 4x4 GenerativeDesign: Beyond Photoshop: Friends Of Ed.

Lieberman, H., & Fry, C. (1997). ZStep 95, A Reversible,Animated Source Code Stepper. In J. Stasko, 1. Domingue,M. Brown & B. Price (Eds.), Software Visualization:Programming as a Multimedia Experience. Cambridge,MA,: MIT Press.

Maeda, J. (1999). Design By Numbers: MIT Press.Maeda, 1. (2000). Maeda@Media: Thames & Hudson.Mamykina, L., Candy, L., & Edmonds, E. (2002).

Collaborative Creativity. Communications of the ACM,45(10),96-99.

Miranda, E. R. (2001). On the Origins and Evolution of Musicin Virtual Worlds. In D. W. Corne & P. 1. Bentley (Eds.),Creative Evolutionary Systems: Morgan Kaufmann.

Mohr, M. (2002). Generative Art. In L. Candy & E. A.Edmonds (Eds.), Explorations in Art and Technology:Springer.

Shneiderman, B. (2002). Creativity Support Tools:Establishing a framework of activities for creative work.Communications of the ACM, 45(10), 116-120.

Steinberg, D. H. (2003). Daddy, Are We There Yet? ADiscussion with Alan Kay, fromhttp://www.openp2p.com/pub/a/p2p/2003/04/03/alan kay.html

Suchman, L. (2002). Replicants and Irreductions: Affectiveencounters at the interface. European Association for theStudy of Science and Technology (EASST).

Terry, M., & Mynatt, E. D. (2002). Recognizing GreativeNeeds in User Interface Design. Paper presented at theCreativity and Cognition Conference 2002, Loughborough,UK.

Wilson, S. (2002). Information Arts: Intersections of Art,Science and Technology. Cambridge, Massachusetts: TheMIT Press.

www.smalltalk.org: Alan Kay. (2003). fromhttp://www.smalltalk.org/alankay.html

51