Intelligent visualization in a planning simulation

Robert St. Amant, Christopher G. Healey, Mark Riedl,Sarat Kocherlakota, David A. Pegram, and Mika Torhola

Department of Computer ScienceNorth Carolina State University

Box 7534Raleigh, NC 27695-7534stamant@csc.ncsu.edu

ABSTRACT

This paper describes visualization techniques for interac-tive planning in a physical force simulation called AFS.We have developed a 3D environment in which texturesare overlaid on a simulated landscape to convey informa-tion about environmental properties, agent actions, andpossible strategies. Scenes are presented, via automatedcamera planning, such that in some cases agent goals canbe induced visually with little effort. These two areas ofvisualization functionality in AFS exploit properties ofhuman low-level and intermediate-level vision, respec-tively. This paper presents AFS, its visualization environ-ment, and an experiment we have run to explore therelationship between AFS visualizations and the high-levelplanning process.

INTRODUCTION

In mixed-initiative planning, users collaborate with anautomated assistant to generate and carry out differentcourses of action. Effective collaboration between ahuman planner and an automated planning system requiresthat the participants work in areas where they performbest, use appropriate representations for communication,and effectively acquire and transfer authority for planningtasks [1, 5]. A number of mixed-initiative systems havebeen developed in planning and natural language process-ing research (e.g., TRAINS [11], TRIPS [12], COL-LAGEN [23], AIDE [24, 25]), and significant progress hasbeen made on abstract models of mixed initiative (e.g.[7]). Nevertheless, although the broad outlines of the areaare gradually becoming better understood, basic questionsabout user interaction techniques for mixed-initiativeassistance remain open.The term mixed-initiative assistance covers a wide rangeof potential activities, including providing timely informa-tion for situation assessment, helping users focus on criti-cal problem areas, making suggestions about appropriateactions, and handling plans and actions delegated by theuser. Many of the systems developed to date have relied onnatural language in interacting with users, an appropriatechoice for many situations. Our interest, in contrast, lies ina different interaction style, one more closely associatedwith conventional graphical user interfaces.

Direct manipulation techniques combined with graphicaldata presentation (which we will refer to here simply asGUI techniques) dominate modern interactive software.The purported benefits of GUIs, in comparison to otherstyles of interaction, include reduced error rates, fasterlearning and better retention, and facilitation of explor-atory behavior. GUI interfaces gain these benefits by offer-ing users a structured, predictable environment: like real-world objects, static software objects remain static overtime; environmental response to a given action is the sameif the action is repeated under the same conditions; actionsare usually taken at the user's direction and pace, ratherthan those of the environment; the environment does notinitiate activity, but rather only responds to user actions[25]. These properties reduce the space of decisions theuser must make to a more manageable level (e.g., timepressure, uncertainty, and environmental instability,including exogenous events, are abstracted away.) Unfor-tunately, the same properties that help users solve prob-lems working alone also limit the role of an assistant. Howcan an assistant contribute effectively to the problem-solv-ing process if it is not allowed, for example, to take visibleactions that the user may not yet have thought of?We believe that GUI techniques can benefit the interactionbetween a user and an intelligent assistant, but that someconcessions must be made in the design of the assistant.Our approach has been to emphasize the visual communi-cation abilities of an assistant, so that it can use the visualGUI environment to guide and sometimes constrain thepotential actions of the user. Our work on mixed-initiativeassistance has focused on strategic, physical planningproblems in AFS, an abstract force simulator [2, 22]. AFSis a general-purpose simulation system that supportsexperimentation with interactive planning techniques andtheir relationship to physical processes. AFS incorporatesan assistant that works behind the scenes to generate planspotentially helpful to the user; a visualization interfacepresents these plans and their supporting information tothe user by graphical means, relying as much as possibleon visual techniques rather than language (i.e., text, sym-bols, or iconic representations) for communication. Ourgoal is not to build an intelligent, collaborative assistantthat relies solely on direct manipulation and graphics

Proceedings Intelligent User Interfaces 2001

S

T

. A

MANT

, H

EALEY

, R

IEDL

ET

AL

.

2

(though intelligent rooms and ubiquitous computingresearch suggest that this is at least conceivable) but ratherto gain a better grasp of the relationship between GUItechniques and mixed-initiative assistance. We believe thata better understanding of this relationship may lead toimproved problem-solving performance and increaseduser acceptance of intelligent assistants.Although our work arises from research on planning in theuser interface, it can also be seen as a form of intelligentvisualization, a staple of research in the intelligent userinterfaces community. Intelligent visualization researchershave built systems for automatic explanation generation,intelligent tutoring, and other tasks [10, 20, 8, 9, 14, 3, 4],relying on many of the same sources we use. The require-ments for our work differ in some ways from these efforts,however: the assistant and the user observe an externalprocess, each able to guide it but without complete con-trol; camera manipulation is viewed as a means of commu-nication, rather than only a mechanism for visualorientation; problem-solving is reactive and opportunistic,with no extended narratives involved; neither party acts ina fixed role in the interaction (e.g., user as commander orstudent, assistant as tutor or information provider.) None-theless similarities to existing systems will be clear. The remainder of this paper is structured as follows. TheAFS section describes the simulation, which provides aconcrete setting for interaction issues relevant to mixed-initiative planning. We describe the visualizations AFSproduces and explain their relationship to models ofhuman visual processing. The experimentation section dis-cusses a study of user consistency in associating visualfeatures with specific physical interpretations; this consis-tency will eventually be exploited by AFS to convey theplanning intentions of the assistant in an unobtrusive, natu-ral-seeming fashion in the physical domain. In the conclu-sion, we tie our research to ongoing efforts to formalizethe concept of affordances in the user interface.

AFS

AFS provides a physical domain in which abstract agents(which we alternatively call “agents,” “force units,” orsimply “blobs”) can interact, based generally on Newto-nian physics. Units and inanimate objects have mass, size,and shape; they may be solid or permeable; they movewith variable friction over a domain-dependent surface;they apply force to one another, causing damage/massreduction.In AFS's Capture the Flag (CTF) domain, two teams offorce units move over a terrain, their travel constrained bymountains, water, and forests. Each team is responsible fordefending a set of stationary flags. A team successfullycompletes a scenario by destroying all the members of theopposing team or capturing all of its flags. Figure 1 showsa sample scenario, in a birds-eye or plan view. In thisdomain, as in all AFS domains, force units rely on a small

set of primitive physical actions: they may move from onelocation to another and apply-force to other units andobjects such as flags. These actions can be specialized andcombined in various ways to form higher level strategies,such as blocking a pass, encircling a flag, attacking anopponent in a group, and so forth. Plan execution andmonitoring is provided by a hierarchical planner at thecenter of the system.Ordinary interaction with AFS is via direct manipulation.The user can direct agents by selecting them and assigningto them either low-level actions or higher-level plans. Onerole of the assistant is to interpret the strategic situation inthe unfolding simulation, to inform the user of significantrelationships or events, and to suggest ways of dealingwith them, in order to help the user make informed deci-sions Visual communication toward these ends is carriedout by two means: texture-based visualizations and sce-nario-based camera planning. We discuss each type inturn.

Texture-based visualizations

Our work on texture-based visualization studies ways ofharnessing the low-level human visual system to the task.When we look at an image, certain visual features can beidentified very quickly, without the need for search. Thesefeatures are often called preattentive, because their detec-tion precedes focused attention in the low-level humanvisual system [27, 30]. Preattentive features include visualproperties like color, brightness, orientation, size, andmotion. When applied properly, these features can be usedto perform exploratory data analysis. Examples includesearching for data elements with a unique feature, identi-fying the boundaries between groups of elements withcommon features, tracking groups of elements as theymove in time and space, and estimating the number of ele-ments with a specific visual feature. Preattentive tasks areperformed very rapidly and accurately; they can often becompleted in a “single glance'' of 200ms or less. The timerequired for task completion is furthermore independent ofdisplay size; users can increase the number of data ele-ments in a display with little or no increase in the timerequired to analyze the display.Our research focuses on identifying such findings in thevision and psychophysical literature, then extending theseresults and integrating them into a visualization environ-ment. To date, we have compiled an interlocking collec-tion of results on the use of color (hue and luminance) [15]and texture (size, density, and regularity) [16, 17] for mul-tidimensional visualization. These results have been usedto visualize a number of real-world applications includingmedical scans [26], weather tracking [16, 17], and scien-tific simulations [19].In our AFS work, we have applied these findings to thepresentation of strategic, spatially distributed informationto assist the user in making planning decisions. The design

Proceedings Intelligent User Interfaces 2001

S

T

. A

MANT

, H

EALEY

, R

IEDL

ET

AL

.

3

of our visualizations in AFS and the work cited above cen-ters around the concept of a perceptual texture element, orpexel. Pexels are graphical icons that collectively conveycolor and texture information: hue, luminance, size, den-sity, and regularity, among other possibilities. Pexelsappear in the visualization in Figure 2 as small verticalstrips of color standing on end over the landscape. In thisvisualization, for example, the height of a pexel representsthe shortest time it will take any agent to reach a givenlocation; the color of a pexel corresponds to the team ofthat agent. As a simulation unfolds, the user sees the localcolors and heights of the pexel field spread and change; itbecomes immediately obvious when a red or blue flag isenveloped by a pexel field of the opposite color, indicatingthat it is in danger of capture.This is not intelligent assistance in any significant sense;however, the ability to manipulate the association betweentextures and strategic information in the simulation doesprovide an assistant with an important communicativetool. We revisit this point in the Experimentation section.

Scenario-based camera planning

The Capture the Flag scenario is presented in an environ-ment that allows the user to navigate using an “eye inhand” interface. The user is free to view the scenario fromany position and angle. The automated system is also ableto manipulate the camera through a camera-planning mod-ule. In particular, the camera can be positioned by the sys-tem to present the scenario such that certain user actionsare more readily realizable that others. Using knowledge

of situated problem solving and visualization, the cameraplanner attempts to situate the camera, and thereby situatethe embodied user, in order to control the affordances per-ceived within the virtual environment. In this way, the AFSsystem can lead the user into taking specific actions.Our approach to camera planning treats the problem as aform of constraint satisfaction. Constraints are placed onwhat the resulting visualization should look like and thecamera planner must meet those constraints to place thecamera in the optimal position. Inconsistent constraintscause conflict, which must be resolved by relaxing one ofthe conflicting constraints to some degree. Based on thehigh-level goals assigned to the force-units in the scenario,a set of visualization constraints are fed into the cameraplanner. The camera planner uses these constraints to eval-uate possible camera positions as it performs a limiteddepth hill-climbing search. Some of the constraints thatcan be passed into the camera planner are given below.Most are related to conveying a suggestion that one agentshould attack a specific opponent or capture a flag.

•

In-scene:

Whether the agents and objects participat-ing in the action are visible.

•

Centered-vertical:

Whether the agents and objectsare centered vertically in the scene.

•

Background:

Whether “distractor” agents are visi-ble.

•

Attack-angle:

How closely the “optimal” view ismet: from an attacking agent to the center of a tar-get object.

•

Agent-occlusions:

Whether any agents obscure par-ticipating agents or objects.

•

Terrain-occlusion:

Whether uneven terrain (e.g. amountain range) obscures agents or objects.

•

Too-low:

Whether the angle of view is too low toshow sufficient context.

•

Viewing-distance:

How closely a specific, constantviewing distance is matched.

These constraints are represented by heuristic functionsthat execute in sequence to evaluate every candidate cam-era position during the search. The constraints returnnumerical values and the camera planner satisfies with thesmallest total score. To reduce the size of the search space,we approximate the solution algorithmically and selectthat as a starting point. The constraints are ordered byimportance and weighted so that more important con-straints are less likely to be relaxed before less importantconstraints. For example, terrain-occlusion is ranked highin importance because it is essential that specified objectsare visible to the user. The weight values for each con-straint were determined experimentally.The constraints coded into the camera planner capture therelationships between objects to be viewed in a givenscene. To accomplish this is a manner consistent with our

Figure 1. AFS plan view

Proceedings Intelligent User Interfaces 2001

S

T

. A

MANT

, H

EALEY

, R

IEDL

ET

AL

.

4

model of visualization in situated environments, we basedour heuristic functions on plausible visual routines in theintermediate layer of vision. The intermediate layer ofvision is responsible for capturing and decoding informa-tion about relationships between objects in the visual field,such as inside/outside, above/below, less than/greater than(angle of body orientation), etc. The visual routines arecomputationally implemented in AFS using geometriccomparisons that simulate the effects of the visual routinesthemselves in order to provide the camera-planning mod-ule with sound, rapidly calculable heuristics.A sample visualization is shown in Figure 3. The assistanthas smoothly moved the camera from its previous positionto the current one, to convey the suggestion that the blueforce unit in the foreground should attempt to capture thered flag immediately in front of it. AFS can also generatevisualizations for comparing opponent agents, to help theuser evaluate a potential suggestion. Further camera-plan-ning visualizations have been designed and user-testedwith paper diagrams, but these have not yet been imple-mented. In contrast to the texture-based visualizations,here the system contributes actual assistance, via sugges-tions and the implicit ruling out of alternatives. The assis-tant currently operates under the restriction that the visiblescene may not be modified to improve a visualization

(e.g., simply removing irrelevant agents, flags, or land-scape features). As we come to a better understanding ofthe capabilities and limitations of the system under thisrestriction, it may be relaxed.

EXPERIMENTATION

We gain some significant advantages in relying on rela-tively low-level perceptual mechanisms in our visualiza-tion techniques—speed, high volume, multi-dimensionaldata integration through texture manipulation, effective(though limited) recognition of simple agent plans byexploitation of visual routines. One of the drawbacks ofthe approach, however, is that the interaction takes placebelow the cognitive level—that is, in AFS we haveadopted the traditional view of planning as search througha problem space, with states represented in symbolicterms. Although users are able to extract properties of thevisualizations efficiently, there is no necessary relationshipbetween these properties and abstract concepts relevant toplanning problems.A straightforward solution is to rely on conventions forvisual representation: the height of a pexel corresponds tosome plan or situation assessment variable x, its hue, lumi-nance, density, and so forth to other variables. These rela-tionships must be learned by the user before the

Figure 2. Texture-based visualization of strategic status in AFS

Proceedings Intelligent User Interfaces 2001

S

T

. A

MANT

, H

EALEY

, R

IEDL

ET

AL

.

5

visualization can be interpreted. We have experimentedwith a different approach, however, one especially well-suited to the physical planning domain. Texture fields suchas the ones shown in Figures 2 and 3 can be viewed asabstractions for conveying information, but that can alsobe interpreted in physical terms. It is possible to see thepexel field as a field of grass, for example, or other, similarvisual texture-producing ground cover. We naturally asso-ciate such textures with our physical interaction with it;that is, we are attuned to its affordances [13]. If users con-sistently relate specific visual texture properties to specificphysical properties, such as ease of movement, direction,or speed, then AFS might exploit this relationship to con-vey physical planning suggestions in visual terms, withoutdepending on the user's learned knowledge of display con-ventions.To explore this issue, we conducted an experiment basedon artificial visualization scenarios. Our 20 subjects werestudents and interns, both men and women, working atNorth Carolina State University. Their ages variedbetween 20 and 30. Each subject was presented with asequence of snapshots consisting of a 3D field of red pex-els surrounding a blue ball (which would represent anagent or blob in AFS), as shown in Figures 4 and 5. Valuesof blob radius, pexel density, and pexel height, which we

will refer to as the variables Radius, Density, and Height,were varied across the snapshots. For each snapshot, thesubject was asked, “If the ball were to be rolled across thefield shown, how fast would it move?” A discrete set ofchoices was available from a pop up menu, ranging fromSpeed 1, the lowest, to Speed 10, the highest. Subjectswere allowed to experiment with a few different snapshotsbefore starting the experiment proper, in order to developan internal calibration of speeds for the textures theywould see.Because of the small number of variables we chose toexamine, a full factorial design was possible. Radius alter-nated between 0.60 and 0.75 units; Height ranged amongfour values of 0.25, 0.40, 0.55, and 0.75 units; Densityranged similarly among four values of 0.25, 0.40, 0.55,and 0.75 units. The units of measurement here are unim-portant; the specific values were chosen after prototypingand testing by the experimenters. Thirty two combinationsof these values are possible (2 x 4 x 4); each subject sawevery combination three times, in randomized order. Thedata resulting from the experiment consisted of a Subjectidentifier, the specific values of Radius, Density, andHeight for each snapshot, the Speed selected, and theDuration of the user's selection action.

Figure 3. Camera planning visualization result in AFS

Proceedings Intelligent User Interfaces 2001

S

T

. A

MANT

, H

EALEY

, R

IEDL

ET

AL

.

6

An analysis of variance shows that all factors have a sig-nificant influence on the mean value of speed (for RadiusF=26.48, p<0.0001; Density, F=51.71, p<0.0001, Height,F=700.05, p<0.0001.) Subjects most strongly associatedHeight with Speed, followed by Density and then byRadius. We found a significant effect of Subject on Speedas well (F=9.80, p<0.0001), which limits the generality ofthis study, but an exploratory examination of the data isnevertheless informative.Figure 6 shows projections through the experimentaldataset. In the top graph, for example, we collected all thevalues for each combination of Subject and Height, merg-ing the different values of Radius and Density, and com-puted the mean of each partition. For each subject we then

see four values, shown as marks on a line, that representthe average speed of the blob for each of the four possiblepexel height values: 0.25, 0.40, 0.55, 0.75. The other twographs are constructed analogously for Radius and Den-sity. We discarded the first subject’s results because of pro-cedural irregularities, but found significant patterns ofsimilarity among the remaining subjects. In general, sub-jects judged that the higher the pexel field, the slower ablob will be able to move. (Subject 2’s results are consis-tent with a reversal of the magnitude of speed choices;talking with subjects afterwards we found this to be aminor source of confusion for others as well.) Except forsubjects 19 and 20, a comparable pattern holds for Radius:larger blobs are judged to be able to move faster thansmaller ones. Finally, a more complex and slightly unex-pected pattern holds for Density. For most subjects, higherpexel density is associated with higher speed, which corre-spond to a physical interpretation in which the blob rollsover the field rather than through the individual pexels Fora few subjects (e.g., 6, 12, 13), however, the lowest densityaffords faster movement as well, producing a U-shapedrelationship between Density and Speed.This exploratory study is a small but promising step in ourresearch. Given experimentally validated relationships ofthis kind, we intend to extend the current assistant in AFSto convey suggestions about agent assignments by the gen-eration of appropriate textures. For example, the assistantcurrently does path planning for force units, and can drawsuch paths in the plan view of the simulation to show itssuggested courses of action. If the assistant’s suggestionsconflict with the user’s intended plans, however, the pathicons are superfluous and possibly confusing. Texturesoffer a less obtrusive solution: the assistant can map lowerheight pexels over the regions over which force units canmove more safely, or possibly to greater strategic effect,without forcing an obvious choice on the user. We are cur-rently running a comparable study of user inference ofdirection of movement based on the orientation of pexels;we will continue by designing more formal experimentalstudies of the relative associations between texture fea-tures and physical behavior. We hope to identify consistentrelationships between visual textures and physical proper-ties such as speed and direction, but also accessibility,safety, vantage point utility, among others.

DISCUSSION

Our work in this area grows out of an interest in situatedproblem-solving and affordances in the user interface [25].Situated problem solving differs from more conventionalforms of problem-solving. When embodied within anenvironment, how one perceives the environment and whatis perceptible within the environment is of paramountimportance to what actions one chooses to make. Theenvironment itself provides cues—affordances—aboutwhat appropriate actions there are, but these cues must

Figure 4. Sample experiment trial (A)

Figure 5. Sample experiment trial (B)

Proceedings Intelligent User Interfaces 2001

S

T

. A

MANT

, H

EALEY

, R

IEDL

ET

AL

.

7

first be perceived and understood within the context ofone’s relationship with elements within the environment.For example, the brink of a gulch may be perceived asaffording falling off of, but if one is moving towards thebrink at a high velocity, the brink may suddenly be per-ceived to afford leaping across [28]. Likewise, one cannotmake use of tools within the environment if they are hid-den from view or out of reach.In our view, the process of situated problem solving is iter-ative in nature, involving three stages: perception, inten-tion, and action. The first stage, perception, involves usingthe senses to determine possible actions that can be madein the environment. This process involves registering theaffordances the environment provides for action. The sec-ond stage, intention, involves taking the results of percep-tion and choosing the best action that will advance theorganism towards a given goal. The final stage, action,occurs when intent is transformed into behavior by inter-acting directly with the environment [21]. Actuators in theenvironment, including the organisms own motor control,are activated in order to bring the organism closer toachieving its goal. Actuators affect the environment andthe organism’s relationship with the environment, result-ing in a new situation. The cycle iterates, starting with per-ception of the environment and how it has been changedby the previous cycle. New intentions are formed and car-ried out.

Intentions are formed when there is a mismatch betweenthe goals internal to the organism and the state of theexternal world [21]. Before intentions can be formed, theorganism must sense its own situation within the surround-ing environment: it must become aware of the environ-ment’s current state as well as its relationship with theenvironment. Needless to say, the perceptual senses playan important role in situated problem solving. The sensoryapparatus, however, cannot be merely instruments forrecording sensory stimuli; they must proactively interpretand transform the sensory stimuli into affordances. The perception of affordances is primarily a cognitiveinteraction with perceptual stimuli. Affordances do notexist without an organism to perceive them. By presentingplans three-dimensionally, viewed from a camera that isallowed to move anywhere within the playing-space, wecall on the metaphor of embodiment within a 3D environ-ment. By embodiment we mean that the user interacts withthe virtual 3D environment as if he were present in theenvironment at the location of the camera. With the userembodied in a 3D virtual environment, we are able tomake use of the same problem-solving strategies one useswhen interacting with the real-world environment.

REFERENCES

1. Allen, J. F. 1994. Mixed initiative planning: Positionpaper. [WWW document].

ARPA/Rome Labs PlanningInitiative Workshop.

Figure 6. Subject consistency in visual interpretation

Proceedings Intelligent User Interfaces 2001

S

T

. A

MANT

, H

EALEY

, R

IEDL

ET

AL

.

8

2. Atkin, M.; Westbrook, D. L.; Cohen, P. R.; andJorstad, G. D. 1998. AFS and HAC: Domain-generalagent simulation and control.

AAAI-98 Workshop onSoftware Tools for Developing Agents,

pp. 89-95. 3. Bares, W. H., and Lester, J. C. 1997. Realtime genera-

tion of customized 3D animated explanations forknowledge-based learning environments.

ProceedingsAAAI-97,

347-354.4. Bares, W. H., and Lester, J. C. 1999. Intelligent multi-

shot visualization interfaces for dynamic 3D worlds,

Proceedings IUI-99,

119-126.5. Burstein, M. H., and McDermott, D. V. 1996. Issues in

the development of human-computer mixed-initiativeplanning. In Cognitive Technology:

In Search of aHumane Interface,

pp. 285-303. Elsevier.6. Chapman, D. 1992. Intermediate Vision: Architecture,

Implementation, and Use,

Cognitive Science16(4):

491-537.7. Cohen, R., Allaby, C., Cumbaa, C., Fitzgerald, M., Ho,

K., Hui, B., Latulipe, C., Lu, F., Moussa, N., Pooley,D., Qian, A., and Siddiqi, S. 1998. What is Initiative?

User Modeling and User-Adapted Interaction 8(3/4):

171-214. 8. Drucker, S., and Zelter, D. 1994. Intelligent camera

control in a virtual environment.

In ProceedingsGraphics Interface ’94,

190-199.9. Drucker, S., and Zelter, D. 1995. Camdroid: A system

for implementing intelligent camera control. Proceed-ings

1995 Symposium on Interactive 3D Graphics,

139-144.10. Feiner, S. K., and McKeown, K. 1991. Automating the

generation of coordinated multimedia explanations.

IEEE Computer 24(10):

33-41. 11. Ferguson, G., and Allen, J. 1996. Trains-95: Towards a

mixed-initiative planning assistant.

Proceedings Artifi-cial Intelligence Planning Systems,

pp. 70-77.

12. Ferguson, G., and Allen, J. 1998. Trips: An intelligentintegrated problem-solving assistant.

ProceedingsAAAI-98.

13. Gibson, J. 1979.

The Ecological Approach to VisualPerception.

Houghton Mifflin. 14. He, L. W., Cohen, M. F., and Salesin, D. H. 1996. The

virtual cinematographer: a paradigm for automaticreal-time camera control and directing.

ProceedingsSIGGRAPH ’96,

217-224. ACM Press.15. Healey, C. G. 1996. Choosing effective colours for

data visualization,

Proceedings Visualization ’96,

pp.263–270.

16. Healey, C. G., and Enns, J. T. 1998. Building percep-tual textures to visualize multidimensional detests,

Proceedings Visualization ’98,

pp. 111–118.17. Healey, C. G., and Enns, J. T. 1999. Large detests at a

glance: Combining textures and colors in scientific

visualization,

IEEE Transactions on Visualization andComputer Graphics 5(2):

145–167. 18. Healey, C. G., Booth, K. S., and Enns, J. T. 1995.

Real-time multivariate data visualization using preat-tentive processing,

ACM Transactions on Modelingand Computer Simulation 5(3):

190–221.19. Healey, C. G. 1998. One the use of perceptual cues

and data mining for effective visualization of scientificdatasets,

Proceedings Graphics Interface ’98,

pp.177–184.

20. Karp, P., and Feiner, S. 1990. Issues in the automatedgeneration of animated presentations.

ProceedingsGraphics Interface ’90

, 39-48.21. Kugler, P.N. and Shaw, R.E. 1990. Symmetry and

symmetry-breaking in thermodynamic and epistemicengines: A coupling of first and second laws. In H.Haken & M. Stadler (eds.),

Synergetics of cognition,

pp. 296-331. Springer-Verlag.22. Pegram, D. A., St. Amant, R., and Riedl, M. O. 1999.

An approach to visual interaction in mixed-initiativeplanning.

AAAI-99 Workshop on Mixed-InitiativeIntelligence.

23. Rich, C., and Sidner, C. L. 1998. COLLAGEN: a col-laboration manager for software interface agents.

UserModeling and User-Adapted Interaction

8(3/4).24. St. Amant, R., and Cohen, P. R. 1998. Interaction with

a mixed-initiative system for exploratory data analy-sis,

Knowledge-Based Systems 10(5):

265–273.25. St. Amant, R. 1999. User interface affordances in a

planning representation.

Human Computer Interac-tion.

26. Tam, R., Healey, C. G., and Flak, B. 1997. Volumevisualization of abdominal aortic aneurysms,

Proceed-ings Visualization ’97,

pp. 43–50.27. Triesman, A., and Gelade, G. 1980. A feature-integra-

tion theory of attention,

Cognitive Psychology 12

, 97–136.

28. Turvey, M. T., and Shaw, R. E., 1979. The primacy ofperceiving: An ecological reformulation of perceptionfor understanding memory.

Perspectives on memoryresearch

, L. G. Nilsson (ed.), pp. 167-222. LawrenceErlbaum.

29. Ullman, S. 1996.

High-level vision.

Cambridge, MA:The MIT Press.

30. Wolfe, J. M. 1994. Guided Search 2.0: A revisedmodel of visual search,

Psychonomic Bulletin &Review 1(2):

202–238.