Cognitively Inspired and Perceptually Salient Graphic ...sara/pubs/fabrikant_etal_an09.pdf · Cognitively Inspired and Perceptually Salient Graphic Displays ... Cognitively Inspired

Cognitively Inspired and Perceptually SalientGraphic Displays for Efficient Spatial Inference

MakingSara Irina Fabrikant,∗ Stacy Rebich Hespanha,† and Mary Hegarty‡

∗Department of Geography, University of Zurich†Department of Geography, University of California Santa Barbara‡Department of Psychology, University of California Santa Barbara

Developing a visual hierarchy in map displays that is congruent with thematic levels of relevance is a fundamentalcartographic design task. Cartographers employ a set of visual variables (e.g., size, color hue, color value,orientation, etc.) for 2-D, static maps to systematically match levels of thematically relevant information toa perceptual hierarchy based on figure–ground relationships. In this article, we empirically investigate therelationship of thematic relevance and perceptual salience in static weather map displays. We are particularlyinterested in how novices’ viewing patterns are modified when thematically relevant items are made perceptuallymore salient through design. In essence, we are asking whether perceptually salient elements draw novice viewers’attention to thematically relevant information, whether or not users have domain knowledge.

In a factorial experiment, we ask novice participants to evaluate the wind direction in weather maps beforeand after training all participants on meteorological principles. Our empirical results suggest that display design(i.e., saliency) does not influence the accuracy of response, whether participants have prior knowledge or not(i.e., training). Analysis of the eye-movement patterns, however, suggests that display design does affect viewingbehavior and response time. These findings provide rare empirical evidence for generally accepted design practiceswithin the cartographic community (e.g., the effects of visual variables). We chose weather map displays as onetypical example of commonly used maps for our study, but the methods employed are generic enough to beapplicable to any spatial display (static or interactive) that might be produced by GIScientists, cartographers,geographic information system (GIS) practitioners, and others. Key Words: empirical study, eye-movement analysis,geographic visualization, spatial inference, weather maps.

Desarrollar una jerarquıa visual en los despliegues del mapa, que sea congruente con los niveles tematicos derelevancia, es una tarea fundamental de diseno cartografico. Los cartografos utilizan un conjunto de variablesvisuales (e.g., tamano, matiz de color, valor del color, orientacion, etc.) para mapas estaticos, de 2-D, a fin dealcanzar de manera sistematica niveles de informacion tematicamente relevante para una jerarquıa perceptualbasada en la relacion figura-realidad. De modo empırico, en este artıculo investigamos la relacion de relevanciatematica y notabilidad perceptual en despliegues de mapas meteorologicos estaticos. Nos interesa, en particular,la manera como se modifican en los novatos los patrones visuales, cuando ıtems tematicamente relevantes sehacen perceptualmente mucho mas salientes por medio de diseno. En esencia, lo que estamos preguntando es si

Annals of the Association of American Geographers, 100(1) 2010, pp. 1–17 C© 2010 by Association of American GeographersInitial submission, March 2008; revised submission, October 2008; final acceptance, November 2008

Published by Taylor & Francis, LLC.

Downloaded By: [Zentralbibliothek Zurich] At: 07:38 5 January 2010

2 Fabrikant, Hespanha, and Hegarty

los elementos que se destacan perceptualmente captan la atencion de observadores novatos hacia la informacionque es relevante desde el punto de vista tematico, sea que los usuarios tengan o no suficiente conocimiento deltema. En un experimento factorial, le pedimos a participantes novatos evaluar la direccion del viento en mapasmeteorologicos, antes y despues de que a todos los participantes se les ensenaran principios meteorologicos.Nuestros resultados empıricos sugieren que el diseno del despliegue (i.e., notabilidad) no influye en la exactitudde la respuesta, ası sea que los entrevistados hayan tenido conocimiento previo o no (i.e., entrenamiento). Elanalisis de los patrones de movimiento del ojo, sin embargo, sugiere que el diseno del despliegue en verdad afectala conducta de la observacion y el tiempo de respuesta. Estos descubrimientos nos brindan una rara evidenciaempırica sobre practicas de diseno generalmente aceptadas por la comunidad cartografica (e.g., los efectos delas variables visuales). Escogimos para nuestro estudio los despliegues del mapa del tiempo, a tıtulo de ejemplotıpico de mapas de uso comun, pero los metodos son lo suficientemente genericos para poder aplicarse a cualquierdespliegue espacial (estatico o interactivo) que sea producido por cientıficos de la informacion y comunicaciongeograficas, cartografos, practicantes de los sistemas de informacion geografica (SIG) y otros. Palabras clave: estudioempırico, analisis del movimiento del ojo, visualizacion geografica, inferencia espacial, mapas meteorologicos.

Emerging and ubiquitous spatial and nonspatialinformation technology (i.e., virtual globe view-ers, location-based services, including informa-

tion search and retrieval capabilities on the Internet,etc.) provides evidence of the “visual turn” in today’sinformation society. The use of complex, highly in-teractive displays has left the narrow domain of ex-perts in science (i.e., scientific visualization) and hasbeen reaching the information society at large, includ-ing broad audiences with varying backgrounds (Card,Mackinlay, and Shneiderman 1999). Many of the avail-able visual products, such as those disseminated throughthe Internet (e.g., mashups), or virtual and augmentedenvironment technologies (e.g., virtual globe viewers)are based on the intuition that people are visuo-spatiallycompetent and have no difficulty extracting meaningembedded in graphic displays for knowledge generationand decision making (Thomas and Cook 2005). GI-Science and related fields operate under the assumptionthat people are able to proactively explore graphics forspatial knowledge discovery, and are able to detect geo-graphic patterns, reveal spatial relationships, and thusmake intuitive sense of what is seen on a geographicinformation system (GIS) display. To this day, though,little is known about the effectiveness of graphic dis-plays for exploratory data analysis, problem solving,knowledge exploration, and learning (Shah and Miyake2005).

A few cognitive scientists and psychologists havestudied how externalized visual representations (e.g.,statistical graphs, organizational charts, maps, anima-tions etc.) interact with people’s internal visualizationcapabilities (Barkowsky et al. 2005). More than tenyears ago, however, Scaife and Rogers (1996) succinctlywarned about the paucity of work on determining howgraphical representations are themselves constructedand how display design decisions interact with high-

level cognitive processes for inference making andknowledge construction. These authors also lamentedthat there has been little progress toward a sound theo-retical and methodological framework that would allowa designer to evaluate existing graphical aids or improvethese aids in the future. Whereas researchers with differ-ent disciplinary backgrounds (i.e., Tufte 1990; Kosslyn1994; Card, Mackinlay, and Shneiderman 1999) havesystematically looked at design issues from various per-spectives, Scaife and Rogers’s (1996) call for a system-atic approach is still relevant today. They proposed thata useful framework would have to include a series ofindependent variables, such as the users’ level of expe-rience with graphic displays, the knowledge domain tobe visualized, and the type of tasks the displays wouldhave to support.

More recently this problem has been reidentifiedas one of the key research challenges for geovisualiza-tion research focusing on cognitive issues and usabilityput forward by the International Cartographic Asso-ciation’s Commission on Geovisualization (ICA Geo-vis; MacEachren and Kraak 2001a). Specifically, theseauthors point to the need to develop a theoretical frame-work based on cognitive principles to support a human-centered approach to geovisualization (MacEachrenand Kraak 2001b). Inspired by this call, Fabrikant andSkupin (2005) proposed a design framework for devel-oping cognitively plausible information visualizations.Extending on cognitive plausibility (Edwards 2001), aterm used by psychologists to assess the accuracy withwhich models are believed to represent human cogni-tion, Fabrikant and Skupin (2005) further suggestedthe notion of cognitively adequate visualizations. Cog-nitively adequate depictions are understood as graphicdisplays that not only support humans’ internal visu-alization capabilities but are able to augment people’smental visualization capabilities for complex reasoning


Cognitively Inspired and Perceptually Salient Graphic Displays for Efficient Spatial Inference Making 3

and problem solving in abstract domains (Fabrikant andSkupin 2005).

Visualization as Internal Representationand as Augmentation of Cognition

Research in cognitive science has shown that staticgraphics can facilitate comprehension, learning, mem-orization, problem solving, and communication, in-cluding inference of dynamic processes (Hegarty 1992;Hegarty and Sims 1994). Previous cognitive visualiza-tion research has focused mainly on identifying howhumans make inferences from graphics (regardless ofthe graphic quality of the display) but has not neces-sarily looked at the possible interplay of humans’ visualinference-making capacities (e.g., visual learning andreasoning) with external visual displays and their spe-cific graphic characteristics (Zhang and Norman 1994;Scaife and Rogers 1996). This is particularly true formaps or GIS displays. Hegarty (2004) suggested threepossibilities regarding the interplay between internalcognition (e.g., mental representations such as images)and external visualization (e.g., graphics, maps) in aneducational/learning context:

1. External visualizations might act as a cognitive pros-thetic for people with limited internal visualizationcapabilities. If this is true, people with low cognitivespatial visualization skills might benefit from well-designed external visualizations (Hegarty 2004).

2. Use of external and internal visualizations might de-pend on internal visualization. That is, a base ca-pacity to internally visualize might be needed totake advantage of external visualizations. If this istrue, people without internal visualization abilitymight not be able to take advantage of externalvisualizations.

3. External visualizations might augment internal vi-sualization for all people, regardless of a person’s in-dividual cognitive skill base (Card, Mackinlay, andShneiderman 1999; Thomas and Cook 2005).

The question remains as to how these three possi-bilities transfer into the exploratory geographic dataanalysis, learning, and spatial knowledge construc-tion domains. Another important question is whethermore abstract (i.e., thematic) maps improve a student’slearning performance, for example, when dealing withintangible environmental processes (e.g., weather phe-nomena) or intangible societal diffusion processes (e.g.,spread of disease). On the surface, one might generally

argue that based on an already established 5,000-year-old success story of map existence and map use, humanshave been quite successful at making inferences fromand with maps. This study, however, aims to answer thefundamental research questions of how and why cartog-raphers can augment (external) visuo-spatial displaysto provide more effective and efficient map displays forvisuo-spatial inference making.

We present a conceptual and methodological frame-work for investigating exactly how display design deci-sions compare with prior knowledge and training ininfluencing nondomain experts’ attention and infer-ence. This framework is supported by empirical datawe collected on an inference task using complexweather map displays. We are particularly interestedin how novices’ viewing patterns are modified whenthematically relevant items are made perceptuallymore salient through cartographic design for one spe-cific inference task relating to (nontangible) weatherphenomena. More generally, we investigate whetherperceptually salient elements draw novice viewers’attention (bottom-up processing) or whether theirattention is directed by domain knowledge of me-teorology (top-down processing). We chose weathermaps as test stimuli because of their potential com-plexity for spatial inference making (i.e., multivari-ate data sets) but also because of their attractivenessand wide dissemination through news media includingthe Internet (Monmonnier 1999). The simplification ofmapped weather data through mimetic icons and brightand colorful designs follows the intuition that moreattractive-looking weather maps would guide noviceusers in extracting the relevant information from com-plex map displays and thus facilitate inferences fromcomplex dynamic weather phenomena. This is just oneexample of how designers try to modify the graphicquality of a display to facilitate visual inference mak-ing. We did not choose weather maps simply for theirpopularity or to test them specifically but because theyare complex enough for spatial inference making. Ourcomplex inference-making task and visually rich stimuliare contrasted by quite a few studies on graphics com-prehension in cognitive science and psychology wherethis kind of work has been done with rather simpletasks (i.e., read-off of values) and on simple graphswith only a dozen or so data points (i.e., line or bargraphs).

Although we chose weather map displays for ourstudy (Figure 1) as one typical example of commonlyused maps, the methods employed are generic enough tobe applicable to any spatial display (static or interactive)



Figure 1. Typical mass media weathermap display example. ( c© Atmos-pheric Sciences, University of Illinoisat Urbana-Champaign. Used withpermission.)

that might be produced by cartographers, graphic de-signers, and others.

Matching Perceptual Salience withThematic Relevance

For a very long time cartographers have thrived in op-timizing the representation of dynamic spatio-temporalphenomena with static, spatial representations in theform of two-dimensional maps (Bertin 1967, 1983).Cartographers have been applying a set of visual vari-ables (e.g., size, color hue, color value, orientation, etc.)to mostly static 2-D maps to systematically match levelsof relevant thematic information to a perceptual hierar-chy based on figure–ground relationships (Bertin 1967,1983). Today these visual variables not only representthe design foundation of all well-designed cartographicmap displays (Dent 1999) but are seen more generallyas the basis for any successful information visualization(Card, Mackinlay, and Shneiderman 1999).

Developing a visual hierarchy in map displays that iscongruent with relevant thematic levels is a fundamen-tal cartographic design task (Dent 1999). Accordingto generic (and cartographic) design principles, a well-designed (cartographic) product’s theme, purpose, andtarget audience should be known before it is created

for maximizing effective and efficient communication,regardless of the target audience’s expertise. Accordingto MacEachren (1995), however, cartographic designtheory has been subjected to only limited empiricalverification. How can cartographers be sure that theirdesigns are indeed cognitively inspired and percep-tually salient? A design framework based on empiri-cal evidence would certainly help cartographers andgraphic designers to systematically evaluate their mapproducts.

Domain-specific design conventions that might notbe cognitively adequate or perceptually salient but haveaccumulated over time or in rapidly changing tech-nological settings might also be issues to consider forcertain (expert) audiences. Arguably, changing well-established design conventions might be often moreproblematic for expert users than for others. For exam-ple, for a long time meteorologists have been specifi-cally trained on and use a well-defined set of symbolsfor their weather forecasts. Similarly, air traffic con-trollers are trained on and rely on a specific vocabularyof graphic codes displayed on their radar screens whenguiding pilots.

Unfortunately, weather maps are rarely designedby cartographers and thus rarely follow long-standingand well-established cartographic design principles orspecific design conventions. The relationship between



Figure 2. Two different map design so-lutions containing the same weather in-formation. (A) Commonly used massmedia map. (B) Cartographically modi-fied map.

perceptual salience and thematic relevance in a static,2-D weather map is illustrated in Figure 2.

The two map designs in Figure 2 are information-ally equivalent; that is, they show the same thematicweather-relevant information (temperature and pres-sure). Informational equivalence, a term coined by Si-mon and Larkin (1987), expresses the idea that allinformation encoded in one representation is also in-ferable from the other, and vice versa. We posit thatdesign decisions (i.e., relationship between thematicrelevance and perceptual salience) affect computationalequivalence, another useful concept proposed by Simonand Larkin (1987) for assessing the effectiveness ofgraphic representation for inference making. Two rep-resentations are said to be computationally equivalentwhen any inference that is easily and quickly drawnfrom the encoded information in one display can beeasily and quickly drawn from the other (information-ally equivalent) display and vice versa. They suggest,for instance, that the advantages of graphics over textin general are computational, not because they con-tain more information but because the presentationof the information can support extremely useful andefficient (computational) inference-making processes.We contend that by systematically making themat-ically relevant information perceptually salient for aparticular inference task, display designers can facilitateuseful and efficient (computational) inference-makingprocesses.

Can we predict which parts of the displays would at-tract a nonexpert viewer’s gaze based solely on design?Applying Bertin’s (1967, 1983) visual variable theory,a cartographer might argue that the temperature in themass media-inspired weather map (a) has more the-matic emphasis, as the visually dominant color shadingis perceptually more salient than the pressure gradientdepicted with black isolines. In the cartographicallymodified map, (b) the pressure system dominates vi-

sually, and a viewer’s gaze might be attracted to itby additionally contrasting the relevant pressure cellswith highly saturated red (Highs) or blue colors (Lows)against the desaturated temperature color scheme. As-suming a task that depends on the information aboutpressure in the map (e.g., inferring wind direction,which depends on pressure), a cartographer could arguethat Figure 2B is the more appropriate display. The the-matically relevant information (e.g., pressure gradient)is purposefully made perceptually more salient throughdesign by applying the visual variable color hue andsize to the pressure system labels and by reducing colorsaturation of the temperature color scale.

To systematically investigate this design principlewe adopt an interdisciplinary approach using exper-imental methods from psychology and cognitive sci-ence, but also grounding our work in a solid designframework borrowed from cartography. In addition tothe commonly employed performance measures in con-trolled empirical experiments such as task response time(i.e., inference speed) and response accuracy (i.e., qual-ity of inference), we measure eye fixations, which al-low us to directly observe whether and how people’sviewing patterns change when thematically relevantitems are made perceptually more salient through de-sign. A compelling advantage of eye-movement record-ings, when compared to traditional experimental data(response time and accuracy), is that they provide rel-atively unobtrusive, real-time measures of visual atten-tion and cognitive information processing (Hendersonand Hollingworth 1998).

Eye-movement behavior reveals where people lookon a visual display. By correlating the response patternof the where (attention focus) and the what (informa-tion content at that location), one can more specifi-cally evaluate design issues in a graphic display used forproblem solving (Grant and Spivey 2003; Thomas andLleras 2007).



Eye-Movement Analysis as a Windowto Internal Visuo-Spatial InferenceMaking

Since the first half of the twentieth century, psy-chologists and other researchers have recorded humaneye movements to learn about how people read textsand view various static graphic displays such as adver-tisements, works of art, charts and diagrams, and so on(Wade and Tatler 2005; Duchowski 2007). The basicprinciple of eye-movement studies is based on the factthat humans move their eyes over the visual field aboutthree to four times per second. These eye movementsare called saccades. Saccades are interrupted by phaseswhere our eyes are relatively static, focusing on an ob-ject of interest. These static phases are called fixations.

Visual acuity is best at the center of one’s retina, thefovea (Irwin 2004). People move their eyes so that thefovea (the center of highest acuity) is directed towardthe location in the visual world to which they wish toattend or to visually process at the highest possible levelof detail (Rayner 1992). Just and Carpenter (1976, 441)proposed the “eye-mind assumption,” which states that“The eye fixates the referent of the symbol currently be-ing processed if the referent is in view.” Although thereare some caveats to this assumption (Irwin 2004), it isstill generally agreed that the location of a person’s gazeis an indication of to what the person is currently at-

tending. As people’s gazes are an overt behavioral mani-festation of attention allocation during visual inferencemaking, cognitive scientists utilize eye-movementrecords to infer knowledge about the cognitive pro-cesses involved in various visual cognition tasks (Grantand Spivey 2003; Henderson and Ferreira 2004). Fur-thermore, cognitive operations involving visuo-spatialprocessing take place during eye fixations only (Irwin2004), which makes eye-movement studies especiallyattractive for investigating spatial cognition issues ingeography, GIScience, and related fields.

In recent research, Grant and Spivey (2003) arguedthat spatio-temporal eye-movement behavior literallyembodies visuo-spatial inference making on graphicdisplays. In line with embodied cognition postulates(Wilson 2002), experiments by Thomas and Lleras(2007) offer strong support for the hypothesis thateye movements can act as an embodied mechanismto guide inference making and problem solving. Em-pirically validating and extending Grant and Spivey’s(2003) ideas on the spatial reasoning–eye movementlinkage, Thomas and Lleras (2007) were able to demon-strate that when actively guiding people’s eye move-ments on a graphic display, by a secondary task, theycould substantially affect people’s chances of problem-solving success.

Figure 3 depicts how a participant’s eyes moved overa map showing weather information during an inference

Figure 3. A gaze plot containing eyefixations and saccades overlaid onto aweather map stimulus.1



task. Eye fixations are shown with graduated circles(larger circles represent longer eye fixations). Eye move-ments between eye fixations (e.g., saccades) illustratethe viewing sequence and are depicted with connect-ing lines.

This example illustrates a typical finding that eye fix-ations are not randomly distributed over a scene. Sinceearly eye movement studies on visual displays (Buswell1935; Yarbus 1967), it has been known that peopleconcentrate their fixations on interesting and infor-mative scene regions (Henderson and Ferreira 2004).Only recently, however, have vision and cognitiveresearchers attempted to formalize and quantify inter-estingness or informativeness by looking at scene fac-tors that might influence people’s viewing behaviors. Inone direction of this research, investigators have begunto build perception-based, neurobiologically inspiredsaliency models to predict human eye movements (Itti,Koch, and Niebur 1998; Itti and Koch 2001; Oliva andTorralba 2001; Rosenholtz et al. 2005; Torralba et al.2006). Most of this work has been based on natu-ral scenes containing real-world objects (sketches, linedrawings, or photographs). The Itti, Koch, and Niebur(1998) model has been quite successful for simple dis-plays used in attention experiments, but it has notshown encouraging results in predicting where peo-ple look in meaningful scenes (Henderson 2003; Tor-ralba et al. 2006). Although this approach has notbeen fully tested on or validated against more ab-stract representations such as maps, diagrams, or otherkinds of more abstract graphic displays, promise of thesaliency map-based model for cartographic work hasbeen demonstrated on static and dynamic thematicmap displays (Fabrikant and Goldsberry 2005) and forremotely sensed images (Swienty, Kurz, and Reichen-bacher 2007).

Cartographers utilized the eye-movement recordingtechnique as early as the 1970s to investigate how peo-ple look at static maps (Steinke 1987). Cartographerswere particularly interested in improving the design oftheir map products based on eye-movement research,thereby creating better and more user-friendly products(Montello 2002). After an increased interest in eye-movement studies on maps during the 1970s and early1980s, the collection of eye-movement data in aca-demic cartography has almost disappeared. Montello(2002) suggested that one of the factors might havebeen that eye-movement analysis tended to be very ex-pensive and difficult to perform. Other critical voicesargued that this kind of data collection technique didnot tell mapmakers anything more than what they al-

ready knew and thus did not warrant the extra effort andexpense (Montello 2002). A final reason for the limitedsuccess of eye-movement studies in cartography mighthave been that researchers tended to focus their studieson where people looked, rather than getting at questionsof how and why or relating eye fixation behavior to spe-cific tasks (Brodersen, Andersen, and Weber 2002). Wesuggest that to interpret eye fixations on maps, and un-derstand at a process level why a certain cartographicdisplay works better than another, cartographers needto frame their studies within cognitive theory, as we dohere.

Saliency Assessment

To systematically investigate the cartographicallyinspired design assumptions made earlier, we first com-pared the maps on the basis of bottom-up models of vi-sual salience (Itti, Koch, and Niebur 1998; Rosenholtzet al. 2005). The Itti, Koch, and Niebur (1998) saliencemodel is a neurobiologically based bottom-up atten-tion model that has been inspired by, and success-fully validated against, experimental evidence for classicvisual search tasks (e.g., pop-out vs. conjunctive search)proposed by Treisman and colleagues (e.g., Treismanand Gelade 1980). Eye-movement predictions based onsaliency maps (SMs) are driven entirely by preattentiveprocesses and do not reflect cognitive (top-down) fac-tors such as knowledge of the task, type of visual display,or domain knowledge (Henderson 2007). For our pur-poses, one function these models can serve is to validatethat our display design solutions indeed vary in percep-tual salience. They also offer predictions of the locationsof eye fixations based on visual salience alone, but ofcourse it is important to realize that with a meaningfultask such as ours, there will also be top-down influences(Henderson 2007).

We applied the computer-based saliency models tosystematically evaluate the salience of different types ofinformation in the created map stimuli before runningthe experiment with participants. The Itti, Koch, andNiebur (1998) saliency model predicts eye movementsbased on the visual variables color hue, color value (in-tensity), and orientation. In this model, three filters areapplied to extract color hue, color value, and orienta-tion contrasts at several levels of image resolutions in avisual scene. Three feature maps (one for each filter) arecomputed based on center-surround comparisons. Fea-ture maps are additionally computed at several imageresolutions and integrated to form a single conspicuitymap for each feature type. A nonlinear normalization



Figure 4. First three eye fixations predicted by a saliency-based attention model. (A) Mass media map. (B) Cartographically modified map.

is applied to each conspicuity map to amplify peaks ofcontrasts relative to noise in the background. In thefinal stage, feature maps are combined to produce a sin-gle SM of the visual scene. In Figure 4 we show anexample of two SMs (with varying gray tones) over-laid onto two respective map stimuli. The more salienta map element (in Figure 4), the lighter its shade inthe SM (i.e.,“spotlight” metaphor). In the mass mediamap (Figure 4A) the lightest zones are distributed infour disjoint areas over the map but not where the the-matically relevant items (pressure systems) are located.In contrast, for the cartographic map (Figure 4B), thecenter of highest saliency (i.e., strongest visual contrast)seems to be only the zone where the three pressure cellsand the arrow are located.

The saliency model also predicts a sequence of loca-tions (ranked saliency peaks in the SM) that will attracta viewer’s gaze in a scene. Predicted initial eye fixations(circles) and the sequence of eye scan paths (lines) inFigure 4 are derived from the grayscale SM. The visualvariables color hue, color value, and orientation em-ployed by the Itti model are the same visual variablesmanipulated by cartographers to generate perceptualsalient figure–ground contrasts.

According to the model, the thematically relevantinformation (pressure) in the cartographically modifiedmap (Figure 4B) is perceptually salient and thus is pre-dicted to be fixated. Figure 4B is actually a special case,as the center low pressure system is so salient that theothers do not show up in the SM. When looking at thepredicted eye-movement sequence (based on the SM),however, all pressure systems are identified as salientand are predicted to be fixated. In contrast, none of the

thematic relevant items are predicted to be salient inthe mass media type map shown in Figure 4A.

The model results depicted in Figure 4 are gratify-ing for cartographers, as they suggest validity for thevisual variables proposed by Bertin (1967, 1983) morethan forty years ago. In other words, the intention ofthe cartographer to visually emphasize the thematicallyrelevant pressure cells for the wind direction task us-ing the visual variable color hue has been successfullyvalidated according to the Itti SM model (i.e., by desat-urating temperature hues and increasing color contrastsof the pressure cell labels).

We examined this prediction with another percep-tion-based saliency model proposed by Rosenholtz,Li, and Nakano (2007). Their proposed feature con-gestion measure of visual clutter rests on the assump-tion that the more cluttered a display is, the moredifficult it would be to add a new item that would reli-ably draw attention or, in other words, be salient. Theclutter measure is based on a statistical saliency model,related to the local variability of color hue, luminance(color value), and orientation contrasts between itemsin a display, similar to those employed by cartographers(Bertin 1967, 1983) and in other SM models from visionscientists (e.g., Itti, Koch, and Niebur 1998). Specifi-cally, the statistical saliency model is based on the ideaof a feature space where the mean of a visual distrac-tor distribution (i.e., color hue, color value, orienta-tion, velocity, etc.) is compared with the value of asearch target. The larger the difference between distrac-tors and target, the more salient the target in a visualscene. The advantage of the Rosenholtz model is thatit has been successfully validated with search tasks with



Figure 5. Clutter maps for (A) mass media map and (B) cartographic map.

meaningful displays, using a variety of graphic stimuli(Rosenholtz and Jin 2005), including even referencemaps (Rosenholtz, Li, and Nakano 2007).

According to the clutter model that includes thesaliency model, the mass media map is less clutteredwith an average clutter value of 3.6 compared to 4.7for the cartographically modified map. In other words,adding a salient graphic element to the cartographi-cally enhanced map would be more difficult than for themass media map. Salient, in this case, means that thosehigh-contrast features—represented in lighter shadesin Figure 5—are statistical outliers to the local dis-tribution of features in the display. One can also seewhy this might be the case as shown in Figure 5. InFigure 5B the thematically relevant items such as thepressure cell labels, and even more so the isobars, fillthe entire display space such that there would be littleroom to add another salient item. In Rosenholtz, Li, andNakano’s (2007) words, too many existing feature con-trasts (i.e., in color hue, color value, and orientation)are already clamoring for attention. This is in contrastto Figure 5A, where more space for attention-grabbingitems is still available; for instance, in the (darker) cen-ter and upper West parts of the American continent.In Figure 5B, the thematically relevant isobars vanishinto the background due to lacking contrasts in colorvalue and color hue. Although the thematically rele-vant pressure cells are salient, the irrelevant tempera-ture legend also seems to unnecessarily attract a person’sgaze.

The results in Figure 5 are again very encouragingfor cartographers, to see that the thematically relevantfeatures in Figure 2B were successfully made perceptu-

ally salient according to a second (statistical) SM modelshown to be predictive of human’s eye movements dur-ing visual search tasks on various images (Rosenholtzet al. 2005).

Supported by the already mentioned saliency models,cartographers potentially do have a sound theoreticallymotivated baseline at hand to develop map displaysor visual analytic tool interfaces based on cognitiveand perceptual principles. In a next step, we compareresults from the computer-simulated models with be-havioral data collected through an eye-movement ex-periment with human participants using the same mapstimuli.

Experiment

In an experiment, we asked novice participants toevaluate the veracity of the wind direction that is indi-cated by an arrow on weather map types such as thoseshown in Figure 2. Eye-movement and response datawere collected for participants as they performed thiswind inference task before and after training on me-teorological principles. Half of the participants workedwith the mass media maps and the other half workedwith the cartographically designed maps. Thus, theexperiment had a two (type of map) by two (beforeand after training) mixed factorial design. Based onthe preceding discussion, we hypothesize the followingeye-movement patterns:

1. Prior to training on the task-relevant weather prin-ciples, and thus less aware of which information isthematically relevant, novice viewers’ attention (as



revealed by eye movements) will be attracted bysalient items in the display (i.e., stimulus-driven,bottom-up factors). For instance, novice viewers’ eyefixations in the mass media map condition will be at-tracted to the salient but task-irrelevant temperaturelegend, especially when temperature is more salientin the display. These predictions are suggested by thepreattentive saliency models and by prior empiricalresearch (e.g., Lowe 1999).

2. After training and demonstrating their knowledgeof the task-relevant weather principles, viewers’ at-tention will be directed to the thematically relevantitems, regardless of their saliency (i.e., knowledge-driven, top-down). For instance, viewers’ gaze in themass media map condition will be attracted to thetask-relevant items (i.e., closest pressure systems tothe arrow) even if they are not salient.

We also predicted that the maps would not be com-putationally equivalent (cf. Simon and Larkin 1987),such that the cartographically enhanced map would bemore efficient for inference making than the mass mediamap. According to the employed saliency models, thecartographic map matches thematic relevance with per-ceptual salience much more effectively than the massmedia map. Thus, it should support more efficient (e.g.,faster or more accurate) inference making.

Method

Participants

Thirty students (thirteen male and seventeen fe-male) from an undergraduate introductory human geog-raphy class, with a mean age of 20.1 years, took part inthe experiment. They received a small amount of coursecredit in return for their participation. The test samplewas judged to be a good sample of the desired novice userpopulation, as the vast majority of the participants (72percent) had not taken more than one geography class(including the first one they were just taking). On aver-age, participants reported that they had used maps onlyoccasionally and had never had training in meteorol-ogy, cartography, GIS, computer graphics, or graphicdesign.

Materials

Participants viewed computer displays that werecreated using Environmental Systems Research Insti-tute’s (ESRI; 2003) ArcMap. The displays consisted of

Figure 6. Sample map stimulus, including test question andresponse boxes.

full-color weather maps including surface temperatureand pressure, inspired by weather maps similar to thatshown in Figure 1. We chose actual weather data fromthe National Oceanic and Atmospheric Administra-tion (NOAA)’s hourly NCEP/NCAR reanalysis DataComposites2 on random days. For each display, par-ticipants needed to evaluate the possible wind direc-tion that was indicated by a gray arrow. To determinewhether the wind direction indicated was indeed thetrue wind direction, it was necessary for participants touse the display to obtain information about the pressuregradient, which was represented by black isolines, andthe two or three neighboring pressure cells labeled H(High) or L (Low; see Figure 6). Wind direction canbe inferred from surface pressure (thematically relevantinformation), but the temperature information is irrel-evant for the task.

Participants’ eye movements were recorded using aniView X RED II by SensoMotoric Instruments (SMI).The experiment was administered on a Windows 2000Pentium IV personal computer using Microsoft Power-Point to display the map stimuli and training materialson a 21-inch color monitor, at 1024 × 768 pixel spatialimage resolution. The GazeTracker software by Eye Re-sponse Technologies (2003) was employed to run theSMI eye tracker. A standard mouse and keyboard wereused to answer questions. Gaze recordings were recali-brated after data collection. This means that we wereable to improve the accuracy of raw fixation locationsafter recording, to overcome initial calibration inaccu-racies and accumulating drift error. Fixations were then



extracted using eyeMAT, a Matlab toolbox codevel-oped by one of the authors.3

Answers were recorded automatically and stored dig-itally, including the time required to respond to thequestions. Response time was measured as the elapsedtime in milliseconds between the trial display appear-ing on the screen and the participant proceeding to thenext trial.

Procedure

Participants were randomly assigned to two map de-sign conditions: mass media map or cartographic map,which varied in the visual salience of the task-relevantinformation. Participants were told they would be pre-sented with two series of thirty U.S. weather maps, andthat each map would include an arrow pointing in a pos-sible direction that the wind might be blowing at thatlocation. Participants were asked to determine whetherthe wind directions indicated by the arrows were true(correct wind direction) or false (wrong wind direc-tion). They were also told that the maps included tem-perature information indicated by color and that theblack lines represented pressure (isobars). They weretold that their response would be timed but to take asmuch time as they needed to answer the questions asaccurately as possible.

Participants then performed six practice trials to be-come comfortable with the test instrument and withhaving their eye movements tracked. For each weathermap display, participants were asked to click on a Trueor False button in response to the statement “The ar-row points to the true direction of the wind” (arrowverification task). Following the practice trials, theyresponded to the main test trials organized into twoseparate blocks, each including thirty different mapsand true–false tasks. The presentation order of the mapdisplays was varied systematically such that the pres-sure cell configuration changed in location and distancefrom the arrow. The orientation of the arrow (wind di-rection) was also varied systematically. Whereas in onethird of the trials the arrow pointed in the correct di-rection of the winds, in the other two thirds of the trialsthe arrow showed an incorrect wind direction. Trialswithin each block were presented in one of three dif-ferent randomized orders. The three display orders weresystematically rotated through the participant pool toavoid potential order confounds.

After the first block of thirty trials, participants weretrained on the principles of surface air movement (i.e.,pressure gradient force, Coriolis force, cyclones, etc.).

During this training phase participants read informationand viewed diagrams that explained how to predict winddirection based on pressure gradient and knowledge ofCoriolis force. After participants viewed the on-screentraining materials on pressure gradients and Coriolisprinciples extracted from meteorology textbooks, theresearcher led them through a set of exercises designedto check their understanding of the materials before thesecond block of trials began. For example, participantswere given three practice problems on the wind veri-fication task that were similar but not identical to theexperimental task. Participants had to verbally explaintheir answers to the experimenter and received feed-back on the quality of their answers. They were notallowed to continue with the experiment until the ex-perimenter was satisfied with the correctness of theiranswers.

The participants were given the second set of thirtytrials only after the experimenter felt that they hadlearned and understood the task-relevant weather prin-ciples. Participants’ eye movements were not recordedduring the training session. After completing the sec-ond block (including eye-movement recording) partic-ipants were debriefed, thanked, and marked down forcredit.

Results

Whereas participants’ accuracy on the inference task(i.e., wind direction assessment) was at chance prior totraining for both map types, their accuracy improvedsignificantly after training for both map types. Meanaccuracy for both map type and training are shownin Figure 7. Accuracy scores were subjected to a two-way (mixed) analysis of variance (ANOVA) havingtwo levels of training (within subject: pre, post) andtwo levels of map types (between subject: mass media,cartographic map).

The within-subject main effect of training yieldeda significant effect, F (1, 28) = 64.72, p < 0.001, suchthat the percentage of correct responses was greater aftertraining (M = 0.74, SD = 0.13) than before training(M = 0.51, SD = 0.13). The between-subject maineffect of map type yielded an F ratio of F (1, 28) =0.315, indicating that map design did not significantlyaffect the mean accuracy scores.

The pattern of participants’ response times in sec-onds on the wind inference task is depicted in Fig-ure 8. For the repeated-measures ANOVA on responsetime, there was a significant effect of training, F (1,



Figure 7. Mean accuracy on the wind direction task as a functionof training and map type. The numbers in brackets show absolutenumber of correct responses (max. = 30). Error bars show standarderrors.

28) = 14.38, p < 0.001, indicating that participantstook significantly longer to respond after training (M =265.03 ms, SD = 143.11) than before (M = 197.08 ms,SD = 87.32). The between-subject main effect of maptype was also significant for this variable, F (1, 28) =7.81, p < 0.001. Although participants in general tooksignificantly longer to respond after training, they re-sponded significantly faster with the cartographicallyenhanced map compared with the mass media map,as predicted. We did not find any correlation betweenresponse time and response accuracy.

The first step in the eye-movement analysis was theidentification of areas of interest (AOIs) based on theirrelevance or irrelevance to the task. Pressure systemswere selected as AOIs as they show high task relevance.The pressure systems decrease in thematic relevancewith increasing distance from the gray (wind direction)arrow (Figure 9).

Figure 8. Mean response times on the wind direction task as func-tion of training and map type. Error bars show standard errors.

The temperature information shown in the legendis also selected as an AOI. It is not considered taskrelevant. Although it is perceptually demoted in thecartographically enhanced map, the temperature legendis perceptually more salient (i.e., a distractor) in themass media map (see also the darker gray shades in theRosenholtz saliency model representing the legend inFigure 5A compared to Figure 5B).

We set the task-relevant AOIs to different sizes, de-pending on the display item.4 We tested the stability ofour results by varying the AOIs’ sizes (i.e., pressure cells,etc.). As we specifically modified the pressure labels toincrease saliency we chose AOI sizes that covered thelabels well but did not extend too far. Based on availabletheory (Henderson and Ferreira 2004; Irwin 2004), sug-gesting fixation times between 50 and 300 ms for variousviewing tasks, we determined a 100-ms threshold for afixation for our study. Again, we tested the stability forour results by varying the sampling threshold.

Overall, participants spent more time looking at theclosest pressure system (e.g., task-relevant information)after training (M = 43.86 ms, SD = 33.56 ms) thanbefore training (M = 21.41 ms, SD = 14.15 ms). Thisincrease is significant, F (1, 28) = 14.67, p < 0.001,indicating that participants spend more time fixat-ing on thematically relevant task items after learningthe weather principles. This pattern is also evident inrelative terms. Before training, participants spent 10.8percent of their fixation time on the closest pressuresystem (see P1-pre in Figure 10), compared with 16.4percent after training (P1-post in Figure 10).

It is noteworthy that proportions of fixation times forthe pressure system AOIs overall decrease with increas-ing distance from the arrow, in analogy to the first lawof geography (Tobler 1970) that closer things are morerelated than distant things, even for inference making(i.e., a distance-decay process). Figure 10 depicts theproportions of fixation times for the examined pressuresystems (P1–P3). P1 is the pressure system closest indistance to the arrow, followed by P2, and P3 is thefarthest.

The proportion of fixation time increase as an effectof training is significant for pressure systems 1 and 2.This is true for both the mass media map type and thecartographic map (indicated with an asterisk in Figure10). The fixation time proportion for pressure system 3 isvery low (less than 3 percent) and this does not changesignificantly after training. We found no interactioneffect of map type.

The main effect of training on the proportion of fix-ation time on the legend is also significant, F (1, 28) =



Figure 9. Selected areas of interestto analyze participants’ gaze patternsduring task completion.

8.28, p < 0.009. Task-irrelevant information such asthe temperature legend is fixated significantly less af-ter training—that is, not at all (M = 0.002, SD =0.005)—than before training (M = 0.02, SD = 0.03;Figure 11). The same pattern is revealed when inspect-ing fixation time totals. Participants spend significantlyless time fixating the temperature legend after training(M = 0.85 ms, SD =1.76 ms) than before training(M = 4.01 ms, SD = 8.79 ms).

The interaction of training and map type is also sig-nificant, F (1, 28) = 9.01, p < 0.007, however, indi-

cating that the fixation duration change on the legendbetween pre- and posttraining is significantly differentfor the mass media map compared to the cartograph-ically enhanced map. In general, in the cartographicmap condition the proportion of fixation time on thelegend is significantly lower than for the mass mediamap. This proportion of fixation time is essentially zeroon this map both before (0.0007; total time: 0.29 ms)and after training (0.0013; total time: 0.49 ms). Theminimal increase after training is not significant. Theproportion of fixation time on the legend in the mass

Figure 10. Proportions of fixation timesfor examined pressure systems (P1–P3).



Figure 11. Prototypical eye-movement behavior during the pretraining phase for (A) mass media map and (B) cartographic map.

media map drops significantly from 0.03 (total time:7.73 ms) before training to essentially zero (0.003; totaltime: 1.21 ms) after training (p < 0.001).

Discussion

To summarize the empirical results, accuracy of re-sponse (inference quality) increases significantly afterparticipants learn the task-relevant weather principles.Participants’ response accuracy does not depend on thetype of graphic display they viewed. Overall, responsetime increases significantly after training. In terms ofthe eye fixation patterns, we find that participants spendmore time fixating on thematically relevant task items(i.e., closer pressure systems) and less on task-irrelevantitems (i.e., legend) after learning the weather principles.

The knowledge gain from the training session sig-nificantly and positively affected people’s quality of in-ference. Whereas participants’ accuracy was essentiallyat chance before training, it rose to 75 percent aftertraining. The inference quality improvement is paral-leled by a significant increase in response time betweenpre- and posttraining sessions. The observation thatparticipants took almost twice as long to respond afterlearning the weather principles might seem counterin-tuitive at first, but it can be explained. Before train-ing, and when they did not know the task-relevantmeteorological principles, people probably guessed, sothat they answered quickly but with low accuracy. Af-ter they learned the principles, participants neededmore time to process the newly acquired information

and to apply it correctly to the changing configurationsshown in the various displays. Knowledge gain has notonly significantly changed what people answer but alsowhere people look in the display. Before training, per-ceptually salient items, such as the temperature legend,seem to have attracted novice viewers’ attention morethan less salient items, as predicted by the cartographicprinciples and validated with the preattentive saliencymodels. This was evident when the visual variable colorhue indicating temperature was more salient in the dis-play. In contrast, after training, and armed with newtask-relevant knowledge, viewers focused entirely onthe thematically relevant items (i.e., pressure cell clos-est to arrow) on the graphic display, regardless of theirvisual properties. It is noteworthy that this significantperformance improvement was accomplished in onlyabout ten to fifteen minutes of training.

Our results generally support the information-reduction hypothesis proposed by Haider and Frensch(1996), which states that the transition from noviceto expert in a task involves learning to separate task-relevant from task-redundant information. This reducespotential information overload by limiting cognitiveprocessing to relevant aspects of the task at hand. In eye-movement studies Haider and Frensch (1999) foundthat task-redundant information is ignored at the per-ceptual and not just at the conceptual level. Our resultsare consistent with this conclusion and suggest thatgood map design might even accelerate informationprocessing for participants learning to become more ex-pert, as further elaborated later.



Our results are also consistent with research by Lowe(1999) on animated weather map displays, in showingthat participants tend to extract information based onperceptual salience rather than thematic relevance.

When comparing the Itti, Koch, and Niebur (1998)SM model predictions (Figure 4) with a typical sam-ple of actual collected eye-movement data (as shownin Figure 11) one can see that whereas the collectedgaze plot for the cartographic map matches the Itti pre-diction well, a viewer’s gaze plot for the mass mediamap does not match the model prediction at all. TheRosenholtz, Li, and Nakano (2007) model shown inFigure 5 provides a better fit of empirical data for bothmap types. The Rosenholtz model has been validatedwith meaningful displays. This is in contrast with theItti, Koch, and Niebur (1998) model, which has notalways been a good predictor of eye fixations in mean-ingful tasks (Henderson 2003), so the Rosenholtz modelmight have more potential for use by cartographers indesigning displays.

The significant difference in the inference efficiency(response speed) between the mass media map and thecartographic map is where the power of perceptuallysalient design lies in this study. In Simon and Larkin’s(1987) words, our results show that the mass media mapand the cartographic map differ significantly in compu-tational equivalence. After training, inferences drawnfrom the cartographic map seem to have been mademore easily and quickly than from the information-ally equivalent mass media map. We contend that thisis because thematic and task-relevant information ispresented in a perceptually more salient manner. Thisefficiency gain from better display design must drawon an existent knowledge base. In other words, peo-ple need a relevant knowledge base for taking bestadvantage of the perceptually inspired display, con-forming to Hegarty’s (2004) relation number 2 thatcomprehension of external displays depends on internalcognition.

Conclusions and Outlook

We have presented a theoretical and methodolog-ical framework, grounded in cognitive theory andcartographic design principles, for constructing and sys-tematically evaluating cognitively and perceptually in-spired depictions for making spatial inferences withcomplex map displays. This framework combines theapplication of saliency models developed in researchon human vision with the assessment of inference pro-

cesses by combining traditional performance measures(accuracy and speed) with eye-movement recordings.

In the first phase, our framework employs stimulus-driven (bottom-up) saliency models for assessing thedesign quality of a graphic display and predicting eye-fixation patterns based on bottom-up stimulus factorsalone. In a second step, the effect of top-down (cogni-tion) factors on inference making is assessed througha controlled experiment, by comparing performance ofindividuals with more and less domain knowledge. Ourempirical results suggest that good display design affectsviewing behavior, especially for naıve participants whodo not yet have the relevant domain knowledge. More-over, matching thematically relevant information withperceptual salience significantly improves the efficiencywith which people make inferences from graphic dis-plays. With the collected empirical evidence we hopeto have provided a better understanding of how goodmap design can help people extract relevant informa-tion from complex thematic maps and make inferencesfrom these displays in the domain of meteorology.

Although visual attention involves a combination ofbottom-up (stimulus-driven) processes and top-down(knowledge-driven) components, the predictions ofsolely bottom-up-based models seem to be promising forcartographers, as they provide a systematic approach tovalidate their map designs based on well-established butrarely empirically validated design principles (Bertin1967, 1983). Cartographers can utilize saliency-basedmodels to get a first insight into their design decisionsand systematically assess effects of visual variables forthe mapping task at hand. This study shows that boththe SM approach and the measurement of eye fixationscan be employed to systematically assess the utility ofBertin’s (1967, 1983) system of seven visual variableswidely used in cartography and more recently discov-ered in information visualization (Card, Mackinlay, andShneiderman 1999). The visual variable system wasdeveloped specifically to help cartographers to bettercontrol the visual salience of symbols on maps. Un-til today, however, it lacked in systematical validationprocedures, which we hope to have provided with thiscontribution.

Acknowledgments

This material is based on work supported by the Na-tional Science Foundation under Grant No. 0350910,by the Swiss National Science Fund under Grant No.200021-113745 (Sara Irina Fabrikant, Stacy Rebich



Hespanha), and the Office of Naval Research GrantNo. N000140310119 (Mary Hegarty). We gratefullyacknowledge Daniel Montello’s continued stimulat-ing feedback, lively discussions, and support on thisproject from its inception. We thank Kirk Goldsberry,Maral Tashjian, and Natalie Campos for their assistancein designing and running the experiments and JoaoHespanha for development of the eyeMAT applicationto facilitate eye-movement data recalibration and anal-ysis. We also thankfully acknowledge Thomas Gross-mann’s valuable contribution on the development ofthe eyeview tool and Laurent Itti and Ruth Rosenholtzfor making the saliency code available to us.

Notes1. Gaze plots shown in this article (i.e., Figures 3 and 11)

were visualized with a developed visual analytics toolcalled EyeView described in Grossmann (2007).

2. Available online at http://www.cdc.noaa.gov/cgi-bin/data/composites/printpage.pl/hour/ (last accessed 24January 2009).

3. This Matlab toolbox was developed by Joao Hespanha ofthe Department of Electrical and Computer Engineeringand Stacy Rebich Hespanha of the Department of Ge-ography at the University of California, Santa Barbara.

4. For example, for the pressure systems we selected circleswith a 92-pixel diameter.

References

Barkowsky, T., C. Freksa, M. Hegarty, and R. K. Lowe, eds.2005. Reasoning with mental and external diagrams: Com-putational modeling and spatial assistance. Menlo Park, CA:AAAI Press.

Bertin, J. 1967. Semiologie graphique: Les diagrammes—lesreseaux—les cartes [Semiology of graphics]. Paris: Mouton.

———. 1983. Semiology of graphics. Madison: University ofWisconsin Press.

Brodersen, L., H. H. K. Andersen, and S. Weber. 2002.Applying eye-movement tracking for the study of map per-ception and map design. Copenhangen, Denmark: Kortand Matrikelstyrelsen, National Survey and CadastreDenmark.

Buswell, G. T. 1935. How people look at pictures. Chicago:University of Chicago Press.

Card, S. K., J. D. Mackinlay, and B. Shneiderman. 1999.Readings in information visualization: Using vision to think.San Francisco: Morgan Kaufmann.

Dent, B. D. 1999. Cartography: Thematic map design.Dubuque, IA: Wm. C. Brown.

Duchowski, A. T. 2007. Eye tracking methodology. Berlin:Springer.

Edwards, G. 2001. A virtual test bed in support of cognitively-aware geomatics technologies. In Spatial information the-ory: Foundations of geographic information science, LectureNotes in Computer Science 2205, ed. D. R. Montello,140–55. Berlin: Springer.

Environmental Systems Research Institute (ESRI). 2003.ArcGIS, Version 8.3. Redlands, CA: ESRI.

Eye Response Technologies. 2003. GazeTracker. Char-lottesville, VA: Eye Response Technologies.

Fabrikant, S. I., and K. Goldsberry. 2005. Thematic relevanceand perceptual salience of dynamic geovisualization dis-plays. Paper presented at the 22nd ICA/ACI Interna-tional Cartographic Conference, Coruna, Spain.

Fabrikant, S. I., and A. Skupin. 2005. Cognitively plausibleinformation visualization. In Exploring geovisualization,ed. J. Dykes, A. M. MacEachren, and M. J. Kraak, 667–90. Amsterdam: Elsevier.

Grant, E., and M. Spivey. 2003. Eye movements and problemsolving: Guiding attention guides thought. PsychologicalScience 14:462–66.

Grossmann, T. 2007. Ansatz zur Untersuchung derWahrnehmung bei geographischen Darstellungen: EinWerkzeug zur visuellen Exploration von Blickreg-istrierungsdaten [An approach to study the perceptionof geographic depictions: A tool for the visual explo-ration of eye tracking data]. Master’s thesis, UNIGISProgram, Salzburg, Austria.

Haider, H., and P. A. Frensch. 1996. The role of informa-tion reduction in skill acquisition. Cognitive Psychology30:304–37.

———. 1999. Eye movement during skill acquisition: Moreevidence for the information-reduction hypothesis. Jour-nal of Experimental Psychology: Learning, Memory, andCognition 2:172–90.

Hegarty, M. 1992. Mental animation: Inferring motion fromstatic displays of mechanical systems. Journal of Ex-perimental Psychology: Learning, Memory, and Cognition18:1084–102.

———. 2004. Diagrams in the mind and in the world:Relations between internal and external visualizations.In Diagrammatic representation and inference, LectureNotes in Artificial Intelligence 2980, ed. A. Blackwell,K. Mariott, and A. Shimojima, 1–13. Berlin: Springer.

Hegarty, M., and V. K. Sims. 1994. Individual differences inmental animation during mechanical reasoning. Memoryand Cognition 22:411–30.

Henderson, J. M. 2003. Human gaze control during real-worldscene perception. Trends in Cognitive Science 7:498–504.

———. 2007. Regarding scenes. Current Directions in Psycho-logical Science 16:219–22.

Henderson, J. M., and F. Ferreira. 2004. Scene perceptionfor psycholinguists. In The integration of language, vision,and action: Eye movements and the visual world, ed. J. M.Henderson and F. Ferreira, 1–58. New York: PsychologyPress.

Henderson, J. M., and A. Hollingworth. 1998. Eye move-ments during scene viewing: An overview. In Eyeguidance in reading and scene perception: Eye guidancewhile reading and while watching dynamic scenes, ed. G.Uderwood, 269–93. Oxford, UK: Elsevier.

Irwin, E. 2004. Fixation location and fixation duration asindices of cognitive processing. In The integration of lan-guage, vision, and action: Eye movements and the visualworld, ed. J. M. Henderson and F. Ferreira, 105–34. NewYork: Psychology Press.

Itti, L., and C. Koch. 2001. Computational modeling of visualattention. Nature Reviews Neuroscience 2:194–203.



Itti, L., C. Koch, and E. Niebur. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEETransactions on Pattern Analysis and Machine Intelligence20:1254–59.

Just, M. A., and P. A. Carpenter. 1976. Eye fixationsand cognitive processes. Cognitive Psychology 8:441–80.

Kosslyn, S. M. 1994. Elements of graph design. New York:Freeman.

Lowe, R. K. 1999. Extracting information from an anima-tion during complex visual learning. European Journal ofPsychology of Education 14:225–44.

MacEachren, A. M. 1995. How maps work: Representation,visualization, and design. New York: Guilford.

MacEachren, A. M., and M.-J. Kraak. 2001a. Research chal-lenges in geovisualization. Special issue. Cartography andGeographic Information Science 28 (1): 3–75.

———. 2001b. Research challenges in geovisualization. Car-tography and Geographic Information Science 28:3–12.

Monmonnier, M. 1999. Air apparent. Chicago: University ofChicago Press.

Montello, D. R. 2002. Cognitive map-design research inthe 20th century: Theoretical and empirical approaches.Cartography and Geographic Information Science 29:283–304.

Oliva, A., and A. Torralba. 2001. Modeling the shape of thescene: A holistic representation of the spatial envelope.International Journal of Computer Vision 42:145–75.

Rayner, K., ed. 1992. Eye movements and visual cognition: Sceneperception and reading. New York: Springer Verlag.

Rosenholtz, R., and Z. Jin. 2005. A computational form of thestatistical saliency model for visual search. Journal of Vi-sion 5:777, 777a. http://journalofvision.org/775/778/777/(last accessed 24 January 2009).

Rosenholtz, R., Y. Li, J. Mansfield, and Z. Jin. 2005. Featurecongestion: A measure of display clutter. In Proceedingsof the SIGCHI Conference on Human factors in ComputingSystems, 761–70. New York: Association for ComputingMachinery Press.

Rosenholtz, R., Y. Li, and L. Nakano. 2007. Measuring visualclutter. Journal of Vision 7:1–22.

Scaife, M., and Y. Rogers. 1996. External cognition: How dographical representations work? International Journal ofHuman-Computer Studies 45:185–213.

Shah, P., and A. Miyake, eds. 2005. The Cambridge handbookof visuospatial thinking. New York: Cambridge UniversityPress.

Simon, H. A., and J. H. Larkin. 1987. Why a diagram is(sometimes) worth ten thousand words. Cognitive Science11:65–100.

Steinke, T. R. 1987. Eye movement studies in cartographyand related fields. Cartographica 24:40–73.

Swienty, O., F. Kurz, and T. Reichenbacher. 2007. Attentionguiding visualization in remote sensing IIM systems. Pho-togrammetrie, Fernerkundung, Geoinformation 4:239–51.

Thomas, J. J., and K. A. Cook. 2005. Illuminating the path:Research and development agenda for visual analytics.Richland, WA: IEEE Press.

Thomas, L. E., and A. Lleras. 2007. Moving eyes and mov-ing thought: On the spatial compatibility between eyemovements and cognition. Psychonomic Bulletin and Re-view 14:663–68.

Tobler, W. R. 1970. A computer movie simulating ur-ban growth in the Detroit region. Economic Geography46:234–40.

Torralba, A., A. Oliva, M. Castelhano, and J. M. Henderson.2006. Contextual guidance of eye movements and atten-tion in real-world scenes: The role of global features onobject search. Psychological Review 113:766–86.

Treisman, A., and G. Gelade. 1980. A feature integrationtheory of attention. Cognitive Psychology 12:97–136.

Tufte, E. 1990. Envisioning information. Cheshire, CT: Graph-ics Press.

Wade, N., and B. Tatler. 2005. The moving tablet of the eye:The origins of modern eye movement research. Oxford, UK:Oxford University Press.

Wilson, M. 2002. Six views of embodied cognition. Psycho-nomic Bulletin and Review 9:625–36.

Yarbus, A. L. 1967. Eye movements and vision. New York:Plenum.

Zhang, J., and D. A. Norman. 1994. Representations in dis-tributed cognitive tasks. Cognitive Science 18:87–122.

Correspondence: Department of Geography, University of Zurich, Winterthurerstr. 190, CH-8057 Zurich, Switzerland, e-mail: [email protected](Fabrikant); Department of Geography, University of California Santa Barbara, Santa Barbara, CA 93106, e-mail: [email protected](Hespanha); Department of Psychology, University of California Santa Barbara, Santa Barbara, CA 93106, e-mail: [email protected](Hegarty).


Documents

Cognitively Inspired and Perceptually Salient Graphic ...sara/pubs/fabrikant_etal_an09.pdf · Cognitively Inspired and Perceptually Salient Graphic Displays ... Cognitively Inspired