Visualizing Scienti c Data - Erik Erhardt · 2015-12-04 · Allen and Erhardt Visualizing Scienti c Data Page 2 of 21 recent survey of nearly 1500 gures from ve neuroscience journals

Visualizing Scientific Data

Elena A. Allena, Erik Barry Erhardtb

aDepartment of Biological and Medical Psychology, University of BergenbDepartment of Mathematics and Statistics, University of New Mexico

Contents

1 Introduction and Motivation 1

2 Designing for Human Abilities and Limita-tions 22.1 Graphical Encoding . . . . . . . . . . . . . 22.2 Short-term and Long-term Memory . . . . . 32.3 Design Principles . . . . . . . . . . . . . . . 3

3 Selecting a Chart Type 43.1 Common Chart Types . . . . . . . . . . . . 43.2 Displaying Variation and Uncertainty . . . 63.3 Multivariate Visualizations . . . . . . . . . 73.4 3D visualizations . . . . . . . . . . . . . . . 8

4 Symbols and Colors 84.1 Using Symbols . . . . . . . . . . . . . . . . 84.2 Using Color . . . . . . . . . . . . . . . . . . 10

5 Supporting Details 125.1 Axes . . . . . . . . . . . . . . . . . . . . . . 125.2 Grids and Guides . . . . . . . . . . . . . . . 135.3 Annotation . . . . . . . . . . . . . . . . . . 135.4 Font Choice . . . . . . . . . . . . . . . . . . 145.5 Numerical Precision . . . . . . . . . . . . . 15

6 Putting it into Practice 156.1 Tools . . . . . . . . . . . . . . . . . . . . . . 156.2 Examples . . . . . . . . . . . . . . . . . . . 166.3 A Checklist for Assessing Visualizations . . 19

List of Tables

1 Decoding accuracy . . . . . . . . . . . . . . 32 Data types . . . . . . . . . . . . . . . . . . 63 Checklist . . . . . . . . . . . . . . . . . . . 20

IChapter to appear in 2016 in Handbook of Psychophysiol-ogy, 4th Edition by Cambridge University Press, edited by Pro-fessor John T. Cacioppo, Professor Louis G. Tassinary and Gary G.Berntson.

Acknowledgments: We thank Lana Chavez for generous helpediting early versions of the chapter. We also acknowledge BangWong and Martin Krzywinski, whose clear writing and illustrationsinspired much of the work presented here.

Email addresses: [email protected] (Elena A. Allen),[email protected] (Erik Barry Erhardt)

List of Figures

1 Perceptual Tasks . . . . . . . . . . . . . . . 22 Difference between curves . . . . . . . . . . 33 Chart types . . . . . . . . . . . . . . . . . . 54 Alternatives to 3D . . . . . . . . . . . . . . 95 Symbol shape and saturation . . . . . . . . 96 Color benefits . . . . . . . . . . . . . . . . . 117 Color cylinder . . . . . . . . . . . . . . . . . 128 Aspect ratio . . . . . . . . . . . . . . . . . . 139 Categorical axis . . . . . . . . . . . . . . . . 1410 Integrate statistical descriptions . . . . . . . 1511 Data visualization process . . . . . . . . . . 1712 Modified designs with real data . . . . . . . 18

1. Introduction and Motivation

Data visualization is more than just the graphical dis-play of information. Rather, it is a form of visual commu-nication intended to generate understanding or insightsabout data. Specific to scientific research, data visual-ization serves many distinct purposes. In exploratorydata analysis, which renowned statistician John Tukey de-scribes as “graphical detective work” (Tukey, 1977), visu-alizations provide the ability to reveal unanticipated pat-terns and relationships. When a priori hypotheses areavailable, visualizations are used to support hypothesistesting and model validation. In presentations and pub-lications, visualizations are the primary medium used toconvey research findings to colleagues. While tables havetheir place for point/value reading in small or moderate-sized datasets, graphs are the superior choice for show-ing trends, summarizing data, and demonstrating rela-tionships (Jarvenpaa and Dickson, 1988).

As with any form of communication, data visualiza-tion can be effective or ineffective. It has been argued byWallgren, Wallgren, Persson, Jorner, and Haaland (1996)that “a poor chart is worse than no chart at all”. With-out consideration of how a visualization will be interpreted(or possibly misinterpreted), you run the risk of confusingyour viewer rather than enhancing their understanding.Unfortunately, there is evidence to suggest that graphicalcommunication in the sciences can be substantially im-proved. An early survey by Cleveland (1984) of 377 graphsin a volume of Science showed that 30% had at least onemajor error compromising interpretation. Our own more

Preprint Handbook of Psychophysiology, 4th Edition, 2016 October 2015

Allen and Erhardt Visualizing Scientific Data Page 2 of 21

recent survey of nearly 1500 figures from five neurosciencejournals found that graphical displays became less infor-mative and interpretable as the dimensions of datasets in-crease (Allen, Erhardt, and Calhoun, 2012). For example,only 43% of graphics displaying higher-dimensional datalabeled the dependent variable (meaning that more thanhalf the time the viewer couldn’t determine what quantitywas being plotted) and only 20% portrayed the statisticaluncertainty of measured or calculated quantities.

The inadequacies of many scientific graphics are notdue to a lack of guidelines or established best practices.Quite the contrary, there are numerous excellent resourcesavailable (e.g., Cleveland, 1994; Tufte, 2001). The prob-lem appears to be that the advice of the experts is notreaching its intended audience (Wickham, 2013). It isincreasingly recognized that data visualization educationmust be integrated into resources broadly accessed by thescientific community. Examples of this paradigm shiftinclude the appearance of relatively simple graphing tu-torials in prominent journals (e.g., Spitzer, Wildenhain,Rappsilber, and Tyers, 2014; Wand, Iversen, Law, andMaher, 2014), and recent development of the strongly rec-ommended “Points of View” column by Bang Wong, Mar-tin Krzywinski, and colleagues, in Nature Methods, whichprovides a high level discussion of data visualization fun-damentals. Education — particularly early on — is crit-ical for the development of visual communication skills,and it is exciting to see innovative resources (such as thisbook) that acknowledge data visualization as a generalresearch method for scientists.

It is our intention that by reading this chapter you willbecome a better communicator of visual information. Wewill guide you through the motivating principles of graph-ical design, how to construct graphs that play to humanperceptual strengths, how to clarify visual messages withannotations, and, finally, how to implement these conceptsin practice.

2. Designing for Human Abilities and Limitations

There are countless ways to represent information graph-ically, however not all of these ways are useful or even in-terpretable to human beings. When designing a graphic,we must consider our perceptual and cognitive capabil-ities. Effective graphics will be those that play to thestrengths of human information-processing abilities andavoid the weaknesses.

2.1. Graphical Encoding

In data visualizations, information is represented notwith numbers, but with geometric objects. Thus, quanti-tative and categorical data must be mapped or encoded tovisual objects, typically through attributes such as posi-tion, size, shape, or color. Communication is effective onlyif the perceptual decoding of information by the viewer issuccessful. Ideally, a visualization is designed that helps

the viewer to decode information effortlessly and accu-rately.

Psychophysics experiments have long shown that someencodings are easier to decode than others (e.g., Fech-ner, 1860). An intensive period of investigation into hu-man perceptual abilities in the 1950s–70s found power-law relationships between the true magnitude of a stimu-lus and its perceived magnitude (Baird, 1970a,b; Feinbergand Franklin, 1975; Stevens, 1975). For example, whilejudgements of length were approximately linear, perceivedbrightness, area, and volume all showed sublinear rela-tionships to true stimulus magnitude, i.e., in order to getthe same increase in perceived area from one object toanother, one must consistently over-represent the actualarea of the object.

In the 1980s, William Cleveland and colleagues pur-sued investigations into perceptual decoding in relation tographical communication, positing that visual attributeswhich could be decoded more accurately would ultimatelyresult in a greater likelihood of perceiving patterns indata and making correct interpretations (Cleveland andMcGill, 1984). The findings of Cleveland and McGill(1985), which have also been reproduced more recentlyby Heer and Bostock (2010), led to a ranking of visual at-tributes based on decoding accuracy, presented in Table 1.Several of these visual attributes are shown in Figure 1,which represents the same set of data encoded differentlyin each column. You can test for yourself whether someattributes are easier to accurately decode than others.

15

?

?

?

?

?

40

VOLUME AREA ANGLE LENGTH POSITIONSATURATIONVALUE

Figure 1: Test your ability to decode quantitative information fromdifferent visual attributes. Each column encodes the same 7 valuesusing a different graphical feature. How accurate is your perception?Values from top to bottom: 15, 30, 50, 10, 32, 24, 40. Adapted fromWong (2010b).

The relative abilities of humans to make accurate per-ceptual judgements has become the basis for recommend-ing particular graphical encodings over others. It is com-monly accepted that position along a common axis shouldbe the default encoding method for displaying quantita-tive information because it is most accurately decoded.Histograms, bar charts, line plots, scatter plots, and manyother forms encode information in this fashion. When asingle common axis is not sufficient to show the data, agood alternative is a series of replicated axes, known assmall multiples (Tufte, 2001). Color hue and saturation,


Visual attribute Rank

Position along a common scale 1 (most accurate)Position along non-aligned scales 2Length 3Angle, Slope 4Area 5Volume, Color saturation 6Color hue 7 (least accurate)

Table 1: A ranking of decoding accuracy for different visual at-tributes, adapted from Cleveland and McGill (1984) and Clevelandand McGill (1985).

which rank as the least accurate visual attributes, shouldbe used cautiously to encode quantitative information, atopic we discuss at length in Section 4.2.

With regard to perceptual limitations, special care shouldalso be taken when visually comparing curves (Haemer,1947a; Cleveland and McGill, 1984). To determine thedifference between curves, we must accurately judge thevertical distance between them. Unfortunately, our brainstend to judge the minimum distance between the curves,rather than the vertical distance. You can test your ownabilities to perceive curve differences in Figure 2 — youmay be surprised by the result. Thus, if the differencebetween curves is of interest, we recommend plotting thisquantity directly.

A B C a

b

ab

a

b

X

Y

How does the difference between curves change with X?

Figure 2: Test your perceptual ability to determine the differencebetween curves. In each panel, how does the difference betweencurves a and b change as a function of x? Answers: (A) a − bincreases exponentially with x; (B) a−b is constant over x; (C) a−bincreases linearly with x. Loosely adapted from Haemer (1947a).

We conclude this section by reminding readers thatwhile computer graphing capabilities advance rapidly, thehuman visual system persists relatively unchanged. Con-sider the enduring perceptual limitations of humans whenimplementing new graphical designs (Cleveland, 1994).

2.2. Short-term and Long-term Memory

Effective designs accommodate not only perceptual lim-itations, but also the constraints of short-term memory.Healthy individuals can keep only three to five objectsin visual short-term memory (Cowan, 2001). Since thiscapacity is not easily increased, we must design graphicsthat do not require the reader to hold too many detailsat once. For instance, graphics commonly place too much

material in a display, overwhelming the viewer, or relegatetoo much information to the figure legend, requiring theviewer to arduously memorize encodings and definitions(Kosslyn, 1985). Improved graphical designs (Sec. 2.3)and integrated annotation (Sec. 5.3) can reduce demandson short-term memory, enhancing and accelerating under-standing.

Considerations of long-term memory involve takingadvantage of past experiences. Comprehending a con-ventional display requires little more than recognizing thechart type and identifying graphical elements. In con-trast, a novel display type must first be deciphered — itis a puzzle to be solved, rather than a graph to be read(Kosslyn, 1985). By following conventions and using fa-miliar chart types (Sec. 3.1), we can exploit the repositoryof knowledge stored in long-term memory and facilitatecomprehension.

2.3. Design Principles

When we write, we choose each word carefully andconsider its integration into the surrounding text. Whenwe design, we follow the same process and choose each vi-sual element based on its role in the graphic. Good designchoices, like good writing, make ideas easy to understand.We present five design principles that will help to guideyour choices.

Principle 1: Determine the goal. Designing avisualization without a goal is a bit like taking a roadtrip without a destination — you may see many beauti-ful things along the way, but you’ll never be sure whenyou’ve arrived. A clear goal can often be specified as aquestion to be answered by the visualization. For exam-ple, a simple goal is to answer “What is the distribution ofcontinuous variable Y ”, with secondary questions, “Doesthe variable require transformation?” and “Are there ex-treme outlying observations?”. With the goals defined,one can implement one or more graphical forms for visu-alizing distributional shape (e.g., see Fig. 3A) that willanswer the associated questions. Designing with a goal inmind will help you to determine which information yourvisualization needs to convey and the appropriate salienceof each visual element (Chambers, Cleveland, Kleiner, andTukey, 1983).

Principle 2: “Let the data dominate” (Wall-gren et al., 1996). A graph includes two parts: thedata and the annotations that put the data in context.The data should always take the leading role, with an-notations as supporting players. Comprehension can bediminished when minor details are highlighted (Wainer,1984). Adjust object size, line thickness, color, satura-tion, etc., to emphasize the data over non-data elements(Kosslyn, 1985; Krzywinski, 2013a). If you are in doubtregarding the visual prominence of elements, try squint-ing at the image from a distance: the data should be mostsalient feature and annotations should fade into the back-ground (Wong, 2011b). For example, in Figure 8B, thedata points are most prominent, followed by the smooth


curve obtained with locally weighted regression (LOESS),and finally the grid and labels.

Principle 3: “Simplify to clarify” (Wong, 2011c).Every element in a graphic competes for our visual at-tention. Good designs use the fewest elements possibleto communicate the message without compromising dataintegrity. Statistician and artist Edward Tufte providestwo pragmatic guidelines to help streamline figures: (1)“maximize the data-ink ratio”, i.e., the majority of inkin a graphic should depict the data, and (2) “erase non-data-ink”, i.e., superfluous elements that fail to conveyvital information should be removed (Tufte, 2001). Whenrefining a graphic, consider the primary goal of a figure,prune it down to its essential parts, eliminate any extra-neous elements, then refine the elements that remain touphold the message.

Principle 4: Consistency trumps creativity. Gooddesign uses consistent encodings, layouts, colors, and otherelements to achieve visual continuity across panels andfigures (Wainer, 2008). A single (well-considered) designwill reduce the burden on your reader to decipher graph-ics and will discourage misinterpretation. Creative andnovel designs have their place, but a conventional or fa-miliar display will require the least effort to comprehend(see also Sec. 2.2). As an example, consider Figure 3.Although each panel displays distinct content, the visualstyle, font families, and font hierarchy is maintained con-sistently across panels, simplifying digestion for the reader.

Principle 5: Group with Gestalt. Gestalt princi-ples are the rules of perceptual organization which helpus to understand and exploit the interaction between theparts and the whole (Ellis, 1999). Although there arenumerous Gestalt principles, we focus on two describinghow humans tend to organize visual elements into groups.First, the proximity principle states that objects that arecloser together tend to be perceived as being part of thesame group. This principle explains why *** *** and** ** ** are seen as two and three groups, respectively.Proximity is commonly applied to achieve effective lay-outs that reveal hierarchical relationships through align-ment and the use of negative space (i.e., blank space). Asan example, Figure 4 uses a relatively subtle addition ofnegative space between rows A and B to encourage theinterpretation that the plots within each row should beconsidered together, but that the rows themselves are dis-tinct. Second, the similarity principle states that objectswith similar visual attributes (e.g., shape, color, size, ori-entation, etc.) tend to be perceived as being part of thesame group. This principle is commonly exploited withsymbol encoding. For example, in the scatter plots dis-played in Figure 5, we easily perceive the presence of twogroups based on the visual (dis-)similarity of plotting sym-bols.

3. Selecting a Chart Type

When faced with visualizing a set of data, building orselecting a chart can be a daunting task. We find it use-ful to first determine dataset characteristics, since theseplace constraints on possible visualizations. One shouldbe well-acquainted with the dataset sample size (i.e., thenumber of observations collected), the data dimensionality(i.e., the number of variables collected for each observa-tion), and whether variables are considered independentor dependent. Equally important is the data type of eachvariable, a simplified taxonomy for which is provided inTable 2. A critical distinction is made between quantita-tive and qualitative (categorical) variables, as this affectsthe types of encodings that can be used.

Once dataset characteristics are established, we canmake choices regarding graphical encodings and transfor-mations into geometric objects, or “geoms” (Wickham,2009). Geoms are the building blocks of all charts andrepresent the mapping between a visual attribute and itsrepresentation with points, lines, and shapes. For exam-ple, if you encode a series of numbers using position alonga common axis (as in Fig. 3C–D), you could use a dot plot(where the geom is the set of points whose positions aredefined by the data), a line plot (where the geom is the linethat connects points defined by the data), or a bar plot(where the geom is the series of bars whose heights aredefined by the data). As we will see in the next section,layering geoms can provide rich and informative chartsthat meet our communication needs.

3.1. Common Chart Types

Common chart types are shown in Figure 3. Eachpanel displays one or more charts that may be appro-priate based on data types, data dimensionality, and thegoals of the visualization. As discussed previously, usingconventional chart types (and geoms therein) and adapt-ing as necessary will typically yield designs that are moreaccessible to viewers.

Take some time to study the charts in Figure 3 andconsider the encodings used to portray the data, and de-coding accuracy thereof. For low dimensional data (toppanels), nearly all common visualizations encode data basedon position along a common axis. As dimensionality in-creases and multiple encodings are required, less optimalencodings (such as area in mosaic plots and color in heatmaps) may be necessary. A notable exception to thisprinciple is the pie chart, which encodes data with an-gles and/or area despite visualizing only a single categori-cal variable. This discrepancy between decoding accuracyand data complexity, along with additional shortcomingsof pie charts (e.g., low data-density and labeling difficul-ties), have encouraged some data visualization experts torecommend strongly against their use (Tufte, 2001; Few,2007).

Although we name and portray a limited number ofchart types, there are in practice infinite ways to commu-

Allen and Erhardt Visualizing Scientific Data Page 5 of 21DAT

A D

IMEN

SIONA

LITY

EMPHASIS OF VISUALIZATIONDATA DISTRIBUTION COMPARISON

SMALL

LARG

E

BBAR CHART

STACKED BAR CHART

PIE CHART

Visualize frequencies or proportions over a single qualitative variable.

AHISTOGRAM

VIOLIN PLOT

BEE SWARM

Visualize the distribution shape, center, and spread of a single quantitative variable.

C BAR PLOTCompare the mean or other summary statistic over categories. The bar height, representing distance from zero, should be meaningful. Error bars can illustrate the uncertainty of the statistic.

D LINE PLOTCompare the mean or other summary statistic over a qualitative or quantitative variable. Comparisons between groups can be made by plotting multiple lines. Error bars/bands can illustrate the uncertainty of the statistic.

SCATTER PLOT

2D HISTOGRAM

E

Visualize the bivariate distribution between two quantitative variables. A scatter plot with multiple symbols allows group comparisons.

SMALL MULTIPLES

Compare any type of visualization over one or more categorical dimensions. Particularly useful when comparing across 4+ categories, which will typically overwhelm single-panel displays.

I

MAPH

Visualize spatial patterns of qualitative or quantitative variables.

SCATTER PLOT MATRIX

X

Y

Z

X Y Z

J

Visualize pairwise relationships between qualitative or quantitative variables in a multivariate dataset. Bivariate distributions are displayed in off-diagonal panels; univariate distributions are along the diagonal.

X Y ZV W

X

Y

Z

V

W

PARALLEL COORDINATE PLOT

GLYPHSRADAR PLOT

K

Visualize multivariate datasets, in particular to compare groups or samples. In parallel coordinate plots dimensions are aligned in parallel lines; in radar plots dimensions are arranged radially. Glyphs can be generated from the hulls created with a radar plot.

HEAT MAP

DENDROGRAM

GRAPH

2D PROJECTION

L

Visualize multivariate datasets to discover data structure. Visualizations portray the similarity among samples, emphasizing relationships and clustering.

11

29

18

5 6

17

2217

7

TABLE

MOSAIC PLOT

F

Visualize frequencies or proportions over two or more qualitative variables.

HEAT MAP

TREEMAPG

Visualize proportions in a hierarchical data structure. Color encoding (a heat map) can further visualize a qualitative or quantitative value for each cell.

Figure 3: Common chart types. Each panel shows possible visualizations based on the type of data, dimensionality, and desired emphasis.Panels A through L are arranged by increasing data dimensionality (top to bottom) and increasing emphasis on comparison (left to right).


Data Type Example Values

Qualitative/CategoricalNominal gender {female, male}Ordinal disease stage {mild, moderate, severe}

Quantitative/NumericalDiscrete number of errors made 8, 1, 5, . . .Continuous cortical thickness 1.72, 3.32, 2.26, . . .

Table 2: Data can be broadly classified as qualitative or quantitative. Qualitative data are categorical. Nominal variables differentiatebetween groups based only on their names, and ordinal variables allows for rank ordering of groups. Quantitative data are numerical withmeaningful intervals between measurements. Discrete variables can take only integer values, and continuous variables can take any real value.

nicate data through various encodings and the creativeuse of points, lines, and shapes. In his highly influen-tial book “The Grammar of Graphics”, statistician LelandWilkinson shuns the notion of a fixed “chart typology”,and instead encourages flexibility by layering geoms thatportray different levels of detail (Wilkinson, 2005). Ex-amples of such layering can be seen in Figure 3. In PanelA, a violin plot layers a box plot on top of a kernel den-sity estimate (smoothed histogram) (Hintze and Nelson,1998), making both the distribution shape and the dis-tributional quartiles accessible to viewers. Of course, onecould replace the box plot geom with a “rug” of the indi-vidual data points (referred to as a “beanplot” (Kampstra,2008)) or with a portrayal of the mean and/or standarddeviation, tailoring the visualization for what is meant tobe communicated. Another effective use of layering is theportrayal of data and a model fit to the data, as shown inFigure 10B. When including a geom that depicts a model,ensure that the data maintains visual prominence (i.e.,follow Principle 2 to let the data dominate).

One should also consider clarity when choosing be-tween geoms. For example, when visualizing a bivariatedistribution between continuous variables, a scatter plot(Panel E, top), may be the natural first choice. However,with a large dataset (e.g., hundreds or thousands of ob-servations) the density of points may obfuscate portrayalof the distribution. In this case, a two-dimensional (2D)histogram or contours of the 2D kernel density estimate(Panel E, bottom) offers greater clarity. (When permittedby the data density, layering a scatter plot on top of the2D contours can provide an attractive visualization thatallows access to the data at high and low levels of detail.)As a second example, a scatter plot can effectively com-pare bivariate distributions between a few categories thatare distinguished by different plotting symbols or colors.However when the number of categories exceeds three orfour, it may be difficult to discern and compare distri-butions. Small multiples can yield a less overwhelminggraphic.

3.2. Displaying Variation and Uncertainty

The concept of layering also facilitates the depictionof statistical variation and uncertainty. As emphasized bystatistician Howard Wainer, an effective data visualization

must “remind us that the data being displayed do con-tain some uncertainty” and must “characterize the size ofthat uncertainty as it pertains to the inferences we have inmind” (Wainer, 1996). Unfortunately, our recent survey ofneuroscience journals demonstrates that many publishedfigures do not meet this standard, particularly as data di-mensionality increases (Allen et al., 2012). When the goalof a visualization is to compare a measured or derivedquantity across categories or conditions, one must includea geom portraying the quantity and a second geom por-traying the uncertainty of the quantity. Note that withoutthe portrayal of uncertainty, accurate visual comparisonis not possible; viewers may draw incorrect or uninformedconclusions.

Variation and uncertainty can be portrayed with a va-riety of geoms, but are most commonly displayed witherror bars. Unfortunately, there is no single standard forwhat quantity the error bar should represent. In fact,there is an overwhelming plurality of possible meanings,e.g., a standard deviation (SD) of the sample, a stan-dard error of the mean (SEM), a range, a parametric100(1−α)% confidence interval (CI) of the mean, a boot-strap CI, a Bayesian probability interval, a prediction in-terval, etc. Each quantity has its own statistical inter-pretation, and poor labeling of error bars has been shownto mislead viewers (Wainer, 1996; Cumming and Finch,2005).

Thus, when using error bars, ensure that (1) the quan-tity encoded by the bar is consistent with the goal of thevisualization, and (2) the quantity is unambiguously de-fined. Regarding the first point, we offer the followingguidelines when using error bars to portray variation of aparameter estimate, or variation of the data.

If the interest is estimating a population parameter,such as the mean or variance, then the variation of theestimate (i.e., the sampling distribution of the statistic)is wanted. Examples of suitable error bars include theSEM or a 95% parametric or bootstrap CI, as seen invisualizations that emphasize comparisons (Fig. 3C–D).Parametric CIs should only be used if data meet the as-sumptions of the underlying model, otherwise a bootstrap(or other strategy for approximating the sampling distri-bution) should be used. Note that many aspects of CIs(and p-values) are often misunderstood, even by experi-


enced scientists (Belia, Fidler, Williams, and Cumming,2005; Hoekstra, Morey, Rouder, and Wagenmakers, 2014).For example, when comparing population parameters be-tween two or more groups, viewers may look to the overlapbetween CI bars to determine whether the parameters arestatistically different. While non-overlapping 95% CIs doindicate a significant difference (under a normal proba-bility model), the converse is not true — depending onsample size, the CI bars may overlap by as much as 50%and still meet significance criteria (Cumming, Fidler, andVaux, 2007; Krzywinski and Altman, 2013). Therefore,after defining the quantity portrayed by an error bar, werecommended interpreting that quantity and the resultsof any relevant hypothesis tests to help your reader reachthe correct conclusions.

Alternatively, if the interest is possible observationaloutcomes, then the variation of the data (i.e., the em-pirical distribution) is desired. In this case, “error bars”might indicate one SD from the mean or the interquartilerange (middle 50% of data) around the median (Cleve-land, 1994). Note, however, that these “error bars” donot reflect error or uncertainty, but rather variation. Assuch, in these cases we prefer to use geoms such as boxplots and violin plots (Fig. 3A) that are intended to em-phasize distributions.

Displaying uncertainty remains an active area of re-search and development, particularly for compact visual-izations of high-dimensional data (e.g., Griethe and Schu-mann, 2006; Potter, Rosen, and Johnson, 2012). For heatmaps,Hengl (2003) suggests using a 2D colormap: color hue isused to visualize observations, and transparency is usedto visualize the error or uncertainty associated with eachobservation. In Section 6.2 we apply this strategy to func-tional brain imaging data and find the visualization to bemuch more informative than the typical design. One con-cern with this approach is how accurately combined hueand transparency encodings can be decoded, particularlygiven the perceptual non-independence of these visual at-tributes (Wilkinson, 2005). We look forward to furtherimprovements and innovations in visualizing uncertainty.

3.3. Multivariate Visualizations

Because readers may be less familiar with the multi-variate visualizations shown in Panels J-L of Figure 3, weprovide a more detailed discussion of their compositionand usage. The scatter plot matrix (Panel J) displaysbivariate relationships between all pairs of variables ar-ranged into a matrix. Originally conceived as a visualiza-tion for only continuous variables, the scatter plot matrixhas since been generalized to include both continuous andcategorical variables and is also referred to as a generalizedpairs plot (Emerson, Green, Schloerke, Crowley, Cook,Hofmann, and Wickham, 2011). The example dataset inPanel J includes three variables, X (categorical), Y (con-tinuous), and Z (continuous). The visualizations of bi-variate distributions make use of simpler geoms depictedin earlier panels and are highly customizable. Since the

full square form of a scatter plot matrix is redundant (i.e.,it displays both X vs. Y and Y vs. X), many choose to in-clude only axes in the upper or lower triangle of the matrix(i.e., above or below the diagonal, respectively). Alterna-tively, one can use the redundancy to provide slightly dif-ferent visualizations of the same bivariate distributions.For example, we plot X vs. Z (bottom left axis) usingviolin plots to emphasize distributional differences, andplot Z vs. X (top right axis) using bee swarms to showindividual observations.

One limitation of scatter plot matrices is that visual-izations are limited to pairs of variables. Panel K showsseveral chart types that help to overcome this limitation.In a parallel coordinate plot, data dimensions are depictedas equally-spaced parallel lines. The example dataset hasfive dimensions (variables V , W , X, Y , and Z) portrayedby parallel lines. Each observation in the dataset is rep-resented as a line or profile which traverses the parallels.Profiles of similar observations will form “bundles”, indi-cating natural clusters in the data and making outliers ap-parent. A related visualization is a radar plot, or star plot,which orients axes radially rather than in parallel. Forboth methods, the scaling and order of axes are arbitrary.Both properties can profoundly affect the appearance ofthe visualization, thus one may consider re-ordering or re-scaling to help reveal patterns. Note, too, that arbitraryscaling and ordering can also make pairwise relationshipsmuch less accessible than in a scatter plot matrix (seeFig. 4B for an example).

Multivariate data can also be visualized with glyphs,which are symbolic or iconic representations wherein eachvariable is mapped to a visual feature. Simple glyphscan be generated directly from the hulls created with aradar plot, where each variable is mapped to the ver-tex position of a polygon (Panel K, bottom). More com-plex glyphs include Chernoff faces (Chernoff, 1973), whichmaps each variable to a different facial feature (e.g., noselength, mouth curvature, eye separation, eyebrow angle,etc.), exploiting the innate ability of humans to recognizeand analyze faces. In neuroimaging, glyphs are commonlyused to display diffusion weighted images: variables de-scribing the orientation and magnitude of diffusion alongfiber tracts are mapped to ellipsoid-like shapes displayedat each voxel location (Margulies, Bottger, Watanabe, andGorgolewski, 2013). A significant advantage of glyphs overscatter plot matrices or parallel coordinate plots is thatpatterns involving several dimensions can be more readilyperceived. However, the accuracy with which glyphs canconvey data is often questioned, and the use of complexglyphs may be best suited for data exploration or quali-tative comparisons (Ward, 2008).

While the chart types in Panels J and K lose their in-terpretability at 10–20 dimensions, the visualizations inPanel L are suitable for very high-dimensional data (e.g.,observations with hundreds of time points or thousands ofpixels). Notably, none of these visualizations display theoriginal data or even reveal the number of variables in the


dataset. Rather, they focus on the similarity or differencesbetween observations, or use data reduction methods toproject the data to a lower-dimensional space. In the ex-ample, we consider a dataset with five observations, eachof which has values over a large number of variables. Wecan visualize the relationships between observations bycomputing the similarity matrix (or distance matrix) anddisplaying the result as a heat map. Reordering the simi-larity matrix (Friendly, 2002) can help to reveal patternsor clusters; in this case we can visually identify two differ-ent clusters with three and two observations, respectively.The similarity matrix can be used to produce a theoreticalnetwork graph, where each observation is represented bya node (circle) and relationships between them are rep-resented by edges (lines). Such visualizations have be-come ubiquitous in the study of structural and functionalbrain connectivity, where methods adapted from graphtheory are used to describe and analyze brain networks(Bullmore and Sporns, 2009). Alternatively, the originaldata can be subjected to hierarchical clustering and dis-played as a dendrogram, where the height of each branchindicates the difference between observations. Finally, wecould also apply dimension reduction techniques to theoriginal dataset, producing an accessible visualization intwo (or three) dimensions. The classical linear data re-duction approach is principle component analysis (PCA),where data are projected onto the dimensions that cap-ture the greatest variance. Nonlinear dimension reductionapproaches, such as Sammon mapping (Sammon, 1969)or t-Distributed Stochastic Neighbor Embedding (t-SNE)(Van der Maaten and Hinton, 2008), determine a mappingto a low-dimensional space by optimally preserving rela-tionships between data present in the high-dimensionalspace. T-SNE, in particular, has been found to be verywell-suited for visualizing high-dimensional data.

While multivariate visualizations can be initially over-whelming and may take more time to interpret, we believethey are under-utilized both for the purposes of data ex-ploration and presentation. Often, multivariate datasetsare unfortunately portrayed with a series of univariate vi-sualizations. This reduces the ability to systematicallystudy relationships between variables or find outliers inthe multivariate space. As a simple example, consider thescatter plot in Panel E. The plot reveals not only a nonlin-ear relationship between variables, but also an outlier thatis only detectable in the 2D space, i.e., the point wouldnot be seen as an anomaly in any portrayal of univari-ate distributions. Likewise, outliers in an n-dimensionalspace may only be visible in visualizations that portray orincorporate multiple dimensions simultaneously.

3.4. 3D visualizations

Note that regardless of dataset dimensionality, noneof the chart types displayed in Figure 3 use a 3D visu-alization. This is not by accident. Portraying data ina 3D space requires additional complexity, often at thecost of clarity. In order to interpret 3D representations,

the visual system relies on depth cues such as occlusion(where objects in the foreground partially conceal thosein the background) or graphical perspective (where par-allel lines converge toward the background) (Gehlenborgand Wong, 2012). These cues can interfere with effectivevisualization by obstructing data or distorting height andlength estimates. Thus, while 3D graphics may be appro-priate in some cases (e.g., portraying 3D objects such asneuroanatomical structures), 3(+)-dimensional data cantypically be visualized more effectively on the 2D plane.As shown in Figure 4, using additional encodings (such ascolor) or selecting chart types suitable for higher-dimensionaldata can easily supplant the need for 3D graphics.

Recommendations against plotting in a 3D space areeven stronger when visualizations include a pseudo-thirddimension or “3D-effect” to create a sense of graphicalsophistication, a prime example of “chartjunk” (Wainer,1984; Tufte, 2001). Examples include 3D-effect bar charts,where each bar is portrayed as a column, or the highly crit-icized 3D pie chart. In these cases the empty third dimen-sion is completely gratuitous and simply degrades decod-ing accuracy. Although data visualization experts haveadvised against such designs for nearly 70 years (Haemer,1947b, 1951), such graphics unfortunately persist.

4. Symbols and Colors

4.1. Using Symbols

When it comes to plotting data, not all symbols arecreated equal. Effective symbols minimize data occlusionand provide intuitive visual contrast between different cat-egories (Krzywinski and Wong, 2013).

For plots with a single category of data, Cleveland(1994) recommends using the open (i.e., unfilled) circle asa plotting symbol. Open symbols are preferred to filledsymbols because they retain their distinctness in instancesof overlap. Additionally, circles are unique in that theirintersection does not form another circle — this is notthe case with squares (which can form additional squareswhen intersecting), triangles, crosses, or other symbols.One downside of the open circle (or any open symbol)is that it does not contain ink at the value being plot-ted. Thus, the eye may be drawn to the periphery of theshape, distorting judgements of length or position. Onecan mitigate this concern by using small symbols, or, incases of low or no overlap, by reverting to the filled circle(e.g., see Fig. 9).

For plots with multiple categories, choose symbols withhigh contrast that are easy to distinguish. Figure 5 com-pares combinations of plotting symbols for two groups. Asseen in Panel A, using open circles and open squares (orany other pair of open polygons) provides relatively poorcontrast — the eye must actively seek out different cat-egories. Discrimination is improved by changing symbolshape (from squares to crosses or filled circles) or symbolsaturation, as shown in panels B-F. Where possible, we


MAP

(mm

Hg)

80

100

120

140

PWV

(m/s

)

Age (years)20 40 60 80

6

9

12

MAP (mmHg)80 100 120 140 Age

(years)MAP

(mmHg)PWV(m/s)Age (years)

Mea

n ar

teria

l pre

ssur

e (m

mH

g)

20 40 60 80

80

100

120

140

PWV (m/s)

4.5 6.5 8.510.5

12.5

10

100

60

140

3

14

Age (years)

15 40 65 90

3D SCATTER PLOT SCATTER PLOT MATRIX PARALLEL COORDINATE PLOT

SCATTER PLOTWITH COLOR ENCODING

B

A

Task difficultyG

enot

ype

1 2 3 4

AA

Aa

aa

12

34

aa

Aa

AA

0

2

4

Reac

tion

time

(s)

Reaction time (s)

1.5 2.0 2.5 3.0 3.5

Task difficultyGenotype

3D BAR PLOT HEAT MAP SMALL MULTIPLESLINE PLOTWITH SYMBOL ENCODING

Task difficulty

Reac

tion

time

(s)

1 2 3 4

1

2

3

4

5

1 2 3 4 1 2 3 4

AAAaaa

0

Task difficulty

Reac

tion

time

(s)

1 2 3 41

2

3

4

aa

Aa

AA

Puls

e w

ave

velo

city

(m/s

)

Age (y

ears)

Mean arterial pressure (mmHg)

2040

6080

60801001201404

7

10

13

mean ±1 SEM

Figure 4: Datasets with 3(+) dimensions need not be visualized in a 3D space. Panels A and B show examples of 3D visualizations (left) aswell as alternative visualizations that portray the same data on the 2D plane. Both datasets are synthetic.

recommend changing symbol color rather than saturation,as color is recognized as the most effective discriminator(Lewandowsky and Spence, 1989); see also Section 4.2 andFigure 6. When varying symbol attributes, be mindfulof the effects on salience. For example, in Panels C, D,and E, one category is more salient than the other. Ifcategories are intended to have equal prominence, we pre-fer panel B (which uses very distinct shapes with similarsalience) or panel F (which balances the ink used by eachcategory).

When plotting categories that have a natural hierar-chy or grouping, choose parallel symbols that reflect thisorganization (Krzywinski and Wong, 2013). The degreeof visual contrast between categories should mirror the in-tended analytic contrast. For example, consider a studycomparing subjects with different diagnoses where datahas been collected at multiple research sites around thenation. You are primarily interested in differences withrespect to diagnosis, but also want to consider the influ-ence of collection site. One approach would be to usedifferent colors to encode diagnosis (since color will pro-vide the greatest visual distinction between categories)and different symbols to encode study site.

With numerous categories to distinguish, visual dis-tinctness may be futile regardless of how strategically sym-

CHA

NGE

SYM

BOL

SATU

RAT

ION

CHANGE SYMBOL SHAPE

BA C

D E F

Figure 5: Increase the visual distinction between symbols by chang-ing symbol shape or saturation. When available, color can also bea powerful discriminator.


bols have been chosen. The data-density and degree ofoverlap may also present a challenge. As mentioned inSection 3.1, a good strategy is to use small multiples andpresent each category (or subset of related categories) ina separate panel.

4.2. Using Color

The trichromatic visual system of humans allows mostindividuals to distinguish millions of different colors, cre-ating a rich sensory experience of the visual world. Incor-porating color into scientific visualizations can increaseexpressiveness and heighten impact, however there are anumber of disadvantages introduced by color that shouldbe addressed.

First, color is a relative (rather than an absolute) medium(Wong, 2010a). Human color perception depends on con-trast with neighboring colors, as is often illustrated withbeguiling optical illusions (Albers, 1975). This depen-dence can create unfortunate interactions between colorand spatial location, biasing our judgments. As demon-strated by Cleveland and McGill (1983), color also inter-acts with other visual features such as area (brighter colorscan make areas appear larger than darker colors).

A second concern in using color is color blindness,which affects seven to ten percent of males. Commonforms of color blindness involve deficiencies in red- orgreen-sensitive cone cells, thus red/green contrast (or anycontrast that involves similar levels of red and green, suchas orange/light-green) should be avoided. To ensure thatyour graphic is accessible to the widest possible audience,use colorblind-friendly palettes (colorbrewer2.org). Ad-ditionally, graphical editing software often includes toolsto simulate red-green color blindess, allowing you to makeadjustments to color while you work. Similar web-basedsimulators, such as Coblis (color-blindness.com) or Vis-check (vischeck.com), invite you to upload your visual-ization and check how it appears to individuals with com-mon color vision impairments.

Thirdly, when working with color you must considerthe different representations of your graphic in digital andprinted media. Digital media (e.g., computer screens, pro-jectors, etc.) operate in the additive RGB (red, green,blue) color space: different intensities of R, G, and B lightare emitted to produce different colors. Print media re-lies on the subtractive CMYK (cyan, magenta, yellow,black) color space: quantities of C, M, and Y pigmentsabsorb wavelengths to produce different colors via reflec-tion. RGB has a wider color gamut than CMYK, meaningthat not all colors visible on the screen can be reproducedin print. The difference in gamut is most noticeable forvibrant colors. You may find your printed graphic appear-ing dull or muted with a loss of discriminability aroundgreen and violet hues. Issues surrounding conversion toprint media are more severe when color printing is not anoption and images are converted to grayscale. Categoriesor features that were once easily differentiated by color

may become impossible to distinguish, though see belowfor strategies to avoid this problem.

Given the disadvantages of color, we recommend try-ing to use other designs or encodings first. If you do decideto use color, make sure that the benefits to your visual-ization outweigh any costs. Figure 6 shows two examplesof incorporating color. Panel A uses color to encode acategorical variable (which we refer to as qualitative colorencoding), and Panel B applies color to a continuous vari-able (quantitative color mapping). Specific considerationsfor each type of encoding are discussed below. In Panel A,color-coded categories are easier to discriminate, despiteour efforts to maximize contrast between grayscale sym-bols with shape and saturation. Because we have doubleencoded (with symbols and colors), the graphic will beaccessible to color blind individuals as well as viewers ofgrayscale reproductions. In Panel B, color has a profoundimpact. Using only grayscale, the salience of negative andpositive values is unbalanced as our attention is drawn to-ward darker regions. Color restores balance because nega-tive and positive values can be represented with equivalentsaturation. Additionally, color encoding allows zero to bemapped to visually neutral white (rather than ambiguousmedium-gray), facilitating pattern detection. The chosenblue-to-red colormap avoids red/green contrast and is ro-bust to a variety of color vision impairments.

Qualitative Color Encoding

Effective qualitative color encoding requires adept choiceof easily-differentiated colors. One strategy for doing so,suggested by artist and color theorist Albert Munsell, isto determine a “harmonic” palette by selecting equally-spaced points along a perceptually uniform color space(Munsell, 1947). Similar recommendations are made morerecently by Cynthia Brewer (Brewer, 1994). The col-ors resulting from this approach have distinct hues withcomparable saturation and brightness, providing a palettethat is unlikely to produce attentional bias. We adoptthis strategy in Figure 6A and use a 4-color qualitativepalette provided by Brewer’s online resource, ColorBrewer(colorbrewer2.org). A second strategy, proposed byBang Wong, is to select colors by spiraling through hueand saturation while varying brightness (see Wong, 2010a,Fig. 2). The resulting colors have distinct hues and bright-ness, thus are discernible even in grayscale reproductions.As with symbol encoding, if your dataset has so manycategories that a palette with easily-differentiated colorsis unobtainable, consider small multiples.

Quantitative Color Mapping

Quantitative color mapping, that is, the mapping ofa numerical variable to a continuous progression of color,is considered controversial by several data visualizationexperts. Tufte (2001) writes that “the mind’s eye doesnot readily give a visual ordering to colors,” and Kosslyn(1985) emphatically states that “differences in quantities


RegionA B C D E F G H I J K L MN O P Q R S T

ABCDEFGHIJKLMNOPQRST

Correlation

−1 −0.5 0 0.5 1

RegionA B C D E F G H I J K L MN O P Q R S T

ABCDEFGHIJKLMNOPQRST

Correlation

−1 −0.5 0 0.5 1

True score (%)

Pred

ictie

d sc

ore

(%)

50 60 70 80 90 10050

60

70

80

90

100

True score (%)50 60 70 80 90 100

50

60

70

80

90

100

Regi

on

task 1task 2task 3task 4

task 1task 2task 3task 4

QUALITATIVE COLOR ENCODING

QUANTITATIVE COLOR MAPPING

A

B

Figure 6: Examples of incorporating color for qualitative encodingof different categories (A) and quantitative mapping of a continuousvariable (B). In A, the addition of color facilitates group discrimina-tion. Colors were chosen based on a 4-color qualitative palette pro-vided by ColorBrewer (colorbrewer2.org). In B, colors facilitatepattern detection and balance the salience of positive and negativevalues.

should not be represented by differences in color . . . shift-ing from red to green does not result in ‘more of some-thing’ in the same way as shifting from a small dot to alarge one does.” We agree that the relationship betweenquantitative value and color is arbitrary (and must beclearly defined), but feel that strategically chosen map-pings can compensate for the lack of an absolute colororder.

Consider four common colormaps displayed in Fig-ure 7, as well as the trajectories these maps take throughthe hue-saturation-value (HSV) color space. (Note thatHSV is equivalent to RGB in terms of gamut and simplyuses a different set of axes to navigate the same space.)HSV is portrayed as a cylinder, where angle around thevertical axis corresponds to hue (the color), radial dis-tance from the vertical axis corresponds to increasing sat-uration (color intensity), and height along the verticalaxis corresponds to value (color brightness). We find thatthe trajectories taken by the colormaps (Paths 1–4) sug-gest natural and specific mappings to different data types.For example, Path 1 circumnavigates the upper rim ofthe cylinder and represents a circular progression throughpure hues, thus is appropriate for cyclical data such asphase angle or time of day. In contrast, Paths 2 and 3 in-tersect the vertical axis representing neutral colors (whiteat Point B; gray at Point E), thus are suitable for datathat diverge from an origin (commonly, zero). These “di-vergent” or “bipolar” colormaps are consistent with ourperceptual expectations that the distance from the originshould correspond to “more of something” — more colorintensity in Path 2 and more brightness in Path 3.

Thus, the choice or design of a colormap must considerthe nature of the data and how this can be aligned withperceptual expectations. If the data contain a discontinu-ity or abrupt change point, construct a colormap with asimilar perceptual discontinuity. Rely primarily on satu-ration or brightness (rather than hue) to reflect changesin quantitative magnitude, since these dimensions do havea natural order. Map neutral grayscale colors to zero orother natural baselines. Most critically, include a coloraxis (i.e., a colorbar) defining your color mapping (Allenet al., 2012). Analogous to any x- or y-axis, the coloraxis should be properly labeled and include sufficient tickmarks to help viewers decode the map. See Figure 6 foran example.

In Figure 7 we also draw attention to the fact thatalthough the colormaps progress smoothly through HSVspace, the color transitions we perceive are non-uniform.For example, in Path 1, a pair of points represent colorsperceived as nearly indistinguishable green hues, while asecond pair of equidistant points are perceived as verydistinct hues (yellow and orange). Similar discrepanciescan be seen in the other paths. In general, a percep-tually smooth colormap must be carefully designed andcalibrated.

Despite numerous limitations, quantitative color map-ping is and will likely remain a preferred choice for com-


Hue

Valu

e

Saturation

Path 1

Path 2

Path 3

Path 4

Valu

e

Saturation

D

A

B

F

C

G

E

H

J

I

C B

A

D FC GH JI

D FC GE

D FB

G DF FC

HUE-SATURATION-VALUE COLOR SPACE

TRAJECTORIES OF COMMON COLORMAPS

SLICE AT A SINGLE HUE

Figure 7: A depiction of the hue-saturation-value (HSV) color space. Four different paths, described by points A–J, denote the trajectoriesthat common colormaps take through HSV space. On each colormap, colors highlighted in circles illustrate that equidistant shifts along thetrajectory are not commensurate with our perceived shifts in color.

pact visualizations of large datasets. Maintain awarenessof these limitations and expect that mappings may intro-duce biases or distortions. If the goal of your visualizationrequires highly accurate judgements, either avoid colormapping altogether or provide additional designs (e.g.,small multiples of relevant “slices” through an image) thatclarify the data through more accurate encodings.

5. Supporting Details

The supporting details of graph construction includethe use of axes, grids and guides, and verbal and graphicalannotations. These details may seem minor, but play aprofound role in clarifying information and guiding atten-tion.

5.1. Axes

Axes are the coordinate system reference around data,orienting viewers to the data’s extent, scale, and precision.Axes should extend slightly beyond the extremes of thedata to avoid concealing symbols or lines, as well as toprovide a visual frame for the viewer’s attention (Wainer,2008). If extreme values dramatically compress the rangeof the majority of your data, consider using axis breaks,scale transformations, or trimming outlying values, andnotate these elements or omissions on the plot (Krzywin-ski, 2013a).

One should also consider the axes aspect ratio andscale the width and height to maximize visual discrimina-tion of trends and change points. Figure 8 displays twodifferent datasets at three different aspect ratios. It is ob-vious from these plots that the accessibility of patternsis strongly impacted by aspect ratio. Cleveland, McGill,and McGill (1988) recommend that axes should be scaledsuch that the mean absolute angle of line segments is 45degrees, since deviations of slopes in this range will be eas-iest to detect. Although the “bank at 45 degrees” guide-line has received some recent criticism and may not beoptimal in all cases (Talbot, Gerth, and Hanrahan, 2012),we feel it is a valuable starting point.

Additional recommendations for axes apply to specificchart types or data types.When working with small multi-ples, determine whether comparisons between axes shouldbe based on relative or absolute differences. If absolutedifferences are most relevant, use uniform scaling acrossthe axes and label a single y- (or x-) axis to emphasizethat the scale is fixed (Krzywinski, 2013a). See Figure 4A(rightmost panel) for an example. When working withcategorical (in particular, nominal) variables, special at-tention should be paid to the order of categories. Fein-berg and Wainer (2011) and many others suggest thatcategories should be ordered based on the data (e.g., byfrequency or median value) and that one should avoid al-phabetical or other arbitrary orders. As seen in Figure 9,a data-driven ordering can greatly enhance one’s ability


Time (seconds)

Pres

sure

(mm

Hg)

ASPECT RATIO 2 : 1

0 2 4 6

120

125

130

Time (seconds)

Pres

sure

(mm

Hg)

ASPECT RATIO 1 : 1

0 2 4 6

120

125

130

Time (seconds)

Pres

sure

(mm

Hg) ASPECT RATIO 1 : 5

0 2 4 6

120

125

130

Error rate (%)

Reac

tion

time

SD (

ms)

ASPECT RATIO 2 : 1

0 5 10

100

150

200

250

300

Error rate (%)

Reac

tion

time

SD (

ms)

ASPECT RATIO 1 : 1

0 5 10

100

150

200

250

300

Error rate (%)Reac

tion

time

SD (

ms)

ASPECT RATIO 1 : 5

0 2 4 6 8 10100

200

300

B

A

Figure 8: Choose an aspect ratio for the axes that maximizes visualdiscrimination. Panels A and B show two different sets of simulateddata plotted at different aspect ratios. In A, an aspect ratio of 1:5(bottom) yields curve segments with slopes close to 45◦ and revealsdetail in the oscillatory pattern that are not apparent at higheraspect ratios. In B, an aspect ratio of 1:1 (top right) yields linesegments with slopes close to 45◦, making the change point clearlyvisible. Loosely adapted from (Cleveland, 1994).

to detect trends and identify categories with specific prop-erties. Note too, that categorical axes often benefit fromvertical orientation. Though this may break with the con-vention of placing dependent variables along the y-axis, itallows for categories to be labeled with horizontal text,eliminating the need for viewers to rotate their heads 90degrees.

5.2. Grids and Guides

Axis ticks, grids, and guides comprise the “naviga-tional elements” that help viewers determine where datalie in the (x, y) plane and in relation to other importantquantities (Heer and Bostock, 2010; Krzywinski, 2013a).Rather than falling victim to software defaults, manuallyset tick mark spacing and grid density in a manner thatis consistent with the data. For example, if your dataare discrete and take on only integer values, remove ir-relevant ticks or grid lines at non-integers (e.g., see Fig-ure 11). When appropriate, tick marks should be spacedby base-10 conventions in multiples of 1, 2, 5, or 10, sincelabels will be more accessible to readers (Wainer, 2008;Wallgren et al., 1996). Choose tick mark and grid densitycarefully: a dense grid suggests high data precision andthat minor differences are important (Krzywinski, 2013a).Furthermore, very dense grids can actually lead to greaterjudgement errors (Heer and Bostock, 2010) or give rise tomoire effects (i.e., the illusion of movement or vibration)(Tufte, 2001). In all cases, ensure that the data retainsprominence over the grid. This can be achieved by adjust-ing grid line width, color, and opacity, or, for bar graphs,by “gridding” only the bar rather than the entire plotregion (see Tufte, 2001, p. 128).

Guides refer to lines or other geoms that provide aspecific reference. Common examples include a point orline at the value of a parameter under the null hypothesis,or a y = x line for scatter plots comparing pre- and post-measurements or true and predicted values (e.g., Fig. 6A).As usual, emphasize the data over the guide.

5.3. Annotation

The majority of verbal annotation in a figure is ded-icated to the labeling and defining of graphical objects.In all cases, labels should be explicit and complete. Forexample, an axis label of “Time (s)” is considerably moreambiguous than “Time since stimulus onset (s)” (Wainer,2008). Labels defining symbol encodings should also beincluded on the graphic, potentially in a legend or key asshown in Figure 6A. Place legends as close as possible tothe plotting symbols they define. As mentioned in Sec-tion 2.2, the separation between labels and the objectsthey are labeling can place unnecessary demand on visualshort-term memory, forcing viewers to toggle back andforth between the data and the legend to recall encod-ings. When possible, omit a legend and label shapes orlines directly, as in Figure 4A (Schmid, 1983). In the caseof crossing lines, ambiguity can be reduced by providinglabels at both ends (Wainer, 2008).


Functional connectivity to posterior cingulate (t−statistic)

ORDERED ALPHABETICALLY

0 10 20 30

Superior temporal LSuperior frontal

Semilunar lobule RSemilunar lobule L

PyramisPons

Parahippocampal RParahippocampal LMiddle temporal RMiddle temporal R

MidbrainMedulla RMedulla L

Medial frontalInferior temporal L

Inferior frontal RInferior frontal RInferior frontal L

Anterior cingulateAngular gyrus RAngular gyrus L

Functional connectivity to posterior cingulate (t−statistic)

ORDERED BY MAGNITUDE

0 10 20 30

PonsMidbrain

Superior temporal LSemilunar lobule R

Inferior frontal RInferior frontal LInferior frontal R

Medulla RPyramis

Medulla LSemilunar lobule L

Parahippocampal RParahippocampal LMiddle temporal RInferior temporal LMiddle temporal RAnterior cingulate

Superior frontalMedial frontal

Angular gyrus LAngular gyrus R

Figure 9: Order nominal variables based on the data. Ordering alphabetically (left) or by other arbitrary criteria yields data patterns thatare difficult to parse. Ordering based on statistical magnitude (right) immediately reveals regions with high and low functional connectivityto the posterior cingulate. Graphic is based on data reported in Tomasi and Volkow (2011).

When error bars or other portrayals of uncertainty areincluded, define the type of uncertainty that is portrayed(e.g., see Fig. 10A). Such information is typically includedin the figure caption, however we recommend it appeardirectly on the graph to avoid the possibility of misinter-pretation (Cumming and Finch, 2005; Vaux, 2004).

Another important use of annotation is the integra-tion of statistical or numerical descriptions. When scien-tific graphs are intended to address a specific question orhypothesis visually, there is often a corresponding statis-tical model that addresses the same question numerically.Placing relevant statistics alongside their graphical coun-terparts can improve understanding of both the graphicand the model and remove the burden on readers to di-vide their attention between the figure and explanatorytext elsewhere (Lane and Sandor, 2009). Commonly inte-grated statistics include correlation coefficients, regressioncoefficients and p-values, however annotations can alsoinclude instructive explanations of the hypothesis beingtested. Figure 10 provides two examples where relativelyelaborate statistical annotation has been integrated intothe visualization. In both cases, annotation is limited todescription of the statistical model(s) and key parametersthat address the hypothesis. One should keep the textconcise and avoid indiscriminately reporting all informa-tion that software may output.

Note that while we and others advocate for the ad-dition of descriptive numerical and verbal annotation infigures (e.g., Tufte, 2001; Lane and Sandor, 2009), thispractice is actively discouraged by some journals, perhaps

to prevent excessive “storytelling” or elaboration. If youfind yourself limited by journal conventions, include sta-tistical descriptions and definitions in the figure captionto maintain proximity to the graphic.

A type of non-verbal annotation that deserves specialmention is the arrow, which is best used as a “visual verb”to describe processes and relationships (Wong, 2011a). Inthis chapter, arrows are used in different scenarios to in-dicate continuation (Fig. 3), interchangeability (Fig. 4),and process flow (Fig. 11). Because arrows can representso many different functions and actions, ensure that yourintended meaning is clear from context.

5.4. Font Choice

The clarity and impact of verbal annotation is affectednot only by what you say, but also how you say it. Thechoice of typeface (more commonly known as font family,e.g., Helvetica) and font style (bold, italic, etc.) shouldbe made based on the purpose of the text as well asits desired tone. Typefaces are broadly classified intotwo groups depending on the presence or absence of ”ser-ifs” (i.e., the small projections attached to the end ofstrokes). Serif typefaces tend to be easier to read inprinted multi-line blocks. Sans-serif (without serifs) fontsare simpler forms which can decrease time for word recog-nition (Moret-Tatay and Perea, 2011), thus are suited forsmaller chunks of text such as labels and headings. Forboth classes, Gestalt principles dictate that labels of agiven type (axis tick labels, line labels, etc.) be of a singletypeface and style. Using multiple typefaces or different


A

B

ERN amplitude (µV)

Accu

racy

(%)

Are the slopes different between groups?

−10 −5 0

80

85

90

95

100

Pupi

l dia

met

er (m

m)

Model 2 (shown):Y = β0 + β1G + β2XR2 = 0.64

mean, 95% confidence interval

Group 1Group 2

How do the groups differ across conditions?

Facial expression conditionneutral angry

2

3

4

5

Group 1Group 2

repeated measures ANOVA: FC (1,18) = 16.0, p < 0.001 FG (1,18) = 0.3, p = 0.6 FC × G (1,18) = 12.7, p = 0.002

Model 1:Y = β0 + β1G + β2X + β3G × Xβ3 = 0.14, t(76) = 0.6, p = 0.6R2 = 0.64

Figure 10: Integrate statistical descriptions related to the hypoth-esis of interest. In each panel, the graphic is designed to address aspecific question or hypothesis. Incorporating corresponding statis-tics can enhance interpretation of both the graphic and the model.Both datasets are synthetic.

font styles within a typeface can be an effective approachto create contrast or establish a visual hierarchy, howeverone should be wary of overwhelming the viewer with in-consistencies. Additionally, consider the context of yourmessage and find a typeface and font style with a comple-mentary mood. For example, throughout this chapter wehave used two different typefaces: to label data, axes, andtick marks we use the freely available sans-serif typeface“Open Sans”, which is easy to read and has a clean andunassuming appearance; for more expository labels, com-ments, and questions, we use a contrasting handwrittentypeface “Architect’s Daughter”, which evokes a friendlyand less austere tone.

5.5. Numerical Precision

When reporting numbers in figures (or tables), a goodstrategy is to round liberally, retaining the fewest sensi-ble significant digits. There are several reasons for this:(1) humans don’t comprehend more than three digits veryeasily, (2) it’s very rare that we care about more than threedigits of absolute accuracy (when using scientific nota-tion), and (3) it’s even rarer that we can justify more thanthree digits of statistical precision (Feinberg and Wainer,2011). As an example, consider the statistical precisionof the Pearson correlation coefficient. While the defaultreporting of correlation by some software is four digits,achieving a standard error less than 0.00005 requires asample size greater than 400 million! Justifying more thanone digit still requires a sample size at least 400, a levelmet by relatively few research studies. (To perform yourown calculations, consider that the standard error of r canbe as large as tanh(1/

√n− 3), where n is the sample size

(Feinberg and Wainer, 2011).) Similar considerations ofsignificant digits should be made when labeling axis ticks.For example, avoid “1.00”, “2.00”, etc., if “1” and “2” willsuffice, and use scientific notation in axes labels to keeptick labels concise (e.g., replace “Time (ms)” with “Time(s)” and change tick labels from “1000”, “2000”, etc. to“1” and “2”).

6. Putting it into Practice

Previous sections have focused on the theory of graph-ical perception and numerous guidelines for effective visu-alizations. In this section we address the practical imple-mentation of this information, from tools of the trade toa straightforward checklist with which to assess visualiza-tions.

6.1. Tools

Paper and Pencil

While some might consider a paper and pencil to becrude or out-dated in the age of tablets and touch-screens,there are few tools that can match the same immediacy ofexpression and ease of use (Wong and Kjærgaard, 2012).A quick doodle allows us to rapidly specify what we do


and do not know, explore design ideas, and exchange con-cepts with our colleagues. Additionally, drawing can helpto assess understanding and gain insights into data or de-signs: just as you may discover the limits of your knowl-edge when you attempt to teach a concept, you may re-alize those same limits when you try to draw it. Finally,doodling focuses your thinking on the content of your cre-ation, rather than on the “how” of creating it.

Plotting Software

Data visualization is often performed with the sametool used for processing and statistical analysis (e.g., MAT-LAB, R, Python, SAS, SPSS, Microsoft Excel, etc.), thoughdata and summaries can typically be exported to be plot-ted with other software. When choosing among tools, con-sider flexibility and accessibility. Nearly all software willproduce ubiquitous chart types such as bar and line plots,however custom encodings of visual attributes and geomsmay be difficult or impossible to achieve with some pack-ages. You may also consider the community of users fordifferent programming languages and the repositories ofsophisticated visualization methods that are readily avail-able. For example, the Grammar of Graphics (Wilkinson,2005) has been implemented in libraries for R and Python(ggplot, Wickham (2009)), and ColorBrewer is accessiblewithin R, Python, and MATLAB. There are also special-ized software packages for analyzing and visualizing par-ticular data types, such as FreeSurfer (freesurfer.net)for cortical surfaces, TrackVis (trackvis.org) for fibertract data, and Gephi (gephi.org) for networks. For anyplotting software, developing a high level of proficiencywill help you be less restricted in implementing and revis-ing your design.

Graphical Editing Software

Assembling a multi-panel figure or fine-tuning a singleplot may require the use of graphical editing software thatprovides the ability to adjust colors and transparency, re-size objects, add annotation, etc. Because data visualiza-tions are almost entirely composed of points, lines, curves,and shapes, they are best edited in vector-based softwareprograms such as Adobe Illustrator, Inkscape, or Corel-DRAW. These programs allow you to manipulate graph-ical objects in their native format, avoiding distortion ordegradation introduced by “rasterizing”. For graphicalelements such as photos or pictures that originate as pix-elated images, editing can be performed with raster-basedsoftware such as Adobe Photoshop, GIMP, or ImageMag-ick. Regardless of the software choice, we again recom-mend investing the time to master it. Graphical editingshould empower you to explore and create, rather thanleave you feeling frustrated.

6.2. Examples

Here we provide two concrete examples of data visu-alization, revisiting and integrating principles introducedearlier.

Our first example focuses on the full process of creat-ing a visualization. We use a simple set of synthetic datacomprising subject reaction times collected over a rangeof task difficulty levels. As illustrated in Figure 11, we be-gin by thinking about the data and the visualization goals.(If ever in doubt about where to start with data visualiza-tion, consider the phrase “think before you ink”.) In Step1, we establish the sample size, the data dimensions, andthe type of each variable. Additionally, we formulate a hy-pothesis relating mean reaction time to subject genotype.Our ability to visually address this hypothesis guides ourdesign choices. In Step 2, we doodle. Putting pen to pa-per allows for experimentation with different chart types,layouts, and encodings. Doodles need not be accurate —their purpose is to explore designs rather than the dataitself. We select one of these designs in Step 3. In thiscase, a heat map is a less optimal choice because it is dif-ficult to incorporate uncertainty in mean reaction times.Small multiples provide a very good choice, as they per-mit portrayal of the data at both group and individuallevels. Ultimately, we choose a line plot with symbol en-coding because the proximity of the lines to each otherencourages the eye to make comparisons between groups(following design Principle 1).

In Step 4 we finally “ink” and create our first versionof the visualization using MATLAB, with the majority offigure properties left at their default values. Step 5 offersa refinement of the visualization, following design princi-ples provided in Section 2.3. The bulk of revisions areimplemented in MATLAB, with small adjustments addedin Adobe Illustrator. Depending on the results of Step4 or 5, one may reconsider the design choice or its im-plementation. As noted by scientist and illustrator Mar-tin Krzywinski, “a good figure, like good writing, doesn’tsimply happen — it is crafted.” Similarly, the custom-ary advice to “revise and rewrite” becomes “revise andredraw” (Krzywinski, 2013b). Typically, one’s initial ideafor a visualization will be immature and unclear. Use it asa starting point, then re-doodle (literally go back to thedrawing board) and re-design as necessary to clarify yourmessage. Once you are satisfied with the visualization, itis important to consider how it integrates into a multi-panel figure or larger body of work such as a manuscriptor presentation (Step 6). For example, if additional pan-els or figures also compare genotypes, the same symbolsshould be used consistently. Follow Gestalt principles andthe recommendations in Section 5 to maintain uniformityof visual styles and typefaces such that your viewers canfocus on data variation, rather than design variation.

Our second example focuses on modifying visualiza-tions of electroencephalographic (EEG) and functional mag-netic resonance imaging (fMRI) datasets which are com-monly seen in psychophysiology and neuroscience research.

Figure 12A presents data from an EEG visual flankertask. Subjects were asked to indicate the direction of avisual target which appeared shortly after the presenta-tion of flanking distractors. Multi-channel EEG record-


DATA CHARACTERISTICS

Mean reaction time profiles over task difficulty depend on subject genotype.

Sample size: 25 subjectsDimensions: 3 • Genotype (nominal, independent) • Task difficulty (ordinal, independent) • Reaction time (continuous, dependent)

HYPOTHESIS

STEP 1 : THINK STEP 2 : DOODLE STEP 3 : CHOOSE A DESIGN

STEP 4 : INKSTEP 5 : REFINESTEP 6 : INTEGRATE

A line plot with symbol encoding

makes comparisons between genotypes

easiest.

Let the data dominate. Remove extraneous grid lines and labels. Annotate data directly, rather than with a legend. Adjust axes to frame the data. Jitter data as necessary to avoid occlusion. Define error bars on the graphic.

Integrate into a larger multi-panel figure or body of work using Gestalt principles. Similar content should have similar visual styles. Consider grouping and alignment. Task difficulty

Reac

tion

time

(s)

1 2 3 41

2

3

4

aa

Aa

AA

THINK BEFORE YOU INK

mean ±1 SEM

1 1.5 2 2.5 3 3.5 41

1.5

2

2.5

3

3.5

4

4.5

Task difficulty

Reac

tion

time

(s)

AAAaaa

REVISE

& R

EDRA

W

Figure 11: An illustration of the data visualization process.

ings were decomposed using independent component anal-ysis and a single component best matching the expectedfronto-central topography for a performance monitoringprocess was selected for further analysis (Eichele, Juvod-den, Ullsperger, and Eichele, 2010). Here, we ask how theextracted event-related potential (ERP) differs accordingto the subject’s response (i.e., correct or error). Panel Aprovides a common portrayal of such results, with themean ERP displayed for each condition. While this vi-sualization clearly contrasts the mean ERPs, it does notprovide sufficient information to determine whether theydiffer. In the modified display (Panel A’), we incorporate95% confidence bands around the average ERPs. Thebands are made slightly transparent to highlight overlapbetween conditions and to maintain the visual prominenceof the means. Confidence intervals clarify that there isgreater uncertainty in the error response than the correctresponse (since subjects make few errors), and that thereis insufficient evidence to conclude a response differenceafter ∼800 ms. We also add several annotations to thegraphic. These define the type of uncertainty that is por-trayed, specify our null and alternative hypotheses as wellas the alpha level chosen to determine statistical signif-icance, indicate the results of the significance tests, andestablish that the timeline is relative to the presentation ofthe target stimulus. The resulting design provides view-ers with more information about the experiment, data,analysis, and results.

Figure 12B portrays results from an auditory odd-ball event-related fMRI experiment. Participants were

asked to respond to target tones presented within a seriesof standard tones and novel sounds. Blood oxygenationlevel-dependent (BOLD) time series at each brain voxelwere regressed onto activation models for the target, novel,and standard stimuli (Kiehl, Laurens, Duty, Forster, andLiddle, 2001). Here, we ask which brain regions mightbe involved in the novelty processing of auditory stimuliand compare beta parameters between novel and stan-dard conditions. Panel B presents voxelwise differencesbetween beta coefficients using a widely reproduced de-sign: functional-imaging results are thresholded based onstatistical significance and overlaid on a high-resolutionstructural image. This design provides excellent spatiallocalization for functional effects, but is not without prob-lems. It does not portray uncertainty and has a remark-ably low data-ink ratio due to the prominent (non-data)structural image and sparsity of actual data (Habeck andMoeller, 2011). Moreover, the design hides results thatdo not pass a somewhat arbitrary statistical threshold. Arich and complex dataset is reduced to little more thana dichotomous representation (i.e., “significant or not?”)that suffers from all the limitations of hypothesis testing(Harlow, Mulaik, and Steiger, 2013).

Rather than threshold results, we suggest the dual-coding approach to represent uncertainty proposed by Hengl(2003). As shown in Panel B’, differences in beta esti-mates are mapped to color hue, and associated paired t-statistics (providing a measure of uncertainty) are mappedto color transparency. Note that for dual-coding, thehue colormap should be restricted to fully saturated col-


−500 0 500 1000−4

−3

−2

−1

0

1

2

3

4

−500 0 500 1000−6

−4

−2

0

2

4

6

Time (ms)

Aver

age

IC p

oten

tial (

mV)

Time (ms)

flankers target

* *

A

Aver

age

IC p

oten

tial (

mV)

Correct

Error

−200

B

A’

B’

Correct

Error

H0: μN = μSHa: μN ≠ μS : p < 0.001

mean, 95% CI

COMMON VISUALIZATION MODIFIED VISUALIZATION

H0: μE = μCHa: μE ≠ μC : p < 0.001

−1.6 +1.6

∆β weight

0

L R

∆β weight

|t|0

≥5

−1.6 +1.60

−0.5

0

0.5

1

1.5

Standard β

Nov

el β

−0.5 0 0.5 1 1.5

n = 28 subjects

1 2

1

2

Z = −39Z = −19

L R

Z = −39Z = −19

Z = 22 Z = 2Z = 2Z = 22

Novel − Standard Novel − Standard

Figure 12: Conventional (A, B) and modified designs (A’, B’) portraying real data. Captions describe the modified designs. (A’) EEGflanker data. Mean ERPs (averaged over 10 subjects) and 95% non-parametric CIs for Error trials (red) and Correct trials (blue). Asterisksindicate significantly different mean ERPs at p < 0.001 (nonparametric randomization test with implicit correction for multiple comparisons).(B’) FMRI auditory oddball data. Axial slices show the difference between Novel and Standard beta weights averaged over 28 subjects.Beta difference is mapped to color hue; t-statistic magnitude is mapped to transparency. Black contours denote significantly different betasat p < 0.001 (two-tailed paired t-tests, corrected with false discovery rate). Scatter plots show Standard versus novel Betas for regions ofinterest (ROIs) 1 and 2. Beta weights are averaged over clusters of contiguous voxels passing significance (ROI 1 = 2426 voxels; ROI 2 =1733 voxels). Gray lines indicate y = x. Adapted from Allen et al. (2012).

ors (i.e., the outer cylindrical surface in Figure 7), sincechanges in saturation will be confounded with changes intransparency. In this example we use the “jet” or “rain-bow” colormap (Path 4 in Figure Figure 7). Compared toPanel B, no information is lost. Transparency is sufficientto determine structural boundaries and statistical signif-icance is indicated with contours. However substantialinformation is gained. The quality of the data is now ap-parent: large and consistent differences in betas are whollylocalized to gray matter, while white matter and ventric-ular regions exhibit very small or very uncertain differ-ences. In addition, isolated blobs of differential activationin Panel B are now seen as the peaks of larger contiguousactivations (often with bilateral homologues) that failed tomeet significance criteria. The modified display also re-veals regions in lateral parietal cortex, medial prefrontalcortex, and posterior cingulate cortex with reduced acti-

vation to novel stimuli compared to standard tones. Thesebrain areas coincide with the default-mode network, a sys-tem preferentially active when subjects engage in internalrather than external processes (Buckner, Andrews-Hanna,and Schacter, 2008). Along with portraying uncertainty,Panel B’ also includes scatter plots displaying the Stan-dard and Novel beta parameters in regions of interest forindividual subjects. Scatter plots allow viewers to accessthe data in greater detail as they indicate the beta esti-mates for each condition (rather than just the difference),reveal the degree of variability across subjects (and theabsence of outliers), and validate a “paired” statisticalapproach since beta values covary across conditions. Ad-ditionally, scatter plots remove dependence on color map-ping and remain perfectly clear when reproduced in blackand white.

In summary, the modified design and use of dual cod-


ing provides substantially more information to viewers,increases data accessibility, and provides clarity regardinghypothesis testing. To encourage the use of this approach,sample Matlab scripts for hue and transparency coding areprovided at mialab.mrn.org/datavis.

6.3. A Checklist for Assessing Visualizations

Table 3 provides a checklist for data visualizations thatcan be used before, during, and after you implement yourdesign (Wallgren et al., 1996). If you can answer “yes” tomost of the questions, your figure is likely to be accessibleand correctly interpreted by others. We hope this check-list will serve as a useful reference for you as you createyour own visualizations, as well as a tool to evaluate andimprove the visualizations of your colleagues.

References

Albers, J. Interaction of Color. Yale University Press, New Haven,Connecticut, 1975.

Allen, E. A., Erhardt, E. B., and Calhoun, V. D. Data visualizationin the neurosciences: Overcoming the curse of dimensionality.Neuron, 74(4):603–608, 2012.

Baird, J. C. A cognitive theory of psychophysics I. ScandinavianJournal of Psychology, 11(1):35–46, 1970a.

Baird, J. C. A cognitive theory of psychophysics II. ScandinavianJournal of Psychology, 11(1):89–102, 1970b.

Belia, S., Fidler, F., Williams, J., and Cumming, G. Researchersmisunderstand confidence intervals and standard error bars. Psy-chological Methods, 10(4):389, 2005.

Brewer, C. A. Guidelines for use of the perceptual dimensions ofcolor for mapping and visualization. In Proceedings of the Inter-national Society for Optical Engineering (SPIE), volume 2171,pages 54–63, San Jose, 1994. International Society for Optics andPhotonics.

Buckner, R. L., Andrews-Hanna, J. R., and Schacter, D. L. Thebrain’s default network. Annals of the New York Academy ofSciences, 1124(1):1–38, 2008.

Bullmore, E. and Sporns, O. Complex brain networks: Graph the-oretical analysis of structural and functional systems. NatureReviews Neuroscience, 10(3):186–198, 2009.

Chambers, J., Cleveland, W., Kleiner, B., and Tukey, P. Graphi-cal Methods for Data Analysis. Wadsworth International Group,Belmont, CA, 1983.

Chernoff, H. The use of faces to represent points in k-dimensionalspace graphically. Journal of the American Statistical Associa-tion, 68(342):361–368, 1973.

Cleveland, W. S. Graphs in scientific publications. The AmericanStatistician, 38(4):261–269, 1984.

Cleveland, W. S. The Elements of Graphing Data. Hobart Press,Summit, New Jersey, revised edition, 1994.

Cleveland, W. S. and McGill, R. A color-caused optical illusion on astatistical graph. The American Statistician, 7(2):101–105, 1983.

Cleveland, W. S. and McGill, R. Graphical perception: Theory,experimentation, and application to the development of graphicalmethods. Journal of the American Statistical Association, 79(387):531–554, 1984.

Cleveland, W. S. and McGill, R. Graphical perception and graphicalmethods for analyzing scientific data. Science, 229(4716):828–833, 1985.

Cleveland, W. S., McGill, M. E., and McGill, R. The shape param-eter of a two-variable graph. Journal of the American StatisticalAssociation, 83(402):289–300, 1988.

Cowan, N. Metatheory of storage capacity limits. Behavioral andBrain Sciences, 24(01):154–176, 2001.

Cumming, G. and Finch, S. Inference by eye: confidence intervalsand how to read pictures of data. American Psychologist, 60(2):170, 2005.

Cumming, G., Fidler, F., and Vaux, D. L. Error bars in experimentalbiology. The Journal of Cell Biology, 177(1):7–11, 2007.

Eichele, H., Juvodden, H. T., Ullsperger, M., and Eichele, T. Mal-adaptation of event-related eeg responses preceding performanceerrors. Frontiers in Human Neuroscience, 4, 2010.

Ellis, W. D. A source book of Gestalt psychology, volume 2. Psy-chology Press, 1999.

Emerson, J. W., Green, W. A., Schloerke, B., Crowley, J., Cook,D., Hofmann, H., and Wickham, H. The generalized pairs plot.Journal of Computational and Graphical Statistics, 22(1):79–91,2011.

Fechner, G. Elemente Der Psychophysik. Breitkopf & Hartel,Leipzig, 1860.

Feinberg, B. M. and Franklin, C. A. Social Graphics Bibliography.Bureau of Social Science Research, Washington, D.C., 1975.

Feinberg, R. A. and Wainer, H. Extracting sunbeams from cucum-bers. Journal of Computational and Graphical Statistics, 20(4):793–810, 2011.

Few, S. Save the pies for dessert. Visual Business IntelligenceNewsletter, pages 1–14, 2007.

Friendly, M. Corrgrams: Exploratory displays for correlation matri-ces. The American Statistician, 56(4):316–324, 2002.

Gehlenborg, N. and Wong, B. Points of view: Into the third dimen-sion. Nature Methods, 9(9):851–851, 2012.

Griethe, H. and Schumann, H. The visualization of uncertain data:Methods and problems. In Proceedings of SimVis, pages 143–156,2006.

Habeck, C. and Moeller, J. R. Intrinsic functional-connectivity net-works for diagnosis: Just beautiful pictures? Brain Connectivity,1(2):99–103, 2011.

Haemer, K. W. Hold that line. The American Statistician, 1(1):25–25, 1947a.

Haemer, K. W. The perils of perspective. The American Statisti-cian, 1(3):19–19, 1947b.

Haemer, K. W. The pseudo third dimension. The American Statis-tician, 5(4):28–28, 1951.

Harlow, L. L., Mulaik, S. A., and Steiger, J. H. What if there wereno significance tests? Psychology Press, 2013.

Heer, J. and Bostock, M. Crowdsourcing graphical perception: Us-ing mechanical turk to assess visualization design. In Proceedingsof the SIGCHI Conference on Human Factors in Computing Sys-tems, pages 203–212. ACM, 2010.

Hengl, T. Visualisation of uncertainty using the HSI colour model:Computations with colours. In Proceedings of the 7th Interna-tional Conference on GeoComputation, pages 8–17, 2003.

Hintze, J. L. and Nelson, R. D. Violin plots: A box plot-densitytrace synergism. The American Statistician, 52(2):181–184, May1998.

Hoekstra, R., Morey, R. D., Rouder, J. N., and Wagenmakers, E.-J.Robust misinterpretation of confidence intervals. Psychonomicbulletin & review, 21(5):1157–1164, 2014.

Jarvenpaa, S. L. and Dickson, G. W. Graphics and managerialdecision making: Research-based guidelines. Communications ofthe ACM, 31(6):764–774, 1988.

Kampstra, P. Beanplot: A boxplot alternative for visual comparisonof distributions. Journal of Statistical Software, 28(1):1–9, 2008.

Kiehl, K. A., Laurens, K. R., Duty, T. L., Forster, B. B., and Lid-dle, P. F. Neural sources involved in auditory target detectionand novelty processing: an event-related fmri study. Psychophys-iology, 38(01):133–142, 2001.

Kosslyn, S. M. Graphics and human information processing: Areview of five books. Journal of the American Statistical Associ-ation, 80(391):499–512, 1985.

Krzywinski, M. Points of view: Axes, ticks and grids. Nature Meth-ods, 10(3):183–183, 2013a.

Krzywinski, M. Points of view: Elements of visual style. NatureMethods, 10(5):371–371, 2013b.

Krzywinski, M. and Altman, N. Points of significance: Error bars.


Before you begin

Is a visualization necessary? Have you considered a table or written description?Have you clarified the goal(s) of the visualization?Are you aware of size and color constraints?

While you are working

DesignIs the visualization consistent with the model or hypothesis being tested?Does the visualization emphasize actual data over idealized models?Where possible, does the graphical encoding method have high decoding accuracy?Have “empty dimensions” or 3D effects been removed?Have encodings and annotations been used consistently?

AxesAre axes scales defined as linear, log, or radial?Do axes limits frame the data?Is the aspect ratio appropriate for the data?Are the axes units intuitive?

UncertaintyDoes the visualization portray uncertainty where necessary?Is the type of uncertainty appropriate for the data?Are the units of uncertainty labeled?

ColorIs color necessary or useful?Can features be discriminated when printed in grayscale?Does the visualization accommodate common forms of colorblindness?

ColormappingIs the colormap consistent with the data type?Does a colorbar fully define the mapping (quantity, units, and scale)?

AnnotationAre all symbols defined, preferably by directly labeling objects?For numerical annotation, are the significant figures appropriate?Are uncommon abbreviations avoided or clearly defined?

When you think you are finished

Have you tested the visualization on a naıve viewer?Did you communicate the intended message?

Table 3: A checklist for data visualizations.

Nature Methods, 10(10):921–922, 2013.Krzywinski, M. and Wong, B. Points of view: Plotting symbols.

Nature Methods, 10(6):451–451, 2013.Lane, D. M. and Sandor, A. Designing better graphs by including

distributional information and integrating words, numbers, andimages. Psychological Methods, 14(3):239, 2009.

Lewandowsky, S. and Spence, I. Discriminating strata in scatter-plots. Journal of the American Statistical Association, 84(407):682–688, 1989.

Margulies, D. S., Bottger, J., Watanabe, A., and Gorgolewski, K. J.Visualizing the human connectome. NeuroImage, 80:445–461,2013.

Moret-Tatay, C. and Perea, M. Do serifs provide an advantage in therecognition of written words? Journal of Cognitive Psychology,23(5):619–624, 2011.

Munsell, A. H. A Color Notation. Munsell color company, Inc.,Baltimore, 12 edition, 1947.

Potter, K., Rosen, P., and Johnson, C. R. From quantificationto visualization: A taxonomy of uncertainty visualization ap-proaches. In Uncertainty Quantification in Scientific Computing,pages 226–249. Springer, 2012.

Sammon, J. W. A nonlinear mapping for data structure analysis.

IEEE Transactions on Computers, 18(5):401–409, 1969.Schmid, C. F. Statistical Graphics: Design Principles and Practices.

Wiley New York, 1983.Spitzer, M., Wildenhain, J., Rappsilber, J., and Tyers, M. Box-

PlotR: A web tool for generation of box plots. Nature Methods,11(2):121–122, 2014.

Stevens, S. S. Psychophysics. Transaction Publishers, 1975.Talbot, J., Gerth, J., and Hanrahan, P. An empirical model of slope

ratio comparisons. Visualization and Computer Graphics, IEEETransactions On, 18(12):2613–2620, 2012.

Tomasi, D. and Volkow, N. D. Association between functional con-nectivity hubs and brain networks. Cerebral Cortex, 21(9):2003–2013, 2011.

Tufte, E. R. The Visual Display of Quantitative Information.Graphics Press, Cheshire, CT, USA, second edition, 2001. ISBN0-9613921-0-X.

Tukey, J. Exploratory Data Analysis. Addison-Wesley series inbehavioral science. Addison-Wesley Publishing Company, 1977.

Van der Maaten, L. and Hinton, G. Visualizing data using t-SNE.Journal of Machine Learning Research, 9(2579-2605):85, 2008.

Vaux, D. L. Error message. Nature, 428(6985):799–799, 2004.Wainer, H. How to display data badly. The American Statistician,


38(2):137–147, 1984.Wainer, H. Depicting error. The American Statistician, 50(2):101–

111, 1996.Wainer, H. Visual revelations: Improving graphic displays by con-

trolling creativity. Chance, 21(2):46–52, 2008.Wallgren, A., Wallgren, B., Persson, R., Jorner, U., and Haaland,

J.-A. Graphing Statistics & Data: Creating Better Charts. Sage,1996.

Wand, H., Iversen, J., Law, M., and Maher, L. Quilt plots: A simpletool for the visualisation of large epidemiological data. PloS One,9(1):e85047, 2014.

Ward, M. O. Multivariate data glyphs: Principles and practice. InHandbook of Data Visualization, pages 179–198. Springer, 2008.

Wickham, H. ggplot2: Elegant Graphics for Data Analysis. SpringerScience & Business Media, 2009.

Wickham, H. Graphical criticism: Some historical notes. Journalof Computational and Graphical Statistics, 22(1):38–44, 2013.

Wilkinson, L. The Grammar of Graphics. Springer Science & Busi-ness Media, second edition, 2005.

Wong, B. Points of view: Color coding. Nature Methods, 7(8):573–573, 2010a.

Wong, B. Points of view: Design of data figures. Nature Methods,7(9):665–665, 2010b.

Wong, B. Points of view: Arrows. Nature Methods, 8(9):701–701,2011a.

Wong, B. Points of view: Salience to relevance. Nature Methods, 8(11):889–889, 2011b.

Wong, B. Points of view: Simplify to clarify. Nature Methods, 8(8):611–611, 2011c.

Wong, B. and Kjærgaard, R. S. Points of view: Pencil and paper.Nature Methods, 9(11):1037–1037, 2012.

Documents

Visualizing Scienti c Data - Erik Erhardt · 2015-12-04 · Allen and Erhardt Visualizing Scienti c Data Page 2 of 21 recent survey of nearly 1500 gures from ve neuroscience journals