The curse of three dimensions: Why your brain is lyingto you

Susan VanderPlassrvanderplas@gmail.com

Iowa State University

Heike Hofmannhofmann@iastate.eduIowa State University

Di Cookdicook@iastate.edu

Iowa State University

Abstract—One of the basic principles of visual graphics isthat the graphic should accurately reflect the data. Tufte’s liefactor [8] was created with the idea that graphs that do notrepresent the underlying data accurately should be avoided. Inthis paper, we examine a second level of graph distortion thatoccurs during the perceptual process. The human visual system islargely optimized for perception of three dimensions. Generally,the brain processes potential ambiguities in the rendering as themost-common three-dimensional object. This can lead to visualdistortions, such as occur with the Necker figure or in the Muller-Lyer illusion. We discuss the underlying psychological mechanismsfor the distortions, examine the effect these distortions have onjudgments, and consider the implications for graph design. Usingthe sine illusion as a case study, we quantify the effects of thedistortion that create a “perceptual lie factor” for the sine illusion.

I. INTRODUCTION

One of the basic principles of visual graphics is that thegraphic should accurately reflect the data. Tufte’s “Lie Factor”,for instance, is calculated based on the ratio of the effectsize shown in the graphic to the effect size in the data [8].If there is a systematic difference, the lie factor will benotably different from 1 (values between .95 and 1.05 aretypically acceptable). While Tufte’s lie factor is an effectivemeasurement of the transition from data to graphics, it is notas effective at measuring the transition from graph to the brain.Ideally, graphics would not only represent the data accurately,but would also allow readers to draw accurate conclusions fromthe graph. While in many cases these two goals can be achievedsimultaneously, there are instances in which human perceptionproduces a psychological distortion that can interfere with theinterpretation of graphics, even if the graphics are accurateportrayals of the numerical data. In this paper, we examine situ-ations in which low-level human perceptual processes interferewith making accurate judgements from displays and suggestan experimental methodology for estimating the psychological“lie factor” of a graphic.

II. THE CURSE OF THREE DIMENSIONS

The human visual system is largely optimized for perceptionof three dimensions. Biologically, binocular vision ensures thatwe have the necessary information to construct a reasonablyaccurate mental representation of the three-dimensional world,but even in the absence of binocular information the brainuses numerous heuristics to parse otherwise ambiguous two-dimensional retinal images into meaningful three-dimensional

information. Predictably, however, these heuristics are not with-out drawbacks; the same two-dimensional neural representationmight correspond to multiple three-dimensional objects, asin the Necker Cube shown in Figure 1. Additionally, thesame three-dimensional object often has infinitely many two-dimensional representations, for instance, when viewed fromdifferent angles. Optical illusions that demonstrate this phe-nomenon are the so-called “impossible objects” such as thePenrose triangle.

Necker Cube Interpretation 1 Interpretation 2

Fig. 1: The Necker Cube is a so-called “ambiguous object”because two different transparent objects could produce thesame retinal image (and thus the same perceptual experience).Commonly, the image seems to transition instantaneously fromone possible mental representation to the other.[5]

In the case of the Necker cube, the two different figuresare equally salient, and thus the brain does not prefer oneinterpretation over the other and instead continuously switchesbetween interpretations. In many other instances experiencewith the real world informs the choice between multiplepossible three-dimensional objects which form the same two-dimensional picture. This indicates that processing occurs “topdown” in that our previous experience influences our currentperceptions. Without this top-down influence, the brain wouldnot be able to map the picture back to a three-dimensionalobject. One of the most well studied examples of the influenceof top-down processing is the Muller-Lyer illusion, shown inFigure 2

One explanation for the Muller-Lyer illusion is that the braininterprets the ambiguous lines as a common three-dimensionalobject: corners of a room: Figure 2A occurs when viewingthe outside corner of a rectangular prism, Figure 2B occurswhen viewing the prism from the inside. In regions whichdo not commonly have rectangular buildings, the illusion is

A B

Fig. 2: The Muller-Lyer illusion. The central segment of FigureA is perceived as shorter than the central segment of FigureB, even though the two are actually the same length.

significantly less pervasive [1]. Figure 3 provides one possiblecontext that would lead to the Muller-Lyer effect. This real-world experience carries with it an inferred perspective - whenthe arrows point inward, the object is typically closer thanwhen the arrows point outward, which causes the brain tointerpret the outward-pointing figure as larger, even thoughthe retinal size is identical for the two objects. This inferred“depth cue” [5] is reasonably consistent across individuals.A similar effect can also be found in the Necker Cube -whichever face appears to be furthest away also seems larger,even though any two parallel faces are equally sized in theimage. The advantages of this approach in the real worldare considerable [5], as pictures of real objects are seldomambiguous and this strategy allow for high performance withlimited neural bandwidth. Figure 3 shows not only the real-world context, but that the perspective cues which contributeto the illusion allow for an accurate neural representation ofthe object in context.

Fig. 3: Real-world context that gives rise to the Muller-Lyerillusion

III. THREE DIMENSIONAL CONTEXT OF THE SINE

ILLUSION

While the classic Muller-Lyer illusion is seldom a factorin statistical graphics, there are other illusions caused by theinterpretation of a two-dimensional stimulus in the contextof three-dimensional objects, leading to a distortion in themental representation of the original stimulus. The sine illusion(also known as the line width illusion) is one example of thisphenomenon which occurs frequently in statistical graphics.Like the Muller-Lyer illusion, it is pervasive and very difficultto “un-see” or mentally correct. Figure 4 shows the sine illusionin its original form, and Figure 5 shows the same illusion in oneof the charts from Playfair’s Statistical Atlas [7]. This illusion,which is also known as the line-width illusion, has also beendocumented in parallel sets plots [6].

Fig. 4: The classic sine illusion. Each vertical line has the samelength, though the lines at the peak and trough of the curveappear longer.

Fig. 5: The sine illusion in Playfair’s graph of trade betweenthe East Indies and England, 1700-1780. The trade balancein 1763 does not appear to be the same size as that in 1745,though the vertical distance is approximately the same.

Cleveland and McGill [3] determined that comparison of thevertical distance between two curves is often inaccurate, as “thebrain wants to judge minimum distance between the curves in

2

different regions, and not vertical distance”. While they do notexplain a reason for this tendency, introspection does readilyconfirm their explanation: we judge the distance between twocurves based on the shortest distance between them, whichgeometrically is the distance along the line perpendicular tothe tangent line of the curve. Day & Stecher [4] suggest thatthe sine illusion is similar in principle to the Muller-Lyerillusion, attributing it to the perceptual compromise betweenthe vertical extent and the overall dimensions of the figure. Thesine illusion is similar to the Muller-Lyer illusion in anotherway, as well - there are three-dimensional analogues of the two-dimensional image that may influence the perceptual context.One of these contexts is shown in Figure 6, generated fromthe same function shown in the two-dimensional analogue,Figure 4, but with the length projected onto a third dimension.While the images do not match exactly, the similarities arestriking. Additionally, the tendency to judge vertical distanceusing the extant width noted in Cleveland et al [2] correspondsto the measurement of depth in the three-dimensional image.One main difference between the three dimensional imageshown in Figure 6 and the original image is that the linesconnecting the top and bottom sections of the curve areslightly angled in the three-dimensional version; this is dueto the perspective projection used to create the image and thecorresponding angles of rotation chosen such that the entiresurface is visible.

Fig. 6: Three dimensional context for the sine illusion.

As the vanishing point moves further away from theviewer and the 3d projection decreases in strength, the three-dimensional reconstruction of the image converges on fig-ure ??. Figure 7 shows a weaker 3-dimensional projection thatis much closer to Figure 4, however, the three-dimensionalcontextual information provided by the shading removes muchof the illusion’s distortions. This is similar to the Muller-Lyer illusion, as Figure 3 is not at all ambiguous because thecontextual depth information provided by the rest of the surfaceof the house is sufficient to remove the illusion that the closercorner is in fact larger due to the perspective.

One difference between the sine illusion and the Muller-Lyer illusion that may influence the tendency to see a three-dimensional “ribbon” instead of the two-dimensional sine curveis that the vertical lines in the sine illusion are ambiguouslyoriented - there is an entire plane of possible three-dimensionalreconstructions for each line, and each possible rotation leadsto a line of different length. It is this facet of the image thatwe believe partially contributes to the ambiguity of the image,though it is not a necessary feature for the illusion to persist,as the illusion also can be found in scatterplots and in “ribbonplots” such as Figure 5.

Fig. 7: Three dimensional context for the sine illusion, with aweaker perspective transformation.

The psychological mechanisms which force three-dimensional context onto two-dimensional stimuli are usefuladaptations to a three-dimensional world [5], but they dohave disadvantages when applied to abstract two-dimensionalstimuli, such as statistical graphics. In order to estimate thepsychological lie factor that occurs due to this illusion, weassessed the strength of the illusion experimentally.

IV. EXPERIMENTAL DETERMINATION OF THE SIZE OF THE

PSYCHOLOGICAL LIE FACTOR IN THE SINE ILLUSION

A. Study Setup

In order to determine the amount of the psychologicaldistortion, participants were presented with different stimuliconsisting of six subplots, similar to Figure 8a. These sets ofcharts were constructed such that line lengths along a curvewere varied to different degrees to counteract the illusion.Figure 8b shows the amount of line correction used in eachof the sub-plots above. Participants were asked to answerthe question: ”In which graph is the size of the curve mostconsistent?”. The phrasing ‘size of the curve’ was chosendeliberately so as not to bias participants to explicitly measureline lengths.

The amount of correction in each sub-plot was chosen suchthat each stimuli set contained a selection of curves corrected tovarious degrees using correction factors determined previouslyin an internal pilot study. The correction factor we use forextending the length of the line segment at loctaion x is givenas

(1−w)+w ·(

1/cos(

tan−1(| f ′(x)|)))

where w is varied between 0 (for no correction) and 1.5(very strong correction). From the previous pilot study, wesuspected values around w = 0.8 to be promising, but didnot know whether this would generalize to those outside ofthe statistical graphics community. This led to twelve stimulisets with correction factors as shown in table I. The difficultylevel of each of the twelve stimuli sets was determined by thesimilarity of the six plots shown.

Three underlying mean functions were tested using theseweight values: sin(x), exp(x), and 1/x.

Each participant was presented with eleven sets of graphs,consisting of one “easy” test chart, five stimuli sets of difficultylevel 1 through 5 with the sine curve as the underlying function,and another five graph sets (also of difficulty levels 1 to 5) witheither the exponential or the inverse curve as the underlyingfunction. Plots were shown to participants in random order.The test chart consists of a set of six sine curves with a very

3

(a) One of the graphs presented to participants through AmazonMechanical Turk.

1 2 3

4 5 6

(b) Line lengths (without the trend) for Figure 8a.

Fig. 8: Stimuli similar to (a) were shown to participants, while(b) shows the corresponding differences in line lengths.

sub-plotset 1 2 3 4 5 6 difficulty

1 0.00 0.20 0.40 0.80 1.25 1.40 test2 0.00 0.15 0.35 0.80 1.20 1.40 test3 0.00 0.20 0.40 0.60 0.80 1.00 14 0.10 0.30 0.50 0.70 0.90 1.10 15 0.00 0.50 0.70 0.80 0.90 1.00 26 0.00 0.45 0.65 0.75 0.85 1.00 27 0.40 0.70 0.80 0.90 1.00 1.30 38 0.30 0.65 0.75 0.85 0.95 1.20 39 0.50 0.60 0.70 0.80 0.90 1.00 4

10 0.55 0.65 0.75 0.85 0.95 1.00 411 0.60 0.70 0.75 0.85 0.90 1.00 512 0.60 0.70 0.75 0.80 0.90 1.00 5

TABLE I: Distortion factors used for the sine curve stimulus.

low level diffcilty level, and is used as an introduction to thetesting procedure.

Though participants were asked to participate in 11 totaltrials, some participants continued to provide feedback beyondthe eleven trials required to receive payment through Amazon.For any subsequent responses we randomly selected one of the32 possible stimuli. This approach allowed us to collect somedata in which a single participant provided responses to allthree underlying functions.

B. Demographics

Participants for the study were recruited through the AmazonTurk web service. Responses from 128 users at 123 unique IPaddresses were collected.

In order to more accurately identify unique individuals,data were grouped by IP address rather than by user-enteredidentification code, as it is common for individuals to havemultiple accounts and somewhat less likely for two AmazonMechanical Turk participants to share the same IP address andcomplete the same study in a reasonably short window. Wetreated responses from the same IP addresses as additionalresponses from the same user.

C. Analysis

a) Psychological “Lie Factor”: As the strength of thecorrection varies across the horizontal range of the curve,we quantify the psychological distortion as the ratio of themaximum line length to the minimum line length for each plotj: D j = lmax/lmin, and denote the chosen ratio as D∗. Withoutany correction, this factor is, as Tufte’s lie-factor, equal to one.Values above one indicate that at least in some areas of thecurve line segments are extended.

We compute this quantity for each plot in each set presentedto the participant. The participant’s choice therefore providesus with an estimate of what value of D constitutes the mostconsistent line length (out of the set shown). As each setof 6 plots is not guaranteed to contain a plot with w = 0,corresponding to constant length, choosing a plot with D = 1.4indicates more distortion if there is a plot with w = 0 andD j = 1 also in the plot than if there is at minimum a plotwith w = 0.4 and D = 1.2. This correction for the set ofD∗ = {D1, ...,D6} that is available to choose from producesan estimate of the overall psychological ‘lie factor’ as

P = D∗/ min1≤ j≤6

D j

for each plot and each participant. That is, P is the ratio of thelie factor of the chosen plot to the smallest lie factor availablein the set of available plots.

By considering each participant’s answers for the plot withthe most consistent line length, we can obtain an estimate ofthe psychological distortion from the sine illusion on an indi-vidual level. Estimating distortion factors for each participantfacilitates comparison of these estimated values to determinewhether the illusion is a product of an individual’s perceptualexperience or whether there is a possible underlying perceptualheuristic for the sine illusion common across the majority ofparticipants. If the illusion is a learned misperception ratherthan an underlying perceptual “bug”, we would expect there tobe considerable variability in the estimated individual lie factorPi for each unique participant i, 1 ≤ i ≤ 123, as it is likelythat personal experience varies more widely than perceptualheuristics and their underlying neural architecture.

Each set of w values as defined in table I corresponds toa value of P as defined above. We test for only a set ofdiscrete values of w, which is reflected directly in the numberof different values of P we can observe. This approach allows

4

Sine Exponential Inverse

0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.15

0.20

0.25

0.00

0.25

0.50

0.75

1.0 1.5 2.0 2.5 1.0 1.5 2.0 2.5 1.0 1.5 2.0 2.5Lie Factor

Den

sity

Fig. 9: Estimated densities for θi, shown in color, with theestimated overall density for θ shown in black. Individualshave extremely similar posterior distribution of θi, and evendifferent functions have similar θ , suggesting a common un-derlying mental distortion.

us to use a finite set of stimuli for testing, so that we canexplicitly control the range of w displayed in each set ofplots. To mathematically model a continuous quantity (the realdomain of possible P values) using discrete data, we employ aBayesian approach to model an overall psychological lie factorθ and individual participant lie factors θi.

Plots used in the experiment have factors P ranging between1 and 2.5, so we can use a truncated normal data model forparticipant i viewing plot j, with P = pi j ∼ N(θi,σ) and inde-pendent flat priors π(θ) = 1/3 and π(σ) = 2.5 for σ ∈ [.1, .5].These “prior distributions” π(.) represent our expectations ofthe values of θ and σ before the experiment; assigning themconstant values indicates that we had little useable knowledgeabout the joint or marginal distributions of θ and σ before theexperiment was conducted. Using Bayesian estimation, we canthen obtain posterior distributions for θi and θ , the individualand overall mean lie factors. We are not particularly interestedin the actual values of σ , but it a useful tool to better estimatepossible values for θ .

D. Results

The posterior density of θ for each function is shown inFigure 9, along with separate posterior densities for eachindividual θi. θ is reasonably similar for all three functions,suggesting that while function type may moderate the sizeof the effect, the illusion occurs regardless of function type.Individual curves have different variability due to the numberof trials completed, and are necessarily more spread out asthere is less data with which to estimate the individual posteriordistributions.

On an individual level, Figure 10 shows the posterior densityfor θi for four of the participants who completed at least6 trials in each category. While in many cases, the mostprobable θi is similar across trials, individuals do seem tohave been somewhat more affected by the illusion when theunderlying function was sinusoidal, though this may reflect

Participant 2(n = 41)

Participant 9(n = 40)

Participant 32(n = 55)

Participant 88(n = 56)

0.00

0.02

0.04

0.06

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4Mean Psychological Lie Factor

Den

sity

Function Type Exp Inv Sin

Fig. 10: Posterior distributions for θi for four of the participantswho completed at least 6 trials of each of the three functiontypes.

Sine Exponential Inverse

0

25

50

75

100

125

1.0 1.2 1.4 1.6 1.0 1.2 1.4 1.6 1.0 1.2 1.4 1.6Mean Lie Factor

Par

ticip

ant I

D

Fig. 11: 95% posterior predictive intervals for θi, calculated foreach stimulus type. Vertical lines indicate the median estimateof the overall θ with a 95% credible interval.

a discrepancy in the number of trials rather than a strongerillusion. Alternately, as the illusion depends on variable slope,it is possible that the monotonic exponential and inverse stimuliinduced a weaker three-dimensional context.

In order to appropriately compare intervals for each par-ticipant’s θi, we simulated 11 new “data points” from ourmodel (thus enforcing a uniform 11 trials per participant foreach function type) to get a single new estimate of θi. Foreach participant, we generated 100 of these θi and used thesesimulated values to calculate the 95% credible intervals shownin Figure 11. These intervals will allow us to consider thevariability in θi due to participant preference rather than thenumber of trials a participant completed during the study.Removing this additional variability provides us with theopportunity to consider whether the sine illusion stems froman individual’s perceptual experiences or from a lower-levelperceptual heuristic.

Posterior predictive intervals for θi as shown in Figure 11

5

Function 95% Credible Interval for θ MedianSin (1.1808, 1.4415) 1.2973Exp (1.1742, 1.4428) 1.2791Inv (1.08, 1.208) 1.1255

TABLE II: Credible intervals for the overall θ for exponential,inverse, and sine stimuli.

w = 0 w = 1 w = 1.05

Sine

w = 0 w = 0.75 w = 1

Inverse

w = 0 w = 0.4 w = 1

Exponential

Fig. 12: Stimuli with uncorrected (w=0), optimally corrected(according to the mean value of θ ), and fully corrected (w=1)

suggest that overall, the θi are similar across individuals. Veryfew of the intervals contain overlap the region (1, 1.05), whichcorresponds to an “acceptable” lie factor according to Tufte.This indicates significant distortion for most participants inour experiment, and the marked overlap of the intervals foreach participant provides evidence consistent with a commonmagnitude of distortion. This suggests that there may besome common psychological strategy that is misapplied to theperception of these stimuli.

Comparison of the Preferred Stimuli: Estimates of θ = E[θ ]for each function are 1.29, 1.13, and 1.3 respectively forexponential, inverse, and sine functions, suggesting a simi-lar psychological distortion even for very different functions,though it seems as if the inverse function causes somewhatless distortion, possibly because the correction factor is not asproportionately large. Credible intervals can be found in TableII. As all three of the credible intervals exclude 1.05, there isevidence that a psychological distortion is occurring; that is,there is evidence of a significant psychological lie factor.

The optimal weight values corresponding to these θ areshown in Figure 12. In all three cases, the optimally correctedplots appear less distorted than the uncorrected plots.

V. CONCLUSIONS

The sine illusion arises from misapplication of three-dimensional context to a two-dimensional stimulus whichresults in nearly unavoidable perceptual distortions that impactthe inferences made from graphics. We have estimated that theilluson produces a distortion of about 135%. This distortionoccurs entirely between the retinal image and the mentalrepresentation of the object; it is not due to the chart, rather,it is an artifact of our perceptual system.

As Tufte advocated for graphics that showed the data withoutdistortion, our goal is to raise awareness of perceptual dis-tortions that occur within the brain itself due to misappliedheuristics. While applying corrections to the data to removethese distortions is somewhat radical, the persistence of theillusion despite awareness of its presence presents a challengeto those seeking to display data visually. In addition, manygraph types can induce this illusion (scatterplots, ribbon plots,parallel sets plots), so avoiding a specific type of graph is not aneffective solution. The best solution to this problem is to raiseawareness: to demonstrate that optical illusions occur withinstatistical graphics, and to understand how these illusions arise.

REFERENCES

[1] A. Ahluwalia. An intra-cultural investigation of sus-ceptibility to “perspective” and “non-perspective” spatialillusions. British Journal of Psychology, 69:233–241, 1978.

[2] W. S. Cleveland and R. McGill. Graphical perception andgraphical methods for analyzing scientific data. Science,229(4716):828–833, 1985.

[3] William S. Cleveland and Robert McGill. Graphicalperception: Theory, experimentation, and application tothe development of graphical methods. Journal of theAmerican Statistical Association, 79(387):pp. 531–554,1984.

[4] Ross H Day and Erica J Stecher. Sine of an illusion.Perception, 20:49–55, 1991.

[5] R.L. Gregory. Perceptual illusions and brain models. Proc.Roy. Soc. B, 171:279–296, 1968.

[6] Heike Hofmann and Marie Vendettuoli. Common angleplots as perception-true visualizations of categorical as-sociations. Visualization and Computer Graphics, IEEETransactions on, 19(12):2297–2305, 2013.

[7] William Playfair. Commercial and Political Atlas. London,1786.

[8] Edward Tufte. The Visual Display of Quantitative Information.Graphics Press, USA, 2 edition, 1991.

6