DispatchR865

Dispatches

Human Perception: Visual Heuristics in the Perceptionof Glossiness

New insights into the perception of surface glossiness embody a conceptualchange in perception research. Instead of estimating the physical properties ofobjects, the brain exploits ‘invariants’ — even though these sometimes makeus get the answer wrong.

Roland W. Fleming

Working out how the brain estimatesthe material properties of surfaces isone of the most exciting and rapidlydeveloping areas of visualneuroscience. The perception ofglossiness is of particular interest,partly because very small changesin the image, such as the addition ofa small highlight, can have radicaleffects on how the brain interpretswhole surfaces. One recurring themein research on this topic is thatfactors other than the physicalglossiness of a surface can have largeand unpredictable effects on itsperceived glossiness [1–7]. Changingthe shape or lighting conditions canoften make a huge difference to howglossy a surface appears; forexample, the apple in Figure 1Aappears less glossy than it does inFigure 1B because the lighting ismore diffuse in A than in B. So far thereis little consensus on when and whythese different effects occur.Sometimes changing the shape makessurfaces appear more glossy,sometimes less, in a seemingly erraticway. Why does this occur? Is theresome unifying principle that canexplain the inconsistent effects ofshape and lighting on perceivedglossiness? A recent study byMarlow and colleagues [8], reportedin this issue of Current Biology,suggests the explanation lies inthe brain’s use of a number ofsimple — but imperfect — heuristics.

The main argument of Marlow et al.[8] is that the visual system doesn’tactually estimate glossiness per se;instead, they argue, the brainmeasures a set of simple ‘proximalstimulus properties’ thatapproximately correlate withglossiness in typical circumstances,but which may get the answer wrong

under other conditions. In other words,rather than estimating the physicalproperties of surfaces, the brainmeasures whatever it can from theimage itself, and identifies usefulstatistical patterns or cues among themeasurements. The authors show thatsuch cues can predict the failures aswell as the successes of human glossperception.

Marlow et al. [8] rendered a set ofstimuli consisting of a glossy materialin the form of bumpy reliefs. They thenvaried both the depth of the reliefs andthe patterns of illumination incidenton the surfaces to create a rangeof different images. All the surfaceshad exactly the same surfacereflectance — they were all made outof the same ‘stuff’ — so that a perfectobserver should report seeing them allequally glossy. But this was not whatthe authors found. As previousauthors have also reported, thedifferences in lighting and shape havelarge spurious effects on the apparentreflectance of the surfaces.Experimental participants reportedseeing a wide range of differentdegrees of glossiness for the differentstimuli. However, the pattern of resultsis complex, and in many casesnon-monotonic. Shallow reliefs appearglossier than deeper reliefs when thelighting is facing the surfaces, but lessglossy when the lighting comes fromabove. Under some conditions theintermediate reliefs appear glossierthan either the shallow or the deepreliefs. Can any sense bemade of theseresults?

Importantly, both the lightingdirection and the relief of thesurfaces have large effects on theproperties of the highlights visible inthe images. When a surface isshallow and illuminated from in front, itappears awash with large blurryhighlights. In contrast, when the relief

has higher curvature, the highlightsappear smaller and focused, likedots. If the visual system uses theextent to which a surface produceshighlights as a proxy for the truephysical surface reflectance, then thepattern of results starts to make moresense. Surfaces that produce large,visible highlights have the ‘look’ thatwe associate with glossy surfaces. Butwhen the same surface produceshighlights that are puny in terms ofsize, contrast or distinctness, then thesurface doesn’t look as glossy,even though the physical reflectanceis the same. In other words, the‘distal’ physical property of thesurface isn’t actually what the braincares about. Rather, it is the extentto which the surface produceshighlights that determinesits appearance.To capture this intuition,Marlow et al.

[8] identified several differentproperties of highlights that representtheir clarity and salience in theimage — size, contrast, sharpnessand binocular separation from thesurface (see Figure 1C–E). Computingthese quantities directly from theimage is not trivial, so instead, theauthors showed all the images tonew participants and asked them tojudge each property independently.The subjects were not askedanything about the glossiness of thesurface. They just had to report howlarge, or high contrast the highlightsappeared.As expected, the participants’

responses also vary systematicallywith lighting and surface relief.Importantly, the authors found thata weighted combination of theratings for the individual cues couldaccount for 94% of the variance in theglossiness ratings from the otherparticipants. In other words, thesimple cues can account for almostall of the seemingly inconsistenteffects of lighting and shape on theperception of glossiness. When askedto judge glossiness, subjects actuallyreport the extent to which a surface

Size +

-Contrast

Sharpness

A

C D E

B

Figure 1. Effects of illumination on the apparent glossiness of an apple, along with three of thecues that Marlow et al. [8] suggest may explain changes in appearance.

In (A) an apple was photographed with the curtains drawn, whereas in (B) the curtains wereopened, creating a clearly visible highlight on the apple. Most people see the apple in (B) tobe somewhat glossier than in (A), even though the physical surfaces are identical. Marlowet al. [8] find that changes in appearance can be well predicted by some simple propertiesof the highlights, such as (C) the size, (D) the contrast and (E) the sharpness.

Current Biology Vol 22 No 20R866

manifests highlights — this is thedefinition of ‘glossiness’ for the humanbrain.

As this was a correlational study,Marlow et al. [8] cannot be certainthat it is exactly these specific cues thatdetermine glossiness perception.There are potentially many other waysof capturing the intuition of bigger,more salient highlights. However, thisis not the important point of thestudy. The important point is thatinstead of representing glossiness inphysical terms, the brain uses a setof imperfect cues or heuristics.These heuristics predict the errorsas well as the successes ofgloss perception in the authors’experiments.

The idea that the brain exploitsheuristics is almost as old as thestudy of perception itself. However,Marlow et al. [8] have captureda change in current thinking aboutthe goals of perception, whichapplies to much more than just glossratings. Many problems in perception,from visual stereopsis to auditory

pitch perception can be posed asa process of estimating physicalparameters — distances, frequencies,and so on. But this may be the wrongway of thinking about the biologicalproblems that the brain solves.Rather than estimating physicalproperties of the world, it may bemore important — and easier — tocompute systematic relationsbetween internal states. In otherwords, it may be better to identifyquantities that can guide decisionsconsistently in the face of changes ofirrelevant variables, whether or notthey map cleanly onto physicalproperties. For example, ‘colourconstancy’ should not be posed, as itusually is, as a problem of estimatingthe spectral reflectance of surfacesunder varying illuminants; rather,colour constancy is the process ofidentifying image measurements thatare as close as possible to invariantacross transformations caused byother scene factors, such as lighting,shape or viewpoint. Only with thisalternative goal in mind does it make

sense that purples and redsare subjectively similar to oneanother — that is, close to one anotherin the hue circle — even though theylie at opposite ends of the visiblespectrum.Other perceptual tasks make this

idea even more explicit. For example,most people can easily recognizefamiliar linguistic accents acrossa wide range of differences betweenspeakers (age, gender, and so on).In other words, we have good‘accent constancy’. At the same time,it probably doesn’t make sense topose accent constancy as theestimation of physical parameters ofthe speaker’s vocal tract. Instead, it ismore likely that the brain identifiesauditory quantities that arediagnostic of accent but relativelystable across age, gender andother vocal attributes. After all,the hidden Markov models used bycomputer speech recognitionsystems identify predictive featuresin the input stream, rather thanmodel the physics of speechproduction. Thus, more generally, ifwe want to understand how thebrain perceives, we should changethe way we pose the aims ofperception.

References1. Nishida, S., and Shinya, M. (1998). Use of

image-based information in judgments ofsurface- reflectance properties. J. Opt. Soc. Am.A 15, 2951–2965.

2. Fleming, R.W., Dror, R.O., and Adelson, E.H.(2003). Real-world illumination and theperception of surface reflectance properties.J. Vis. 3, 347–368.

3. Vangorp, P., Laurijssen, J., and Dutre, P. (2007).The influence of shape on the perception ofmaterial reflectance. ACM Trans. Graph. 26, 1–9,(Article 77).

4. Ho, Y., Landy, M.S., and Maloney, L.T. (2008).Conjoint measurement of gloss and surfacetexture. Psych. Sci. 19, 196–204.

5. Wijntjes, M.W., and Pont, S.C. (2010). Illusorygloss on Lambertian surfaces. J. Vis. 10, 1–12.

6. Olkkonen, M., and Brainard, D.H. (2011). Jointeffects of illumination geometry and objectshape in the perception of surface reflectance.i-Perception 2, 1014–1034.

7. Doerschner, K., Boyaci, H., and Maloney, L.T.(2010). Estimating the glossiness transferfunction induced by illumination change andtesting its transitivity. J. Vis. 10, 1–9.

8. Marlow, P.J., Kim, J., and Anderson, B.L. (2012).The perception and misperception ofspecular surface reflectance. Curr. Biol. 22,1909–1913.

Department of Psychology, University ofGiessen, Otto-Behaghel-Str. 10/F, 35394,Giessen, Germany.E-mail: roland.w.fleming@psychol.uni-giessen.de

http://dx.doi.org/10.1016/j.cub.2012.08.030