29
7/29/2019 Rollins 1999_Pictorial Representation http://slidepdf.com/reader/full/rollins-1999pictorial-representation 1/29 This article was downloaded by: [UZH Hauptbibliothek / Zentralbibliothek Zürich] On: 04 July 2013, At: 11:47 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Philosophical Psychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/cphp20 Pictorial representation: When cognitive science meets aesthetics Mark Rollins Published online: 19 Aug 2010. To cite this article: Mark Rollins (1999) Pictorial representation: When cognitive science meets aesthetics, Philosophical Psychology, 12:4, 387-413, DOI: 10.1080/095150899105657 To link to this article: http://dx.doi.org/10.1080/095150899105657 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub- licensing, systematic supply, or distribution in any form to anyone is expressly

Rollins 1999_Pictorial Representation

  • Upload
    3468who

  • View
    224

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 1/29

This article was downloaded by: [UZH Hauptbibliothek / ZentralbibliothekZürich]On: 04 July 2013, At: 11:47Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Philosophical PsychologyPublication details, including instructions for authors

and subscription information:

http://www.tandfonline.com/loi/cphp20

Pictorial representation:

When cognitive science meetsaestheticsMark Rollins

Published online: 19 Aug 2010.

To cite this article: Mark Rollins (1999) Pictorial representation: When cognitive

science meets aesthetics, Philosophical Psychology, 12:4, 387-413, DOI:

10.1080/095150899105657To link to this article: http://dx.doi.org/10.1080/095150899105657

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.

However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, orsuitability for any purpose of the Content. Any opinions and views expressedin this publication are the opinions and views of the authors, and are not theviews of or endorsed by Taylor & Francis. The accuracy of the Content shouldnot be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions,claims, proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly in connection

with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly

Page 2: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 2/29

forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 3: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 3/29

P H IL O SO P H ICAL P S Y CH O L O GY  , VO L . 1 2, NO . 4, 1999

Pictorial representation: when cognitive

science meets aesthetics

M AR K  R OLLINS

A B S T R A C T   Pictorial representation is a subject of interest to both cognitive science and aesthetics.

Standard theories of depiction often draw on vision science, and vision science must give an account of picture perception. I offer a critical overview of standard theories of depiction and argue that none

of them is adequate. I then describe ways in which new theories of perception blend elements of  

representationalism with an emphasis on attention and motor control. Such theories, in effect, limit 

the reliance on mental representation in perceptual tasks. This work provides the basis for a theory

of depiction in which pictorial representation is explained in terms of both mental representations and 

 perc ep tu al stra te gies. I ar gue th at, in th e ca se , th e m en ta l re pr ese nt at ion s ar e m os t pl au sibly

individuated by the functional and conceptual roles, rather than by causal links to the external world.

Introduction

What is a picture? And how do pictures have content? Pictures are representations

of some sort; the ®rst question is about how to distinguish them from other forms

of representation. Individual pictures are identi®ed by the contents they have; the

second question asks how to distinguish one picture from another. Pictures are, of 

course, often works of art, and their artistic properties depend on the fact that they

are representations. Thus, pictorial representation is a matter of natural interest in

aesthetics. There are, in aesthetics, ®ve standard types of theory. They explain

depiction in terms of convention, resemblance, causal relations, mental construc-

tions, and information, respectively. Four of those theories explain depiction in

term s of perception. It is here that aesthetics and cognitive science m eet; for 

perception is a central subject of both. Moreover, picture perception is arguably a

test case for any theory of vision; an adequate theory will have to account for it.

Standard aesthetic accounts sometim es reveal im portant dim ensions of picture

perception. Thus they can pose a challenge to cognitive science, as m uch as the

other way around.

Unfortunately, theories of depiction in aesthetics h ave so far not been systemat-ically aligned with recent work in vision science. To draw out those connections is

my ®rst goal. However, it is not my intention to suggest that a particular theory of 

M ark Rollins, Department of Philosophy, W ashington University, Campus Box 1073, One Brookings

Drive, St. Louis, MO 63130-4988, email: [email protected].

0951-5089/99/040387±27Ó1999 Taylor & Francis Ltd

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 4: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 4/29

38 8 M . ROLLINS

FIG . 1. Theories of pictorial representation.

depiction can be shown to be correct, merely by pointing to the currently accepted

theory in vision science. F or one thing, there is no o ne accepted scienti®c theory; for 

another, different theories of depiction have been grounded on the same perceptual

theory and set of facts. N onetheless, there is a need for an analysis of this sort,precisely to disentangle the threads w ith wh ich cognitive science and aesthetics h ave

been interwoven; entanglements that have fostered certain confusions.

Therefore, in the initial part of this essay, I discuss several theories of depiction

identi®ed according to the following scheme. First, perceptual accounts are con-

trasted with no nperceptual theories. The latter include conventionalism and resem-

blance theory, wh ere the resemblance is between the picture and another object. But

resemblance can also be construed as a relation between the visual experiences of 

picture and object, in which case it supports a perceptual theory. Thus, under the

banner of perceptual accounts fall four types of theory: resemblance (reconstrued),

causal, mental construction, and information-based accounts. Perception-based

theories of depiction can then be further subdivided along two dimensions. On the

most general dimension, there are theories that draw on indirect  an d direct  models of 

perception, respectively. The former make perception depend on complex internal

representations, whereas the latter do not. And within the indirect camp, there are

modular  an d nonmodular  accounts: those that sharply distinguish perception from

c og nit io n v ers u s th os e th at d o n ot . T h e ® rs t t hre e ty p es o f d ep ic tio n th eo ry

(resemblance, causal, and constructivist) presuppose an indirect theory of percep-tion. On the last type (information-based), perception is direct. Among the indirect

accounts, the ®rst two accept a m odularity thesis, while the third rejects it (see

Figure 1).

In a dd it io n , it is im p or ta nt t o n ot e t ha t s om e o f t he in d ir ec t t he or ie s go

hand-in-hand with familiar theories of mental content. Speci®cally, causal theories

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 5: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 5/29

PICTORIAL REPRESENTATION  38 9

of depiction correspond naturally to causal theories of mental content, and construc-

tivist accounts ®t well with conceptual role semantics (which makes mental content

depend on relations among mental states). I stress this because, while part of our 

concern is to understand what sets pictures apart from other forms of representation,

it need not be thought that what sets them apart is a special account of perceptual

content. Some theories of depiction stand or fall with the theories of mental contentthey presuppose.

However, as it will turn out, al l  of these approaches face serious problems. T his

suggests that it is time to try a new tack. That is the aim in the latter part of this

essay. In recent years, several theories of perception have emerged that attempt to

limit the reliance on mental content, w ithout dispensing with it altogether; in effect,

they combine elem ents of indirect and direct accounts. The com bination takes

several forms, referred to variously as ªactive,º ªdirected,º or ªutilitarianº theories

of vision (e.g. Cutting, 1986; Ballard, 1991; Ram achandran, 1990). There are

important differences among the variants, but generally the idea is that vision does

not require the construction of detailed, complete mental representations of a scene

or object; partial representations will suf®ce. The reason is that the visual system can

use attentional and other strategies to facilitate the performance of p erceptual tasks.

These strategies allow the visual system to take representational shortcuts. T hus I

will treat these approaches together under the general heading of   strategic design

theories.

It is essential that strategic design theories be applied to pictures. Proponents of 

this approach sometimes argue that it is not always necessary (or helpful) to storecomplex representations in memory; representations that might, in turn, have an

im pact on perception. The reason is that the environm ent contains structured

information in various ways and thus can serve as a kind of external memory store

to which the p erceiver can return at w ill. Pictures, in particular, are rich repositories

of information. It seems quite obvious that we rely on them to remind us of things

we do not or cannot keep in mind. But this argument presupposes that extracting

information from pictures themselves does not require the production of complex

mental representations.

What I want to show is that it is not necessary to explain that fact in terms of 

the forms that mental representations take, as has often been argued. A more fruitful

approach can be based, instead, on an understanding of the tasks and processing

principles that are characteristic of picture perception. Taking this approach will

distinguish the hybrid theory of depiction that I defend from other recent moves in

the direction of a compromise.

Nonperceptual theories

Convention

Pictures have been said to be ªconventionalº in more than one respect. However,

conventionalism in its strongest and m ost fully developed form is due to the w ork of 

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 6: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 6/29

39 0 M . ROLLINS

Nelson Goodm an (1976). On his account, pictures are language-like arti®cial

symbol systems. Both their status as pictures and their contents are determined by

the syntactical and semantic properties they are deemed by common agreement to

have. Because these properties are assigned by convention, their assignment is said

to be arbitrary. On this view, pictures are distinguished from words by the fact that

they represent the properties of objects in a continuous, analog form ; that is,between any two marks representing features there will be a third mark representing

another feature or feature dimension, and there are no distinct grammatical roles

among marks that allow for the expression of discrete propositions. However, there

is no natural connection between a representation having that form and what it

represents. Of course, depicted objects will look more realistic if the properties

attributed to them are fam iliar ones; but any object±property association can

becom e fam iliar or (in Goodm an’s term ) ªentrenched.º More im portantly, the

means by which such associations become entrenched does not constrain them in

any way that contributes to the speci®cation of pictorial content. Thus, the role of 

perception in becoming familiar with the ªlanguageº of paintings or ph otographs (or 

ªreadingº representations in it) is only of tangential interest.

This theory is appealing on several counts. First, pictures clearly do have

language-like semantic properties: they can denote objects and are typically mean-

ingful compositions. Second, it would be gratifying to have a uni®ed approach to

verbal language and visual art, both of which are form s of com m unication. And

t hir d, t he re is , in f ac t, m u ch d iv e rs it y ac ro ss c u lt ure s in s ys tem s o f p ic to ria l

representation, which could be explained if such systems were wholly conventional.Nonetheless, there are several reasons to reject this account.

O n e p ro b le m is t ha t th e e vid en ce is n o t a ll o n t he c on ve nt io na lis t’s s id e.

Pictorial systems are simply not as varied as the theory would lead us to believe.

While it may be true that anything can denote anything, it is not true that anything

can depict anything, just as a matter of fact.

The second problem is related: picture±object relations are not arbitrary. It is

important to recognize how radical Goodman’s conventionalism is in this regard. It

is sometimes said, rightly, that an interest in the psychology of learning is consistent

with conventionalism and that the latter is compatible with a perceptual theory that

explains visual experience in term s of acquired knowledge. But if an account of  

depiction is actually grounded on such a perceptual theory, then picture±object

relations will not be arbitrary; rather, they will depend on the laws of psychology.

And while a conventionalist of Goodm an’s type may agree that we learn to recognize

picture±object relations in accordance with some psychological laws, it does not

follow that the relations are determined by those laws for him. On the contrary, the

facts of human psychology do not matter for such a conventionalist, because those

facts could be something quite different than they happen to be, and his formal,logical analysis of symbol systems would remain unaffected.

But this is precisely where Goodman’s conventionalism goes astray. Recent

years have seen a shift toward a naturalistic semantics for verbal language. This

means, for example, that linguistic reference and meaning are seen as capacities that

depend on psychological processes, in terms of which they must be understood. If 

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 7: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 7/29

PICTORIAL REPRESENTATION  39 1

that shift of focus is on the right track for verbal language, then a similar change in

regard to the visual ªlanguageº of art should be warranted as well.

 R esem blanc e

According to this second theory, what a picture represents depends on what it lookslike. More precisely, pictures are said to represent in virtue of similarity, and

representing in that way is distinctive of the class of representations to which they

belong. Thus a portrait is said to be a picture of Rembrandt because it resembles

him. The name ªRem brandt,º on the other hand, does not represent the man in that

way.

This theory, too, has intuitive appeal. Pictures often do resemble their objects.

And the experience of picture recognition does seem, on the face of it, to be one of 

seeing similarities between the picture and what it represents. But resemblance

theory faces som e notorious problem s. First, as Plato pointed out, everything

resembles everything in some respect or other. Thus, the appeal to resemblance is

empty, unless one can answer the question, ªresemblance with respect to what?º

Pinning down the relevant similarity would seem to require something other than

resemblance; an interpretation, a description, or a title, perhaps. Second, it is a

well-known point of Goodman’s (1976) that resemblance does not distinguish the

representation from what it represents. Resem blance is both sym m etrical and

re¯exive: a picture and its object each resemble the other; a picture resembles itself 

m ore than anything else. But objects do not represent pictures, nor do picturesrepresent them selves. Therefore, resem blance is not a suf®cient condition for 

pictorial representation. It is not a necessary condition either if any one of the other 

four standard accounts are correct. For each of them shows how something might

count as a picture, despite the absence of any signi®cant similarity to its object.

Perceptual theories

 R esem blanc e (r econstr ue d)

One way of pinning down the respect in which a picture resembles its object is to

invoke a perceptual theory; a theory of how viewers see picture±object similarities.

Such a theory has been defended recently in terms of David Marr’s theory of vision

( M arr , 1 98 2 ; G il m an , 1 99 2 ; s ee a ls o P e ac oc ke , 1 99 2 , fo r a re la te d a cc ou n t).

According to that theory, vision depends on certain basic operations shared by all

perceivers; that is, operations that are not affected by differences in background

knowledge. Early vision is thus modularized , i.e. segregated from other sense modal-

ities and higher cognitive processes. Vision is informationally encapsulated (anyknowledge on which it depends is restricted to basic visual knowledge), and the

processes that de®ne it are activated only by a limited type of (visual) stimuli. On

this view, not only is vision unchanged by differences in the beliefs, values, or 

theories held by the viewer; it is also isolated from other senses. Marr’s argument,

then, is that when the task is object recognition, the same modularized operations

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 8: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 8/29

39 2 M . ROLLINS

that are used in ordinary perception are also used in picture perception. It is in that

sense that pictures ªlook likeº their objects: information is extracted from them in

order to identify their objects by the very same mechanisms that would be brought

into play by the objects themselves. This is supposed to translate into similarities in

perceptual experience (Gilman, 1992, p. 186). Thus a human ®gure is recognized

both in the ¯esh and in Picasso’s Rites of spring  because its contours and basiccomponents are registered through processes that are in both cases, and for every

viewer, the same [1].

Appeals to the modularity thesis in aesthetics have lately become widespread,

and they are not lim ited to visual art. For exam ple, Arthur Danto (1981, forth-

com ing) argues that two pictures can be visually identical, yet have different

contents, depending on their place in the world of artists, viewers, patrons, and

dealers. He thus rejects resemblance as a suf®cient condition for depiction. But the

assumption that two pictures can be visually identical, despite these differences, rests

on the modularity thesis. Raffman (1993) has employed the notion of modularity in

her account of music; Spolsky (1993) applies it to literature. However, there are

important reasons to believe that, at least in a strong form, the modularity thesis is

probably false (see Churchland, 1988; Rollins, 1994, for further discussion).

First, psychological evidence shows that even basic visual functions can be

affected by background knowledge, and studies of the primate visual system, along

with human brain imaging research, indicate that there are reciprocal, top±down

pathways that reach areas of visual cortex devoted to early visual processes. It is

reasonable to assume that these pathways link psychological functions, and thus thatthey are p ossible avenues by which early vision is affected by higher order cognition.

In addition, there appears to be communication across different sense modalities.

Finally, it is clear that the brain is highly plastic: damage to an area of the brain

normally responsible for a particular perceptual task need not result in a loss of 

function, because other areas can take over that function and compensate for the

loss. That implies that structures in the normal brain are not simply and perma-

nently dedicated to a single function.

In any case, even if early vision were strongly m odular, that would not be

enough to motivate a resemblance theory of pictorial representation in any interest-

ing sense. The reason is that early visual mo dules impose only limited constraints on

higher order conscious processes, constraints that can be effectively superceded

(Wollheim, 1993; Churchland, 1988). That is, they provide certain kinds of input

to subsequent processes, e.g. elem entary shape representations, on the basis of  

which objects are recognized. But the more complex representations, concepts, and

b elie fs o n w h ic h p erc ep tu al e xp erie n ce p re su m ab ly d ep en ds c an va ry w id e ly

nonetheless. In light of that, any assurance of a perceived resemblance between a

picture and its object largely disappears. Because higher order processes do haveaccess, in principle, to all sorts of background knowledge, objects in pictures may

not look similar to the same objects in the world. Knowledge may simply be brought

into play in different ways when the picture and its object are perceived.

Ironically, this is a conclusion that a perceptually oriented resemblance theory

is f orc ed to e m bra ce , in o rd er to d is tin gu is h th e re pre se nta ti on fr om w h at it

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 9: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 9/29

PICTORIAL REPRESENTATION  39 3

represents. Gilman (1992) argues that there are ways to tell visually which is the

picture and which the original object. Among other things, one is usually in a frame

and the other is not. But we cannot sim ply rely on the correct observation that

pictures are usually distinguished from other sorts of objects with ease; for that fact

is precisely what needs an explanation. Thus a psychological theory of resemblance

is forced to say that we can recognize differences between a picture and its objectbased on w hat we know. H owever, if that is so, then what is left is only a similarity

in the responses produced by the picture and the object; responses at the level of 

brain activity and elementary processes. This is compatible with the picture and the

depicted object being experienced in very different ways. Thus, even if there were

modularized early visual processes, they could not sustain the weight that a resem-

blance theory requires them to have.

Causal relations

Traditional accounts

The next approach to pictorial representation explains depiction in terms of a causal

chain. The chain begins w ith the object or scene toward wh ich a camera is pointed

or an artist directs his attention during picture production. It ends with the picture

perceiver’s response. It seems almost obvious that the content of a picture should be

identi®ed w ith the object or scene that stands at the b eginning of this chain (Walton,

1984; Thomas, 1997). For example, the surface of a prize-winning pumpkin re¯ectsthe light that enters the camera lens and results in a certain pattern on the ®lm. The

salient cause in this chain of events is the pumpkin (or its surface); therefore, that

is what the photograph is supposed to depict. W hat else, we m ight ask, would a

photograph represent, if not the object toward which the cam era was, in fact,

pointed when the shutter was snapped? Moreover, if the object in the photograph is

unrecognizable to us, it makes sense to decide what is depicted by tracing its actual

causal history back to the object present during production.

However, the account is so far incom plete. The reason is that, in speaking

simply of ªwhat a picture represents,º we have glossed over a standard distinction

between two components of representation: pictorial reference and pictorial meaning .

The former involves the picture standing for a particular object or individual of a

certain type; the latter concerns the attribution of properties to the individual in

question. For exam ple, a picture of a cat m ay refer to a real individual nam ed

Morris. And it may represent him as being orange, having a striped tail, or wearing

a sailor’s hat. That these are distinct (yet p ossibly related) components is seen in the

fact that two pictures can refer to the same individual, but show different aspects or 

features of him.This distinction raises dif®cult questions about how to weight the relative

importance of the two components and understand the relations between them. But

the point for now is that establishing the referent of a picture by tracing its casual

history to an individual is not enough to specify the picture’s content. Indeed, in

some cases, the actual referent even seems irrelevant. Does it really matter who the

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 10: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 10/29

39 4 M . ROLLINS

anonymous model posing as Jesus was, for a description of the picture’s content?

Probably not always. In other cases, there will be no referent at all; for example, in

paintings of imaginary objects.

One solution to these shortcom ings m ight be to say that pictures refer to

instances of  properties, as well as to individuals. In that case, the content of a picture

could in principle be speci®ed in traditional causal terms, i.e. by describing thecausal history of the properties as they are depicted. This also opens the door to an

account of imaginary objects. Mental representations in the artist or perceiver are,

in that case, composites; i.e. they are composed of more basic representations of 

previously perceived features, which are attributed to the depicted object. T hose

features are individuated by causal stories of their own [2].

However, even if we overcome the problems of incompleteness and imaginary

objects in this way, an important objection still remains: it is not obvious how the

ªactual causeº of a picture can be described in any case. Any effect will be the result

of a chain of causes, and items in the chain can be described in many ways. Which

item, then, under what description, should be said to stand at the beginning of the

chain and provide the content of the picture? It is true that som e of the causal

in d ete rm in a cy c an b e o ve rc om e b y k no w in g t he n at ure o f th e v is ua l s ys tem

(Dretske, 1986). The causal relation involves a lawlike covariation: one sort of  

change (the presence or absence of the depicted object) corresponds regularly with

another (the presence or absence of an image on ®lm, and then, of a certain kind of 

neural, psychological, or behavioral response in the perceiver). But each of these

changes occurs in a medium, so to speak, the properties of which impose limitationson it (e.g. the structure of the camera or the perceiver’s brain). This establishes a

range, at least, of possible causes. However, establishing points in the causal chain

to which the visual system is responsive does not tell us how best to describe the

items found at those points. This is as true for properties as it is for objects.

We might try to avoid this problem by shifting the emphasis from the cause of 

a picture to its effects on perceivers. However, the idea that a signi®cant part of a

picture’s content depends on the perceiver’s response leads to a causal theory

of a rather different sort than the traditional account we have so far considered.

 R ecogn ition theo ries

According to Flint Schier (1986), what a picture represents is determ ined by a

perceiver’s ability to recognize objects in it; i.e. pictorial content derives from the use

of ordinary perceptual abilities. A picture, P, represents an object, O, if a perceiver 

interprets P rightly, based just on his ordinary perceptual ability to recognize O.

Although some learning and conventions are involved in interpreting pictures, they

are limited. In contrast to language learning, picture perception is distinguished bythe fact that, once an initial successful interpretation has been made, the perceiver 

who made it can then interpret novel pictures without further training. This is a

distinctive feature of pictures called natural generativity. Given this feature, Schier 

says, ªif you can see it, the chances are you can see it in picturesº (1986, p. 43).

This is a causal theory in two respects. F irst, it presupposes a causal theory of 

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 11: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 11/29

PICTORIAL REPRESENTATION  39 5

reference. What makes the perceiver’s response a correct or veridical one is the fact

that P refers to O, which is determined by the fact that O stands in the proper causal

relation to P (Schier, 1986, pp. 91±93; cf. Peacocke, 1992; Lopes, 1993). Second,

although it is not required by their basic construal of pictures, recognition theories

of depiction have been grounded on a causal theory of perception (Schier, 1986,

p. 49; Lopes, 1993). In both philosophy and cognitive science, object recognition isusually explained in terms of percepts or other internal representations. According

to the most in¯uential account, the content of those representations is determined

by the items with which they causally covary. The recognitional abilities involved in

picture perception can be described in similar terms [3].

For exam ple, on this account, a picture of   Mt. Saint Victoire refers to that

mountain if it stands in the appropriate causal relation to it. That a mountain is w hat

is depicted depends on the fact that the picture activates the same mental represen-

tations that mountains ordinarily activate; for instance, a certain perceptual proto-

type. It is because mountains ordinarily activate this type of mental representation

that it has the identity that it has. Thus, in general term s, w e m ight say: if the

perceiver were to see a certain object, O, a certain m ental representationÐ the

O-representationÐwould ordinarily be activated. A picture represents an O if it

activates the O-representation, too.

I postpone until later a discussion of what sort of mental representations play a

role in picture recognition. What matters for now is that recognition depends on

some kind of mental representations, which are individuated causally; for it is here

that recognition theoriesÐas much as the more traditional causal accountsÐbeginto founder. The reason is that causal indeterminacy now simply reappears to plague

the individuation of mental states. It is just as hard to single out the relevant cause

of a mental representation as it is of a picture.

Indeed, causal indeterminacy now takes on a new form, one familiar from the

philosophy of mind. In this form, the attempt to describe the contents of mental

states in terms of their typical causes makes it impossible to say how a picture might

misrepresent  it s o bje ct. T h is p oi nt is im p o rt an t fo r t w o re as on s: ( 1) it is a fa ct

that pictures can m isrepresent, and som e account of that fact is required; and

(2) the concept of representation includes the possibility of accuracy, and the

n o ti on o f a cc ura c y m a ke s n o s en se w it ho u t th e p os sib ilit y o f in ac cu ra cy a s a

concomitant.

The problem can be illustrated in the following w ay. Suppose that a picture

represents a sperm whale as having a horn in its forehead. We would like to say that

this picture shows features that real sperm whales do not have. Perhaps this is

because either the individual artist or people generally in the region have misper-

ceived the whales, confusing them with narwals. As a consequence, viewers of the

picture misperceive them, too. In terms of recognition theory, this means that thepicture elicits sperm whale prototypes, concepts, or percepts, and encourages a

m istaken association of them with aspects or features of the narwal. But here is

where the problem arises. Strictly speaking, if a mental representation is sometimes

caused by sperm whales and at other times by narwals (or narwal pictures), then the

class of objects that usually (most frequently) cause such a mental representation

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 12: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 12/29

39 6 M . ROLLINS

will include both whales and narwals and, indeed, anything else that happens to

produce a representation of the relevant sort. Such a mental representation will

always refer to a sperm w hale-or-narwal-or-¼ as the best description of its typical

cause. In that case, it can never m isrepresent, because the description will be

satis®ed by the presence of any one of its disjuncts (cf. Dretske, 1986).

The standard solution to this causal indeterminacy for mental states is to adda teleological  component. An item is singled out as the content of a mental represen-

tation by appealing to the ªproper functionº or adaptational role of the representa-

tion in question (Millikan, 1984). Mistakes happen when a representation occurs

under conditions in which it cannot perform its proper function. Mental content is

thus established by identifying something in the environment with which a certain

type of representation co-occurred in the past; that something being an object or 

scene, the presence of which conferred an evolutionary advantage on the species in

which the representation is present. But such a representational content might be

produced later in an individual in the absence of the relevant object or scene. The

mental states that bear that content might then be said to misrepresent.

T hu s a p ic tu re w ill re pr es en t a p um p ki n if it g iv es ris e to a p erc ep t th at

corresponded often enough to real pumpkins during evolution for perceivers to use

them as food [4]. (Of course, no one is likely to try to consume the picture, because

it gives rise to other percepts, as well as beliefs about the status of the picture as a

representation, which protect against m isguided action.) But if the pum pkin is

depicted, e.g. as blue, then we may say that it misrepresents pumpkins, because it

gives rise to a pum pkin percept under abnorm al conditions, namely, when theperceived object is blue.

The problem with this appeal to teleology is that there is never just one way to

describe a depicted object as adaptive. Pumpkin pictures might be said to simply

represent fruit, if pumpkin detectors are no more advantageous than fruit detectors

happen to be. This problem is m ade worse by the fact that evolution favors an

economical use of scarce resources. Is it really necessary to distinguish the pumpkin

f ro m its c lo se r ela tiv e s? W h o is t o s ay ? I n o rd er t o d ec id e t he is su e, w e n ee d a

comprehensive understanding of what would count as an ef®cient use of representa-

tional resources within the context of all representational activities and goals. It is

unlikely that we will get such an understanding in the foreseeable future (cf. Fodor,

1990, 1991, p. 293ff).

Moreover, even if a maximally adaptive pictorial content could be identi®ed,

there is no guarantee that pictures will have it. Much recent evidence suggests that

the visual system comprises several different subsystems and that these subsystems

c an b e c om b in ed in a v ar ie ty o f w ay s to p er fo rm a t as k. T h u s, p er ce iv ers c an

employ m any distinct perceptual strategies to achieve the same end result. This means

that ªdifferences in cognitive styles [can] lead subjects to select certain types of strategies even if they are not particularly effective for a given taskº (Kosslyn, 1994,

p. 400). The point is that relatively ineffective strategies need not be idiosyncratic;

they can be characteristic of an individual and even shared in a culture. If so, then

pictorial content cannot be speci®ed in terms of an idealized adaptational role. This

is not to say that an appeal to evolution will play no role in a theory of perceptual

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 13: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 13/29

PICTORIAL REPRESENTATION  39 7

functions, but only that it is dif®cult to individuate representational content in those

terms.

Mental constructions

As an alternative to causal accounts, we might consider explaining pictures in termsof relations among mental states. There are three major theories that take this line.

They emphasize make-believe, illusion, a n d ªseeing-in,º respectively, as the modes of 

mental activity that play the central roles in picture perception. The focus of these

theories has generally not been on how to individuate the mental states in question.

H o w ev er, in s om e c as es, c on st ru ctiv is t t he or ie s o f d ep ic tio n h av e a ls o b ee n

grounded on constructivist theories of perception. Such perceptual theories lend

t he m se lv es w e ll t o a c ert ai n n o nc au sa l a cc ou nt o f m e nt al c on te n t; n am e ly , a

conceptual role sem antics, which individuates m ental contents in term s of the

relations among mental states.

Make-believe

According to Kendall Walton (1984), pictures come in two basic types: on the one

h an d , th er e ar e p h ot og ra p hs , w h ic h re fe r to t he ir o b je cts in v irt u e o f a c au sa l

connection; on the other hand, there are handmade pictures, which do not refer at

all. Whether they depict existing entities or not, all of the latter are non-literal

representations that are de®ned by their roles in a game of make-believe. O n thisview, a picture represents a certain object, O, if it enjoins its perceivers to pretend

or im agine that they are seeing O. It is im portant to note that the gam e of visual

make-believe has rules, and within it, certain propositions are true. In this sense, it

is analogous to standard forms of make-believe: it is true, for example, that grif®ns

have wings (in a ®ctional context); it is not true that they have ®ns or scales.

In so far as it severs the causal links that would establish reference to objects in

the world, and because it makes depiction depend on propositional attitudes and

higher order cognitive abilities, Walton’s theory of make-believe is constructivist in

spirit. Of course, his is an institutional  account that emphasizes social practices,

perhaps more than perceptual processes. And, on this view, pictures have features

that guide the visual imagination. But the rules of the practice can be internalized,

and pictures function largely as props. We are thus left with a kind of com plex

mental activity that is something other than object recognition, namely, imagining

that one is seeing the object.

However, one problem with this theory is the lack of psychological evidence

that picture perception involves mental pretending in a game-like, rule-governed

way. In fact, it is not clear even from ordinary experience that picture perceptiondepends on imagination in any elaborate sense [5]. Further, pictures often provide

us with information. It seems unlikely that, in order to extract that information, we

h av e to e ng ag e in a g am e of m en ta l p re ten d. In lig h t o f th es e c on ce rn s, it is

appropriate to turn to constructivist theories of depiction that are more explicitly

based on constructivist perceptual psychology.

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 14: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 14/29

39 8 M . ROLLINS

Illusion

Perceptual constructivism is usually identi®ed with the New Look psychological

movement (Bruner, 1957). On that approach, perception depends on inference,

which in turn depends on knowledge possessed by the perceiver. This implies that

all perceptual processes are, in effect, theory-laden . The knowledge on which percep-

tion depends consists, not of isolated bits of information, but of an interrelated set

of ideas drawn from an integrated conceptual framework or scheme. In a somewhat

different sense, Gestalt psychology can also be viewed as constructivist. In this case,

perceptual content is the product of mental activity grounded on brain events that

are isomorphic to forms found in the visual stimulus. What is important here is that

these forms are organized wholes (exemplifying certain principles of symmetry,

proximity, continuation, etc.) that have emergent properties. These properties are

irreducible to mere concatenations of m ore elementary parts, w hich are transformed

by their combination into larger units.Both versions of the constructivist approach have had a signi®cant impact on

aesthetics. Ernst Gombrich (1960) draws heavily on the New Look school. The idea

that perception is theory-laden (or at least highly knowledge-dependent) can be seen

at wo rk in his famous declaration that ªthere is no innocent eye.º G estalt psychology

is e m bo die d in th e w riti ng s o f R u do lp h A rn he im ( 19 66 ) o n v is ua l a rt . O n h is

account, picture perception depends on various visual concepts and pictorial atti-

tudes that differ in their em phasis on aspects of visual form . However, Gestalt

psychology is no longer as in¯uential as it once was. I shall concentrate, therefore,

on recent work in cognitive science that underwrites a theory of picture perception

in the tradition of Gombrich.

Three ideas are central to Gom brich’s account of art: schema-and-correction,

seeing-as, and illusion. According to the ®rst idea, perceptual recognition depends on

matching the visual stimulus to familiar forms, which are modi®ed over time if the

®t is repeatedly too inexact. A lthough Gomb rich views schemas as invented pictorial

techniques, these clearly rest on psychological mechanisms. The matching-and-

correction process re¯ects a ªprinciple of the adapted stereotype,º which suggests

that schem as are like perceptual prototypes; not strict rules or de®nitions for 

classifying objects, but overlapping sets of features that provide criteria for judging

degrees of similarity among members of a class.

The idea of schema-and-correction is used by Gombrich to explain the history

of art. That, he claims, is driven by a process of hypothesis-and-testing. Gombrich

follows Popper in grounding the selection of visual hypotheses on a principle of 

falsi®ability. This ®ts well with his use of the concepts of information and illusion,

which at ®rst may seem inconsistent. His view is that, in so far as pictures represent

correctly, a perceiver w ill derive no false information from them. Thus, one mayproperly apply certain concepts to the picture and see it as the sort of object it

represents. In that sense, art is illusion. But the illusion is only ªof the eyeº and not

of the mind. The perceiver ordinarily does not believe that the picture is the object

he sees it as. This does not mean, of course, that the perceiver makes-believe that

the picture is its object. If I pretend that one thing is another, then I harbor no

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 15: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 15/29

PICTORIAL REPRESENTATION  39 9

illusions about them. Seeing P as O is quite different from pretending or imagining

that P is O.

But do we always see P as O? One obvious objection to the illusion theory is

that it overgeneralizes from a small subset of pictures, in applying the claim about

illusion to the entire range. In the case of many paintings, there are reasons to think 

that, even if we apply the concept of a certain O to them, their surface properties socom pel our attention that illusion is really out of the question. In m ore general

terms: even if picture perception is as dependent on concepts, theories, or schemas

as Gombrich suggests, that by itself does not warrant the view that art is illusion.

While pictorial content may derive from what perceivers know, a better understand-

in g o f p erc ep tu al d yn am ic s m a y s ho w th a t, fo r m o st p ic tu re s, t he re is n e ve r a

moment in which the eye is simply tricked.

At any rate, it is not clear that picture perception is actually theory-laden or fully

concept-dependent. If it isn’t, then a m ajor prem ise in the case for illusion is

undercut. There are two lines of argument against this view, one theoretical and the

other empirical.

The theoretical objection concerns the way in which mental content is m ost

plausibly identi®ed on this account. Pictorial representation on Gombrich’s theory

is a function of mental representations formed by picture perceivers; the contents of 

the latter are a function of their relations to other mental states. Although his eclectic

use of psychological theories and philosophical ideas makes it hard to identify a

single theory of mental representation, the central tenets of his account suggest a

conceptual role semantics for mental states and, by extension, for pictures.To be speci®c: the hallmark of conceptual role semantics is its holism . Mental

contents can never be identi®ed in isolation, but only as part of an interrelated

network of mental states. This interdependence of schemas, concepts, and beliefs is

apparent in Gombrich’s account when he speaks of the ªtendency of our minds to

classify and register our experience in terms of the known,º on the one hand; and

ªthe in¯uence which acquired patterns or schemata have on the organization of our 

perception,º on the other (1960, p. 168). Such a view stands in contrast to the

resemblance and causal theories we have considered. Like constructivist accounts,

they treat pictorial content in terms of complex mental representations. But they also

impose important constraints on perception that are missing from constructivism. In

particular, certain ªassumptionsº are said to be built into perceptual processes by

natural selection; for example (in Marr’s theory), the assumption that objects are

m ore or less rigid. These are necessary for perceptual system s to get a unique,

correct identi®cation of an object. Building them in is nature’s way of tuning our 

sensory receptors to the kind of world in w hich we live. Such assumptions are, in a

sense, the axioms from w hich we derive more elaborate models, theories, and beliefs

about the visual world. They m ust, therefore, be construed as elem ents that areunaffected by their use in further information processing. To that extent, the views

we have considered are atomistic rather than holistic, at least in regard to early stages

of perception.

The motivation for this sort of atomism comes from the need to provide some

grounding for higher order mental contents. If all knowledge and memory is fully

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 16: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 16/29

40 0 M . ROLLINS

embedded in a holistic conceptual framework, then the analysis of mental content

w il l b e re gr es siv e o r c irc u la r, a nd it w il l b e im p o ss ib le t o ® n d a ® xe d p o in t o f  

reference from which to attribute mental content to a perceiver. The ®rst objection

to treating picture perception as thoroughly knowledge-dependent, then, is that it

lacks a grounding in just this sense. The result is that pictorial content will be as

unstable as m ental content is, if we accept Gom brich’s account. What a picturerepresents will depend on an elaborate network of schemas, concepts, knowledge,

and beliefs. This network is ever changing, both within the individual perceiver and

across viewers in different places and times [6].

The empirical objection to this account is typically based on two points: Marr’s

theory of vision, according to which perception sometimes depends on nonconcep-

tual representations or processes; and the commonplace observation that we can see

and discriminate shapes for which we have no concepts. If vision is nonconceptual,

it can be argued, then picture perception is not a matter of seeing-as, nor, therefore,

of illusion. It should be noted, however, that the fact that shapes are not brought

under a concept when they are encoded, processed, and discriminated, does not, by

itself show that shape recognition is independent of the in¯uence of concepts

altogether. It must also be the case that shape perception is cognitively impenetrable.

Nonconceptual recognition processes would be cognitively impenetrable, just in

virtue of being nonconceptual , only if we accept a certain assumption; i.e. that cognitive

effects m ust be due to inference, w here inference is de®ned in terms of relations

among propositional representations. That assumption needs to be defended. The

effects of theory on skills, or of background knowledge on perceptual categoriesthat are encoded in nonpropositional forms (e.g. images), cannot simply be dis-

counted as part of the argum ent that perception is concept-dependent. And if  

nonconceptual shape recognition processes can be affected by background knowl-

edge, then illusion in Gombrich’s sense remains a possibility: seeing-as can be a

higher order effect.

Nonetheless, there are other reasons to question the claim that perception is

theory-laden. These reasons can be brought out by considering two related objec-

tions that have been raised against constructivism, one in cognitive science, the other 

in aesthetics. First, the perceptual plasticity that is supposed to show that perception

is theory-laden has not always been due to the effects of theory or concepts (Fodor,

1984). For instance, a perceiver’s ability to switch between different ways of seeing

an ambiguous ®gure (such as a Necker cube or duck±rabbit) often depends more on

a redirection of attention than on background knowledge [7]. Second, Wollheim

(1973) has criticized Gombrich for ignoring our ability to detect visually (and not

just un derstand conceptually) differences between surface properties of paintings

and their objects, while sim ultaneously seeing objects ªinº them . This ability

requires dividing attention, rather than just switching it back and forth betweensurface properties and representational content. Both objections suggest that, in

order to understand how pictures represent, we m ust know m ore about the con-

straints that are im posed on perception by attentional m echanism s. These con-

straints are at odds with the idea that art is illusion, which makes picture perception

depend primarily on inferential and conceptual relations among mental states.

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 17: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 17/29

PICTORIAL REPRESENTATION  40 1

Seeing-in

According to Wollheim ’s (1973) m odel of picture perception, it is necessary to

conceptualize pictorial content; but conceptualizing does not require that the per-

ceiver believe (or m ake-believe) that the picture is its object; nor must he see the one

as the other. Instead, one can see in the picture whatever object it represents; which

is to say that one applies the concept of the object to the picture, while at the same

time attending to, and being aware of, e.g. the color and shading, the layers of paint

or lines of ink, etc. as design features of the painting or drawing. Seeing-in is not

limited to picture perception; we can also see the shapes of objects in water stains

and clouds. But pictures are distinguished by the fact that, in order to see something

c or re ct ly in t he m , w e m u st s ee w h at th e p ic tu re p ro d uc er in te nd ed . S till, th e

question remains: in what does seeing-in consist?

Wollheim does not himself offer a full account of seeing-in. This is an obvious

weakness in the argum ent for the theory. However, som e recent m odels in psy-chology allow for the simultaneous division of attention, in which information about

meaning or content can be processed along with information about shapes, sounds,

or locations (e.g. Treisman & Gelade, 1980). These open the door to an empirically

grounded analysis of seeing-in.

At the same time, integrating theories of attention with a mental construction

account also points to a hybrid  model in which elements of both indirect and direct

theories of perception are combined. It is on this sort of model that I want to focus.

But before discussing it, we must consider Gibson’s account of depiction, which is

borne out of a view of perception as entirely direct, i.e. unm ediated by m ental

representations.

Information

According to James Gibson (1978), pictures contain the same information as the

scenes they d epict. In particular, both pictures and the light surrounding the o bjects

they represent em body gradients of texture density. These gradients are said to

ªspecifyº properties of the objects. For instance, size is a function of the number of 

textural units displaced by the object, which is constant over different locations in

space. This is because texture com pression varies regularly with distance. On

G ib s on ’s d ir ec t re alis t a cc ou nt , s uc h in fo rm at io n c an b e ª pic ke d u pº b y th e

perceiver without any intervening mental processes [8].

This information theory initially faces the same kind of symmetry problem that

resem blance theory confronts: if the same inform ation can be picked up from a

picture and the object it represents, and pictorial representation is explained as being

due to that fact, then the object would seem to count as a representation of thepicture [9]. However, the problem is avoided by an appeal to attention: a perceiver 

can notice aspects of a painting not found in the scene or object it depicts. While

G ib s on c an n ot in vo ke th e ore tic al k no w le d ge in t he m in d o f th e p erc eiv er to

distinguish picture from object, he can cite practical know-how, i.e. skills or abilities

like attention. M oreover, G ibson’s ecological approach allows him to argue that

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 18: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 18/29

40 2 M . ROLLINS

pictures h ave higher o rder relational properties not found in other artifacts, and vice

versa. These properties fall under the heading of  affordances. Gibson was liberal in

describing properties that could be directly perceived: a cake, for instance, might b e

seen as edible, whereas a picture of a cake would not. Thus the real cake affords its

perceiver o pportunities not available with the picture. T he picture, in turn, presents

possibilities for perceptual training which the ordinary object does not present.Information theory is appealing, because it seems to correspond well to our 

sense that picture perception is a natural, unlearned ability that we all use without

thinking. And it emphasizes the fact that both the world of objects and pictures of 

them are rich in prestructured information that does not depend on mental con-

structive activity in its perceivers. No netheless, Gibson’s theory is un able to explain

how pictures have a particular content. As Gibson himself notes, two surfaces at

different distances, but slanted at appropriate angles, could have textural features

that are represented in pictures in the same way. Pictures can thus be ambiguous.

C o nc ern s o f th is s ort le d G ib so n to e m ph as iz e t he im p o rt an c e o f p er ce iv e r’ s

movements through his environment to help pick up disambiguating information.

But that is a source of information that is unavailable in pictures, where the point

of view and spatial perspective are ®xed by the artist. Shifting attention over the

surface of the picture, through m ovem ents of the eyes and head, can lead us to

discover new details; but it cannot change that perspective.

Although Gibson’s account cannot explain picture perception on its own,

certain models of perception continue to draw upon Gibsonian ideas, in ways that

p ro m is e a m o re a de qu ate a cc ou nt o f p ic tu re s t ha n t he t he or ie s w e h av e s o f ar  considered. I turn now from the critique of available theories to an alternative

positive proposal in that vein.

Hybrid theories

Fueled by the failures of existing accounts, recent developments have moved in the

direction of a compromise. Several proposals have been made for a hybrid theory of 

pictorial representation. However, hybrids come in different forms. I shall ®rst

discuss certain w ays in which a c ompromise might be pursued and then focus on o ne

form, arguing that it works best.

It should be noted that there is som e precedent for the com prom ise effort:

elements of hybridization can already found in existing accounts. For instance, while

taking illusion to be the de®ning feature of visual art, Gom brich also holds that

different styles exploit techniques differently. He claims that there is a continuum

along which styles can be arranged, according to the degree to which resemblance

or conventional symbols are used to achieve the pictures’ effects. At one end of the

spectrum, art is ªconventionalº because it relies heavily on symbols that are formal,codi®ed, of®cially sanctioned, or otherwise institutionalized [10]. For example, a

lamb may represent Christ in certain religious traditions. But Gombrich’s theory is

also conventional in a second sense. His view that schemas are acquired through

perceptual learning m eans that picture perception depends on experiences and

memories that arise within a particular cultural context. Pictorial content is thus

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 19: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 19/29

PICTORIAL REPRESENTATION  40 3

conventional in that it is relative to a cultural context in which shared schemas are

accepted by tacit or explicit agreement. Schemas may be articulated and taught in

schools of art or simply accepted unconsciously as part of common practice. In this

second sense, al l  of the styles on the continuum are conventional [11].

 R ecogn ition -based hyb rids

However, several lines of thought lately suggest that conventionalism can be recon-

ciled, not with constructivist accounts, but with some version of recognition theory

(e.g. Lopes, 1993, emphasizes aspect recognition; see also discussions of reference

in Evans, 1982; Schier, 1986.) The result is a third and somewhat stronger sense in

which pictures are ªconventionalº: they are similar to conventional linguistic sym-

bols [12]. The argument is that visual art is like a language, in the sense that both

verbal and visual representations have semantic properties, and at least some of 

those properties are grounded on perception in both cases. In particular, some forms

of verbal and pictorial reference are said to require perceptual contact with the

referent. For instance, it has been argued that both demonstrative and indexical

reference are grounded on the speaker’s location of himself, which depends on his

perception of space. Similarly, pictorial reference is mediated by aspects that depend

on the perspective or point of view from which the picture represents its objects.

These efforts are, in my view, generally on the right track. Unfortunately, they

are derailed in the end by concentrating on the wrong dim ension of perceptual

contact. In effect, recognition theories aim to ground a hybrid theory of depiction(one that emphasizes both perception and convention) on an account of the forms

that mental representation takes, the type of representation it is, or on constraints

governing its computational relations to other representations. And this, I believe, is

a mistake. Part of the problem is that the evidence strongly suggests that perception

does not, in fact, depend on the necessary types and constraints. Beyond that,

however, if we are to defend the claim that pictorial representations, as perceptual

objects, have language-like semantic properties, I believe that we must ®rst intro-

duce a hybrid account in a different sense: not one that simply combines conven-

tionalism with a recognition (or some other indirect) theory of perception, but an

analysis of pictures that rests on a m ixture of elem ents drawn from direct and

indirect accounts. As it turns out, I will argue, the m ixture works best when the

indirect component is provided by a theory of representation closer to construc-

tivism than to the sort of causal model associated with recognition accounts.

To see the m isstep that recognition hybrids take, let us return now to the

question of w hat sorts of m ental representation are required for picture recognition.

In so far as what are recognized are objects, it is natural to think that representations

of perceptual categories will be needed: prototypes, concepts, and possibly evenbeliefs regarding the object in question. However, here there is controversy. It

centers on the fact that recognitional ability is ªaspect-relativeº (Schier, 1986,

p. 91). That is, any object m ay be recognized correctly in som e guises by som e

perceivers and in other guises by others. Object recognition in pictures will always

be m ediated by aspect recognition. Thus, in addition to the representations of  

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 20: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 20/29

40 4 M . ROLLINS

perceptual categories that are eventually brought into play, there will also be mental

states that encode the properties that make up aspects. The question is, on what sort

of representation does aspect recognition depend?

We might think that, in order for a picture to represent an object under a certain

aspect (e.g. Morris with a striped tail), it is necessary for the perceiver to hold certain

beliefs (and thus possess the relevant concepts); beliefs about the properties of thetype of object in question (e.g. that cats have tails). H owever, depending on the

notion of an ªaspectº used, it need not always be the case that aspect recognition

requires concepts and beliefs. For instance, Lopes (1993) and Evans (1982) de-

scribe ways in which aspect recognition might be belief-independent, nonconcep-

tual, and cognitively impenetrable. As I have noted (in the discussion of G ombrich),

it does not follow from the mere fact that aspect recognition is nonconceptual that

it is also cognitively im penetrable. And as I have argued (in the discussion of  

resemblance accounts), the evidence runs against cognitive impenetrability. I now

want to suggest why aspect recognition cannot simply be said to be nonconceptual,

and thus, why it is somewhat misleading to ground a hybrid depiction theory on the

a ss um p tio n th at it is . M y ar gu m en t d ra ws ® rs t o n r es ea rc h o n p erc ep tio n a nd

episodic memory.

In the literature on picture identi®cation, there has often been a focus on the

role of abstract mental representations that encode memory for general characteris-

tics. Concepts, prototypes, and schemas are abstract representations in this sense, as

ar e b as ic c om po ne nts t ha t r em ai n th e s am e , r ega rd le ss o f th e w ay s t he y ar e

combined in pictures (as in Marr and Biederman). Structures that represent thegeneral features of depicted objects have been termed pictogens (Warren & Morton,

1982). The assumption has commonly been made that picture processing requires

the use of such structures. One reason to accept that assumption is that amnesiacs

can recognize a de graded version of a picture they have previously seen, even though

they have no recollection of having previously seen it. They are apparently able to

use general semantic memo ry, yet lack any memory of particular pictures they might

have seen.

H o we ve r, t he re is a ls o s tr on g e vid e nc e t ha t m e m or ie s o f e n co u nte rs w it h

speci®c pictures affect later picture identi®cation. That is, picture identi®cation

depends on episodic memory. For example, Jacoby et al  . (1989) showed subjects

intact line drawings of familiar objects and asked the subjects to name the objects

(using basic level nam es like ªdogº). Other subjects were shown degraded line

drawings and asked to clarify them by pressing a key that changed the signal-to-noise

ratio of a computer image; then they named the depicted object. Later, both groups

were asked (1) to freely recall the names of presented pictures, and (2) to clarify and

then name degraded versions of pictures, som e of which were identical to ones

previously seen, some of which o nly shared the same name with earlier pictures, andsom e of which were new.

If picture identi®cation always depends on the priming of pictogens, then the

experience of clarifying the initial set of images should have no special effect on

either the free recall of names or later visual identi®cation of pictures; that is, it

should produce no advantage over simply naming pictures. But there was an effect:

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 21: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 21/29

PICTORIAL REPRESENTATION  40 5

previous experience with degraded images made later identi®cation of those images

easier than having simply named the pictures before. Speci®cally, less clari®cation of 

the subsequent degraded images was required.

The reason, according to Jacoby et al ., is that the clari®cation process promoted

a greater attention to surface features and visual details than naming did. However,

the effect is not simply due to the accidental discovery of some diagnostic featurethat happens to appear in both the earlier and the later degraded image. Rather, it

reveals the operation of a basic principle of episodic memory, namely, the principle

of encoding speci®city. According to that idea, performance on a memory (and, in this

case, a perceptual) task is enhanced to the extent that the processing that occurs

during study of a picture is similar to the processing that occurs during the later test.

Beyond the mere fact that the clari®cation technique was used in both conditions,

its use added more dimensions in which processing could be similar from study to

test. It did this by encouraging attention to detail.

In one sense, of course, these results support the claim that picture perception

often depends on nonconceptual psychological processes. They show that episodic

effects, which do not require abstract memory representations, are important, even

for picture identi®cation (which is supposed to be a semantic memory task). Thus

a recognition theory of pictures can claim that a knowledge of pictorial reference is

mediated by the picture perceiver’s episodic memories and not by concepts stored

in semantic memory (as in Lopes, 1993, p. 137).

There is, however, a problem with that argument. First, some evidence shows

t ha t a b en e ®c ia l e ff ec t o n p er fo rm a n ce is a ls o p ro d uc ed b y a n e arlie r n am in gexperience. Previously named pictures are later named more accurately and quickly

th an n ew o ne s a re . S uc h an e ffe ct is p re su me d t o be d ue t o a bst ra ct m en ta l

representations, because such representations w ould encode features that pictures

with the sam e nam e have in comm on. This evidence raises questions about the

purported nonconceptual nature of picture identi®cation, at least as a general

account. In light of that, we m ight try to analyze the occasional contribution of  

episodic mem ory to sem antic tasks, and at the same time, p reserve a distinct role for 

abstract representations (Warren & Morton, 1982; Posner & Keele, 1968). On some

models that follow that line, episodic memo ries are used to classify depicted objects,

e.g. by comparing them to a memory of a particular picture, taken as an exemplar 

of the class. On other models, the exemplars are further used to generalize to new

patterns. T hese analyses aim to save the episodic-semantic distinction by de®ning

two ways a picture can be identi®ed: using exemplars, on the one hand, and using

pictogens, on the other. This implies that perceiver can choose between two strategic

options.

I believe that this last point is important. However, dual-option models tend to

be somewhat vague and unparsimonious: the alternate ways of performing the task are not clearly distinguished, and the reasons for the options being available are not

apparent. Thus, especially where exemplars are used in generalization, ªthe distinc-

tion between episodic and semantic memory is largely lostº (Jacoby et al  ., 1989,

p. 279). This has led Jacoby et al . to argue that these mixed models of memory miss

the point, which concerns the importance of processing speci®city, rather than the

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 22: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 22/29

40 6 M . ROLLINS

form that representation takes. On their view, picture perception cannot be ex-

plained simply in terms of  either  pictogens or exemplars, especially if the latter are

taken to be literal representations of previously studied pictures: ªMem ory for 

studied pictures is not literal but rather, depends on prior processingº (p. 280).

Thus, the evidence of Jacoby e t a l  . casts doubt on the conceptual±nonconceptual

distinction, not because it shows us what concepts are, but because it reveals thateffects like those due to concepts can result from prior processing episodes. One

might conclude that concepts are sometimes unnecessary, or alternatively, that

concepts are exemplars. But a different conclusion is that whether picture recogni-

tion depends on concepts or not is not the important (or not the only important)

issue. That is my position.

Now denying the centrality of the distinction between episodic and semantic

m e m ory in t hi s w a y m ig ht s ee m to re m ov e th e p os sib ilit y t ha t p erc eiv er s c an

alternate between strategies and thus undercut a point I have taken to be important.

Nonetheless, I w ould argue that the appeal to episodic effects simp ly opens up a new

range of strategic options. In so far as attention to details plays a role in processing,

whatever variations in scan paths, attention shifting, visual routines, or selective

enhancement are available for picture perception will de®ne processing conditions

that may or may not match from one encounter with a picture to the next. These are

strategies (although they need not be consciously employed), and the point is that

the processing speci®city that is so im portant to perform ance on a task can be

described in terms of them.

The central conclusion that emerges from a look at episodic effects on pictureperception, then, is that perform ance on picture identi®cation tasks cannot be

explained entirely in terms of mental representations. It requires an understanding

of processing conditions as well. Putting this point the other way around, we can say

that picture perceivers need not generate detailed or literal mental constructions in

order to do well at picture recognition. To som e extent, task perform ance is a

function of the processing strategies the perceiver (or the perceiver’s visual system)

uses during study and on the task; strategies that both de®ne processing similarities

and depend on other processing conditions present at the relevant times. It follows

that variability in what perceivers can recognize in a picture does not always depend

on differences in what the perceivers know or believe, or on the theories they hold.

It is due, to some extent, to the particulars of processing, including the strategies

they consciously or unconsciously adopt.

Strategic design theories

That conclusion is very m uch in line with other work on perception that is not

speci®cally concerned with episodic effects. This work is characterized, to a largeextent, by a rejection of the view that object recognition and spatial location require

the construction of detailed representations of features. In many cases, the visual

system performs its tasks by actually disregarding information. Of course, theories of 

vision in the tradition of Helmholz and Marr also explain how objects and pictures

of them can be recognized on the basis of incomplete information: poverty-of-the-

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 23: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 23/29

PICTORIAL REPRESENTATION  40 7

stimulus arguments have been the lynchpin of cognitive science and are one primary

basis for positing mental representations and computational processes. But what is

distinctive of strategic design theories is the idea that perceptual tasks need not be

performed by enriching the stimulus mentally through inferring to a more complete

representation of it; indeed, details are often ignored. How might this be accom-

plished?Several possibilities have been proposed. These include: attending to diagnostic

features of objects; using psychological resources not speci®cally designed for the

task; moving through the environment or using the motor system in other ways; and

exploiting assum ptions about the correlation of one type of inform ation with

another. As an example of the last alternative, it has been shown that the shape of 

an object can be recognized without a full delineation of its contours, because the

shading of its surface will take distinctive forms, depending on the location of the

lig h t ( L eh ky & S e jn ow s ki, 1 98 8) . R e pr es en ti ng s ha pe in s ha din g is a f am ilia r  

pictorial technique, suggesting that objects are not only recognizable in pictures

based on partial information, but that some pictures are designed to exploit that fact.

This raises a question: which perceptual strategies might be especially appropri-

ate, when the object of perception is a picture? The evidence for episodic effects on

picture identi®cation and the reliance on shape-from-shading suggest two possibili-

ties. I now want to consider two more. These concern (1) the role of images or other 

iconic mental states in picture perception, and (2) the perception of object features

in moving pictures. I single these examples out because, I shall argue, they support

a surprising conclusion about how to individuate perceptual contents. The con-clusion is that external causal factors, taken in isolation from internal psychological

processes, play no role in the individuation of mental representations. It follows that

causal theories of perceptual and pictorial reference are off the mark. I base this

conclusion, in part, on the formidable problems that causal theories face, problems

I have already discussed. But, in addition, it will emerge that some of the central

problems for the alternative conceptual or functional role semantics can be over-

come when the contribution of strategies is rightly understood.

The ®rst line of research makes it clear that even iconic representations depend

on semantic memory, as well as playing a role in semantic tasks. At the same time,

t he re se ar ch re ve al s h ow a n a cc ou nt o f im a ge ry m u st b e s up ple m e nte d b y a n

understanding of perceptual strategies. This topic follows nicely on the heels of the

d is cu ss io n o f e pis od ic m e m or y, b ec au s e it h as b e en s ho w n t ha t im a ge ry c an

c on tri bu te t o th e p ro ce ss in g c on diti on s o n w h ic h t he p ri nc ip le o f e nc od in g

speci®city depends.

According to a theory called Recognition-by-Components (RBC), objects are

recognized by being parsed visually into basic elements (called ªGeonsº on Bieder-

man’s 1987 model). Stephen Kosslyn has employed RBC to explain picture percep-t io n. O n h is v ie w , ª on e’s k no w le d ge an d b elie fs a pp ar en tl y c an a ffe ct v is ua l

processing, alm ost from the start;º that is, it is theory-laden (1994, p. 87). In

particular, objects are often depicted in pictures from a nonstandard perspec-

tive. When that happens, their parsing requires a hypothesis-and-testing process,

w hic h in vo lv es gene rating an im ag e o f a p ossib le o bject an d searchin g

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 24: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 24/29

40 8 M . ROLLINS

associative memory for information that supports the image as an interpretation of 

the stimulus. This is, so far, essentially a constructivist account.

The problem is that the hypothesis-and-testing does not always work. For 

example, when subjects are presented with an ambiguous picture, they should draw

upon memory in the generation and testing of interpretations of the picture. But the

evidence shows that it is very dif®cult, and in some cases impossible, to interpret anambiguous picture from memory. When Deborah Chambers and Daniel Reisberg

(1985) asked subjects to recall standard ambiguous ®gures, not one subject could

recognize any content other than the one they had originally ascribed when the

p ic tu re w as in it ia ll y v ie we d . C ha m be rs an d R eis be rg a rg u e th at , in th is c as e,

attention to the details of the image that would support a new interpretation was

blocked. The block was removed only by a supplementary motor  process, in which

the subjects redrew the picture themselves.

This experiment is important because, in effect, it isolates the contribution

made by hypothesis-and-testing to object recognition in pictures and shows that,

while there may be such a component, sometimes it does not suf®ce. This raises the

possibility that picture recognition m ay ordinarily depend on som ething m ore.

Speci®cally, the experiment dissociates the inductive process from strategies that

involve attention and m otor skills resulting in a publicly viewable picture. The

implication is that picture perception and content must be explained in terms of 

such strategies as well. Even though ambiguities might b e readily perceived in public

pictures, picture perception cannot be assumed to rely on hypothesis testing and

background knowledge alone. If so, a constructivist account is, at best, incomplete.The research of C hambers and Reisberg suggests that, although it is known that

picture memory is very powerful (i.e. the number of pictures we can recall is very

large), it does not follow that all or even most of the information contained in the

pictures is available in memory representations. This, it would seem, is the result of 

attentional strategies employed either in encoding or storing pictorial information,

and it is by similar strategies that the de®cit is overcome. What is interesting here is

the possibility that, in the absence of the original picture, some help is provided by

the motor control system, when we redraw or simulate redrawing the picture [13].

T he e vid en ce t ha t im a ge s c an b e s ele ct iv e , e ve n if t he y a re ic on ic , le n ds

empirical support to the argument that selective mental representations need not be

descriptive; i.e. their selectivity does not depend on abstracting away from details, in

order to express a limited proposition. Ned Block (1983) has articulated, in this

vein, an account of several modes of pictorial selectivity. F or instance, in addition to

simply omitting details that he might have included, an artist may also be prevented

from representing certain features of an object by virtue of the other features he does

include. This will often be a function of point of view: one cannot see the spots on

a leopard when one is looking it in the face. Selectivity in this sense is compatiblewith an account of pictorial representation in terms of episodic memories and/or 

nonconceptual perceptual content.

However, the selectivity of im ages, icons, or pictures (in term s of Block’s

analysis) is equally compatible with a strategic design account. As the Reisberg and

Cham bers results suggest, the inform ation included in an im age m ay be also a

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 25: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 25/29

PICTORIAL REPRESENTATION  40 9

function of covert attention or m otor control. It is thus natural to think that a

picture’s ªcommitmentsº (the sum total of features it includes and excludes, either 

intentionally or not) is not necessarily due to its actual composition, but may result,

instead, from the kind of perceptual control strategies it tends to compel.

Further evidence for that conclusion comes from research on ªmotion capture.º

R a ma ch an dra n ( 19 90 ) h as s ho w n th at p er ce iv ers o f a n e nc lo se d c on to u r th atappears and disappears alternately at two locations will see the contour as moving

back and forth (which is an illusion). If the contour contains spots, they will be seen

to move as well, and in unison; i.e. their locations in relation to one another remain

t he s am e . In t hi s c as e, p erc eiv er s c an no t c ou nt o r re p ort t he n u mb er o f s po ts

because their presentation is too brief. Yet they m ove in synchrony and so m ust

be processed in som e way. Ram achandran argues that the num ber and location

of individual spots are essentially disregarded by the visual system, but their rela-

tive positions are somehow captured by the moving contour. This is possible, he

claims, because the visual system has adapted its resources based on an ecologically

speci®c assumption: that spots do not normally jump off of surfaces. As a result, it

is not necessary to represent and track the locations of the spots in detail. The

application of this analysis to the perception of motion pictures in more familiar 

forms is clear.

Pictorial semantics

As I have so far described them, strategic design theories are essentially theories of perception that can be shown to shed light on how pictures are perceived. But the

central question of interest here is how, more exactly, they explain pictorial repre-

sentation; i.e. how they account for pictorial sem antics. On a strategic design

approach, pictorial content will derive from perceptual content. I suggest that the

most plausible account of perceptual content, in that case, is not a causal theory, but

conceptual or functional role semantics.

In part, that conclusion derives from the fact that picture recognition appears

not to be insulated, at any stage, from the effects of background knowledge or to be

entirely independent of a conceptual scheme. That is so, even if the relevant mental

states are iconic or represent basic spatial relations. M oreover, as we have seen, such

states can play an important role in semantic tasks, where the cognitive effects are

most likely to be pronounced. It is certainly not necessary to adopt a causal theory

merely because perception depends on episodic memory. On the contrary, the role

of imagery in episodic effects, as well as various problems of false memory, suggest

that the purported recollection of events is based largely on mental constructions.

To be sure, these facts do not entail  a fully holistic theory of mental content. For 

example, they are consistent with a ªtwo factorº model. According to that model,the referent of any mental representation is due to a causal relation between the

representation and an item in the external world. But the fuller meaning associated

with the representation is the result of its internal relations to other representational

states. In this case, the appeal to neither factor is dependent on the stage at which

the representation occurs in visual processing; so cognitive effects are possible all the

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 26: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 26/29

41 0 M . ROLLINS

way down. However, aside from then facing standard problems for both causal and

constructivist theories, the two factor approach has the additional problem of 

explaining how the two factors ®t together.

I would thus argue that the view that ®ts better with a strategic design theory

is this: two factors do contribute to the explanation of performance on perceptual

tasks, but only one of them serves to determine what a perceptual state (and thus apicture) represents. Attention, motor control, and other facets that de®ne perceptual

strategies constitute one ingredient in explanation, but those facets are not them-

selves identi®ed with inferential and conceptual relations. The other ingredient is

mental content, and it is individuated in terms of those relations. It is these two

ingredients working together that makes conceptual role semantics a real possibility

for picture perception. Because strategies are identi®ed separately from mental

representation, yet constrain the representation’s use, they provide us with an

independent basis from which to attribute content to a holistic system. An under-

standing of strategies thus removes some of the motivation for positing a nonconcep-

tual, c ognitively impenetrable, atomistic foundation.

In what sense, then, might a hybrid theory of perception of this type also lend

s up po rt t o a h yb ri d th eo ry o f d ep ic tio n , i. e. o n e w h ic h c om b in e s a n ap p ea l to

perception with the idea that art is like a language? The answer depends, in part, on

h ow it is u sed to e xp la in p erc ep tu al a nd p ic to ria l re fe re n ce . T he re a re th re e

possibilities. (1) Pictures might be said not to refer; i.e. as Walton suggests, we might

conclude that the notion of reference really has no proper application to them. (2)

Reference might be explained in terms of the strategic factor. It can be argued, for instance, that reference depends, not on a particular mode of representation, but on

attention and motor control; for it is by such mechanisms that perceptual contact is

maintained (cf. Ballard, 1991). And (3) reference, as much a meaning, might be

determined by functional relations among mental representations. Reference would

be successful, and representation accurate, not because of an appropriate causal link 

to the referent, b ut because the representation fosters effective behavior and per-

formance on tasks. This is not precluded by the fact that pictures and perceptual

states often serve an information bearing function. As Gibson suggests, a state may

carry information about matters to which it is not causally connected, information

that may both result from and be extracted by constructive mental processes. I am

inclined to defend this third alternative, but will not develop the argument here. It

is c on sis te nt w ith th e id e a t ha t a rt is lik e a la n gu ag e in v irt u e o f its s em an ti c

properties.

Conclusion

Pictures are perceptual objects. The challenge for a theory of pictorial representationis to ®nd the right theory of perception. This is true for sculpture, ®lm, theatre, and

even literature as well, to the extent that these are representational arts that depend

on visual and auditory perception. Although m uch is know n, in particular, about the

visual system, there is currently no clear consensus regarding the nature and extent

of m ental representation in perception. However, recent research points in the

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 27: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 27/29

PICTORIAL REPRESENTATION  41 1

direction of compromise. Some of the work discussed here suggests that perception

depends on both internal representations and attentional or other strategies. This

opens the door to a better understanding of picture perception and thus of pictorial

representation.

Notes

[1] Marr himself does not argue for a resemblance theory of depiction. As I will show later, if it is

separated from the claim that pictures represent in virtue of experienced similarities, this same

argument and theory of vision can be used to support a ªrecognitionº model of depiction. I treat

that as a type of causal theory. All of the arguments that follow against the modularity of vision

undercut causal theories that rely on it, as well as resemblance accounts.

[2] This is not to say that property terms mu st be like names. While the correct use of a property term

might be traced back to some original perceptual encounter with an instance of the property, the

causal relation that matters mostÐas we will seeÐis between properties and representations in

pictures (or minds) with which they typically covary.[3] In that case, the standard causal theory of reference is modi®ed, so that pictorial reference is

explained in terms of a perceptual theory in which reference depends on nondescriptive percep-

tual states, which are themselves individuated causally. Linking external causes to internal

perceptual responses in this way supports a generally functionalist construal of mental states; i.e.

the states are de®ned in terms of their causal powers, in relation to input. Thus they supervene

on brain states. But a functionalist ontology at that level of generality need not be taken to entail

a functional or conceptual role semantics in the sense described below. It is compatible with a

causal covariance account.

[4] Note that ªoften enoughº here need not mean ªtypically,º and the required correspondence need

not involve a causal relation only. A mental representation of a fatally poisonous mushroom mightbe produced most often by innocuous mushrooms. But the false positives are harmless, and the

true ones are vital, so the occasional co-occurrence of the representation with the poisonous type

is all that is needed to establish it as the representation’s content. Further, what matters in this

case is not so much the causal connection between the real mushroom and the representation of 

it, but the way in w hich the representation is used by the perceiverÐthe contribution it makes to

suc ce ssful pla nning, tho ught, a nd a c tion. T hus it is e asy to e xpla in ho w the c o ntent o f a

representation and its cause might be mismatched.

[5] The fact that mental images might play a role in perception, which I discuss below, is not enough

to support Walton’s make-believe theory of imagination.

[6] I defend constructivism below against this charge; but the defense depends on its being part of ahybrid account in which certain aspects of picture perception are explained in terms other than

those describing constructed mental representations.

[7] Note that one can grant this without asserting that these effects are entirely noncognitive. See

Rollins (1994).

[8] It is worth noting that Gibson’s theory is not a causal theory. This is in part what distinguishes

his use of the notion of information from that of Evans and Lopes, who describe recognitional

abilities in terms of ªinformational statesº that depend on a causal connection. As Dretske (1981,

p. 31) says: ªan event may carry information about events to which it stands in no regular causal

relationship.º Although Dretske’s use of the n otion o f information is later and more technical than

Gibson’s, they (and, indeed, Gombrich, who draws on Gibson) are, in this respect, alike. Evans

and Lopes also posit a technical analysis of information states, but not in the same sense as

Dretske’s.

[9] Of course, to say the pictures convey the ªsame informationº as the scenes they depict is not to

say that a picture resembles the scene. The same information can be embodied in different forms.

Further, the claim of resemblance is driven by the purported fact that picture perception and

ordinary object perception depend on the same type of mental representations. There are no such

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 28: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 28/29

41 2 M . ROLLINS

representations on Gibson’s account. Finally, perceived similarities between pictures and objects

do not, for him, explain the perceiver’s response to the picture, which is a matter of picking up

the information in it.

[10] A similar appropriation of ªconventionº can be seen in Schier’s claim that interpreting some

visual art, namely, ®ctive art, depends on an acceptance of rules of a game of pretending, as

Walton suggests. These special cases are nonetheless set against a background perceptual theory

in terms of recognition.

[11] It might be argued that conventional symbols at one end of the continuum are ªarbitraryº in the

way that Goodman suggests. After all, in another tradition, a lamb might represent the secular 

government, and a snake could stand for Christ. But pictorial symbols are associated with speci®c

qualities (e.g. meekness or benevolence), the understanding of w hich requires a recognition of 

their perceptible properties. For Gombrich, this means that the psychology of illusion must be

brought into play: the marks on the canvas must ®rst be seen as a lamb. Further, symbols function

only in the context of a system of beliefs. And Gombrich proposes a distinct psychological process

by which perceptual beliefs are acquired. In making this empirical claim, he departs from

Goodman.

[12] As Goodman suggests. But pictures are still not conventional, on this account, in the fourth and

strongest sense; they are not arbitrary.

[13] I should point out that Kosslyn, too, employs the notion of strategy to explain imaging and

perceiving; indeed, it plays a central role in his account. Where he disagrees with Chambers and

Reisberg is regarding the bearing of that fact on the claim that imaging is like perceiving. See

Rollins (forthcoming) for a discussion.

References

A RNHEIM , R. (1966). Art and visual perception. Berkeley, CA: University of California Press.

B ALLARD, D. (1991). Animate vision. Arti®cial Intelligence, 48, 57±86.B IEDERMAN , I. (1987). Recognition-by-Components: a theory of hum an image understanding. Psycho-

logical Review, 94 , 115±147.

B LOCK  , N. (1983). The photographic fallacy in the debate about mental imagery. Nous, 17 , 651±661.

B R UNER  , J. (1957). On perceptual readiness. Psychological Review, 64, 123±152.

C HAMBERS, D. & R  EISBERG , D. (1985). Can mental images be ambiguous? Journal of Experimental 

Psychology: Human Perception and Performance, 11 , 317±328.

C H UR CH LA ND, P. (1988). Perceptual plasticity and theoretical neutrality: a reply to Jerry Fodor.

Philosophy of Science, 55 , 167±187.

C UTTING , J. (1986). Perception with an eye for motion. Cambridge MA: MIT Press.

DA NTO

, A. (1981). The trans®guration of the commonplace. Cambridge MA: Harvard University Press.D A NTO, A. (forthcoming). Seeing and showing. Journal of Aesthetics and Art Criticism.

D R ETSK E , F. (1981). Knowledge and the ¯ow of information. Cambridge MA: MIT Press.

D R ETSK E , F. (1986). Misrepresentation. In R. B OGM A N  (Ed.) Belief: form, content, and function. Oxford:

Oxford University Press.

E VANS, G. (1982). In J. M CD OWELL  (Ed.) Varieties of reference. Oxford: Oxford University Press.

F ODOR , J. (1984). Observation reconsidered. Philosophy of Science, 51, 23±43.

F ODOR , J. (1990). A theory of content and other essays. Cambridge MA: MIT Press.

F ODOR , J. (1991). Replies. In B. L OEWER   & E. L E PO RE  (Eds) Meaning and mind: Fodor and his critics.

Oxford: Blackwell.

G IBSON , J.J. (1978). The ecological approach to the perception of pictures. Leonardo, 11 , 227±235.

G ILMAN , D. (1992). A new perspective on pictorial representation. Australian Journal of Philosophy, 70 ,

174±186.

G OMBRICH, E. (1960). Art and illusion. Princeton, NJ: Princeton University Press.

G OODM A N , N. (1976). Languages of art . Indianapolis: Hackett

 J ACOBY , L., BA K ER  , J.G. & B R OOK S, L. (1989). episodic effects on picture identi®cation: implications for 

theories of concept learning and theories of memory. Journal of Experimental Psychology, 2 , 275±281.

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3

Page 29: Rollins 1999_Pictorial Representation

7/29/2019 Rollins 1999_Pictorial Representation

http://slidepdf.com/reader/full/rollins-1999pictorial-representation 29/29

PICTORIAL REPRESENTATION  41 3

K OSSLYN , S. (1994). Image and brain. Cambridge, MA: MIT Press.

L EHKY , S . R & SEJNOWSKI, T.J. (1988). Network model of shape from shading. Nature, 333 , 452±454.

L OPES, D. (1993). Understanding pictures. Oxford: Oxford University Press.

M AR R , D. (1982). Vision. San Francisco: Freeman.

M ILLIKAN , R. (1984). Language, thought, a nd other biological categories. Cambridge, M A: MIT Press.

PEACOCKE , C. (1992). A study of concepts. Cambridge, MA: MIT Press

POSNER  , M. & K EELE , S.W. (1968). On the genesis of abstract ideas.Journal of Experimental Psychology

,77

,

353±363.

R AFFMAN , D. (1993). Language, music, and mind . Cambridge, MA: MIT Press.

R AMACHANDRAN , V.S. (1990). Interaction between motion, depth, color, and form: the utilitarian theory

of perception. In C. BLA K EM OR E (Ed.) Vision: coding and ef®ciency. Cambridge: Cambridge Univer-

sity Press.

R OLLINS, M. (1994). Deep plasticity: a general encoding approach. Philosophy of Science, 61 , 39±59.

R OLLINS, M. (forthcoming). The strategic eye: Kosslyn’s theory of imagery and perception. Mind and 

Machines.

SCHIER , F. (1986). Deeper into pictures. Cambridge: Cambridge University Press

SPOLSKY , E. (1993). Gaps in nature: literary interpretation and the modular mind . Albany: State University

of New York Press.

T H OM A S, L. (1997). Junk representations. British Journal of Philosophy of Science, 48, 345±361.

T RIESMAN , A.M. & G ELA DE , G. (1980). A feature integration theory of attention. Cognitive Psychology,

12 , 97±136.

W A LTON , K. (1984). Critical inquiry. Chicago: University of Chicago Press.

W ARREN , C. & M  OR TON , J. (1982). The effects of priming on picture perception. British Journal of 

Psychology, 73 , 117±129.

W OLLHEIM , R. (1973). Re¯ections on Art and Illusion. In On art and the mind . Cambridge, MA: Harvard

University Press.

W OLLHEIM , R. (1993). Danto’s gallery of indiscernibles. In M. R OLLINS (Ed.) Danto and his critics.

Oxford: Blackwell.

 D o w n l o a d e d b y

 [ U Z H  H a u p t b i b l i o t h e k / Z e n t r a l b i b l i o t h e k Z ü r i c h ] a t 1

 1 : 4 7 0 4 J u l y 2 0 1 3