75
What are some of the first What are some of the first steps in connecting vision steps in connecting vision with the world with the world The central operation is that of “picking out” or selecting and the usual mechanism that is appealed to in explaining this selection is attention (sometimes called focal attention or selective attention). Why do we need to select? This is a nontrivial question and we will consider several different answers: We need to select because we can’t process all the information available. This is the resource-limitation reason (channel capacity). We need to select because of the way relevant information in the world is packaged. It gives rise to the Binding Problem We need to select because certain patterns cannot be computed without first selecting (“marking”) certain elements of a scene We need to select because selection is the first line of contact between the mind and the world – and precedes all conceptualizing and encoding <But I will not talk about that in this class>

What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Embed Size (px)

Citation preview

Page 1: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

What are some of the first steps in What are some of the first steps in connecting vision with the worldconnecting vision with the world

The central operation is that of “picking out” or selecting and the usual mechanism that is appealed to in explaining this selection is attention (sometimes called focal attention or selective attention).

Why do we need to select? This is a nontrivial question and we will consider several different answers: We need to select because we can’t process all the information

available. This is the resource-limitation reason (channel capacity). We need to select because of the way relevant information in the

world is packaged. It gives rise to the Binding Problem We need to select because certain patterns cannot be computed

without first selecting (“marking”) certain elements of a scene We need to select because selection is the first line of contact

between the mind and the world – and precedes all conceptualizing and encoding <But I will not talk about that in this class>

Page 2: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Attention as SelectionAttention as Selection Focus on the Selection or Filtering aspects. Ask yourself:

1. Why do we need to select anyway? Because our processing capacity is limited?

The Big Question: In what way is it limited? (Miller, 1957) We will return to this core question after some preliminaries on

the early study of attention as selection and the filter theory.

2. On what basis do we select? Some alternatives: We select according to what is important to us (e.g., affordances) We select what can be described physically (i.e., “channels”) We select based on what can be encoded without accessing LTM We “pick out” things to which we subsequently attach concepts:

i.e., we pick out objects (but what do we do if they move?)

3. What happens to what we have not selected?

Page 3: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Big Question #1: Why do we Big Question #1: Why do we need to select anyway?need to select anyway?

Human information processing is limited. But along what dimensions (in what respect) is it limited?

Channel capacity: Shannon-Hartley Theorem

Capacity measured in some sort of “chunks” (Miller) Capacity measured in terms of the number of

arguments that can be simultaneously bound to cognitive routines (Newell)

To what things in the world can the arguments of visual predicates be bound?

Page 4: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Amount of information in terms of the Amount of information in terms of the Information-theoretic measure (entropy)Information-theoretic measure (entropy)

Amount of information in a signal depends on how much one’s estimate of the probability of events is changed by the signal.

H = -pi Log2 (pi) … information in bits

“One of by land, two if by sea” contains one bit of information if the two possibilities were equally likely, less if they were not (e.g., if one was twice as likely as the other the information in the message would be ⅓ Log ⅓ + ⅔ Log ⅔ = 0.92 bits)

The amount of information transmitted depends on the potential amount of information in the message and the amount of correlation between message sent and message received. So information transmitted is a type of correlation measure (without regard to any ordinal properties of messages).

Page 5: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Information transmitted in a typical Information transmitted in a typical absolute judgment experimentabsolute judgment experiment

Information transmitted in an experiment in which subjects were presented with tones drawn from a known practiced set (of a given size, which determines the value of input information) and had to name the tones from a learned name set.

The information transmitted was always around 2.5 bits or an average of 6.25 equiprobable alternatives!

Page 6: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Example of the use of Example of the use of chunkingchunking

•To recall a string of binary bits – e.g., 00101110101110110101001

•People can recall a string of about 8 binary integers. If they learn a binary encoding rule (000, 011, 102, 113) they can recall about 8 such chunks or 18 binary bits. If they learn a 3:1 chunking rule (called the Octal number system) they can recall a 24 bit string, etc

Binary

Octal

Hex

…..?

Page 7: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Why can we retain vastly different amounts Why can we retain vastly different amounts of “information” just by using a different of “information” just by using a different

encoding vocabulary size?encoding vocabulary size? Answer: The architecture of the cognitive system has the

property that it can deal with a fixed maximum number of items, regardless of what the items are.

This property can be exploited to get around the bottleneck of the short-term memory. We do this by recoding the input into a smaller number of discrete units, called chunks.

There is also evidence that it takes additional time to encode and decode chunks, so the recoding technique is a case of time-capacity tradeoff or what is known in CS as a compute-vs-store tradeoff. Allan Newell’s novel model to account for the time taken

in the Sternberg memory scan experiment attributes the observed RT to encoding or chunking.

Page 8: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Early studies of attention: The Early studies of attention: The “Cocktail Party Problem”“Cocktail Party Problem”

What determines how well you can select one conversation among several? Why are we so good at it?

The more controlled version of this study used dichotic presentations – one “channel” per ear.

It was found that when attention is fully occupied in selecting information from one ear (through use of the “shadowing” task), almost nothing is noticed in the “rejected” ear.

More careful observations shows this was not quite true Change in spectral properties (pitch) is noticed You are likely to notice your name spoken Even meaning is extracted, as shown by involuntary ear

switching and disambiguating effect of rejected channel content

Page 9: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Broadbent’s Filter TheoryBroadbent’s Filter Theory

Broadbent, D. E. (1958). Perception and Communication. London: Pergamon Press.

Limited Capacity Channel

Effectors

Store of conditional probabilities of past events (in LTM)

Filt

erMotor planner

Ver

y Sh

ort T

erm

Sto

re

Sens

es

Rehearsal loop

Page 10: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Problems with the Filter TheoryProblems with the Filter Theory The filter “leaks.” Work by Treisman, Lackner, and many others

shows that the filter could not be eliminating parts of the input using a physically-defined channel, because the properties on the basis of which the input is filtered require a high level of processing (e.g., determination of meaning). Consequently such information must have to have gotten through the filter!

Many solutions to this conundrum have been proposed, none of which are entirely satisfactory, but each of which embodies some ideas that may be part of the story.

What all these alternatives do is assume that the filter is responsive to top-down expectancy and prediction effects. But the evidence is against this sort of knowledge-based selection as a general property of perception (Pylyshyn, 1999), since perception is a modular function (i.e., early stages of vision are insensitive to cognitive factors – they are cognitively impenetrable)

Page 11: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Visual analogues illustrating the Visual analogues illustrating the two-channel selection problemtwo-channel selection problem

In these examples you are to read only the text in shadows and ignore the rest. Read as quickly as you can and when you are finished, close your eyes or look away from the text.

Page 12: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Visual analogue #1 illustrating the Visual analogue #1 illustrating the two-channel selection problemtwo-channel selection problem

In performing an experiment like this one on man attention car it house is boy critically hat important she that candy the old material horse that tree is pen being phone read cow by book the hot subject tape for pin the stand relevant view task sky be read cohesive man and car gramatically house complete boy but hat without shoe either candy being horse so tree easy pen that phone full cow attention book is hot not tape required pin in stand order view to sky read red it nor too difficult.

Page 13: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Visual analogue #2 illustrating the Visual analogue #2 illustrating the two-channel selection problemtwo-channel selection problem

It is important that the subject man be car pushed slightly boy beyond that his normal limits horse of tree competence open for be only in phone this cow way book can hot one tape be pin certain stand that snaps he with is his paying teeth attention in to the the empty relevant air task and rather minimal than to the attention candy to horse the tree second or peripheral task.

Page 14: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Stroop EffectStroop EffectBaseline: Name the colors of the inkBaseline: Name the colors of the ink

Page 15: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Stroop Effect in English Stroop Effect in English Name the colors of the inkName the colors of the ink

RED GREEN BLUE PINK BROWN ORANGE GREEN PINK RED YELLOW GREEN YELLOW RED BROWN RED BLUE BROWN GREEN RED ORANGE RED BLUE YELLOW PINK ORANGE GREEN BLUE BROWN PINK RED YELLOW GREEN YELLOW RED BROWN PINK RED YELLOW GREEN YELLOW RED PINK ORANGE GREEN BLUE BROWN PINK RED YELLOW GREEN YELLOW RED BROWN RED BLUE GREEN BROWN YELLOW GREEN YELLOW RED PINK ORANGE GREEN RED BLUE BROWN GREEN RED ORANGE RED BLUE YELLOW YELLOW GREEN YELLOW RED BROWN PINK RED YELLOW GREEN PINK RED YELLOW

Page 16: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Stroop Effect in PortugueseStroop Effect in Portuguese Name the colors of the inkName the colors of the ink

VERMELHO VERDE AZUL MARROM ROSA ALARANJADO VERDE ROSA VERMELHO AMARELO VERDE AMARELO VERMELHO MARROM VERMELHO AZUL MARROM VERDE VERMELHO ALARANJADO VERMELHO AZUL AMARELO ROSA ALARANJADO VERDE AZUL MARROM ROSA VERMELHO AMARELO VERDE AMARELO VERMELHO MARROM ROSA VERMELHO AMARELO VERDE AMARELO VERMELHO ROSA ALARANJADO VERDE AZUL MARROM ROSA VERMELHO AMARELO VERDE AMARELO VERMELHO BROWN VERMELHO AZUL MARROM VERDE AMARELO VERDE AMARELO VERMELHO ROSA ALARANJADO VERDE VERMELHO AZUL MARROM VERDE VERMELHO ALARANJADO VERMELHO AZUL

Page 17: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Degree of Interference of the attended Degree of Interference of the attended message, as well as its interpretation, shows message, as well as its interpretation, shows that the rejected message was that the rejected message was understoodunderstood Moral: Although the rejected channel appears to be rejected,

it is being processed enough to understand the words! The semantic interpretation of attended message depends on

the meaning content of the rejected message. Subjects were asked to paraphrase the attended message in: Channel 1 (attended): “I think I will go down to the bank but I will

be back for dinner” Channel 2 (rejected): “The election results will depend on the value

of the dollar against the Euro and on the state of the domestic economy”

OR Channel 2 (rejected): “The rain has resulted in erosion by the overflowing river”

(Lackner, J. R., & Garrett, M. F. (1972). Resolving ambiguity: Effects of biasing context in the unattended ear. Cognition, 1, 359-372.)

Page 18: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

The special case of visual attentionThe special case of visual attention

Visual working memory and visual selection What is the nature of the input, storage and

information processing limitations in vision?

Page 19: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Studies of the capacity of Visual Studies of the capacity of Visual Working MemoryWorking Memory (Luck & Vogel, 1997)(Luck & Vogel, 1997)

People appear to be able to retain about 4 properties of an object (4 colors, 4 shapes, 4 orientations, etc) over a short time

People can also retain the identity of 4 objects for a short time.

Luck and Vogel found that as long as there are not more than 4 properties per object, people can retain large numbers of properties (a phenomenon that is reminiscent of Miller’s “chunking hypothesis” except the chunks are objects).

Page 20: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Luck & Vogel findingLuck & Vogel finding

People can retain about 4 properties of a visual display in their VSTM

People can retain the identity of about 4 objects in their VSTM

If the properties are associated with different objects people can retain 4 properties per object – a much higher total number of properties

Page 21: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

What does What does visualvisual attention select? attention select? (What are the bases for selection?)(What are the bases for selection?)

If attention is selection, what does visual attention select? An obvious answer is places. We can select places by moving

our eyes so our gaze lands on different places. When places are selected, are they selected automatically? Must we always move our eyes to change what we attend to?

Studies of Covert Attention-Movement: Posner (1980). How does attention switch from one place to another? Is it always the case that we attend to places? Can we attend to

any other property? Can we select on the basis of color, depth, spatial frequency, affordances, or the property a painting has of having been painted by Da Vinci (A property to which Bernard Berenson was able to attend extremely well). cf Gibson

Page 22: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

How else can visual attention select? How else can visual attention select?

Can we control the size and shape of the region that is selected, or is selection always punctate and data-driven? Zoom Lens model of spatial attention (Eriksen & St James, 1986). We control where attention moves:

Is this automatic or voluntary? How do we know where to direct our attention? How do we

specify a location prior to attending to it? We need a way to specify where or what prior to attending to it!

Keep this conundrum in mind – we will return to it later! How narrowly can we focus our attention? Can we make it pick

out one out of several objects? Are there special conditions under which we are able to pick out

individual things? We will return to “attentional resolution” or the minimum spacing for selecting individual things.

Page 23: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Covert movements of attentionCovert movements of attention

Example of an experiment using a cue-validity paradigm for showing that the locus of attention moves without eye movements and for estimating its speed. Posner, M. I. (1980). Orienting of Attention. Quarterly Journal of Experimental Psychology, 32, 3-25.

Page 24: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Recall Posner’s demonstration of exogenous attention switchRecall Posner’s demonstration of exogenous attention switch

Does the improved detection in intermediate locations entail that the “spotlight of attention” moves continuously through empty space?

Uncued

Cued

CueFixationframe

Target-cueinterval Detection target

*

Along thepath

*

*

Page 25: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Sperling & Weichselgartner (1995) “Episodic” or Sperling & Weichselgartner (1995) “Episodic” or Quantal Theory of Attention switching Quantal Theory of Attention switching

Assumes a quantal “shift” in attention in which the spotlight pointed at location -2 is extinguished and, simultaneously, the spotlight at location +2 is turned on. Because extinction and onset take a measurable amount of time, there is a brief period when the spotlights partially illuminate both locations simultaneously.

Page 26: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

But there are empirical reasons why But there are empirical reasons why objectsobjects are a are a better target for attentional selection than better target for attentional selection than locationlocation

There is experimental evidence that attention attaches to things rather than places

The Posner evidence of analog movement of focal attention, when attention is exogenously summoned, can be explained by a punctate object-based theory of attention-allocation – Sperling & Weichselgartner (1995)

Page 27: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

This This object-basedobject-based view of attentional view of attentional selection is an important recent discoveryselection is an important recent discovery

There are good reasons on both empirical and theoretical grounds for supposing that attention attaches itself to objects rather than locations

But first let’s look at some other ways that attention can be allocated in vision

Page 28: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

We can select a shape even when it is We can select a shape even when it is intertwined among other similar shapesintertwined among other similar shapes

Are the green items the same? On a surprise test at the end, subjects were not able to recall shapes that had been present but had not been attended in the task

But this should not be possible if we allocate attention to locations

Page 29: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

The time-course of attention:The time-course of attention:Inhibition of returnInhibition of return

If we vary the time between the cue and target in a modified Posner paradigm, we find that when the Cue-Target-Onset-Asynchrony (CTOA) gets to around 300-900 ms, reaction time to the target begins to increase. This is called Inhibition-of-return (Klein, 2000).

To get this effect we actually have to attract attention to the target location and then attract it back to the origin. IOR is one of many examples of an inhibition effect being produced by attention.

Page 30: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Other examples of attentionally Other examples of attentionally induced inhibitioninduced inhibition

Negative Priming (Treisman & DeShepper, 1996).

Is there a figure on the right that is the same shape as the figure on the left? When the figure on the left is one that had appeared as an ignored figure on

the right in a previous trial, Reaction Time is longer and accuracy poorer. This “negative priming” effect persisted over 200 intervening trials and was

reported to last for a month!

Page 31: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Another negative Another negative attention effect: attention effect: Inattentional Inattentional BlindnessBlindness

Page 32: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Inattentional BlindnessInattentional Blindness The background task is to report which of two arms of the + is

longer. One critical trial per subject, after about 3,4 background trials. Another “critical” trial presented as a divided attention control.

25% of subjects failed to see the square when it was presented in the parafovea (2° from fixation).

But 65% failed to see it when it was at fixation!

When the background task cross was made 10% as large, Inattentional Blindness increased from 25% to 66%.

It is not known whether this IB is due to concentration of attention at the primary task, or whether there is inhibition of outside regions.

Mack & Rock (1988)

Page 33: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

In what other ways might our visual In what other ways might our visual information capacity be limited?information capacity be limited?

There are obviously limitations on the input side of vision that depend on the acuity of the sensors and the range of physical properties to which they respond.

But there is a limitation beyond that of acuity: The perceptual system is limited in what it can individuate and how many of these individuals it can deal with at one time. The capacity to individuate is different from the capacity to discriminate. Some reason for thinking that individuating is a distinct

process

Page 34: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Exploring the limits of attention and the Exploring the limits of attention and the units over which selection operatesunits over which selection operates

It appears that the human information-processing bottleneck cannot be expressed perspicuously in terms of information-theoretic measures, nor can it be specified in physical parameters (e.g., in terms of locations or spatio-temporal regions), although such measures often do capture important aspects of attention (e.g., visual attention often moves continuously through space).

But there are other possible ways one might consider expressing the limits of attention. Over the past 25 years evidence has been accumulating that the

human attention system is, at least in part, tuned to individual objects in the world. This would certainly make sense from an evolutionary perspective. But what does this mean?

Page 35: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

The increasingly important role played by The increasingly important role played by ‘Objects’‘Objects’ in studies of visual attention in studies of visual attention

Miller’s ‘Magic Number 7’ has continued to haunt us even beyond studies of short-term memory (STM).

There is a limitation in visual information processing that is beyond the limitation of acuity and of STM capacity: The perceptual system is limited in what it can individuate and how many of these individuals it can deal with at one time.

The capacity to individuate is different from memory capacity and discrimination capacity.

This notion of individuating and of individuals may be related to Miller’s “chunks”, but it has a special role in vision which I can only sketch very briefly at this time

First some reasons why individuating is a distinct process

Page 36: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Individuating is different from discriminatingIndividuating is different from discriminating

Page 37: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Individuals and patternsIndividuals and patterns Vision does not recognize patterns by applying a template

but by parsing in into parts (recognition-by-parts) A pattern is encoded over time (and often over saccades),

therefore the visual system must keep track of the individual parts and merge descriptions of the same part at different times and stages of encoding

Thus in order to recognize a pattern, the visual system must pick out individual parts and bind them to the representation being constructed – keep track of them

It must do so before it has recognized any properties of the parts – it must individuate prior to recognizing Examples include what are called “visual routines”

Page 38: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Are there collinear items (n>3)?Are there collinear items (n>3)?

Page 39: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Several objects must be picked out at Several objects must be picked out at once in making relational judgmentsonce in making relational judgments The same is true for other relational judgments

like inside or on-the-same-contour… etc. We must pick out the relevant individual objects first.

Page 40: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

When items cannot be individuated, When items cannot be individuated, patterns over them cannot be recognizedpatterns over them cannot be recognized Do these figures contain one or two distinct curves? Individuating these curves requires a “curve tracing” operation, so Number_of_curves (C1, C2, …) takes time proportional to the length of the shortest curve.

Page 41: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

The figure on the left is one continuous The figure on the left is one continuous curve, the one on the right is two distinct curve, the one on the right is two distinct

curves – as shown in color.curves – as shown in color.

Page 42: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Another example: Subitizing Another example: Subitizing vsvs Counting. Counting. How many squares are there? How many squares are there?

Subitizing Subitizing is fast, accurate and only slightly is fast, accurate and only slightly dependent on how many items there aredependent on how many items there are. Only the . Only the

squares on the right can be subitized.squares on the right can be subitized.

Concentric squares cannot be subitized because individuating them requires curve tracing, just as it did in the previous example.

Page 43: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Signature subitizing phenomena only Signature subitizing phenomena only appear when objects are automatically appear when objects are automatically

individuated and indexedindividuated and indexed

Page 44: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Example of subitizing popout Example of subitizing popout and non-popout featuresand non-popout features(Count Pink vs. Count Online)(Count Pink vs. Count Online)

Page 45: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Encoding conjunctions of propertiesEncoding conjunctions of properties

Experiments showing the special difficulty that vision has in detecting conjunctions of several properties have provided a basis for understanding an important problem in in visual analysis

Page 46: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

How are conjunctions of features detected?How are conjunctions of features detected?

Read the vertical line of digits in the following display

Under these conditions Conjunction Errors are very frequent

Page 47: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Rapid visual search Rapid visual search (Treisman)(Treisman)

Find the following simple figure in the next slide:

Page 48: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

This case is easy – and the time is independent of how many nontargets there are – because there is only one red item. This is called a ‘popout’ search

Page 49: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

This case is also easy – and the time is independent of how many nontargets there are – because there is only one right-leaning item. This is also a ‘popout’ search.

Page 50: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Rapid visual search Rapid visual search (conjunction)(conjunction)

Find the following simple figure in the next slide:

Page 51: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism
Page 52: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Single Feature Single Feature vsvs Conjunction-feature search Conjunction-feature search

Page 53: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Serial vs parallel search?Serial vs parallel search? Finding an element that differs from all others in a scene

by a single feature – called a single-feature search – is fast, error-free and almost independent of how many nontargets there are;

Finding an object that differs from all others by a conjunction of two or more features (and that shares at least one feature with each object in the scene) – called a conjunction search – is usually slow, error-prone, and is worse the more nontargets there are in the scene*.

These results suggest that in order to find a conjunction, which requires solving the binding problem, attention has to be scanned serially to all objects.

* This way of putting is simplifies things. Under certain conditions the serial-parallel distinction breaks down

Page 54: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

The ‘The ‘attention-as-glue’ attention-as-glue’ hypothesis has a hypothesis has a corollary: in computing conjunctions of corollary: in computing conjunctions of properties attention must be directed properties attention must be directed

primarily at primarily at objects objects since it is objects that since it is objects that have the conjoined propertieshave the conjoined properties

Instead of being like a spotlight beam that can be scanned around a scene and can be zoomed to cover a larger or smaller area, perhaps attention can only be directed towards occupied places – i.e., to visual objects.

Page 55: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

An alternative view of how we An alternative view of how we solve the binding problemsolve the binding problem

If we assume that only properties of indexed objects are encoded and stored in Object Files, then properties that belong to the same object are stored in the same Object File, so the binding problem does not arise This is the Object-Based Attention view exemplified by

FINST Theory

The assumption that only properties of indexed objects are encoded raises the problem of what happens to properties of the other (unindexed) objects or unencoded properties in a display

I will return to this conundrum later.

Page 56: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

What happens to unattended What happens to unattended objects in vision (esp tracking)?objects in vision (esp tracking)?

There are three possibilities1. No properties other than of indexed objects are encoded

It may be that the richness of visual phenomenology is illusory! Visual information without experience & vice-versa

2. Other properties are encoded by are only available within modules (e.g., two visual systems)

3. Unattended (unindexed) objects are tracked but access to them is inhibited

Mack & Rock MOT research

Page 57: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Evidence for attentional selection Evidence for attentional selection based on based on ObjectsObjects

Single Object Advantage: pairs of judgments are faster when both apply to the same perceived object

Entire objects acquire enhanced sensitivity from focal attention to a part of the object

Single-Object advantage occurs even with generalized “objects” defined in feature space

Simultanagnosia and hemispatial neglect show object-based effect

Attention moves with Moving Objects IOR Object Files MOT

Page 58: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Single-object superiority even Single-object superiority even when the shapes are controlledwhen the shapes are controlled

Page 59: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

““Objects” endure over timeObjects” endure over time

Several studies have shown that what counts as an object (as the same object) endures over time and over changes in location; Certain forms of disappearances in time and

changes in location preserve objecthood.

This gives what we have been calling a “visual object” a real physical-object character and partly justifies our calling it an “object”.

Page 60: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Yantis use of the “Ternus Configuration” to Yantis use of the “Ternus Configuration” to demonstrate the early visual effect of demonstrate the early visual effect of

objecthoodobjecthoodShort time delays result in “element motion”(the middle object persists as the “same object” so it does not appear to move)

Page 61: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Long time delays result in “group motion” because Long time delays result in “group motion” because the middle object does not persist but is perceived the middle object does not persist but is perceived

as a new object each time it reappearsas a new object each time it reappears

Page 62: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Inhibition of return appears Inhibition of return appears to be object-basedto be object-based

Recall that Inhibition-of-return is the phenomenon whereby an object that has been attended (and then attention is moved away from it) is less likely to attract attention again in a period of 300 ms to 900 ms after it is first attended. The attended item is said to be inhibited. This is thought to help in visual search since it prevents

previously visited objects from being revisited

The original study used static objects. Then

(Tipper, Driver & Weaver, 1991) showed that IOR

moves with the inhibited object.

Page 63: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Inhibition of Return appears to be object-based Inhibition of Return appears to be object-based (it travels with the object that was attended)(it travels with the object that was attended)

Page 64: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Simultanagnosic (Balint Syndrome) patients Simultanagnosic (Balint Syndrome) patients only attend to one object at a timeonly attend to one object at a time

Simultanagnosic patients cannot judge the relative length of twolines, but they can tell that a figure made by connecting the endsof the lines is not a rectangle but a trapezoid (Holmes & Horax, 1919).

Page 65: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Multiple Object TrackingMultiple Object Tracking

One of the clearest cases illustrating object-based attention is Multiple Object Tracking

Keeping track of individual objects in a scene requires a mechanism for individuating, selecting, accessing and tracking the identity of individuals over time These are the functions we have proposed are carried out by

the mechanism of visual indexes (FINSTs)

We have been using a variety of methods for studying visual indexing, including subitizing, subset selection for search, and Multiple Object Tracking (MOT).

Page 66: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Multiple Object TrackingMultiple Object Tracking In a typical experiment, 8 simple identical objects are

presented on a screen and 4 of them are briefly distinguished in some visual manner – usually by flashing them on and off.

After these 4 “targets” have been briefly identified, all objects resume their identical appearance and move randomly. The subjects’ task is to keep track of which ones had earlier been designated as targets.

After a period of 5-10 seconds the motion stops and subjects must indicate, using a mouse, which objects were the targets.

People are very good at this task (80%-98% correct). The question is: How do they do it?

Page 67: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Keep track of the objects that flash

Page 68: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

How do we do it? What properties of individual objects do we use?

Page 69: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Keep track of the objects that flash

Page 70: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

How do we do it? What properties of individual objects do we use?

Page 71: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Basic finding: People (even 5 year old children) can track 4 to 5 individual objects that have no unique visual properties

How is it done?

We have shown that it is unlikely that the tracking is done by keeping a record of target locations, and updating them while serially visiting the objects.

I have proposed that individuating and keeping track of certain kinds of individuals is a primitive visual operation and uses the mechanism of visual indexes or FINSTs.

Explaining Multiple Object TrackingExplaining Multiple Object Tracking

Page 72: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Schema for how FINSTs Schema for how FINSTs function in visual-motor controlfunction in visual-motor control

Page 73: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

The study of Visual IndexingThe study of Visual Indexingis the main goal of my researchis the main goal of my research

It happens that Visual Indexes are of special interest to philosophers concerned with how our conceptual representation of the world can connect with particular things in the perceived world

More on another occasion!!

Page 74: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism

Don’t forget to check the Don’t forget to check the UPDATE link under the UPDATE link under the

Proseminar for your test!!Proseminar for your test!!The instructions are on the linked pageThey include the instruction to select 2 of the 4

questions and to write no more than 300 words on each

Submit answers to [email protected](one answer per email, with the subject field indicating which question you chose to answer)

Page 75: What are some of the first steps in connecting vision with the world The central operation is that of “picking out” or selecting and the usual mechanism