94
Bottom-up and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute for Brain Science Center for Vision Research

Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

  • Upload
    lamdan

  • View
    221

  • Download
    3

Embed Size (px)

Citation preview

Page 1: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bottom-up and top-down processing in visual perception

Thomas Serre

Brown University

Department of Cognitive & Linguistic SciencesBrown Institute for Brain Science Center for Vision Research

Page 2: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘what’ problem monkey electrophysiology

Page 3: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘what’ problem monkey electrophysiology

Willmore Prenger & Gallant ’10

Page 4: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘what’ problem monkey electrophysiology

Willmore Prenger & Gallant ’10

shape factors (if any) are uniquely and consistently associatedwith neural responses. This has not been attempted before becauseof the intractable size of three-dimensional shape space. In this

virtually infinite domain, a conventional random or systematic(grid-based) stimulus approach can never produce sufficiently densecombinatorial sampling.

–1–1

a b

Response = 0.4A + 0.0B + 49.0AB + 0.0

38

10

0

1

Relative x position

Rel

ativ

e y

posi

tion

Rel

ativ

e y

posi

tion

Relative z positionMaximum curvature

Min

imum

cur

vatu

re

0

180

180 360

Angle on xy plane (deg)

Ang

le o

n yz

pla

ne

(deg

)

0

i

5 deg

c d

0

20

40

Rotation angle (deg)900–90–4 –2 0 2 4

10

20

30

Depth (deg)

0 0

10

20

30

1 2.10.48Size(x)x position (deg)

y po

sitio

n (d

eg)

–10 100

–10

10

0

0

10

20

30

0

10

20

30

Shadin

g

Textu

reNon

e

0

10

20

30

e f g h

Textu

reNon

e

Stereo Nonstereo

900–900

10

20

30

Light angle (deg)Sha

ding

0

10

20

30

0

10

20

30

900–90

900–90

10

20

30

0

10

20

30

0

–4 –2 0 2 4

–4 –2 0 2 4

0

10

20

30

0

10

20

30

1 2.10.48

1 2.10.48

0

20

40

0

20

40

900–90

900–90

–10

10

0

–10

10

0

–10 100

–10 100

40

j

0 spikes s–1

Res

pons

e (s

pike

s s–1

)

Res

pons

e (s

pike

s s–1

)

Res

pons

e (s

pike

s s–1

)

Res

pons

e (s

pike

s s–1

)0 spikes s–1

Res

pons

e (s

pike

s s–1

)

1–1 0 1–1

0

1

–1 0 1–1

0

1

Figure 2 Neural tuning for three-dimensional configuration of surface fragments. (a) Top 50 stimuli across eight generations (400 stimuli) for a single ITneuron recorded from the ventral bank of the superior temporal sulcus (17.5 mm anterior to the interaural line). (b) Bottom 50 stimuli for the same cell.(c) Responses to highly effective (top), moderately effective (middle) and ineffective (bottom) example stimuli as a function of depth cues (shading, disparityand texture gradients, exemplified in Supplementary Fig. 10). Responses remained strong as long as disparity (black, green and blue) or shading (gray) cueswere present. The cell did not respond to stimuli with only texture cues (pale green) or silhouettes with no depth cues (pale blue). (d) Response consistencyacross lighting direction. The implicit direction of a point source at infinity was varied across 1801 in the horizontal (left to right, black curve) and vertical(below to above, green curve) directions, creating very different two-dimensional shading patterns (Supplementary Fig. 11). (e) Response consistency acrossstereoscopic depth. In the main experiment, the depth of each stimulus was adjusted so that the disparity of the surface point at fixation was 0 (that is, theanimal was fixating in depth on the object surface). In this test, the disparity of this surface point was varied from !4.51 (near) to 5.61 (far). (f) Responseconsistency across xy position. Position was varied in increments of 4.51 of visual angle across a range of 13.51 in both directions. (g) Sensitivity to stimulusorientation. As with all neurons in our sample, this cell was highly sensitive to stimulus orientation, although it showed broad tolerance (about 901) to rotationabout the z axis (rotation in the image plane, blue curve); this rotation tolerance is also apparent among the top 50 stimuli in a. Rotation out of the imageplane, about the x axis (black) or y axis (green), strongly suppressed responses. (h) Response consistency across object size over a range from half to twice thatof the original stimulus. (i) Linear/nonlinear response model based on two Gaussian tuning functions (details as in Fig. 2d,e). (j) The tuning functions areprojected onto the surface of a high-response stimulus, seen from the observer’s viewpoint (left) and from above (right). Error bars indicate s.e.m. in all panels.

1354 VOLUME 11 [ NUMBER 11 [ NOVEMBER 2008 NATURE NEUROSCIENCE

ART ICLES

Yamane Carlson Bowman Wang & Connor ’08

Page 5: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘what’ problem monkey electrophysiology

Hung Kreiman et al ’05

Willmore Prenger & Gallant ’10

shape factors (if any) are uniquely and consistently associatedwith neural responses. This has not been attempted before becauseof the intractable size of three-dimensional shape space. In this

virtually infinite domain, a conventional random or systematic(grid-based) stimulus approach can never produce sufficiently densecombinatorial sampling.

–1–1

a b

Response = 0.4A + 0.0B + 49.0AB + 0.0

38

10

0

1

Relative x position

Rel

ativ

e y

posi

tion

Rel

ativ

e y

posi

tion

Relative z positionMaximum curvature

Min

imum

cur

vatu

re

0

180

180 360

Angle on xy plane (deg)

Ang

le o

n yz

pla

ne

(deg

)

0

i

5 deg

c d

0

20

40

Rotation angle (deg)900–90–4 –2 0 2 4

10

20

30

Depth (deg)

0 0

10

20

30

1 2.10.48Size(x)x position (deg)

y po

sitio

n (d

eg)

–10 100

–10

10

0

0

10

20

30

0

10

20

30

Shadin

g

Textu

reNon

e

0

10

20

30

e f g h

Textu

reNon

e

Stereo Nonstereo

900–900

10

20

30

Light angle (deg)Sha

ding

0

10

20

30

0

10

20

30

900–90

900–90

10

20

30

0

10

20

30

0

–4 –2 0 2 4

–4 –2 0 2 4

0

10

20

30

0

10

20

30

1 2.10.48

1 2.10.48

0

20

40

0

20

40

900–90

900–90

–10

10

0

–10

10

0

–10 100

–10 100

40

j

0 spikes s–1

Res

pons

e (s

pike

s s–1

)

Res

pons

e (s

pike

s s–1

)

Res

pons

e (s

pike

s s–1

)

Res

pons

e (s

pike

s s–1

)0 spikes s–1

Res

pons

e (s

pike

s s–1

)

1–1 0 1–1

0

1

–1 0 1–1

0

1

Figure 2 Neural tuning for three-dimensional configuration of surface fragments. (a) Top 50 stimuli across eight generations (400 stimuli) for a single ITneuron recorded from the ventral bank of the superior temporal sulcus (17.5 mm anterior to the interaural line). (b) Bottom 50 stimuli for the same cell.(c) Responses to highly effective (top), moderately effective (middle) and ineffective (bottom) example stimuli as a function of depth cues (shading, disparityand texture gradients, exemplified in Supplementary Fig. 10). Responses remained strong as long as disparity (black, green and blue) or shading (gray) cueswere present. The cell did not respond to stimuli with only texture cues (pale green) or silhouettes with no depth cues (pale blue). (d) Response consistencyacross lighting direction. The implicit direction of a point source at infinity was varied across 1801 in the horizontal (left to right, black curve) and vertical(below to above, green curve) directions, creating very different two-dimensional shading patterns (Supplementary Fig. 11). (e) Response consistency acrossstereoscopic depth. In the main experiment, the depth of each stimulus was adjusted so that the disparity of the surface point at fixation was 0 (that is, theanimal was fixating in depth on the object surface). In this test, the disparity of this surface point was varied from !4.51 (near) to 5.61 (far). (f) Responseconsistency across xy position. Position was varied in increments of 4.51 of visual angle across a range of 13.51 in both directions. (g) Sensitivity to stimulusorientation. As with all neurons in our sample, this cell was highly sensitive to stimulus orientation, although it showed broad tolerance (about 901) to rotationabout the z axis (rotation in the image plane, blue curve); this rotation tolerance is also apparent among the top 50 stimuli in a. Rotation out of the imageplane, about the x axis (black) or y axis (green), strongly suppressed responses. (h) Response consistency across object size over a range from half to twice thatof the original stimulus. (i) Linear/nonlinear response model based on two Gaussian tuning functions (details as in Fig. 2d,e). (j) The tuning functions areprojected onto the surface of a high-response stimulus, seen from the observer’s viewpoint (left) and from above (right). Error bars indicate s.e.m. in all panels.

1354 VOLUME 11 [ NUMBER 11 [ NOVEMBER 2008 NATURE NEUROSCIENCE

ART ICLES

Yamane Carlson Bowman Wang & Connor ’08

Page 6: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘what’ problem human psychophysics

Thorpe et al ’96; VanRullen & Thorpe ’01; Fei-Fei et al ’02 ’05; Evans & Treisman ’05; Serre Oliva & Poggio ’07

Page 7: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Vision as ‘knowing what is where’

Aristotle; Marr ’82

Page 8: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Ungerleider & Mishkin ‘84

‘What’ and ‘where’ cortical pathways

Page 9: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

ventral ‘what‘ stream

Ungerleider & Mishkin ‘84

‘What’ and ‘where’ cortical pathways

Page 10: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

ventral ‘what‘ stream

dorsal ‘where’ stream

Ungerleider & Mishkin ‘84

‘What’ and ‘where’ cortical pathways

Page 11: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

ventral ‘what‘ stream

dorsal ‘where’ stream

Ungerleider & Mishkin ‘84

?How does the visual system combine information about the identity and location of objects?

‘What’ and ‘where’ cortical pathways

Page 12: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

ventral ‘what‘ stream

dorsal ‘where’ stream

Ungerleider & Mishkin ‘84

?How does the visual system combine information about the identity and location of objects?

➡ Central thesis: visual attention (see also Van Der Velde and De Kamps ‘01; Deco and Rolls ‘04)

‘What’ and ‘where’ cortical pathways

Page 13: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

• Ying Zhang

• Ethan Meyers

• Narcisse Bichot

• Tomaso Poggio

• Bob Desimone

Part I: A ‘Bayesian’ theory of attention that solves the ‘what is where’ problem

• Predictions for neurophysiology and human eye movement data

Part II: Readout of IT population activity

• Attention eliminates clutter

• Sharat Chikkerur

• Cheston Tan

• Tomaso Poggio

Page 14: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

• Ying Zhang

• Ethan Meyers

• Narcisse Bichot

• Tomaso Poggio

• Bob Desimone

Part I: A ‘Bayesian’ theory of attention that solves the ‘what is where’ problem

• Predictions for neurophysiology and human eye movement data

Part II: Readout of IT population activity

• Attention eliminates clutter

• Sharat Chikkerur

• Cheston Tan

• Tomaso Poggio

Page 15: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Perception as Bayesian inference

Mumford ’92; Knill & Richards ‘96; Dayan & Zemel ’99; Rao ’02 ’04; Kersten & Yuille ‘03;

Kersten et al ‘04; Lee & Mumford ‘03; Dean ’05; George & Hawkins ’05 ’09; Hinton ‘07; Epshtein

et al ‘08; Murray & Kreutz-Delgado ’07

S

I

visual scene description

image measurements

Page 16: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

P (S|I) ∝ P (I|S)P (S)

Perception as Bayesian inference

Mumford ’92; Knill & Richards ‘96; Dayan & Zemel ’99; Rao ’02 ’04; Kersten & Yuille ‘03;

Kersten et al ‘04; Lee & Mumford ‘03; Dean ’05; George & Hawkins ’05 ’09; Hinton ‘07; Epshtein

et al ‘08; Murray & Kreutz-Delgado ’07

S

I

visual scene description

image measurements

Page 17: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

P (S|I) ∝ P (I|S)P (S)

Perception as Bayesian inference

Mumford ’92; Knill & Richards ‘96; Dayan & Zemel ’99; Rao ’02 ’04; Kersten & Yuille ‘03;

Kersten et al ‘04; Lee & Mumford ‘03; Dean ’05; George & Hawkins ’05 ’09; Hinton ‘07; Epshtein

et al ‘08; Murray & Kreutz-Delgado ’07

S

I

visual scene description

image measurements

Page 18: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

P (S|I) ∝ P (I|S)P (S)

Perception as Bayesian inference

Mumford ’92; Knill & Richards ‘96; Dayan & Zemel ’99; Rao ’02 ’04; Kersten & Yuille ‘03;

Kersten et al ‘04; Lee & Mumford ‘03; Dean ’05; George & Hawkins ’05 ’09; Hinton ‘07; Epshtein

et al ‘08; Murray & Kreutz-Delgado ’07

S

I

visual scene description

image measurements

S = {O1, O2, · · · , On, L1, L2, · · · , Ln}

object identities

object locations

Page 19: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Perception as Bayesian inference

Mumford ’92; Knill & Richards ‘96; Dayan & Zemel ’99; Rao ’02 ’04; Kersten & Yuille ‘03;

Kersten et al ‘04; Lee & Mumford ‘03; Dean ’05; George & Hawkins ’05 ’09; Hinton ‘07; Epshtein

et al ‘08; Murray & Kreutz-Delgado ’07

S

Iimage

measurements

S = {O1, O2, · · · , On, L1, L2, · · · , Ln}

object identities

object locations

visual scene description

P (S, I) = P (O1, L1, O2, L2, · · · , Ln, Ln, I)

Page 20: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Assumption #1: Attentional spotlight

Broadbent ’52 ‘54; Treisman ‘60; Treisman & Gelade ‘80; Duncan & Desimone ‘95; Wolfe ‘97;

and many others

O,L

Iimage

measurements

• To recognize and localize objects in the scene, the visual system selects objects, one object at a time P (O,L, I)

visual scene description

Page 21: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Assumption #2: ‘what’ and ‘where’ pathways

O,L

Iimage

measurements

• Object location L and identity O are independent

ventral ‘what‘ stream

dorsal ‘where’ stream

P (O,L, I)

visual scene description

Page 22: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

P (O,L, I) = P (O)P (L)P (I|L,O)

O

I

object

image measurementslocation

L

Assumption #2: ‘what’ and ‘where’ pathways

• Object location L and identity O are independent

ventral ‘what‘ stream

dorsal ‘where’ stream

Page 23: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Assumption #3: universal features

• Objects encoded by collections of ‘universal’ features (of intermediate complexity)

- Either present or absent

- Conditionally independent given the object and its location

O

I

object

image measurementslocation

L

see Riesenhuber & Poggio ’99; Serre et al ’05 ’07

Page 24: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Assumption #3: universal features

• Objects encoded by collections of ‘universal’ features (of intermediate complexity)

- Either present or absent

- Conditionally independent given the object and its location

see Riesenhuber & Poggio ’99; Serre et al ’05 ’07

Page 25: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Assumption #3: universal features

see Riesenhuber & Poggio ’99; Serre et al ’05 ’07

Page 26: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Assumption #3: universal features

Serre et al ’05 David et al ’06

Cadieu et al ’07Willmore et al ’10

V2/V4

see Riesenhuber & Poggio ’99; Serre et al ’05 ’07

Page 27: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Assumption #3: universal features

Serre et al ’05 David et al ’06

Cadieu et al ’07Willmore et al ’10

V2/V4

see Riesenhuber & Poggio ’99; Serre et al ’05 ’07

Page 28: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Assumption #3: universal features

Serre et al ’05 David et al ’06

Cadieu et al ’07Willmore et al ’10

V2/V4

Serre et al ’07IT

see Riesenhuber & Poggio ’99; Serre et al ’05 ’07

Page 29: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Serre et al ’07

Predicting IT readout

TRAIN

TEST

3.4ocenter

Size:Position:

3.4ocenter

1.7ocenter

6.8ocenter

3.4o2o horz.

3.4o4o horz.

0

0.2

0.4

0.6

0.8

1

Cla

ssifi

catio

n pe

rform

ance

IT Model

Model: Serre et al ‘05 Experimental data: Hung* Kreiman*et al ‘05

Page 30: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Serre Oliva & Poggio ’07

behavior

Page 31: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Serre Oliva & Poggio ’07

behavior

Page 32: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Serre Oliva & Poggio ’07

behavior

feedforward (bottom-up)

sweep

Page 33: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

Serre Oliva & Poggio ’07

behavior

feedforward (bottom-up)

sweep

Page 34: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

object

Fi

retinotopic features

O

position and scale tolerant

features

N

location

image measurements

• Goal of visual perception: to estimate posterior probabilities of visual features, objects and their locations in an image

• Attention corresponds to conditioning on high-level latent variables representing particular features or locations (as well as on sensory input), and doing inference over the other latent variables

Page 35: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

Fi

O

N

ventral / what

dorsal / where

Page 36: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

Fi

O

N

ventral / what

dorsal / where

P (Xi|I) =P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

�Xi

�P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

Page 37: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

Fi

O

N

ventral / what

dorsal / where

feedforward (bottom-up)

sweep

feedforward input

P (Xi|I) =P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

�Xi

�P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

Page 38: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

Fi

O

N

ventral / what

dorsal / where

top-down spatial attention

top-down feature-based attention

attentional modulation

feedforward (bottom-up)

sweep

feedforward input

P (Xi|I) =P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

�Xi

�P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

Page 39: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

Fi

O

N

ventral / what

dorsal / where

top-down spatial attention

top-down feature-based attention

attentional modulation

feedforward (bottom-up)

sweep

feedforward input

P (Xi|I) =P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

�Xi

�P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

suppressive drive

Xk

Page 40: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bayesian inference and attention

L

Xi

I

Fi

O

NE A

S

P (Xi|I) =P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

�Xi

�P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

Neuron

Review

The Normalization Model of Attention

John H. Reynolds1,* and David J. Heeger21Salk Institute for Biological Studies, La Jolla, CA 92037-1099, USA2Department of Psychology and Center for Neural Science, New York University, New York, NY 10003, USA*Correspondence: [email protected] 10.1016/j.neuron.2009.01.002

Attention has been found to have a wide variety of effects on the responses of neurons in visual cortex. Wedescribe amodel of attention that exhibits each of these different forms of attentional modulation, dependingon the stimulus conditions and the spread (or selectivity) of the attention field in the model. The model helpsreconcile proposals that have been taken to represent alternative theories of attention. We argue that thevariety and complexity of the results reported in the literature emerge from the variety of empirical protocolsthat were used, such that the results observed in any one experiment depended on the stimulus conditionsand the subject’s attentional strategy, a notion that we define precisely in terms of the attention field in themodel, but that has not typically been completely under experimental control.

IntroductionAttention has been known to play a central role in perceptionsince the dawn of experimental psychology (James, 1890).Over the past 30 years, the neurophysiological basis of visualattention has become an active area of research, yielding anexplosion of findings. Neuroscientists have utilized a variety oftechniques (single-unit electrophysiology, electrical microstimu-lation, functional imaging, and visual-evoked potentials) to mapthe network of brain areas thatmediate the allocation of attention(Corbetta and Shulman, 2002; Yantis and Serences, 2003) and toexamine how attention modulates neuronal activity in visualcortex (Desimone and Duncan, 1995; Kastner and Ungerleider,2000; Reynolds and Chelazzi, 2004). During the same period oftime, the field of visual psychophysics has developed rigorousmethods for measuring and characterizing the effects of atten-tion on visual performance (Braun, 1998; Carrasco, 2006; Cava-nagh and Alvarez, 2005; Sperling andMelchner, 1978; Verghese,2001; Lu and Dosher, 2008).

We review the single-unit electrophysiology literature docu-menting the effects of attention on the responses of neurons invisual cortex, and we propose a computational model to unifythe seemingly disparate variety of such effects. Some resultsare consistent with the appealingly simple proposal that atten-tion increases neuronal responses multiplicatively by applyinga fixed response gain factor (McAdams and Maunsell, 1999;Treue and Martinez-Trujillo, 1999), while others are more inkeeping with a change in contrast gain (Li and Basso, 2008; Mar-tinez-Trujillo and Treue, 2002; Reynolds et al., 2000), or witheffects that are intermediate between response gain andcontrast gain changes (Williford and Maunsell, 2006). Otherstudies have shown attention-dependent sharpening of neuronaltuning at the level of the individual neuron (Spitzer et al., 1988) orthe neural population (Martinez-Trujillo and Treue, 2004). Stillothers have shown reductions in firing rate when attention wasdirected to a nonpreferred stimulus that was paired witha preferred stimulus also inside the receptive field (Moran andDesimone, 1985; Recanzone and Wurtz, 2000; Reynolds et al.,1999; Reynolds and Desimone, 2003). These different effectsof attentional modulation have not previously been explained

within the framework of a single computational model. Wedemonstrate here that a model of attention that incorporatesdivisive normalization (Heeger, 1992b) exhibits each of thesedifferent forms of attentional modulation, depending on the stim-ulus conditions and the spread (or selectivity) of the attentionalfeedback in the model.In addition to unifying a range of experimental data within

a common computational framework, the proposedmodel helpsreconcile alternative theories of attention. Moran and Desimone(1985) proposed that attention operates by shrinking neuronalreceptive fields around the attended stimulus. Desimone andDuncan (1995) proposed an alternative model, in which neuronsrepresenting different stimulus components compete and atten-tion operates by biasing the competition in favor of neurons thatencode the attended stimulus. It was later suggested that atten-tion instead operates simply by scaling neuronal responses bya fixed gain factor (McAdams and Maunsell, 1999; Treue andMartinez-Trujillo, 1999). Treue and colleagues advanced the‘‘feature-similarity gain principle,’’ that the gain factor dependson the match between a neuron’s stimulus selectivity and thefeatures or locations being attended (Treue andMartinez-Trujillo,1999; Martinez-Trujillo and Treue, 2004). Spitzer et al., 1988proposed that attention sharpens neuronal tuning curves, andMartinez-Trujillo and Treue (2004) explained that sharpening ispredicted by their ‘‘feature-similarity gain principle.’’ Finally, Rey-nolds et al., 2000 proposed that attention increases contrastgain. Indeed, the initial motivation for the model proposed herederived from the reported similarities between the effects ofattention and contrast elevation on neuronal responses (Rey-nolds and Chelazzi, 2004; Reynolds et al., 1999, 2000; Reynoldsand Desimone, 2003).The proposed normalization model of attention combines

aspects of each of these proposals and exhibits all of theseforms of attentional modulation. Thus, the various models out-lined above are not mutually exclusive. Rather, they can all beexpressed by a single, unifying computational principle. Wepropose that this computational principle endows the brainwith the capacity to increase sensitivity to faint stimuli presentedalone and to reduce the impact of task irrelevant distracters

168 Neuron 61, January 29, 2009 ª2009 Elsevier Inc.

R(x, θ) =A(x, θ)E(x, θ)

S(x, θ) + σ

Page 41: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Multiplicative scaling of tuning curves by spatial attention

see also Heeger & Reynolds ’09

McAdams and Maunsell ‘99 Model

P (L = x) = 1/|L|

P (L = x) ≈ 1

P (Xi = x|I) ∝�

F i,L

P (Xi = x|F i, L)P (I|Xi)P (F i)P (L).

Page 42: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Contrast vs. response gainsee also Heeger & Reynolds ’09

Trujillo and Treue ‘02 Mc Adams and Maunsell ’99

P (Xi|I) =P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

�Xi

�P (I|Xi)

�F i,L P (Xi|F i, L)P (L)P (F i)

Page 43: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Feature-based attention

0 20 40 60 80 100 120 140 160 180 2000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1P stim/P cueNP stim/ P.cueP.stim/NP cueNP stim/NP cue

0

1

2

3

Nor

mal

ized

spi

kes

Nor

mal

ized

resp

onse

Bichot et al. ‘05

P (Xi = x|I) ∝�

F i,L

P (Xi = x|F i, L)P (I|Xi)P (F i)P (L).

neural data from Bichot et al ’05

Page 44: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Learning to localize cars and pedestrians

L

Xi

I

Fi

O

N

Page 45: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Learning to localize cars and pedestrians

L

Xi

I

Fi

O

N

learning object priors

Page 46: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Learning to localize cars and pedestrians

L

Xi

I

Fi

O

N

learning object priors

Context

Page 47: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Learning to localize cars and pedestrians

L

Xi

I

Fi

O

N

learning object priors

learning location priors (global contextual cues, see Torralba, Oliva et al)

Context

Page 48: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

• Eye movements as proxy for attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

Page 49: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

• Eye movements as proxy for attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

Page 50: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

Page 51: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 52: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 53: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 54: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 55: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 56: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 57: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 58: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 59: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 60: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

Page 61: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The experiment

cars pedestrians0.5

0.75

1

Uniform priors (bottom-up)Feature priors Feature + contextual (spatial) priorsHumans

1st three fixations• Eye movements as proxy for

attention

• Dataset:

- 100 street-scenes images with cars & pedestrians and 20 without

• Experiment

- 8 participants asked to count the number of cars/pedestrians

- block design for cars and pedestrians

- eye movements recorded using an infra-red eye tracker

RO

C a

rea

*similar (independent) results by Ehinger Hidalgo Torralba & Oliva ’10

Explains 92% of the inter-subject agreement!

Page 62: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Bottom-up saliency and free-viewing

L

Xi

I

Fi

N

Method ROC area

Bruce & Tsotos ’06 72.8%

Itti et al ’01 72.7%

Proposed 77.9%

human eye data from Bruce & Tsotsos

Page 63: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Summary I

Page 64: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Summary I

• Goal of vision: - To solve the problem of ’what is where’ problem

Page 65: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Summary I

• Goal of vision: - To solve the problem of ’what is where’ problem

• Key assumptions: - ‘Attentional spotlight’ → recognition done sequentially, one object at a time

- ‘What’ and ‘where’ independent pathways

Page 66: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Summary I

• Goal of vision: - To solve the problem of ’what is where’ problem

• Key assumptions: - ‘Attentional spotlight’ → recognition done sequentially, one object at a time

- ‘What’ and ‘where’ independent pathways

• Attention as the inference process implemented by the interaction between ventral and dorsal areas

- Integrates bottom-up and top-down (feature-based and context-based) attentional mechanisms

- Seems consistent with known physiology

Page 67: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Summary I

• Goal of vision: - To solve the problem of ’what is where’ problem

• Key assumptions: - ‘Attentional spotlight’ → recognition done sequentially, one object at a time

- ‘What’ and ‘where’ independent pathways

• Attention as the inference process implemented by the interaction between ventral and dorsal areas

- Integrates bottom-up and top-down (feature-based and context-based) attentional mechanisms

- Seems consistent with known physiology

• Main attentional effects in the presence of clutters

- Spatial attention reduces uncertainty over location and improves object recognition performance over first bottom-up (feedforward) pass

Page 68: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

• Ying Zhang

• Ethan Meyers

• Narcisse Bichot

• Tomaso Poggio

• Bob Desimone

Part I: A ‘Bayesian’ theory of attention that solves the ‘what is where’ problem

• Predictions for neurophysiology and human eye movement data

Part II: Readout of IT population activity

• Attention eliminates clutter

• Sharat Chikkerur

• Cheston Tan

• Tomaso Poggio

Page 69: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘readout’ approach

Page 70: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘readout’ approach

Page 71: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘readout’ approach

neuron 1

neuron 2

neuron 3

neuron n

Page 72: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘readout’ approach

neuron 1

neuron 2

neuron 3

neuron n

Pattern Classifier

Page 73: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘readout’ approach

neuron 1

neuron 2

neuron 3

neuron n

Pattern ClassifierPrediction

Page 74: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

The ‘readout’ approach

neuron 1

neuron 2

neuron 3

neuron n

Pattern ClassifierPrediction

“Correct”

Page 75: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

++

train readout classifier on isolated object

Page 76: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

++

train readout classifier on isolated object

++

test generalization in clutter

Page 77: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

++

test generalization in clutter

unattended

++

train readout classifier on isolated object

Page 78: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

attended

IT readout improves with attention

++

test generalization in clutter

unattended

++

train readout classifier on isolated object

Page 79: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

attend away

attended

IT readout improves with attention

++

test generalization in clutter

unattended

++

train readout classifier on isolated object

Page 80: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

Page 81: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 82: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 83: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 84: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 85: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

Page 86: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 87: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 88: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 89: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 90: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 91: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 92: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

IT readout improves with attention

Zhang Meyers Bichot Serre Poggio Desimone unpublished data

+

Page 93: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

• Robert Desimone (MIT)• Christof Koch (CalTech)• Laurent Itti (USC)• Tomaso Poggio (MIT)• David Sheinberg (Brown)

Page 94: Bottom-up and top-down processing in visual … and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute

Thank you!