Chapter 12:

Outputs of Visual Processing

IT: Inferior Temporal Visual Cortex LPF/PF: (Lateral) Prefrontal Cortex RC: Recurrent Collateral Connections Attractor Networks: Feedback like system

We will discuss:

Ventral visual processing stream IT output provides distributed representation of…. “what” (Parietal sends “where” info)

Both IT & P output to STM in what region… PFC (Prefrontal Cortex)

why STM system must be separate from IT & P system interaction based on “attractor networks”

There are both superficial and deep cortical neuron layers in each attractor network Superficial layers (s)

project onto self or higher cortical level (like s for superman; up up and away!)

Deep layers (d) project onto self or downwards (d for downward)

How does STM work???

Network must maintain firing rate after stimulus

For how long? In a monkey IT, usually between 0.1 – 10 seconds; in PF even longer

How is it maintained? (also see figure) The collateral connections of (nearby) pyramid cells create

feedback loops. These undergo associative modification. These collateral feedback loops are called: recurrent collateral

connections

Intramodular connections

This model also says connections between networks must be weak. This parameter, g, indicates the relative strength of inter-modular to

intramodular connections.

PFIT

Intramodular connections

Weak g is needed for PF to maintain firing rate (ie. a percept) while several intervening stimuli pass through the posterior IT networks

PFIT

What is the Two Network Model ofthe visual-STM system???

Separate networks are needed for both perception and STM to work simultaneously. So where are they??

Network in IT: perceptual functions

Network in PF: for maintaining STM during intervening stimuli

These networks are coupled; ie. they communicate with each other with forward and back projections

How do we know…..

Stimulus Match to Sample Task Used to understand STM.

Response in RC of PF is activated by both visual input of matching stimulus & backprojected memory of sample stimulus

Two networks firing in synchrony augment response choice

Stimulus Match to Sample Task This model says that for

continuous perception to occur, there must be a module that maintains representation of the stimulus during intervening stimuli (ie. a STM must exist)

Two network model allows planning which is dependent on ability to maintain several short term memories simultaneously.

Visual Search Task

Iai is current for neuron i in module a

a = {IT, PF} (ie. a is either IT or PF)

Iai *decreases in time with constant

*increases proportionally to the activity

elsewhere in the network

is the synaptic efficacy between two neurons. (ie. between neuron ai and presynaptic neuron j of module b.)

vbj is the weight of neuron j of module b (ie. firing rate)

is current from the stimulus that is external to network

Yummy, equations :-p

abijJ

)(

,

)( extaibj

jb

abij

aiai hvJtI

dt

dI

)(extaih

This models how current (ie. firing rate)

changes in time

P is number of binary patterns of active neurons that are possible both for IT and PF

is a neuron with value 1 or 0 with probabilities f and (1-f)

Nt is the normalization constant

},{;))(()1( 1

0),( PFITajiffNff

JJ u

aj

Puai

t

aaij

This model computes synaptic efficacy(firing rate correspondence)between neurons within a module (either IT or PF)

Synaptic Modification This is process of setting up neurons to create an

attractor which can hold an item as STM

Once set up it may be reused when triggered by appropriate cue, even without further synaptic modification

Part II: Visual Outputs to LTM

IT projects to the Hippocampus to create LTMs

In the hippocampus objects may be located as part of a spatial scenes. This is called… episodic memory especially primates, not so much rats Primate object-memory representation could be

generated by a continuous attractor

Damaged Hippocampus

Anterograde amnesia. What causes this in humans??bilateral damage to HC and nearby parts of IT (Squire)

Which LTM memories located in HC HC shown necessary to learn declarative memories, epsiodic Not spatial processing; eg. An object-place memory task which

requires memory of object and environment or context in form of snapshot

LTM functions not affected by HC damage? procedural memory, anterograde amnesia does not impair

procedural memory

Rats and Monkeys

Rat HC pyramidal cells recognize a previously learned “scene” only from the same location and orientation: egocentric (body based) Learning a new scene for rat HC pyramidal cells

takes about 10 minutes A group of cells can map 2 spaces simultaneously,

and is able to process/represent only one them

Rats also have task related HC cells cells respond to olfactory stimuli with particular

behavior response

Rats and Monkeys

Primate HC has spatial view cells that respond to where monkey is looking, from any vantage point: allocentric (world based)

Idiothetic cues to trigger memory when scene’s details are obscured:

eye position head direction linear and axial whole body motion

Why Study Spatial View Cells Primates have spatial view cells:

They enable object-place memory (a kind of episodic memory) some HC neurons respond to combo of object info and spatial

info

HC cells respond both to presented image and location for response: this is called memory convergence (2 stimuli 1 memory)

Show figure

Rats and Monkeys compared:

Rats Monkeys

place cells spatial view cells

wide visual foveate vision

field

Perhaps cells use same computational process and respond according to their different input apparatuses

Hippocampus Models CA3 Recurrent Collateral Connections create auto-association networks

enabling…. Every CA3 neuron to associate with any other CA3 neuron involved in the same memory

The number of different memories p in the CA3 system:

RC = recurrent connectionsa = sparsenessk ≈ synaptic efficacy

Eg. For a rat: CRC = 12000; this translates to between 12k – 36k memoriesLow sparseness (ie .02) more memories

Same idea going between networks:back projections down to CA1

ka

Cp

a)ln( 1

RC

BP

BPBP

BPBP k

aa

Cp

)1

ln(

CA3 RC Network

Mossy fibres help learning only, not recall

Continuous Attractors To represent spatial patterns use continuous attractor networks HC represents spatial patterns uses continuous attractor networks

Continuous networks can explain how in the dark, rats can still maintain place cell firing (or even update the response with idiothetic inputs)

Now known, attractors can store both continuous and discrete patterns locations in space = continuous objects = discrete

HC receives and combines both input types: For an event with both discrete and continuous aspects: spatial info can be

retrieved from object cues and vice versa

Cool?Thanks!

I am intrigued by Rolls et al.'s research about specializedhippocampal cells (p.416). For example, they find cells that respond to a combination of which picture is shown and where the response must be made. Cells like these, that integrate perception of stimuli with what response is needed, seem like they could be close to feeding straight into consciousness (i.e. sound homunculus-like). However, since these cells are not tuned to respond to any particular stimuli or to coordinate any particular response, I can't get a handle on quite how they could work. Is that known? -S.R.

Rolls and others have found neurons responding to many strange stimuli that I should never have thought of: for instance, hippocampal cells that register not only where the animal is (rat) but where it is looking (primate): cells that respond to the physical properties of a stimulus (sweet vs. salty), others that respond to its reward contingencies.

We take as an article of faith that every mental event has an accompanying brain event.  Does every mental event have a corresponding single-neuron event?  Perhaps yes, if Barlow is right and grandmother cells exist.  No, if many 'events' have a distributed code that spreads them across many neurons.

In the present state of the art we can record events in single neurons (with electrodes) and in rather coarsely defined brain areas (with fMRI).  Between the microscopic and macroscopic techniques lies a mesoscopic gap - we cannot record events of the size of cell assemblies.  Is this where most of the brain action is? -S.A.

Meanwhile, what would we find in a catalogue of every stimulus that is known to excite single neurons in the brain? It would include faces, localities, reward contingenciesŠ  Would it contain every imaginable stimulus, and if not, what would it leave out?  Of these left-out items, would cell assemblies respond to them?  I'm really wondering what we could conclude from such a catalogue.  If it contains everything, does it really tell us anything?  And do we know of any instructive omissions from such a catalogue? (dogs that did not bark during the night).

I've heard many informal criticisms of Rolls' book  --   too concentrated on his own work, ignoring the work of others; in particular, hard to read and hard to follow.  But what do we think, now that we have (in theory) read the book?  Is most of it "going nowhere", or is Rolls really "on to something" that we ignore at our peril?  Maybe his insights will give us huge breakthroughs, that we lack the intelligence or perseverance to descry?  What does everybody here think? -S.A.

In section 12.4, the authors point out how translation invariance in IT is a problem for the issue of how information about an object's spatial coordinates is made available to the motor system. They propose that to solve this problem, the motor system simply directs action toward the object foveated, since in most cases that is the object of interest.

Although this explanation does seem possible, I am still not sold. I think a much more parsimonious explanation would be that even after translation invariance takes place in IT, some sort of representation of the target's spatial location is preserved within the system. Why should the motor system start from scratch when information about the spatial coordinates has already been extracted, albeit at an earlier level of visual processing? – L.L.

1. The authors assume that thresholds are exceeded. Is there any mention in the book about how this may occur.

2. The authors assume there is an "activation function." How might this be derived from fundamerntal principles of neuronal activity rather than just be a convenient assumption that makes the model fit some data?

3. The measure of "sparseness" is claimed to have a value that can only occur in a very extreme and limited citcumstance. Is "sparseness" necessary? If so, what would be a better way to compute it.

4. Multiplication occurs in equations throughout the book. How can multiplication occur, physically?

5. Signals passing through the nervous system are representable as independent stochastic random variables. But when a threshold is posited the sum of such signals is no longer a sum of independent RVs because the sum is limited (ie must at least equal the threshold). The conventional algrbra of independent random variables no longer applies. What effect can this have on the theory? On any theory?

6. Although the signal transmission system of the nervous system is known to be a stochastic process, and therefore the algebra of probability theory applies to models that represent the nervous system, the book uses real numbers and real algrebra and essentially linear system for its modelling. Is there something very wrong here? –S.L.

Page 416:  "These primate 'spatial view' hippocampal cells encode information in allocentric (world-centred, as opposed to egocentric) coordinates."

There are two ways in which (for instance) a ship codes its own position:  visually, by measuring nearby visual landmarks: or from dead reckoning, by recording and remembering all its own motions since it left its home port.  Dead reckoning is subject to cumulative drift, whilst visual sighting is probably not.  (I ignore Harrison's chronometric measurements of longitude, and GPS methods).  Which of these two methods, or what combination thereof, do (or could) these hippocampal cells use? – S.A.