Embed Size (px)
Citation preview
Active Vision
Carol Colby
Rebecca BermanCathy Dunn
Chris GenoveseLaura HeiserEli Merriam
Kae Nakamura
Department of NeuroscienceCenter for the Neural Basis of Cognition
University of Pittsburgh
Department of StatisticsCarnegie Mellon University
Hermann von Helmholtz
Treatise on Physiological Optics,1866
Why does the world stay still when we move our eyes?
“Effort of will”
1) Remapping in monkey area LIP and extrastriate visual cortex
1) Remapping in monkey area LIP and extrastriate visual cortex
2) Remapping in split-brain monkeys
Behavior Physiology
1) Remapping in monkey area LIP and extrastriate visual cortex
2) Remapping in split-brain monkeys
Behavior Physiology
3) Remapping in human cortex
Parietal cortex Striate and extrastriate visual cortex Remapping in a split brain human
LIP memory guided saccade
Stimulus On Saccade
Stimulus appears outside of RF
Saccade moves RF to stimulus location
Single step task
Spatial updating or remapping
The brain combines visual and corollary discharge signals to create a representation of space that takes our eye movements into account
LIP Summary
Area LIP neurons encode attended spatial locations.
LIP Summary
Area LIP neurons encode attended spatial locations.
The spatial representation of an attended location is remapped when the eyes move.
LIP Summary
Area LIP neurons encode attended spatial locations.
The spatial representation of an attended location is remapped when the eyes move.
Remapping is initiated by a corollary discharge of the eye movement command.
LIP Summary
Area LIP neurons encode attended spatial locations.
The spatial representation of an attended location is remapped when the eyes move.
Remapping is initiated by a corollary discharge of the eye movement command.
Remapping produces a representation that is oculocentric: a location is represented in the coordinates of the movement needed to acquire the location.
LIP Summary
Area LIP neurons encode attended spatial locations.
The spatial representation of an attended location is remapped when the eyes move.
Remapping is initiated by a corollary discharge of the eye movement command.
Remapping produces a representation that is oculocentric: a location is represented in the coordinates of the movement needed to acquire the location.
Remapping allows humans and monkeys to perform a spatial memory task accurately.
V1
LGN
Retina
V2
V3A
LIP
FEF
SC
Oculomotor System
V3
Stimulus appears outside of RF
Saccade moves RF to stimulus location
Stimulus alone control Saccade alone control
Single step task
Extrastriate Summary
Remapping occurs at early stages of the visual hierarchy.
Extrastriate Summary
Remapping occurs at early stages of the visual hierarchy.
Corollary discharge has an impact far back into the system.
Extrastriate Summary
Remapping occurs at early stages of the visual hierarchy.
Corollary discharge has an impact far back into the system.
Remapping implies widespread connectivity in which many neurons have rapid access to information well beyond the classical receptive field.
Extrastriate Summary
Remapping occurs at early stages of the visual hierarchy.
Corollary discharge has an impact far back into the system.
Remapping implies widespread connectivity in which many neurons have rapid access to information well beyond the classical receptive field.
Vision is an active process of building representations.
1) Remapping in monkey area LIP and extrastriate visual cortex
2) Remapping in split-brain monkeys
Behavior Physiology
3) Remapping in human cortex
Parietal cortex Striate and extrastriate visual cortex Remapping in a split brain human
Stimulus appears outside of RF
Saccade moves RF to stimulus location
What is the brain circuit that produces remapping?
The obvious pathway for visual signals:forebrain commissures
Are the forebrain commissures necessary for updating visual signals across the vertical meridian?
Behavior in double step task
Physiology in single step and double step task
Attain fixationFP
T1 appearsFP T1
T2 flashes brieflyT1
T2
FP
Saccade to T1T1
Saccade to T2
T2
Attain fixationFP
T1 appearsFP T1
T2 flashesT1
T2
FP
WITHIN
T1
T2
T2
Transfer of visual signals
T2
WITHIN
T1
T2
T2’
VISUAL-ACROSS
T2
T1
T2
WITHIN
T1
T2
T2 T2’
VISUAL-ACROSS
T2
T1
T2 T2’
WITHIN
T1
T2
T2 T2’
WITHIN
T1
T2
Is performance impaired on visual-across sequences in
split-brain monkeys?
VISUAL-ACROSS
T2
T1
T2 T2’T2 T2’
Central
Across Within
Central
Within Across
Day 1: Initial impairment for visual-across
Within AcrossCentral WithinAcross Central
Monkey C
MonkeyE
correctincorrect
TRIALS
1-10
Within Central Across WithinCentralAcross
120-130
60-70
Horizontal eye position (degrees)
Ver
tical
eye
pos
ition
(de
gree
s)
Monkey C
First day saccade endpoints
Monkey E
Horizontal eye position (degrees)
Ver
tical
eye
pos
ition
(de
gree
s)
Monkey E
Monkey C
Last day saccade endpoints
Monkey E
Are the forebrain commissures necessary for updating spatial information across the vertical meridian?
Are the forebrain commissures necessary for updating spatial information across the vertical meridian?
No. The FC are the primary route but not the only route.
Are the forebrain commissures necessary for updating spatial information across the vertical meridian?
No. The FC are the primary route but not the only route.
What are LIP neurons doing?
Stimulus appears outside of RF
Saccade moves RF to stimulus location
SINGLESTEP
STIMULUS ALONE
SACCADE ALONE
Population activity in area LIP
SINGLESTEP
DOUBLE STEP
Split Brain Monkey Summary
The forebrain commissures normally transmit remapped visual signals across the vertical meridian but they are not required.
Split Brain Monkey Summary
The forebrain commissures normally transmit remapped visual signals across the vertical meridian but they are not required.
Single neurons in area LIP continue to encode remapped stimulus traces in split-brain animals.
1) Remapping in monkey area LIP and extrastriate visual cortex
2) Remapping in split-brain monkeys
Behavior Physiology
3) Remapping in human cortex
Parietal cortex Striate and extrastriate visual cortex Remapping in a split brain human
Functional Imaging Predictions
1) Robust activation in cortex ipsilateral to the stimulus.
Functional Imaging Predictions
1) Robust activation in cortex ipsilateral to the stimulus.
2) Ipsilateral activation should be smaller than the contralateral visual response.
Functional Imaging Predictions
1) Robust activation in cortex ipsilateral to the stimulus.
2) Ipsilateral activation should be smaller than the contralateral visual response.
3) It should not be attributable to the stimulus alone or to the saccade alone.
Functional Imaging Predictions
1) Robust activation in cortex ipsilateral to the stimulus.
2) Ipsilateral activation should be smaller than the contralateral visual response.
3) It should not be attributable to the stimulus alone or to the saccade alone.
4) Ipsilateral activation should occur around the time of the saccade.
Contralateral Visual Response
Ipsilateral Remapped Response
Ipsilateral Remapped Response
Visual and Remapped Responses
Human Parietal Imaging Summary
Remapping in humans produces activity in parietal cortex ipsilateral to the visual stimulus.
Human Parietal Imaging Summary
Remapping in humans produces activity in parietal cortex ipsilateral to the visual stimulus.
Remapped activity is lower amplitude than visual activity.
Human Parietal Imaging Summary
Remapping in humans produces activity in parietal cortex ipsilateral to the visual stimulus.
Remapped activity is lower amplitude than visual activity.
It cannot be attributed to the stimulus or the saccade alone.
Human Parietal Imaging Summary
Remapping in humans produces activity in parietal cortex ipsilateral to the visual stimulus.
Remapped activity is lower amplitude than visual activity.
It cannot be attributed to the stimulus or the saccade alone.
It occurs in conjunction with the eye movement.
1) Remapping in monkey area LIP and extrastriate visual cortex
2) Remapping in split-brain monkeys
Behavior Physiology
3) Remapping in human cortex
Parietal cortex Striate and extrastriate visual cortex Remapping in a split brain human
Contralateral Visual Response
Ipsilateral Remapped Response
Remapping in Multiple Visual Areas
1) Remapping in monkey area LIP and extrastriate visual cortex
2) Remapping in split-brain monkeys
Behavior Physiology
3) Remapping in human cortex
Parietal cortex Striate and extrastriate visual cortex Remapping in a split brain human
Intact Subjects Split Brain Subject
Strength of Parietal Responses in Split Brain and Intact Subjects
Human Imaging Summary
Remapping in humans produces activity in the hemisphere ipsilateral to the stimulus.
Human Imaging Summary
Remapping in humans produces activity in the hemisphere ipsilateral to the stimulus.
Remapped activity is present in human parietal, extrastriate and striate cortex.
Human Imaging Summary
Remapping in humans produces activity in the hemisphere ipsilateral to the stimulus.
Remapped activity is present in human parietal, extrastriate and striate cortex.
Remapped visual signals are more prevalent at higher levels of the visual system hierarchy.
Human Imaging Summary
Remapping in humans produces activity in the hemisphere ipsilateral to the stimulus.
Remapped activity is present in human parietal, extrastriate and striate cortex.
Remapped visual signals are more prevalent at higher levels of the visual system hierarchy.
Remapping occurs in parietal and visual cortex in a split brain human subject.
Conclusions
Remapping of visual signals is widespread in monkey cortex.
Conclusions
Remapping of visual signals is widespread in monkey cortex.
Split-brain monkeys are able to remap visual signals across the vertical meridian.
Conclusions
Remapping of visual signals is widespread in monkey cortex.
Split-brain monkeys are able to remap visual signals across the vertical meridian.
Remapped visual signals are present in area LIP in split-brain monkeys.
Conclusions
Remapping of visual signals is widespread in monkey cortex.
Split-brain monkeys are able to remap visual signals across the vertical meridian.
Remapped visual signals are present in area LIP in split-brain monkeys.
Remapped visual signals are robust in human parietal and visual cortex.
Conclusions
Remapping of visual signals is widespread in monkey cortex.
Split-brain monkeys are able to remap visual signals across the vertical meridian.
Remapped visual signals are present in area LIP in split-brain monkeys.
Remapped visual signals are robust in human parietal and visual cortex.
In a split-brain human, remapped visual signals are found in parietal and visual cortex.
Conclusions
Remapping of visual signals is widespread in monkey cortex.
Split-brain monkeys are able to remap visual signals across the vertical meridian.
Remapped visual signals are present in area LIP in split-brain monkeys.
Remapped visual signals are robust in human parietal and visual cortex.
In a split-brain human, remapped visual signals are found in parietal and visual cortex.
Vision is an active process of building representations from sensory, cognitive and motor signals.
WithinAcross
Central
Within Across
Central
Learning? Or a monkey trick?
no monkey tricks..
Monkey EM Monkey CH
Both monkeys really update the visual representation
Magnitude of Remapped Response
