Upload
mitchell-hampton
View
218
Download
0
Embed Size (px)
Citation preview
1
Computational Vision
CSCI 363, Fall 2012Lecture 31
Heading Models
2
Motion over a Ground Plane
3
Image motion for Translation plus Rotation
TRANSLATION WITH ROTATION
4
Models for Computing Observer Motion
€
vx −−TX + xTZ
Z
⎛ ⎝ ⎜
⎞ ⎠ ⎟
⎛ ⎝ ⎜
⎞ ⎠ ⎟∑2
+ vy −−TY + yTZ
Z
⎛ ⎝ ⎜
⎞ ⎠ ⎟
⎛ ⎝ ⎜
⎞ ⎠ ⎟2
1. Error minimization: Minimize the equation
2. Template models:Use a group of "templates" that correspond to the flow fields that
would be created by a given set of translation and rotation parameters. Find the template that matches the flow field best.
3. Motion Parallax models:Make use of the fact that image velocities due to translation are
dependent on Z. Image velocities due to rotation are not.
5
Image Velocities
vx = (-TX + x TZ) / Z - (1 + x2) RY + xy RX + y RZvy = (-TY + y TZ) / Z + (1 + y2) RX - xy RY - x RZ
Translation ComponentDepth Dependent
Rotation ComponentDepth Independent
x and y components of image velocity:
TY
TX
TZ
RZ
R X
RY
Y
X
Z
P O
y
xp
Longuet-Higginsand Prazdny, 1980
NearFar
Diff
6
Neurons vs. Pure Math
+
−+
−+
−
Middle Temporal Area Difference Vectors
AsymmetricSurround
Circularly-SymmetricSurround
1. Spatially extendedreceptive fields.
2. Response is tunedto speed and direction.
3. Center and surroundtend to have the samebest direction.
1. Vector subtractionat a single point.
2. Accurate velocitymeasurements assumed.
3. Vectors may differsubstantially in direction.
VectorSubtraction
7
Motion-subtraction by neurons
Odd Symmetric Even Symmetric
8
Computing Heading
Visual Field
Operator GroupReceptive Field Spacing
9
Layer 2 is a Template
Maximally RespondingOperators
Translational Heading Template
Template & MST cells both:1. Have large receptive fields.2. Respond to expansion/contraction.3. Are tuned to center of expansion.
10
Motion toward a 3D cloud
Observer
11
Translation + Rotation
Flow Field
12
Operator Responses
13
Model Heading Estimates
Mod
el R
espo
nse
14
Moving Objects
Demo
15
Hea
ding
Bia
s (d
eg)
16
Model vs. Experiment.
-0.25
0
0.25
0.5
0.75
-10 -5 0 5 10
Center Position (deg)
Right Object Motion
-1
-0.5
0
0.5
-5 0 5 10 15
Left Object Motion
Model
Psychophysics
Response Bias, Model vs. Psychophysical Data
17
Radial Optic Flow Field
Scene Focus of Expansion (FOE)
18
Lateral Flow Field
19
Illusory Center of Expansion(Duffy and Wurtz, 1993)
Scene Focus of Expansion (FOE)
Perceived Focus of ExpansionDemo
20
Difference Vectors for Illusion
Center of Difference Vectors
21
Model Response to Illusion
-15
-10
-5
0
5
10
15E
stim
ated
Cen
ter
-15 -10 -5 0 5 10 15
Lateral Dot Speed
Model
Calculated
22
Model vs. Human Response
-15
-10
-5
0
5
10
15
Est
imat
ed C
ente
r
-15 -10 -5 0 5 10 15
Lateral Dot Speed
Model
Calculated
Average
23
Conclusions1. A model based on motion subtraction done by neurons in
MT can accurately compute observer heading in the presence of rotations.
2. The model shows biases in the presence of moving objects that are similar to the biases shown by humans.
3. The model responds to an illusory stimulus in the same way that people do.
4. The fact that the model responds in the same way as humans with stimuli for which it was not designed provides evidence that the human brain uses a mechanism similar to that of the model to compute heading.
24
How does the brain process heading?
•It is not known how the brain computes observer heading, but there are numerous models and hypotheses.
•One of the simplest ideas is based on template models: Neurons in the brain are tuned to patterns of velocity input that would result from certain observer motions.
•Support for this idea:•Tanaka, Saito and others found cells in the dorsal part of the Medial Superior Temporal area (MSTd) that respond well to radial, circular or planar motion patterns.
•Since then, people have assumed that MSTd is involved in heading computation.
25
Visual Pathway
26
Types of Responses in MSTd
(from Duffy & Wurtz, 1991)
27
Combinations of Patterns•Duffy and Wurtz (1991) tested cell responses to planar, circular and radial patterns. •They found some cells responded only to one type of pattern (e.g. only to circular). Others responded to two or three types of patterns (e.g. both planar and circular).
Single Component Double Component Triple ComponentRadial Plano-Radial Plano-Circulo-RadialCircular Plano-CircularPlanar
•They did not suggest a model of how these might be involved in heading detection.
•They also showed there is not a simple way that MST receptive fields are made from inputs from MT cell receptive fields.
28
Spiral Patterns
Graziano et al. (1994) showed that MSTd cells respond to spiral patterns of motion:
29
Does MST compute heading?
Prediction: If MST is involved in heading computation, one would expect to find cells tuned to a particular position for the center of expansion.Duffy and Wurtz (1995) tested this prediction.
30
Do MSTd cells use eye-movement information?
•Psychophysical experiments showed that humans can make use of eye movement information to compute heading.
•Some MSTd cells have responses that are modulated by eye movements.
•Do eye movements affect the responses of MSTd cells to compensate for rotation?
•This was tested in an experiment by Bradley et al (1996).•They recorded from MSTd cells while showing flow fields that consisted of an expansion plus a rotation. The rotation was generated by real or simulated eye movements.
31
Real eye movement condition
32
Simulated eye movement condition
33
Results
No eye movement
Eye movement in preferred direction.
Eye movement in anti-preferred direction.
This cell seems to take into account eye movements.
The effect was not consistent among all cells tested.