View
214
Download
0
Category
Preview:
Citation preview
1
Computational Vision
CSCI 363, Fall 2012Lecture 21
Motion II
2
Gradient Models
The gradient models use the "Contrast Brightness Assumption".In 1 spatial dimension, this states:
€
I(x,t) = I(x +∂x,t +∂t)
I
xx0
t0 I
xx0 + x
t0 + t
3
The Gradient Constraint Equation
€
∂I∂x∂x +
∂I
∂t∂t = 0
Let u =∂x
∂t, Ix =
∂I
∂x, I t =
∂I
∂t then
Ixu + I t = 0
The Gradient Constraint Equation in 1 dimension:
€
∂I∂x∂x +
∂I
∂y∂y+
∂I
∂t∂t = 0
Let u =∂x
∂t,v =
∂y
∂t, Ix =
∂I
∂x, Iy =
∂I
∂y, I t =
∂I
∂t then
Ixu + Iyv+ I t = 0
The Gradient Constraint Equation in 2 dimensions:
4
The Aperture Problem•The gradient constraint equation for a 2D image is 1 equation with 2 unknowns (u and v).
•To solve for u and v, we must make measurements of Ix, Iy, and It at 2 locations where they are not all identical.
•If our view is limited to an edge seen through an aperture, we cannot solve for both u and v independently. We can only find the component of motion perpendicular to the edge.
Aperture
EdgePerpendicular velocitycomponent
5
The Aperture Problem is Fundamental
•The aperture problem is a fundamental problem when one is trying to measure image velocity using local detectors.
•This is true in biological vision (neurons have local receptive fields).
•This is also true in machine vision (intensity is detected locally by photodetectors).
6
Solving the Aperture problem1. Assume pure translation of the object.
The true velocity may lie anywhere along this "constraint" line.
2. Make two separate local measurements.
v1
v2 Replot in velocity space
v2
v1vx
vy
The true velocity is at the intersection of the constraint lines.
7
The Smoothness ConstraintThe previous solution to the aperture problem requires a rigid object that is not rotating.
If the object is rotating or changing shape (deforming), we need another constraint to solve for velocity.
The "smoothness constraint" states that the velocity along a boundary (or within a 2D area of the image) varies smoothly.
Note: This is violated across the boundary of a moving object.
8
Measuring Velocity along a Contour
S
v
C
vy
vx€
∂v∂S
€
Length of ∂v
∂S is ∂v
∂S
Total variation over the curve is:
€
∂v∂S
2
C∫
To impose the smoothness constraint, we find the velocity field that minimizes the above integral.
9
Do Humans use a smoothness constraint?
The model incorporating the smoothness constraint finds the correct result for:
1) Pure translation of an object2) General motion of a rigid object with straight edges.
Nakayama and Silverman developed a stimulus that shows that humans integrate along contours and over small 2D area:
Oscillating contourLooks non-rigid
Add line-breaksLooks rigid
Add linesLooks rigid
10
Motion Illusions
The model fails for several cases where humans also do not see the correct velocity:
1) Rotating spirals (look like they are expanding)http://www.michaelbach.de/ot/mot_adaptSpiral/index.html
2) The Barberpole illusion (looks like it is moving up)http://www.123opticalillusions.com/pages/barber_pole.php
3) "Wobbling" ellipses.
The model computes velocities along the contours that are consistent with human perception.
Note: A few experiments have shown this is not exactly true all the time.
11
Rotating spirals
True velocity vectors
Initial measurements
Smoothest velocity fieldfrom initial measurements
12
Barberpole illusion
True velocity vectors
Smoothest velocity field
13
Motion Energy Models
An alternative way to think about 2D motion detection involves using spatio-temporal frequency filters.
This type of model relies on filters that are combined to detect a certain range of spatial and temporal frequencies.
Various people have developed versions of these models:van Santen & SperlingWatson & AhumadaAdelson & Bergen
14
Motion as orientation
In x-t space, motion is an oriented line. The slant depends on speed.In x-y-t space, motion becomes an oriented slab within a volume.
x-t
x-y-t
15
Orientation Detectors in Space-Time
Filters oriented in space time can detect a moving stimulus.The orientation of the filter relates to its preferred speed of motion.
These filters can detect sampled motion as well.
Oriented Spatio-temporal filters:
16
Separable Spatio-temporal filters
A Spatio-temporal filter can be created as the product between a spatial filter and a temporal filter.
Spatial impulse response = HS(x)Temporal impulse response = HT(t)Spatio-temporal impulse response: HST(x, t) = HS(x)HT(t)
17
Response to a Moving Edge
t1t2 t3
There is little response at t1 and t3.There is largest response at t2 during the edge motion
18
Oriented spatio-temporal filters
The previous filter was not selective for direction of motion.We can develop an oriented filter that is selective for direction, by creating a spatio-temporal Gabor filter:
€
g(x,t) =1
2πσ xσ t
e−
t 2
2σ t2
+x2
2σ x2
⎛
⎝ ⎜ ⎜
⎞
⎠ ⎟ ⎟
sin(2πωx x + 2πω t t)
-+
-
Filter selective for leftward motion
19
Response of oriented filter
Non-oriented
Oriented
Moving edgestimulus
Filter Response
20
Problems with Gabor filter
The Gabor filter by itself results in several problems:
1) It is phase sensitive: It depends on a particular alignment of the pattern with the filter at a given time. (The response to a drifting sinewave is an oscillation).
2) The sign of the response depends on the stimulus contrast (e.g. white on black gives opposite response to black on white).
21
Solution: Motion Energy
Motion energy filters are constructed with 2 gabor filters, one of which uses a sine and the other uses a cosine (a "quadrature pair").
If you square the outputs of the gabors and sum, the result is motion energy.
22
Motion Energy Responses
With motion energy filters:
The response is always positive.
The response is the same for a black-white edge as for a white-black edge.
The response to motion is independent of contrast.
The response is constant for a drifting sinewave.
Recommended