1

Computational Vision

CSCI 363, Fall 2012Lecture 24

Computing Motion

2

Measuring Motion from 2 points

Recall intersection of constraints:

vp1

vp2vp2

vp1vx

vy

Vp1

p2

€

Vx px1 +Vy py1 − vp1 = 0

Vx px2 +Vy py2 − vp 2 = 0

We can solve for Vx and Vy:

€

px =nxnx

2 + ny2

py =ny

nx2 + ny

2

v p =−dC

dt

⎛

⎝ ⎜

⎞

⎠ ⎟

nx2 + ny

2

€

dC

dt=C2 (x,y) −C1(x,y)

nx =C2 (x,y) −C2 (x −1,y)

ny =C2 (x,y) −C2 (x,y−1)

3

The Problem of NoiseProblem:Measurements are often inaccurate because:

1) Noise in the image measurements2) Inaccuracies due to the discrete (pixelated) image.

Solution:Make multiple measurements and find V that best matches all the

measurements.

The effects of noise will tend to average out.

4

Finding the best fitBecause of noise, there will be no single V that satisfies all the local measurements.

We want to find a V that fits the measurements best, i.e. it gives the least error when compared with predictions from measured quantities.

€

Error = Vx ,Vy( ) ⋅ pxi ,pyi( ) − vpi[ ]2

Predicted perpendicularvelocity

Measured perpendicularvelocity

5

Add up the errorsFor a given V = (Vx, Vy), we can find the total error by summing all the error terms for all the measured perpendicular velocities.

€

E = Vx ,Vy( ) ⋅ pxi ,pyi( ) − vpi[ ]2

i

∑

= Vx pxi +Vy pyi − vpi[ ]2

i

∑

Goal: Find V that gives the minimum error, E.

6

Finding the Minimum

To find the minimum, we find the point where the derivative is zero.

€

dE

dVx= 2 Vx pxi +Vy pyi − v pi[ ]

i

∑ pxi = 0

dE

dVy= 2 Vx pxi +Vy pyi − v pi[ ]

i

∑ pyi = 0

Rearranging, we have:

€

Vx pxi2

i

∑ +Vy pxipyii

∑ = vpipxii

∑

Vx pxipyii

∑ +Vy pyi2

i

∑ = vpipyii

∑

7

The Solution

€

a1 = pxi2

i

∑ a2 = b1 = pxipyii

∑ b2 = pyi2

i

∑

c1 = vpipxii

∑ c2 = vpipyii

∑€

a1Vx + b1Vy = c1

a2Vx + b2Vy = c2

The previous equation has the form:

where

Solving, we obtain:

€

Vx =c1b2 − c2b1

a1b2 − a2b1

Vy =a1c2 − a2c1

a1b2 − a2b1

8

Recall: Motion Energy

Motion energy filters are constructed with 2 gabor filters, one of which uses a sine and the other uses a cosine (a "quadrature pair").

If you square the outputs of the gabors and sum, the result is motion energy.

9

Reverse PhiIf a pattern of white and black lines is moved rightward in small steps, people see rightward motion.If the contrast is reversed with each step (white becomes black and vice versa), people see leftward motion. (The Reverse Phi Effect)Demo: http://psy2.ucsd.edu/~sanstis/Stuart_Anstis/Reverse_phi.html

EnergyWhite energy =>rightward motion.

Dark energy =>leftward motion.

Move pattern insteps.

Reverse Phi:Move pattern insteps while reversing contrast

10

Fluted Square Wave

When the fluted square wave is shifted to the right in 90 deg steps, it appears to move left!

A square wave that is shifted to the right in 90 deg steps, appears to move right.

90 deg step to right

A square wave with the fundamental frequency component removed is a fluted square wave. The highest amplitude component is 3f.

11

Fluted Square Wave

For a fluted square wave, the highest amplitude component is 3f.When the square wave (frequency f) moves 90 deg to the right, the 3f component is being shifted 270 deg to the right, which appears as 90 deg to the left.

Why? When a square wave that is shifted to the right in 90 deg steps, its fundamental frequency moves right in 90 deg steps.

12

Energy Response to a fluted square wave

x

t

Energy

Square wave

FlutedSquare wave

White = Right

Black = Left

13

Moving Plaid Demo

Demo of a moving plaid grating:

+ =

http://www.viperlib.com

Demo:

(Search on "plaid", choose coherence.mp4)

14

Motion Energy for 2D images

€

g(x,y,t) =1

2π( )3

2σ xσ yσ t

e−x2

2σ x2

+y2

2σ y2

+t 2

2σ t2

⎛

⎝

⎜ ⎜

⎞

⎠

⎟ ⎟

⎡

⎣

⎢ ⎢

⎤

⎦

⎥ ⎥

sin(2πωx x + 2πωy y+ 2πω t t)

For a 2D image, we use a 3-D gabor filter:

Selects frequency range within an ovoid in spatio-temporal frequencyspace:

sfxsfy

tf

15

Velocity lies on a PlaneFor a 1D image, all measurements of the same velocity lie along a line in SF-TF space (because v = TF/SF)

For a 2D image, all measurements of the same velocity lie on a plane in SF-TF space.€

ω t = vωxsf

tf

sfx

sfy

tf

€

ω t = uωx + vωy

Find the plane by making multiple measurements and finding best fit.

16

Extra-striate visual areas

Folded Cortex Flattened Cortex

17

Dorsal and Ventral Streams

18

Evidence for two processing streams

Evidence for separate streams of processing comes from three areas:

1) Lesion studies.Lesions in the ventral areas cause selective deficits in color and

orientation discrimination abilities. They can also cause deficits in object or face recognition.

Lesions in the dorsal areas cause selective deficits in judgments of motion (e.g. speed or direction). Can also cause deficits in localization of objects.

2) Psychophysics: Hard to see motion at "isoluminance".

3)Connection patterns:Parvocellular->4C->Superficial cortical layers (color and form)Magnocellular->4C->4B-> MT

19

Motion Processing in V1

In V1, some simple cells and complex cells are tuned to direction of motion. I.e. they respond most strongly to motion in a given direction and their response falls off as the motion deviates from that direction.

180o120o 240o

FiringRate

Direction of MotionDirection TuningPolar Plot(tuning for zero deg)

Tuning for 180 deg

20

V1 neurons tuned to temporal frequency

V1 neurons appear to be tuned to temporal frequency.Their preferred speed depends on the spatial frequency of the pattern.

FiringRate

Temporal Frequency

v = ωt/ωx

Neurons in V1 behave like motion energy filters.