12
Accuracy of velocity estimation by Reichardt correlators Ron O. Dror,* David C. O’Carroll, and Simon B. Laughlin Department of Zoology, University of Cambridge, Downing Street, Cambridge CB2 3EJ, UK Received January 3, 2000; accepted September 7, 2000; revised manuscript received September 20, 2000 Although a great deal of experimental evidence supports the notion of a Reichardt correlator as a mechanism for biological motion detection, the correlator does not signal true image velocity. This study examines the accuracy with which realistic Reichardt correlators can provide velocity estimates in an organism’s natural visual environment. The predictable statistics of natural images imply a consistent correspondence between mean correlator response and velocity, allowing the otherwise ambiguous Reichardt correlator to act as a prac- tical velocity estimator. Analysis and simulations suggest that processes commonly found in visual systems, such as prefiltering, response compression, integration, and adaptation, improve the reliability of velocity es- timation and expand the range of velocities coded. Experimental recordings confirm our predictions of cor- relator response to broadband images. © 2001 Optical Society of America OCIS codes: 330.4060, 330.4150, 330.4270, 150.4620. 1. INTRODUCTION Since its introduction by Hassenstein, Reichardt, and Varju ´, 1 as the first mathematical model of biological mo- tion detection, the Reichardt correlator model has gained widespread acceptance in the invertebrate-vision commu- nity. Most spatiotemporal energy models, currently the dominant models for motion detection in vertebrates, are mathematically equivalent to correlator models. 2 Cor- relator models have been applied directly to explain mo- tion detection in humans, birds, and cats. 35 After 40 years of physiological investigation, a funda- mental issue raised by Reichardt and his colleagues in their initial studies remains unanswered. Although both insects and humans appear capable of estimating image velocity, 6,7 the basic correlator model does not function as a velocity estimator. It reliably indicates directional mo- tion of sinusoidal gratings, but the response depends on contrast (brightness) and spatial frequency (shape) as well as on velocity. 8 The correlator is a nonlinear sys- tem, and its response to a moving broadband image, such as a natural scene, varies erratically as a function of time. In the absence of additional system components or as- sumptions, the raw output of a basic Reichardt correlator provides an inaccurate, ambiguous indication of image ve- locity. Some authors have concluded that animals that are capable of uniquely estimating velocity must possess either collections of differently tuned correlators 9 or an al- ternative motion-detection system that does not suffer from these problems. 6 Before discarding the Reichardt correlator as a velocity estimator, however, one must consider the behavior of a realistic correlator in a natural environment. Previous experimental studies have typically focused on responses to laboratory stimuli such as sinusoidal or square grat- ings; theoretical studies have modeled the response of an idealized correlator to these artificial stimuli. In this study, we first examine the responses of a Reichardt cor- relator to motion of natural broadband images. Recent work has shown that certain image statistics are highly predictable in the natural world. 1013 We show that natural-image statistics cause motion-detection systems based on biological Reichardt correlators to respond more reliably to typical natural images than to arbitrary im- ages (Section 3). Second, we study the effects of addi- tional physiological components. Previous work has ex- perimentally described and computationally modeled various forms of spatial and temporal prefiltering, satu- ration, integration, and adaptation within the motion- detection system; 1,3,1423 we consider their implications for accurate velocity estimation. We find that these com- ponents may improve the performance of the correlator- based system in response to a variety of stimuli, including complex natural images (Section 4). In Section 5 we summarize the results of an experimental study, to be published separately, 24 that confirms many of our predic- tions. Whereas the present study applies to Reichardt corr- elators in general, we have chosen the fly as a model or- ganism for computational simulations and experiments because of the abundance of behavioral, anatomical, and electrophysiological data available for its motion- detection system. In particular, the models of the follow- ing sections are based on electrophysiological, anatomical, and optical data for large flies such as the blowfly Calli- phora and the hoverflies Volucella and Eristalis. 2. CORRELATOR RESPONSE TO NARROW- BAND IMAGE MOTION Figure 1(a) shows a simplified version of the correlator model. Receptors A and B are separated by an angular distance Df. The signal from A is temporally delayed by the low-pass filter D before multiplication by the signal from B. This multiplication produces a positive output in response to rightward image motion. To achieve similar sensitivity to leftward motion and to cancel excitation by Dror et al. Vol. 18, No. 2 / February 2001 / J. Opt. Soc. Am. A 241 0740-3232/2001/020241-12$15.00 © 2001 Optical Society of America

Accuracy of velocity estimation by Reichardt correlators

  • Upload
    simon-b

  • View
    212

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Accuracy of velocity estimation by Reichardt correlators

Dror et al. Vol. 18, No. 2 /February 2001 /J. Opt. Soc. Am. A 241

Accuracy of velocity estimation by Reichardtcorrelators

Ron O. Dror,* David C. O’Carroll,† and Simon B. Laughlin

Department of Zoology, University of Cambridge, Downing Street, Cambridge CB2 3EJ, UK

Received January 3, 2000; accepted September 7, 2000; revised manuscript received September 20, 2000

Although a great deal of experimental evidence supports the notion of a Reichardt correlator as a mechanismfor biological motion detection, the correlator does not signal true image velocity. This study examines theaccuracy with which realistic Reichardt correlators can provide velocity estimates in an organism’s naturalvisual environment. The predictable statistics of natural images imply a consistent correspondence betweenmean correlator response and velocity, allowing the otherwise ambiguous Reichardt correlator to act as a prac-tical velocity estimator. Analysis and simulations suggest that processes commonly found in visual systems,such as prefiltering, response compression, integration, and adaptation, improve the reliability of velocity es-timation and expand the range of velocities coded. Experimental recordings confirm our predictions of cor-relator response to broadband images. © 2001 Optical Society of America

OCIS codes: 330.4060, 330.4150, 330.4270, 150.4620.

1. INTRODUCTIONSince its introduction by Hassenstein, Reichardt, andVarju,1 as the first mathematical model of biological mo-tion detection, the Reichardt correlator model has gainedwidespread acceptance in the invertebrate-vision commu-nity. Most spatiotemporal energy models, currently thedominant models for motion detection in vertebrates, aremathematically equivalent to correlator models.2 Cor-relator models have been applied directly to explain mo-tion detection in humans, birds, and cats.3–5

After 40 years of physiological investigation, a funda-mental issue raised by Reichardt and his colleagues intheir initial studies remains unanswered. Although bothinsects and humans appear capable of estimating imagevelocity,6,7 the basic correlator model does not function asa velocity estimator. It reliably indicates directional mo-tion of sinusoidal gratings, but the response depends oncontrast (brightness) and spatial frequency (shape) aswell as on velocity.8 The correlator is a nonlinear sys-tem, and its response to a moving broadband image, suchas a natural scene, varies erratically as a function of time.In the absence of additional system components or as-sumptions, the raw output of a basic Reichardt correlatorprovides an inaccurate, ambiguous indication of image ve-locity. Some authors have concluded that animals thatare capable of uniquely estimating velocity must possesseither collections of differently tuned correlators9 or an al-ternative motion-detection system that does not sufferfrom these problems.6

Before discarding the Reichardt correlator as a velocityestimator, however, one must consider the behavior of arealistic correlator in a natural environment. Previousexperimental studies have typically focused on responsesto laboratory stimuli such as sinusoidal or square grat-ings; theoretical studies have modeled the response of anidealized correlator to these artificial stimuli. In thisstudy, we first examine the responses of a Reichardt cor-relator to motion of natural broadband images. Recent

0740-3232/2001/020241-12$15.00 ©

work has shown that certain image statistics are highlypredictable in the natural world.10–13 We show thatnatural-image statistics cause motion-detection systemsbased on biological Reichardt correlators to respond morereliably to typical natural images than to arbitrary im-ages (Section 3). Second, we study the effects of addi-tional physiological components. Previous work has ex-perimentally described and computationally modeledvarious forms of spatial and temporal prefiltering, satu-ration, integration, and adaptation within the motion-detection system;1,3,14–23 we consider their implicationsfor accurate velocity estimation. We find that these com-ponents may improve the performance of the correlator-based system in response to a variety of stimuli, includingcomplex natural images (Section 4). In Section 5 wesummarize the results of an experimental study, to bepublished separately,24 that confirms many of our predic-tions.

Whereas the present study applies to Reichardt corr-elators in general, we have chosen the fly as a model or-ganism for computational simulations and experimentsbecause of the abundance of behavioral, anatomical, andelectrophysiological data available for its motion-detection system. In particular, the models of the follow-ing sections are based on electrophysiological, anatomical,and optical data for large flies such as the blowfly Calli-phora and the hoverflies Volucella and Eristalis.

2. CORRELATOR RESPONSE TO NARROW-BAND IMAGE MOTIONFigure 1(a) shows a simplified version of the correlatormodel. Receptors A and B are separated by an angulardistance Df. The signal from A is temporally delayed bythe low-pass filter D before multiplication by the signalfrom B. This multiplication produces a positive output inresponse to rightward image motion. To achieve similarsensitivity to leftward motion and to cancel excitation by

2001 Optical Society of America

Page 2: Accuracy of velocity estimation by Reichardt correlators

242 J. Opt. Soc. Am. A/Vol. 18, No. 2 /February 2001 Dror et al.

stationary stimuli, a parallel delay-and-multiply opera-tion takes place with a delay on the opposite arm. Theoutputs of the two multiplications are subtracted to give asingle time-dependent correlator output R.

Although the correlator is nonlinear, its response tosinusoidal stimuli is of interest. If the input is a sinu-soidal grating that contains only a single frequency com-ponent, the oscillations of the two subunits cancel and thecorrelator produces a constant output. For any linear de-lay filter, the output level depends separably on spatialand temporal frequency.8 If the delay filter D is first or-der low pass with time constant t, as in most modelingstudies,8,19 a sinusoid of amplitude C and spatial fre-quency fs traveling to the right at velocity v produces anoutput

R~t ! 5C2

2pt

ft

f t2 1 1/~2pt!2

sin~2pfsDf!, (1)

where ft 5 fsv is the temporal frequency of the inputsignal.8 At a given spatial frequency, the magnitude ofcorrelator output increases with temporal frequency up toan optimum ft,opt 5 1/(2pt) and then decreases mono-tonically as the velocity continues to increase. Outputalso varies with the square of C, which specifies gratingbrightness or, in the presence of preprocessing stages,grating contrast. A physical luminance grating musthave positive mean luminance, so it will contain a dc com-ponent as well as an oscillatory component. In this case,the output will oscillate about the level given by Eq. (1).

3. CORRELATOR RESPONSE TOBROADBAND IMAGESSince the correlator is a nonlinear system, its response toa generic stimulus cannot be represented as a sum of re-sponses to sinusoidal components of the input. In par-ticular, the response to a broadband image, such as anatural scene, varies dramatically with time, despite thefact that image velocity is constant [see Fig. 6(a) below].

Fig. 1. (a) Minimal Reichardt correlator with a potential inputluminance signal, in this case a sinusoid with amplitude C andwavelength l 5 1/fs moving at velocity v. (b) A more realisticcorrelator model, considered in Section 4, includes spatial prefil-ters, S’s; temporal prefilters, T’s; compressive nonlinearities, r’sand u; temporal integration, M and spatial summation, (.

A. Evaluation of Correlator PerformanceTo compare the performance of various systems, we quan-tify two basic requirements for accurate velocity estima-tion:

1. Image motion at a specific velocity should alwaysproduce approximately the same response.

2. The response to motion at a given velocity should beunambiguous over the range of functionally relevant ve-locities.

We restrict the range of potential input stimuli by focus-ing on responses to rigid, constant-velocity motion as ob-served by an eye undergoing rotational motion.

Given a large image moving at a particular constant ve-locity, consider an array of identically oriented correlatorssampling the image at a dense grid of points in space andtime. Define the mean response value R as the averageof the ensemble outputs, and the relative error as thestandard deviation of the ensemble divided by the meanresponse value. We call the graph of R as a function ofvelocity the velocity response curve. To satisfy require-ment 1, different images should have similar velocity re-sponse curves and the relative error should remain small.Requirement 2 implies that the velocity response curveshould be monotonic in the range of motion velocitiesamong which the correlator system must discriminate.

Figure 2(a) shows velocity response curves for two

Fig. 2. Velocity response curves for the simple correlator modelof Fig. 1(a). (a) Response curves for horizontal sinusoidal grat-ings of 100% contrast at two different spatial frequencies; (b) re-sponse curves for the natural images shown in Fig. 3; (c) thesame four curves, normalized so that their maximum values areidentical.

Page 3: Accuracy of velocity estimation by Reichardt correlators

Dror et al. Vol. 18, No. 2 /February 2001 /J. Opt. Soc. Am. A 243

Fig. 3. Examples of the natural images used in simulations throughout this work. Images a and b are panoramic images that repre-sent the visual scenes in locations where Episyrphus chooses to hover. Images were obtained by using a video camera with a linear CCDelement, then digitized, calibrated to real luminance units, and mosaicked into panoramas spanning a full circle horizontally and ap-proximately 23° vertically (one spatial degree corresponds to 4.64 pixels). Images c and d are samples of the image set acquired byDavid Tolhurst12 with a still camera and then digitized and corrected for luminance nonlinearities in the film. Images in this set mea-sure 256 pixels both horizontally and vertically, with approximately 10 pixels to 1°.

simulated sinusoidal gratings of different spatial frequen-cies. The curves for the two gratings differ significantly,so the mean response level indicates velocity only if thespatial frequency is known. In addition, the individualvelocity response curves peak at low velocities, abovewhich their output is ambiguous. Appendix A describesmodel parameters and simulation techniques usedthroughout this paper.

B. Simulation with Natural ImagesOne can perform similar simulations with natural im-ages. In view of the fact that the characteristics of natu-ral images depend on the organism in question and on itsbehavior, we worked with two sets of photographically ac-quired images. The first set consisted of panoramic im-ages collected from the habitat of the hoverfly Episyrphusbalteatus in the woods near Cambridge, UK. Episyrphusspends much of its time hovering in specific stationary po-sitions, which were selected as panorama centers; two ex-amples appear in Fig. 3. The second set of photographs,acquired by David Tolhurst,12 includes both scenes thatmight be considered ‘‘natural’’ for most animals, such astrees and landscapes, and scenes more typical of a humanenvironment, such as people, buildings, and a computer.These photographs were taken at various distances fromthe objects of interest and under various conditions of il-lumination. Figure 3 includes two sample images fromthis set.

The images in both sets contained only luminance(gray-level) information. The motion-detection pathway

in the fly appears to be dominated by the R1-6 photore-ceptors of each ommatidium, which all have a commonbroadband spectral sensitivity.25 Neither set of imageswas photographed through a filter matched to this spec-tral sensitivity, but previous studies found that, althoughcontrast of natural scenes varies significantly in differentparts of the electromagnetic spectrum, the distribution ofspatial frequencies does not.10 Different spectral sensi-tivities will therefore lead primarily to overall amplitudescaling of the response.

We normalized each image by scaling the luminancevalues to a mean of 1.0, in order to model the responses ofphotoreceptors, which adapt to mean luminance level andsignal the contrast of changes about that level.26 In theabsence of such luminance normalization, the overall re-sponse amplitude of the motion-detection system woulddepend not only on contrast (Subsection 3.C below) butalso on luminance.

Figure 2(b) shows velocity response curves for the im-ages of Fig. 3. The most notable difference between thecurves is their relative magnitude. When the curves arenormalized so that their peak values are equal [Fig. 2(c)],they share not only their bell shape but also nearly iden-tical optimal velocities. We repeated these simulationson most of the images in both sets and found in nearly allcases that whereas velocity response curves for differentimages differ significantly in absolute magnitude, theirshapes and optimal velocities vary little. This similarityin shape is important because, if the motion-detectionsystem could normalize or adapt its response to remove

Page 4: Accuracy of velocity estimation by Reichardt correlators

244 J. Opt. Soc. Am. A/Vol. 18, No. 2 /February 2001 Dror et al.

the difference in magnitude between these curves, themean correlator response would provide useful informa-tion on image velocity independently of the visual scene.In Subsection 3.C we establish the basis of the similarityin shape, and both modeling and experiment (Subsections3.C, 4.C, and 5) suggest that normalization occurs. Thuscorrelators can obtain reasonable velocity estimates fromnatural broadband stimuli, even though they cannot ex-tract the velocity of a sinusoidal grating when the spatialfrequency is unknown.

C. Mathematical Analysis of Mean Response toBroadband ImagesIn this section we develop a general mathematical rela-tionship between the power spectrum of an image and themean correlator response, explaining the empirical simi-larities in shape and differences in magnitude betweenvelocity response curves for different images. Naturalimages differ from sinusoidal gratings in that they pos-sess energy at multiple nonzero spatial frequencies, sothat they produce broadband correlator input signals.As an image moves horizontally across a horizontally ori-ented correlator, one row of the image moves across thetwo correlator inputs. One might think of this row as asum of sinusoids representing the Fourier components ofthat row. Because of the nonlinearity of the multiplica-tion operation, the correlator output in response to themoving image will differ from the sum of the responses tothe individual sinusoidal components. The response to asum of two sinusoids of different frequencies f1 and f2consists of the sum of the constant responses predicted byEq. (1) to each sinusoid individually, plus oscillatory com-ponents of frequencies f1 1 f2 and uf1 2 f2u. Sufficientspatial or temporal averaging of the correlator output willeliminate these oscillatory components. The correlatortherefore exhibits pseudolinearity or linearity in themean,27 in that the mean output in response to a broad-band image is equal to the sum of the responses to eachindividual sinusoidal input component.

This pseudolinearity property implies that the meanresponse of a simple Reichardt correlator to a single rowof an image depends only on the power spectrum of thatrow. Using Eq. (1) for correlator response to a sinusoidand the fact that ft 5 fsv, we can write the mean cor-relator output as

R 51

2ptE

0

`

P~ fs!fsv

~ fsv !2 1 1/~2pt!2 sin~2pfsDf!dfs ,

(2)

where P( fs) represents the power spectral density of onerow of the image at spatial frequency fs . Each velocityresponse curve shown in Fig. 2(b) is an average of themean outputs of correlators exposed to different horizon-tal image rows with potentially different power spectra.This average is equivalent to the response of the cor-relator to a single row whose power spectrum P( fs) is themean of the power spectra for all the rows of the image.

If P( fs) were arbitrary, Eq. (2) would provide little in-formation about the shape of the velocity response curve.However, the power spectra measured from a wide vari-ety of natural images are highly predictable, being gener-

ally proportional to f 2(21h), where f is the modulus of thetwo-dimensional spatial frequency and h is a smallconstant.10–13 If an image has an isotropic two-dimensional power spectrum proportional to f 2(21h), theone-dimensional power spectrum of any straight-line sec-tion through the image is proportional to f 2(11h).28

Overall contrast, which determines overall amplitude ofthe power spectrum, varies significantly among naturalimages and among orientations.29 The best value of halso depends on image and orientation, particularly forimages from different natural environments. Van derSchaaf and van Hateren29 found, however, that a modelthat fixes h 5 0 while allowing contrast to vary sufferslittle in its fit to the data compared with a model that per-mits variation in h.

The similarities in natural image power spectra lead topredictable peak response velocities and to similarities inthe shapes of the velocity response curves for different im-ages. Figure 4 shows velocity response curves predictedfrom hypothetical row power spectra P( fs) 5 f s

21, f s21.25 ,

and f s20.75 , corresponding to h 5 0, 0.25, 20.25, respec-

tively. The theoretical curves match one another and thesimulated curves closely below the peak response value;in this velocity range the velocity response is insensitiveto the value of the exponent in the power spectrum. Thesimulated velocity response curves show greater variationat high velocities because the image takes less time tomove over the correlator array, so that responses were av-eraged over a shorter period of time. Figure 11 below il-lustrates that broadband images with significantly differ-ent power spectra have significantly different velocityresponse curves.

Contrast differences between images explain the over-all amplitude differences between the curves. Figure 5shows horizontal power spectral densities for the imagesof Fig. 3, computed by averaging the power spectral den-sities of the rows that constitute each image. On log–logaxes the spectra approximate straight lines with slopesclose to 21, although the spectrum of image (d) has no-ticeable curvature. The relative magnitudes of the spec-tra correspond closely to the relative magnitudes of thevelocity response curves of Fig. 2(b), as predicted by Eq.(2). Differences in the magnitude of the velocity responsecurves correspond to differences in overall contrast, ex-cept that image (d) has the largest response even thoughits contrast is larger than that of curve (b) only for fre-

Fig. 4. Velocity response curves predicted by Eq. (2) for rowpower spectra P( fs) 5 f s

2(11h) . Simulated velocity responsecurves from Fig. 2(c) are shown by thin dotted curves for com-parison. Predicted peak response velocities are 32, 35, and 40°/sfor h 5 20.25, 0, and 0.25, respectively. Predicted curves havebeen normalized to a maximum value of 1.0 through multiplica-tion by 1.26, 1.0, and 0.92, respectively.

Page 5: Accuracy of velocity estimation by Reichardt correlators

Dror et al. Vol. 18, No. 2 /February 2001 /J. Opt. Soc. Am. A 245

quencies near 0.1 cycle/°. This reflects the fact that somespatial frequencies contribute more than others to thecorrelator response.

For the mean correlator response to be used as a reli-able indicator of velocity, the visual system needs to com-pensate for these contrast variations. Our simulations(Subsection 4.C below) and experimental results (Section5) indicate that contrast saturation early in the motion-detection pathway eliminates significant differences incontrast. A form of contrast normalization akin to thatobserved in vertebrate vision systems30 may also removecontrast differences between images. For example,Osorio31 observed adaptive responses to contrast in the lo-cust medulla.

4. FUNCTIONAL ROLE OF ADDITIONALSYSTEM COMPONENTSAlthough the simple correlator model of Fig. 1 producesmore meaningful estimates of velocity for natural imagesthan for arbitrary sinusoids, it suffers from two majorshortcomings. First, the standard deviation of the cor-relator output is huge relative to its mean, with relativeerror values ranging from 3.3 to 76 [Fig. 6(b)]. Second,the mean correlator response for most natural imagespeaks at a velocity of 35–40°/s [Fig. 2(b)]. Because thevelocity range below the peak response corresponds to themost probable range of inputs, we assume that in the ab-sence of contradictory information a correlator response isinterpreted as the lower of the two putative velocities.Image velocities above the peak will therefore be misin-terpreted. A shorter delay filter time constant wouldraise the peak response velocity, but experimentally de-scribed time constants23 are not sufficiently low to ac-count for the fact that insects may turn and track targetsat velocities up to hundreds of degrees per second.32

In this section we show that additional physiologicalcomponents help to overcome these problems, raising thepeak response velocity and lowering the relative error ofthe correlator output. Figure 1(b) illustrates an elabo-rate correlator, including spatial and temporal prefilter-ing, compressive nonlinearities, and output integration.

Fig. 5. Horizontal power spectral densities of the images in Fig.3. Each spectrum is an average of power spectral densities ofthe rows that constitute the image. Images c and d have highermaximum and nonzero minimum frequencies because Tolhurst’simages are more finely sampled and span a smaller portion of thehorizontal visual field than the panoramic images. Images aand b roll off in power at frequencies above 1.2 cycles/° as a resultof averaging in the image acquisition process. Fortunately, thesampling lattice of the fly’s photoreceptors has a Nyquist fre-quency near 0.5 cycle/°, and optical blur effects in the fly’s eyereject almost all spatial frequency content above 1 cycle/° (seeSubsection 4.B.).

In determining the functional significance of these effects,we utilize models proposed in earlier experimentalstudies.1,3,14–21

A. Generalized Mathematical AnalysisEquation (2) generalizes naturally to include linear spa-tial and temporal prefilters, denoted S and T in Fig. 1(b).In the absence of prefilters, the temporal input signals tothe correlator are the intensity values along rows of theimage. Prefilters modify these input signals. For mo-tion at a given constant velocity, one might consider anequivalent model in which the prefilters simply modifythe original image before it passes over the correlator.One can then determine the effects of prefilters on the ve-locity response curves by determining how the prefilterschange the mean horizontal power spectrum P( fs) for theimage.

The effect of a temporal prefilter on the equivalentpower spectrum depends on velocity, whereas that of aspatial prefilter does not. A linear temporal filter withfrequency response T( ft) produces an output whosepower spectrum can be computed by multiplying the in-put power spectrum by T2( ft). The power spectrum of aspatially filtered image is S2( fs)P( fs), where S( fs) rep-resents the frequency response of a one-dimensional fil-tering operation with an effect on the horizontal powerspectrum equivalent to that of the two-dimensional filterof interest.28,33 The mean horizontal power spectrum ofthe prefiltered image in the presence of both spatial andtemporal prefilters is therefore T2( fsv)S2( fs)P( fs). Ac-cording to Eq. (2), the velocity response curve is given by

R~v ! 51

2ptE

0

`

@S2~ fs!P~ fs!sin~2pfsDf!#

3 FT2~ fsv !fsv

~ fsv !2 1 1/~2pt!2Gdfs . (3)

Fig. 6. (a) Time response of a simple correlator to image b ofFig. 3, moving at 19.6°/s. The dotted curve indicates the meanresponse level. (b) Relative error curves for the four natural im-ages of Fig. 3.

Page 6: Accuracy of velocity estimation by Reichardt correlators

246 J. Opt. Soc. Am. A/Vol. 18, No. 2 /February 2001 Dror et al.

The integrand is a product of terms each of which de-pends either only on spatial frequency fs or only on tem-poral frequency ft 5 fsv. Equation (3) allows one tomake qualitative predictions about the effects of systemcomponents independently of precise parameter values,confirming the generality of effects observed in simula-tions. Appendix B outlines a graphic analysis methodthat simplifies this process.

B. Effects of Physiological PrefiltersLuminance signals are filtered spatially and temporallyin the early visual system. The retinal signal sampled bythe photoreceptors has already been blurred spatially ow-ing to the diffraction effects of the lens optics and theproperties of the photoreceptors themselves.34 Photore-ceptors act as low-pass temporal filters because they de-pend on chemical transduction processes that cannot re-spond instantly to changes in the luminance signal.26

The large monopolar cells, (LMC’s), which are generallyassumed to feed motion-detector inputs,35 perform high-pass filtering, particularly in the temporal domain.18 Toa first approximation, these filters are linear and sepa-rable into spatial and temporal components.18 Spatialand temporal prefilters have been included in correlatormodels by numerous authors;1,3,14 we wish to determinetheir effect on accuracy of velocity estimation for naturalimages.

Figure 7(a) shows simulated velocity response curvesfor a natural image in the presence and in the absence ofa low-pass spatial prefilter in addition to the correspond-ing predictions of Eq. (3). We used a circularly symmet-ric Gaussian filter with half-width 1.48°, which approxi-mates the acceptance function of typical flyphotoreceptors.15 The low-pass filter raises the peak re-sponse velocity from 37°/s to 60°/s, expanding the velocityrange coded in the monotonically increasing portion of thevelocity response curve.

Figure 7(b) shows that low-pass spatial prefilteringdramatically decreases the relative error of the correlator,by a factor of 3 at low velocities. Spatial frequenciesabove 1 cycle/° contribute little to R, even in the absenceof a spatial prefilter, but these frequencies contribute sig-nificantly to variance in the output signal. By filteringthem out, one reduces the output variance much morethan its mean. At very high velocities the relative errorbegins to rise, even in the presence of spatial prefiltering,because spatial frequencies passed by the prefilter gener-ate high-frequency inputs to the correlator.

We examined the effects of low-pass temporal filteringby photoreceptors with a temporal impulse response ofthe log-normal form proposed by Payne and Howard:16

p~t ! 5 expH 2@loge~t/tp!#2

2s2 J , (4)

where tp represents the time to peak of the curve and s isa dimensionless parameter that determines the curve’swidth.16 We used tp 5 7.8 ms and s 5 0.22 for light-adapted conditions and tp 5 26 ms and s 5 0.32 for dark-adapted conditions, based on the results of Tatler et al.17

for Calliphora at 24 °C.

Unlike a low-pass spatial prefilter, a low-pass temporalprefilter decreases peak response velocity. Under light-adapted conditions, temporal prefiltering by the photore-ceptor has little effect because the response rolls off onlyat very high frequencies, and these frequencies are alsoblocked by the correlator’s delay filter. Under dark-adapted conditions the photoreceptor response rolls off atlower frequencies, causing a small decrease in peak re-sponse velocity. Both light- and dark-adapted photore-ceptors significantly reduce the relative error at high ve-locities by rejecting high temporal frequencies, whichcontribute to temporal variations in the output.

We based our simulation of high-pass temporal filter-ing on the work of James,18 who characterized the tempo-ral response of Eristalis LMC’s to continuously varying(white noise) stimuli. James found that the responsewas dominated by a linear component whose impulse re-sponse could be modeled as a difference of two log nor-mals of the form of Eq. (4) but with different time con-stants. At high light levels, he found typical values oftp 5 10.3 ms and s 5 0.236 for the positive log normal,and tp 5 15.6 ms and s 5 0.269 for the negative log nor-mal. Recordings by Laughlin26 show that LMC’s virtu-ally eliminate the dc component in their response, so weadjusted the relative weights of the two log normals togive a zero-mean impulse response. This temporal pre-filter incorporates the effects of both the LMC and thelight-adapted photoreceptor and therefore represents abandpass filter.

Figure 8(a) shows that this temporal prefilter morethan quadruples the peak response velocity, such that themonotonically increasing portion of the curve extends tovelocities greater than 200 °/s. Higher image velocitiesmight result from high-speed turning motions, but such

Fig. 7. Effect of low-pass spatial prefiltering due to optical blur-ring. (a) Simulated velocity response curves for the natural im-age of Fig. 3b with and without the spatial prefilter, comparedwith the analytical predictions of Eq. (3) and assuming a hori-zontal image power spectrum P( fs) 5 fs

21.1 . These curves arenormalized to have a maximum value of 1.0. The simulation re-sults closely match the analytical predictions. (b) Relative errorcurves with and without the effects of optic blurring.

Page 7: Accuracy of velocity estimation by Reichardt correlators

Dror et al. Vol. 18, No. 2 /February 2001 /J. Opt. Soc. Am. A 247

fast rotational egomotion would likely be disambiguatedby the response of the Dipteran halteres, which have ac-celeration sensors that are analogous to the semicircularcanals of humans.36 Addition of the LMC temporal filterto the model also decreases relative error by at least a fac-tor of 2 [Fig. 8(b)]. Very low temporal frequencies, likevery high temporal frequencies, contribute to variation inoutput without contributing to the mean response.

James’s recordings show that some LMC’s performhigh-pass spatial filtering of their inputs, perhaps by alateral inhibition mechanism, but the phenomenon isweaker and more variable than high-pass temporalfiltering.18,26 Such weak high-pass spatial filteringwould slightly decrease both the peak response velocityand the relative error.

While both spatial and temporal prefiltering decreaserelative error, they have qualitatively opposite effects onthe velocity response curves. Low-pass spatial filteringincreases the peak response velocity, whereas low-passtemporal filtering reduces it. The physiological systemexhibits a predominance of spatial low-pass filtering andtemporal high-pass filtering, both of which increase peakresponse velocity and therefore broaden the range of ve-locities coded in the monotonically increasing portion ofthe response curve. This suggests that prefilters are de-signed to improve the accuracy of velocity estimation inaddition to optimizing information transmission in thepresence of noise.26

C. Response Compression and SaturationCompressive nonlinearities, which are ubiquitous in neu-ral receptors and synapses, occur at multiple points in themotion-detection system. As stimulus contrast increasesabove a few percent, correlator response amplitude levelsoff rather than increasing quadratically,19 an experimen-

Fig. 8. Effect of high-pass temporal prefiltering by an LMC withimpulse response as described in the text. Other model param-eters, including optical blurring, are as in Fig. 7. (a) Simulatedand analytically predicted velocity response curves in the pres-ence and in the absence of LMC temporal filters, normalized tohave maximum value 1.0. (b) Simulated relative error curves inthe presence and in the absence of the temporal prefilters.

tally observed effect that we term contrast saturation.Following Egelhaaf and Borst,19 we modeled contrastsaturation by including a dominant compressive nonlin-earity directly before the multiplication operation on allcorrelator arms [r in Fig. 1(b)]. We used a hyperbolictangent function of the form

r~C ! 5 tanh~sC !, (5)

where the constant s was chosen such that a sinusoidalgrating of 10% contrast at optimal spatial and temporalfrequency elicits 70% of the maximum response, in accor-dance with the recordings of O’Carroll et al.20 from HSneurons in Volucella.

Figure 9 shows the velocity response and relative errorcurves for two natural images in the presence and in theabsence of contrast saturation. The figure illustratesthree primary effects of contrast saturation on themotion-detection system. First, the velocity responsecurves for the two images are much closer together inoverall amplitude for a correlator model that includescontrast saturation than for one that does not. Second,contrast saturation decreases the peak response velocity.Third, contrast saturation decreases relative error at allbut very high velocities.

These effects stem from the tendency of a compressivenonlinearity to convert a zero-mean input into an outputthat resembles a binary signal that alternates between amaximum and a minimum value. As long as the ampli-tude of the input signal is large enough for the output toremain saturated to its maximum or minimum valuemost of the time, increasing the amplitude of the inputsignal further will have only a minor effect on the output.Hence, contrast saturation decreases the dependence ofthe velocity response curve on the overall contrast level ofthe visual scene, except at very low contrasts. As notedin Subsection 3.C, such a contrast-independent responseenables the correlator output to provide a useful indica-tion of image velocity.

Fig. 9. Effect of contrast saturation on velocity response and er-ror curves for images a and b of Fig. 3. The correlator modelincludes the spatial and temporal prefilters of Fig. 8. Mean re-sponses for images a and b are expressed in identical units.

Page 8: Accuracy of velocity estimation by Reichardt correlators

248 J. Opt. Soc. Am. A/Vol. 18, No. 2 /February 2001 Dror et al.

The outputs of the wide-field neurons also approachsaturation owing to shunting of the membranepotential,21 which might be modeled as a compressivenonlinearity following spatial integration [u in Fig. 1(b)].Such an effect will flatten the peaks of the velocity re-sponse curves, allowing the neuron to use more of its dy-namic range to signal low velocities. This may be crucialfor correlators with high peak velocities, since in the ab-sence of saturation the response curves are fairly flat atlow velocities (see, e.g., Fig. 8).

D. Output IntegrationSpatial or temporal integration of correlator outputs willnot affect the velocity response curve, which by definitionis the mean of the correlator response over all space andtime. However, integration will decrease the variance ofthe output signal, as illustrated by Single and Borst.37

Figure 10 shows the effects of spatial integration akin tothat performed by an HS neuron. In the absence of con-trast saturation, such integration decreases the relativeerror by approximately a factor of 3 to values near 1.0.Output variance decreases even more in the presence ofsaturation, with relative errors as low as 0.2 for velocitiesnear the peak response velocity. Saturation reduces theeffects of contrast variations from one part of the image toanother. Such regional variations become particularlysignificant in the presence of integration, which elimi-nates the effects of fluctuations of a more localized nature.Error levels of 0.2–1.0 are in line with temporal fluctua-tions observed in the outputs of HS neurons in responseto stimulation with patterns that are moving at constantvelocity.20

Although integration improves the accuracy of a veloc-ity estimator, it reduces the resolution. Various neuralpathways integrate correlator outputs to various extentsin space and time, depending on the requirements of aparticular task. Wide-field neurons specifically measureegomotion as part of the optomotor pathway.38 In suchcases, temporal resolution is more critical than spatialresolution, so the system integrates with respect to spacerather than time. Chasing a conspecific, on the other

Fig. 10. Effect of spatial integration on relative error for imageb of Fig. 3, with and without contrast saturation. Model param-eters are as in Fig. 9. We integrated outputs of an array of cor-relators arranged on an 8 3 18 grid with 1° separation betweencorrelators, corresponding to a small HS cell. For simplicity, weused a square rather than a hexagonal grid. Integration of 144independent outputs would lead to a 12-fold reduction in noise.Because the error signals in different correlators are correlated,relative error decreases by a smaller factor of 3 to 5.

hand, requires a local motion estimate, so spatial integra-tion must be limited, although some temporal integrationmay be permissible.

E. Additional System Components and FurtherConsiderationsIn addition to deviations in output resulting from thestructure of the visual scene itself, a physiological cor-relator also suffers from random noise, including photonnoise and intrinsic noise generated by the neurons andsynapses that constitute the correlator. Laughlin26

showed that, in the LMC’s, photon noise dominates in-trinsic noise up to moderate light intensities and equalsintrinsic noise in magnitude even at the highest light in-tensities. We simulated correlator output in response tonatural images in the presence of photon noise, modelingphoton absorption by the photoreceptor as a Poisson pro-cess. Our results showed that while photon noise leadsto a slight increase in relative error, its contribution is anorder of magnitude less than the relative error of thenoise-free simulations for our set of panoramic images,which were recorded in a forest near the lowest light lev-els at which Episyrphus remains active. We concludethat, even under low lighting conditions, random noisehas a relatively minor effect on the performance of themodels discussed in this paper.

Our analysis of the accuracy with which a realistic Rei-chardt correlator can estimate velocity in a natural envi-ronment remains incomplete for at least two reasons.First, we considered a restricted class of visual signalscorresponding to constant-velocity motion. Time-varyingmotion leads to additional differences between the re-sponse of the basic Reichardt correlator and that of anideal velocity estimator.39 Egelhaaf and Borst14 showedthat appropriate low-pass spatial prefiltering improvesthe representation of sinusoidal velocity fluctuations bythe correlator. Drawing on the findings of the presentstudy, one might examine correlator response to time-varying motion of natural images by taking into accountstatistics of natural image sequences, which extend be-yond the statistical properties of individual images.40

Second, numerous nonlinear and adaptive phenomenabeyond those included in Fig. 1(b) have been observed ex-perimentally in the motion-detection system. For ex-ample, the output level to which the wide-field neuronsaturates with increasing pattern size depends on stimu-lus velocity.21 Although the details of this gain controlare still uncertain, the effect allows the wide-field neuronto signal changes in image velocity despite saturation ofits output signal. Maddess and Laughlin22 and Harriset al.23 described a motion-induced decrease in correlatorresponse to motion. Such motion adaptation may allowthe correlator to respond sensitively to small changes inmotion at low velocities while releasing it from saturationat higher velocities, effectively increasing its dynamicrange. Harris et al. also showed that the delay filter’stime constant does not adapt to motion. We have there-fore kept the delay filter’s impulse response constant inall simulations.

In their original paper describing the spatiotemporalenergy model of motion detection, Adelson and Bergen2

proved that any basic Reichardt correlator, with arbitrary

Page 9: Accuracy of velocity estimation by Reichardt correlators

Dror et al. Vol. 18, No. 2 /February 2001 /J. Opt. Soc. Am. A 249

spatial and temporal prefilters, has an exact equivalentspatiotemporal energy model. Although precise equiva-lence may break down as a result of nonlinearities in thesystem or for spatiotemporal energy models with complexfilter combinations, Adelson and Bergen pointed out thatthe behavior of the two model classes will remain nearlyindistinguishable in suprathreshold conditions. Henceour qualitative results apply to both types of model.

5. EXPERIMENTAL VERIFICATIONTo verify the analytical and computational predictions ofthis study, we recorded the responses of wide-field neu-rons in the fly to motion of broadband images.33 Theseexperiments will be detailed in a separate paper,24 but inthis section we compare some of the results with the ana-lytical and computational predictions of previous sections.

Our experimental equipment was not capable of dis-playing moving photographic images at a frame rate suf-ficiently high for the motion to appear smooth to a fly.Instead of natural images, we designed random texturefields consisting of horizontal rectangular texture ele-ments (texels), each chosen independently with an equalprobability of being illuminated at the display’s maxi-mum intensity or of not being illuminated at all. Thehorizontal power spectrum of such a texture may be com-puted analytically. Although it differs from the powerspectrum of a natural image, it is distinctively broadband,remaining flat at low frequencies and dropping off as 1/f2

at higher frequencies. We can alter the power spectrumby adjusting the texture density; the spectrum of thecoarsest texture available (0.106 texel/°) begins to drop offnear 0.1 cycle/°, whereas that of the finest texture (1.64texels/°) is essentially flat (white) over the bandwidth ofthe hoverfly visual system.

Figure 11(a) shows velocity response curves predictedanalytically from the power spectra for random texturesof different densities. This correlator model includedspatial blurring by the optics and temporal prefiltering bylight-adapted photoreceptors and LMC’s, with param-eters as in Fig. 8.

We made intracellular recordings of the membrane po-tential of individual horizontal-system (HS) wide-fieldneurons from male specimens of the hoverfly Volucella torandom textures of several densities moving at differentvelocities. Recordings were carefully controlled for theeffects of adaptation.33 Output channels of individualcorrelators have not been conclusively identified anatomi-cally, but the wide-field neurons sum the outputs of mul-tiple local correlators in their receptive fields.8,37 Be-cause the wide-field neurons perform extensive spatialintegration and because we averaged recorded outputover time, we effectively measured velocity responsecurves for the biological system, as modeled above.

Figure 11(b) shows experimental velocity responsecurves for HS neurons with equatorial receptive fields,measured at six texture densities. Because the modelparameters used in Fig. 11(a) were not tuned to the or-ganism or the cells in question and because these predic-tions do not incorporate nonlinear effects such as satura-tion and adaptation, we do not expect the predictedcurves to match the experimental data quantitatively.

However, the data qualitatively match the analytical pre-dictions in several important respects. Individual curveshave the general shape predicted by analysis, risingmonotonically to a peak response velocity and then fallingoff. As texture density increases, the curves shift to theleft, with the peak response velocity decreasing from circa270°/s to 85°/s over the range of densities that we used.All of these peak response velocities are higher than theoptimal velocity for a 0.1-cycle/° sinusoidal grating, whichfor these cells was measured at approximately 58°/s.

In a separate experiment we determined velocity re-sponse curves of an HS neuron for textures of identicaldensity but different contrasts. While we observed qua-dratic contrast dependency at very low contrasts, the re-

Fig. 11. (a) Velocity response curves for the model of Fig. 8 forrandom-texture images of various densities, as predicted by Eq.(3) from analytically computed power spectra. Each curve isnormalized to a maximum value of 1.0. (b) Temporally averagedresponse of HS neurons with equatorial receptive fields to tex-tures of corresponding densities, as a function of velocity. Eachcurve represents the mean and the standard error (SE) of the re-sponses of five or six neurons, as indicated by the legend. Mem-brane potential was originally measured in millivolts, but thecurves presented here have been normalized for comparison.

Page 10: Accuracy of velocity estimation by Reichardt correlators

250 J. Opt. Soc. Am. A/Vol. 18, No. 2 /February 2001 Dror et al.

sponse was nearly contrast independent at higher con-trasts. In particular, increasing contrast from 5.9% to94% caused response amplitude to change by approxi-mately 10%. This supports the conclusion of Subsection4.C that contrast saturation improves velocity estimationby ensuring that different natural images have similarvelocity response curves.

6. CONCLUSIONSThe analysis presented in this study leads to two impor-tant conclusions: First, natural image statistics play animportant role in the motion-detection system. Themean response level of a correlator array to motion in anatural environment is far more predictable than the re-sponse to arbitrary images with unknown statistics. Sec-ond, experimentally supported elaborations of the basicReichardt correlator enhance its reliability as a velocityestimator in natural environments. In particular, low-pass spatial filtering and high-pass temporal filtering in-crease the range of velocities coded by the correlator,whereas bandpass prefiltering, contrast saturation, andoutput integration decrease the variability of correlatoroutput.

These principles illuminate the functionality of biologi-cal motion-detection systems. Although we worked withmodels based on data from insect vision, these conclu-sions also apply to models of vertebrate vision such as theelaborated Reichardt Detector3 and the spatiotemporalenergy model.2 The implications also extend to machinevision, both directly for artificial systems modeled on theinsect motion-detection system41,42 and indirectly for amuch wider range of computer vision algorithms. Likeits biological counterpart, a machine vision system couldtake advantage of natural image statistics, employing al-gorithms that perform poorly with general inputs buttend to succeed for visual inputs typical of itsenvironment.43

APPENDIX A: COMPUTATIONALMETHODSThe correlators in all simulations had a first-order low-pass delay filter with t 5 35 ms, which matches the tem-poral frequency tuning observed experimentally in typicallarge flies such as Calliphora, Eristalis, and Volucella.23

We set the interreceptor angle to Df 5 1.08°, near thecenter of the physiologically realistic range for flies.44

These parameter choices do not critically influence our re-sults.

All simulations were implemented in Matlab. Toavoid unnecessary upsampling of the input signals and totake advantage of Matlab’s fast matrix processing capa-bilities, we applied spatial and temporal filters directly tothe original image. Whereas spatial prefilters require atwo-dimensional filtering operation, temporal prefiltersand delay filters in the case of horizontal image motionamount to filtering operations of individual rows of theimage, with the filter impulse response dependent on im-age velocity. Temporal prefilters were implemented inthe frequency domain, and the delay filter in the time do-main with an antialiased impulse response.

To implement the correlation operation, we multipliedthe delay-filtered image with a horizontally shifted ver-sion of the original image; this operation was performedwith opposite shifts for the two correlator subunits, andthe outputs were subtracted. The resulting output takesthe form of a two-dimensional image. One might think ofthis image as the output of a dense array of correlators atone point in time (with each pixel in the prefiltered imagefeeding the left input of one correlator). Equivalently,each row of the output image represents the output of onecorrelator in a vertical array as a function of time. Be-cause we are interested in steady-state response levels,we ignored the correlator output that corresponds to thefirst five time constants of the delay filter, after which anytransients would have fallen to less than 1% of their ini-tial level. When necessary for this purpose, we increasedthe limited spatial extent of Tolhurst’s images by mirror-ing them about their edges.

Saturation simply corresponds to a nonlinear, pixel-wise transformation of the image after the appropriatestage of processing. We implemented output integrationas a filtering operation on the output image of the cor-relator. Simulation of photon noise requires constructionof two separate noisy images, representing inputs to theleft and right arms of the correlator, because the noise inthe two arms is independent.

APPENDIX B: GRAPHIC ANALYSISMETHODOne can use Eq. (3) to confirm that results observed incomputational simulations are independent of particularmodel parameter values. To facilitate this process, wedeveloped a graphic method that allows one to predict theeffects of prefilters, image statistics, and correlator pa-rameters on the velocity response curve. We rewrite Eq.(3) as

R~v ! 5 kE2`

`

F~ fs!W~ fsv !d~ log10 fs!, (B1)

where

F~ fs! 5 fsS2~ fs!P~ fs!sin~2pfsDf!, (B2)

W~ fsv ! 51

tT2~ fsv !

fsv

~ fsv !2 1 1/~2pt!2 , (B3)

and k 5 loge(10)/(2p) is constant. Integration nowtakes place with respect to the logarithm of frequency.In other words, the integral corresponds to the area undercurve FW on a logarithmic spatial frequency axis. As ve-locity increases, W( fsv) shifts rigidly to the left, whileF( fs) remains stationary. Changes to the temporal pre-filter T or the frequency response of the delay filter D af-fect W. Changes to the horizontal image power spectrumP( fs), the spatial prefiltering S( fs), and the intereceptorangle Df affect F. This method allows one to see intu-itively the effects of changing any of these quantities. Inparticular, we used it to confirm that, in general, low-passtemporal prefiltering and high-pass spatial prefilteringdecrease the peak response velocity, whereas low-passspatial and high-pass temporal prefiltering increase it.

Page 11: Accuracy of velocity estimation by Reichardt correlators

Dror et al. Vol. 18, No. 2 /February 2001 /J. Opt. Soc. Am. A 251

Equations (1) and (3), as well as the graphic analysismethod, generalize in a straightforward manner to cor-relator models with arbitrary linear delay filters.

ACKNOWLEDGMENTSWe thank David Tolhurst for sharing his set of imagesand Miranda Aiken for recording the video frames thatformed the panoramic images. Rob Harris, Brian Bur-ton, and Eric Hornstein contributed valuable comments.This work was funded by a Churchill Scholarshipawarded to R. O. Dror and by grants from the Biotechnol-ogy and Biological Sciences Research Council and theGatsby Foundation.

*Present address, Department of Electrical Engineer-ing and Computer Science, Massachusetts Institute ofTechnology, Room 35-427, 77 Massachusetts Avenue,Cambridge, Massachusetts 02139; e-mail [email protected].

†Present address, Department of Zoology, University ofWashington, Box 351800, Seattle, Washington 98195-1800.

REFERENCES AND NOTES1. W. Reichardt, ‘‘Autocorrelation, a principle for the evalua-

tion of sensory information by the central nervous system,’’in Sensory Communication, A. Rosenblith, ed. (MIT Press,Cambridge, Mass. 1961), pp. 303–317.

2. E. H. Adelson and J. Bergen, ‘‘Spatiotemporal energy mod-els for the perception of motion,’’ J. Opt. Soc. Am. A 2, 284–299 (1985).

3. J. P. H. van Santen and G. Sperling, ‘‘Elaborated Reichardtdetectors,’’ J. Opt. Soc. Am. A 2, 300–321 (1985).

4. F. Wolf-Oberhollenzer and K. Kirschfeld, ‘‘Motion sensitiv-ity in the nucleus of the basal optic root of the pigeon,’’ J.Neurophysiol. 71, 1559–1573 (1994).

5. R. C. Emerson, M. C. Citron, W. J. Vaughn, and S. A. Klein,‘‘Nonlinear directionally selective subunits in complex cellsof cat striate cortex,’’ J. Neurophysiol. 58, 33–65 (1987).

6. M. V. Srinivasan, S. W. Zhang, M. Lehrer, and T. S. Collett,‘‘Honeybee navigation en route to the goal: visual flightcontrol and odometry,’’ J. Exp. Biol. 199, 237–244 (1996).

7. S. P. McKee, G. H. Silverman, and K. Nakayama, ‘‘Precisevelocity discrimination despite random variations in tempo-ral frequency and contrast,’’ Vision Res. 26, 609–619(1986).

8. M. Egelhaaf, A. Borst, and W. Reichardt, ‘‘Computationalstructure of a biological motion-detection system as re-vealed by local detector analysis in the fly’s nervous sys-tem,’’ J. Opt. Soc. Am. A 6, 1070–1087 (1989).

9. G. A. Horridge and L. Marcelja, ‘‘On the existence of fastand slow directionally sensitive motion detector neurons ininsects,’’ Proc. R. Soc. London, Ser. B 248, 47–54 (1992).

10. G. J. Burton and I. R. Moorhead, ‘‘Color and spatial struc-ture in natural scenes,’’ Appl. Opt. 26, 157–170 (1987).

11. D. J. Field, ‘‘Relations between the statistics of natural im-ages and the response properties of cortical cells,’’ J. Opt.Soc. Am. A 4, 2379–2394 (1987).

12. D. J. Tolhurst, Y. Tadmor, and T. Chao, ‘‘Amplitude spectraof natural images,’’ Ophthalmol. Physiol. Opt. 12, 229–232(1992).

13. D. L. Ruderman, ‘‘The statistics of natural images,’’ Net-work Comput. Neural Syst. 5, 517–48 (1994).

14. M. Egelhaaf and A. Borst, ‘‘Movement detection in arthro-pods,’’ in Visual Motion and Its Role in the Stabilization ofGaze, J. Wallman and F. A. Miles, eds. (Elsevier, Amster-dam, 1993), pp. 53–77.

15. R. C. Hardie, ‘‘Functional organization of the fly retina,’’ in-

Progress in Sensory Physiology, D. Ottoson, ed. (Springer-Verlag, Berlin, 1985), Vol. 5, pp. 1–80.

16. R. Payne and J. Howard, ‘‘Response of an insect photorecep-tor: a simple log-normal model,’’ Nature 290, 415–416(1981).

17. B. Tatler, D. C. O’Carroll, and S. B. Laughlin, ‘‘Tempera-ture and temporal resolving power of fly photoreceptors,’’ J.Comp. Physiol. A 186, 399–407 (2000).

18. A. C. James, ‘‘White-noise studies in the fly Lamina,’’ Ph.D.thesis (Australian National University, Canberra, Austra-lia, 1990).

19. M. Egelhaaf and A. Borst, ‘‘Transient and steady-state re-sponse properties of movement detectors,’’ J. Opt. Soc. Am.A 6, 116–127 (1989).

20. D. C. O’Carroll, S. B. Laughlin, N. J. Bidwell, and R. A.Harris, ‘‘Spatio-temporal properties of motion detectorsmatched to low image velocities in hovering insects,’’ VisionRes. 37, 3427–3439 (1997).

21. S. Single, J. Haag, and A. Borst, ‘‘Dendritic computation ofdirection selectivity and gain control in visual interneu-rons,’’ J. Neurosci. 17, 6023–6030 (1997).

22. T. Maddess and S. B. Laughlin, ‘‘Adaptation of the motion-sensitive neuron H1 is generated locally and governed bycontrast frequency,’’ Proc. R. Soc. London Ser. B 225, 251–275 (1985).

23. R. A. Harris, D. C. O’Carroll, and S. B. Laughlin, ‘‘Adapta-tion and the temporal delay filter of fly motion detectors,’’Vision Res. 39, 2603–2613 (1999).

24. Information is available from David C. O’Carroll, Depart-ment of Zoology, University of Washington, Box 351800, Se-attle, Wash. 98195-1800; [email protected].

25. N. Franceschini, A. Riehle, and A. le Nestour, ‘‘Direction-ally selective motion detection by insect neurons,’’ in Facetsof Vision, D. G. Stavenga and R. C. Hardie, eds. (Springer-Verlag, Berlin, 1989), pp. 360–390.

26. S. B. Laughlin, ‘‘Matching coding, circuits, cells and mol-ecules to signals: general principles of retinal design inthe fly’s eye,’’ Prog. Retinal Res. 13, 165–195 (1994).

27. T. Poggio and W. Reichardt, ‘‘Visual control of orientationbehaviour in the fly. II. Towards the underlying neuralinteractions,’’ Q. Rev. Biophys. 9, 377–438 (1976).

28. J. H. van Hateren, ‘‘Processing of natural time series of in-tensities by the visual system of the blowfly,’’ Vision Res.37, 3407–3416 (1997).

29. A. van der Schaaf and J. H. van Hateren, ‘‘Modelling thepower spectra of natural images: statistics and informa-tion,’’ Vision Res. 36, 2759–2770 (1996).

30. D. J. Heeger, ‘‘Normalization of cell responses in cat striatecortex,’’ Visual Neurosci. 9, 181–197 (1992).

31. D. Osorio, ‘‘Mechanisms of early visual processing in themedulla of the locust optic lobe: how self-inhibition,spatial-pooling and signal rectification contribute to theproperties of transient cells,’’ Visual Neurosci. 7, 345–355(1991).

32. M. F. Land and T. S. Collett, ‘‘Chasing behaviour of house-flies (Fannia canicularis): a description and analysis,’’ J.Comp. Physiol. A 89, 331–357 (1974).

33. R. O. Dror, ‘‘Accuracy of visual velocity estimation by Rei-chardt correlators,’’ Master’s thesis (University of Cam-bridge, Cambridge, UK, 1998).

34. A. W. Snyder, ‘‘The physics of vision in compound eyes,’’ inComparative Physiology and Evolution of Vision in Inverte-brates: Invertebrate Photoreceptors, Vol. 6A of Handbookof Sensory Physiology, H. Autrum, ed. (Springer-Verlag,Berlin, 1979), pp. 225–313.

35. J. K. Douglass and N. J. Strausfeld, ‘‘Visual motion detec-tion circuits in flies: peripheral motion computation byidentified small-field retinotopic neurons,’’ J. Neurosci. 15,5596–5611 (1995).

36. G. Nalbach and R. Hengstenberg, ‘‘The halteres of the blow-fly Calliphora,’’ J. Comp. Physiol. A 175, 695–708 (1994).

37. S. Single and A. Borst, ‘‘Dendritic integration and its role incomputing image velocity,’’ Science 281, 1848–1850 (1998).

Page 12: Accuracy of velocity estimation by Reichardt correlators

252 J. Opt. Soc. Am. A/Vol. 18, No. 2 /February 2001 Dror et al.

38. K. Hausen and M. Egelhaaf, ‘‘Neural mechanisms of visu-alcourse control in insects,’’ in Facets of Vision, D. G.Stavenga and R. C. Hardie, eds. (Springer-Verlag, Heidel-berg, 1989), pp. 391–424.

39. M. Egelhaaf and W. Reichardt, ‘‘Dynamic response proper-ties of movement detectors: theoretical analysis and elec-trophysiological investigation in the visual system of thefly,’’ Biol. Cybern. 56, 69–87 (1987).

40. D. W. Dong and J. J. Atick, ‘‘Statistics of natural time-varying images,’’ Network Comput. Neural Syst. 6, 345–358 (1995).

41. N. Franceschini, J. M. Pichon, and C. Blanes, ‘‘From insect

vision to robot vision,’’ Philos. Trans. R. Soc. London Ser. B337, 283–294 (1992).

42. R. Sarpeshkar, W. Bair, and C. Koch, ‘‘An analog VLSI chipfor local velocity estimation based on Reichardt’s motion al-gorithm,’’ in Advances in Neural Information ProcessingSystems, S. Hanson, J. Cowan, and L. Giles, eds. (MorganKauffman, San Mateo, Calif., 1993), Vol. 5, pp. 781–788.

43. R. O. Dror, D. C. O’Carroll, and S. B. Laughlin, ‘‘The role ofnatural image statistics in biological motion estimation,’’Springer Lect. Notes Comput. Sci. 1811, 492–501 (2000).

44. M. F. Land and H. M. Eckert, ‘‘Maps of the acute zones offly eyes,’’ J. Comp. Physiol. A 156, 525–538 (1985).