Stability of Binocular Depth Perception with Moving Head and Eyes

V R V 3 1( 0 E ScienceLtd.All rightsr

S ““ P G B0 $ + 0

Stability of Binocular Depth Perception withMoving Head and EyesRAYMOND van EE,*~ CASPER J. ERKELENS*

R F 1 r f D 1 infinalform M 1

sp o

r h h ar p

dm

p m ae @

Binocular disparity Binocular vision Depth perception Stereopsis Robot vision

I

The separationof the human eyes causeseach eye to see adisparate image of the outside world. Generally, it hasbeen accepted that positional disparitiesare sufficienttogenerate a three-dimensional(3D) percept (e.g. Whets-tone, 1838; Ogle, 1950; Julesz, 1971). Wheatstone’sdevelopmentof the stereoscopein 1838was based on thisidea. Recently, this knowledgehas been used in the fieldof binocular robots. However, many phenomenarelatingto disparity and perception of depth are still not under-stood, including the fact that binocular vision is largelyunaffected by eye and head movements (Westheimer &McKee, 1978 concerning lateral eye movements; Stein-man et al., 1985 and Patterson & Fox, 1984 concerninghead movement).

In binocular robots the quality of 3D analysis isseverely reduced by the instability of the cameras (thedisparity acquisition system; Eklundh, 1993). By ana-logy, one would expect the stability of human binocularvision to be reduced by eye and head movements.In thecase of a simpleobject like a chessboardit is immediatelyclear that the imagesof the chessboardon our two retinaediffer according to whether the chessboard is positionedin front of us or eccentrically.Since the disparityfield iscomposed of the positional differences between theretinal images, the disparity field will depend on theposition of the object. On the other hand, if the object isstatic but the binocular observer makes an eye or head

*Utrecht University, Faculty of Physics and Astronomy, HelmholtzInstituut,Princetonplein5, NL-3584CC,Utrecht,The Netherlands.

+Towhom all correspondenceshouldbe addressed.

movement, the disparity field before and after themovements will also change. However, this time thedisparityof the object and environmentchange together.In short, during eye and head movements, the images ofthe entirevisualworld are continuouslychangingon bothretinae, which means that there are also continuouslychanging disparities. One would expect these changingdisparitiesto reduce the stability of stereopsis.

In principle, the visual system can utilize the signalsthat control the eye and neck muscles (efference copies)in order to correct stereopsis for disparities induced bycontrolled eye and head movements. However, dispa-rities are not only due to controlled eye and headmovements,they can also be due to uncontrolledeye andhead movements. These uncontrolled movements arecaused by noise in the motor system. Experiments havedemonstrated large discrepancies between the level ofstereoacuity and the relative sloppiness of oculomotorcontrol. Optimal stereoacuity thresholds in the foveatypically attain mean standard deviationsof the order of5 sec of arc (Berry, 1948;Westheimer & McKee, 1978;McKee, 1983),which is about one-sixth of the diameterof the smallest foveal cones (Westheimer, 1979a).Thesethresholds for stereoacuity can be obtained even for a200-msec exposure (Westheimer & McKee, 1978) andare similar in magnitude to the best monocular hyper-acuities for motion displacement, vernier tasks andrelative width (Westheimer & McKee, 1979; McKee etal., 1990b). Given these values, binocular vision can bevery sensitive. It can be regarded as a hyperacuitymechanism.On the other hand, duringnaturalbehaviour,vergence position errors of up to 1–2 deg (Collewijn &Erkelens, 1990),vergence velocity errors of up to 1 deg/

3827

3828 R. van EE and C. J. ERKELENS

sec (Steinman & Collewijn, 1980) and errors incyclovergenceof 10 min arc (Enright, 1990;van Rijna 1994) are easily generated and introduce disparitiesthat are similar in size to the errors. The measuredsloppinessof oculomotorcontrol is not due to artefactsinexperimental methods (Ferman a 1987). Besidesoculomotor system instability, there is another factor ofuncertaintywhich affects the interpretationof disparities,namely the exact orientation of the head relative to thebody.Head stabilityis no better than oculomotorstability(Schor et al., 1988).

Although oculomotorand head control is sloppy, it isneverthelesspossiblethat a feedback systemis at work inbinoculardepth perception.* The noise in the oculomotorsystem, though producing (cyclo)vergenceerrors, couldbe known to the visual system (for instanceby means ofmuscle sensors) and utilized in order to interpretdisparities. There is evidence, however, that there is nosuch feedback system in binocular depth perception.Firstly, fast side-to-siderotationsof the head, or pressingagainst the eyeball, do not influence depth perception(Steinmanet al., 1985).Secondly,the resultspresentedinseveral reports (Foley, 1980; Erkelens & Collewijn,1985a, b; Regan a 1986; Collett et al., 1991;Cumming et al., 1991; Logvinenko & Belopolskii(1994); Rogers & Bradshaw, 1995 and Backus et al.,1996) show that in situations where (conflicting) eyemuscle information is available changing eye posturedoes not lead to changingperceptionof depth in the caseof large field stimuli (large displays)or they lead to onlyweak perception of depth in the case of small fieldstimuli.

The discrepancies between the sensitivity of stereo-scopic vision and the sloppiness of oculomotor controlmean that oculomotor stability is at least one order ofmagnitude less precise than measured stereoacuity (alsoreported by Nelson, 1977 and Collewijn et al., 1991).Even if we assume that subjects can obtain very goodstereoacuity by using relative depth differences (whichare unaffected by noisy eye and head movements;Westheimer, 1979b; Erkelens & Collewijn, 1985a, b)we still do not know why the stereoacuity stimulus as awhole does not tremble in depth as a result of thetrembling eye and head movements.

A possible way for the visual system to deal with theeffects of sloppy motor control is to utilize all availableretinal information.Frisby, Mayhew & co-workers haveproposed that gaze (eye posture)parameterstheoreticallycan be calibrated by “shape-from-texture”. However,recently they showed that this hypothesis was notconfirmed experimentally (Frisby et al., 1995). More

*Binocular3D vision involvesperceptionof directionsand perceptionof depth. Regarding binocular perception of directions there isevidence that vestibular and proprioceptiveinformationis used tomaintain stability (Howard, 1982; Carpenter, 1988).For instance,fast side-to-side rotations of the head (Steinman a 1985), orpressing against the eyeball, impair the correct couplingbetweenextra-retinal signals and perceived directions and result inimpairmentof the stability of the visual world in lateral directions.

importantly,Mayhew& Longuet-Higgins(1982)showedthat information about gaze parameters in principle canbe calculated from the horizontaland vertical disparities.This gaze parameter information could then be used tointerpret disparities. In addition, G&ding et al. (1995)proposed a decompositionof the disparity interpretationprocess into disparity correction, which is used tocompute three-dimensional structure up to a relieftransformation, and disparity normalization, which isused to resolve the relief ambiguity to obtain metricstructure.Discussingthe existing literaturebased on thisdecomposition into disparity correction and disparitynormalization, they showed that in relief tasks depthperceptionexhibits a large and stable dependenceon thestructure of the vertical disparity field, whereas metrictasks are hardly affected. Glirding et al. (1995) alsoreported on the fact that visual tasks that actually requirea full metric reconstruction of the three-dimensionalvisualworld are fairly uncommon.The relief transforma-tion preservesmany importantpropertiesof visual shape,notablythe depthorder as well as all projectivepropertiessuch as coplanarity and collinearity. Therefore, adisparity processing system that computes a reconstruc-tion of the three-dimensional visual world relying onretinaldisparitiesaloneis very attractiveeven if it doessoup to a relief transformation.

Important exceptions to the idea that the metric tasksare hardly affected by vertical disparitiesare the studiesof Rogers& Bradshaw(1993, 1995).Rogers& Bradshaw(1993)showedthat subjectscan use vertical disparitiesinorder to estimate the perceived peak-to-troughdepth ofcorrugationsfor large-fieldstimuli.However, the amountof perceiveddepth in the full-disparity-cueconditionwasvery much less than would be required for completedepth constancy. In the Appendix of their 1995 paper,Rogersand Bradshawshowedthat absolutedistancefromthe observer is altered by modifying vertical disparities.[See also the paper by Friedman et al. (1978) who alsofound that vertical disparity influencesmetrical percep-tual tasks.]Yet, thesestudieshave notbeen conductedforlimited observationperiods.

Despite these findings about a disparity processingsystem that computes a metrical reconstruction there isno evidence yet that such a system is effective in humanvision on a short time-scale.In the next sectionwe reporton perceptual studies using simple stereograms whichshow that several classes of basic stimuli which mimicreal world stimuli (containing both realistic horizontaland vertical disparities)do not elicit reliable perceptionof metric aspectsof depth for limitedobservationperiods(up to the order of seconds).On the otherhand it hasbeenreported that relief tasks in stereopsis can be effectiveeven to the order of milliseconds(e.g. Kumar & Glaser,1993;Uttal et al., 1994).

Stereogramsand depthperception

Our knowledge about binocular depth perception isobtained to a large extent from experimentswith stereo-grams. In such experimentsthe subjectviews (with static

..

STABILITYOF BINOCULARDEPTH PERCEPTION 3829

FIGURE 1. An example of a horizontal scale and a horizontal sheartransformation between the observed half-images. Horizontal scalebetween the half-images of the stereogram leads to perceived slantabout the vertical axis. Horizontalshear leads to perceived slant aboutthe horizontal axis. M is the magnitude of the horizontal scaletransformation expressed as a fraction, /l is the magnitude of the

horizontal shear transformation,expressed as an angle.

head) two half-imagesof a stereogram, one transformedrelative to the other, projected on a screen. In accordancewith geometricalrules, local horizontallateral shiftof thehalf-images relative to each other alters the perceiveddistance. Horizontal scale between parts of the half-images of a stereogramleads to perceived slant about thevertical axis. Local horizontal shear, on the other hand,leads to perceived slant about the horizontalaxis. Figure1 shows an example of the horizontal scale and sheartransformation.

Lateral movements of the entire half-images of astereogramrelative to each other lead to vergence of theeyes (with a gain unequal to one), but are not interpretedas changes in distance (Erkelens & Collewijn, 1985a,b;Regan et al., 1986). In contrast, differential movementsof parts of the half-imagesgive rise to vivid perceptionofmotion in depth. In addition,Regan et al. (1986) showedthat under stabilized retinal conditionsabrupt changes inthe image vergence angle produced no impression of astep change in depth. These authors suggested that theexplanation for their results may be that the brain inter-prets lateral shifts between the entire parts of the stereo-grams as movementsof the eyes and thereforethese shiftsare best ignored as signals for depth. (As we will seebelow this explanation is not entirely correct.)

a)

d

Slant ,. b)h /

i,

,/

LE half- ‘i ~

Another perceptual study (Howard & Zacher, 1991)showedthat differentialrotationof the entire half-imagesof a stereogram induces cyclovergence with a gainunequal to one (cyclodisparity)but elicits poor percep-tion of depth. Again, two different cyclodisparities,simultaneouslypresent in the visual field,are requiredforreliabledepthperception.Collewijnet al. (1991)reportedthat thresholdsfor perception of depth caused by cyclo-disparity increase by a factor of 7 when the visualreference is removed. Cyclodisparities can have con-siderable magnitudes and can occur frequently duringnatural behaviour. Howard (1993) suggested thatwhole-fieldcyclodisparitiescould indicate that the eyesare misalignedand that thereforethe perceptualsystemisinclined to ignore these cyclodisparities when judgingslant.

Finally, slant from horizontal scale and horizontalshear between the entire half-images of a stereogram isrelatively poorly perceived (Shipley & Hyson, 1972;Mitchison & Westheimer, 1984, 1990; Stevens &Brookes, 1988; Gillam et al., 1988). Recently, van Ee& Erkelens(1996a)investigatedtemporalaspectsof slantperception induced by whole-field horizontal scale andhorizontal shear. They quantitatively corroborated theearlier findingthat when observationperiods last up to afew seconds, perception of slant caused by whole-fieldhorizontal scale and shear is relatively poor. Usingargumentssimilar to those presented in the previous twoparagraphs we now attempt to relate this experimentalresult to the orientation of the head. Head rotation in astationaryvisualworld shouldcause similardisparitiesasrotationof the entire visual world about the centre of theheadwhen the head is stationary.This idea is explainedinFig. 2 where two similar drawings are depicted. Thedrawing in Fig. 2(a) representsthe geometryof viewing ahorizontallyscaled stereogram.The drawing in Fig. 2(b)represents the geometry of viewing an (initially) frontalplane with rotated head (initially, fixation was ateccentricityu). In the horizontalplane the retinal imagesof the two situations are the same. Analogously, oneexpectsdisparitiescausedby forwardrotationof the head

? ! iHeadrotation

FIGURE2. (a) Showsthe geometryof a horizontallyscaled stereogramwith unrotatedhead. (b) Showsthe geometryof an (initially) frontafplanewith rotated head. The retinal images in the horizontalplane are identical in both situations. LE and RE denote left and right eye, respectively.Note that the geometryof a frontal plane at a distance ofzoviewed after a head rotationover a deg is similar to a slantedplane over u degbut at a

distance of z~cos


to correspondto disparitiescaused by horizontalshear ofthe half-images of a stereogram. The argumentswe useare similar to those used by Erkelens & Collewijn(1985a) and Howard et al. (1993). We suggest that thereasonwhy depth perceptionof one linear transformationwithin the stereogramis poor and depthperceptionof twodifferent linear transformations is vivid is that thedisparity field caused by only one linear transformationis ambiguous.In otherwords, head rotationscould inducethe same disparity fields as the scaled and shearedstereograms.We argue that the disparityfieldscaused byhorizontalscale and shear are thereforeprimarilyignoredas signals for perception of slant. We also argue that thedisparity field caused by two different, simultaneouslypresent, linear transformationscannotbe similar to a fieldcausedby ego-movementand, therefore,such a fieldis aneffective stimulus for the slant perception of one planerelative to the other.

HypothesisThus, during (noisy) eye and head movements the

fX51

poor depth perception. So far, this knowledge has notbeen suppliedby the literature.

Headcentric coordinatesand head movement

In order to identify a test point P in three-dimensionalspace relative to the head we define a right-handedorthogonalcoordinate system with the origin above thevertebralcolumnand at the same level as the eyes.The x-axis points from right to left parallel to the interocularaxis, the y-axis points vertically upwards, and the z-axispoints in the primary direction (straight ahead). After ahead rotation or translation the headcentric coordinates(#,~,#) of test point P are (x~,y~, #). Headtranslation correspondsto a trivial coordinate modifica-tion.For example,a head translationalong they-axis overan arbitrary distance (ad) modifiesy$ coordinates into# = ~ – ad. Head rotation is not trivial. The ccror&-nates before and after a head rotation are related to eachother by an Euler rotation matrix:

/E)

(cos~cos~ + sin@’sin@sinq!+’ cos@sin@ –sinq#cos@ + cos@sinVsinq!P’RH = –cos@sin@ + sin@sin@cos@+’ cos~cosfl

)

sin@F’sin~ + cos@sin@cos@ , (1)sin@cos# –sin@ cos@cosOH

disparity field changes continuously. Why do we notperceive a visual world trembling in depth as a result ofour trembling disparity acquisition system? One couldthink of two opposite hypotheses. Either the visualsystem compensates completely for the disparitiesinduced by these (noisy) eye and head movements orthe visual system is blind for these disparities. Thefindings about (1) using the signals that control the eyeand head muscles(efferencecopies),(2) using a feedbackloop based on muscle sensors and (3) using all theavailable (horizontal and vertical) disparities, suggestthat the compensation hypothesis does not provide asufficientanswer to our question, at least not for limited(realistic) observationperiods.

Taken together, the above-mentioned suggested ex-planations for the poor sensitivityof depth perception toseveral transformationsbetween half-imagesof a stereo-gram lead to a generalized hypothesis.We hypothesizethat a possibleway for the visual system to deal with theeffects of sloppy eye and head movementsis to use onlythat part of disparityinformationwhich is invariantundereye and head movements. Investigations about thevalidity of this hypothesis require precise knowledgeabout what sort of disparity is induced, on the one handby eye and head movements and on the other hand bytransformed stereogramswhich are known to elicit only

where ~H, ~ and ~H denote angles of head rotationabout the vertical, horizontal and primary direction,respectively. The signs of the angles are again definedaccordingto a right-handedcoordinatesystem.The orderof rotations is described in a Fick manner, which meansthat the head is first rotated about the vertical axis, then

x

FIGURE3. In Fick’s coordinatesystem a target is uniquely identifiedrelative to the left eye by its longitude O: and its latitude ~~. In thisfigure the eye points to a fixationpoint which is at infinity.The originof the oculocentric coordinate system is located at the centre of theeyeball. The x-axis of the coordinate system points from right to left,the y-axis points vertically upwards, and the z-axis points in theprimarydirection. In the directionof the arrow the sign of the angle is

definedas positive.

.—

STABILITYOF BINOCULARDEPTHPERCEPTION 3831

about the horizontal axis and lastly about the primary infinity,which means that the visual axis coincideswithdirection.*

the primary direction.As shown in Fig. 3 the coordinates

Headcentric coordinatesand stereograms (~, fi,z$) of a test point P relative to the left eye are

If stereogramsare involved,then a separatecalculation parametrizedin a Fick mannerby its longitude~P and itshas to be performed to find the headcentric coordinatesfor each of the two transformed half-images: latitude ~:

# - c ) -— COS(0,54Y)+ horscale –sin(O.5t/P)+ horshear

y? 1.J3eyeimuge sin(O.5w)

(# ( )( )– cos(-O.5479 –sin(–O.5~) #

[)

o.5shift

y~ r i – sin(–O.5@) cos(–O.5#P) M righteyeirnage+ 0 ‘(2)

where @, shift, horscale, horshear denote the rotation,lateral shift, horizontal scale and horizontal shearbetween the entire two half-images of the stereogram,respectively.For simplicitywe assume that there is onlyone transformation at a time between the parts of thestereogram relative to each other.

Oculocentriccoordinates

In addition to defininga coordinate system relative tothe head, we also have to define retinal coordinatesystems.As before, thex-axispointsfrom right to left, they-axis points vertically upwards, and the z-axis points inthe primary direction. The centre of the oculocentriccoordinate system is positioned in the centre of the eye.Initially, we assume that the eye fixates a target at

where ,Z1is the distance between the centre of theoculocentric coordinate system and the headcentriccoordinate system along the z-axis. 1 denotes theinteroculardistance.Similarnotation is used for the righteye.

The directionof a new fixationpoint relative to the lefteye is denoted by longitude #F and latitude @F.Thecoordinatesof a point before and after an eye rotation tothe new fixation point are related by an Euler matrixsimilar to the one given above. After an eye rotation tothe fixationpoint the coordinatesof pointP relative to theleft eye are (xf,yf, Y~)

(cose~cos~~ + sin@~sin@Fsin!/$ cos+$sin~~ ‘sin~~cos+’$ + cos@~sint@intiFRL = –cos@Fsin@F+ sin$$fsin~~cos~~ cos@Fcos@F

)

sin@Fsin@~+ cos$$Fsin@Fcos$$~. (4)sin@Fcos#F –sin@F cos@Fcos@F

*Fick’s coordinate system (Fick, 1854) and Helmholtz’s coordinate From these three equationsthe longitudeOf and latitudesystem (von Helmholtz, 1911) originate from eye movementstudies. Rotations do not commute under summation. Decisions ~~ in the rotated eye coordinatesystemcan be calculatedshould be made about the order in which rotations should be for arbitrary test points and fixation points. An identicalperformed.In Fick’s system, the vertical axis of the eye ball isassumed to be fixed to the skull and the horizontal axis of the eye procedurehas to be performedfor the righteye in order toballisassumedtorotategimbal-fashionaboutthe vertical axis. In find 4$ and r9$. Disparity is computed from theHelmholtz’ssystem it is the horizontalaxis which is assumedto befixed to the skull (Howard, 1982; p. 181). Which system is

differencesbetween retinal coordinates in the two eyes.

preferablewilldependonthesituation.TheadvantageofFick’s Horizontal (in fact longitudinal)disparity is defined bysystemis thatiso~ergencesurfacesareequivalentto isodisparitysubtracting #p from q$~.Vertical (latitudinal)disparityissurfaces.TheadvantageofHelmholtz’ssystemisthatit is based onepi-polar geometry. obtained by subtracting8$ from 19~.


a)

Her.disp.[arcmin]1

b) EI[deg] 40

dlVerf.

L fd2 disp. 11

[arcm!; 2

FIGURE4. Horizontal (a) and vertical disparity(b) of a frontal plane at a distance of 250cm (all, white patch) and 50 cm (d2,grey patch) in front of the eyes. ~ denotes longitude,19denotes latitude. Both angles are taken relative to the head.

N C

The above-mentioneddefinitionsare implementedin acomputer program in which we compute disparity fieldsgenerated by planar surfaces. The disparity fields arecomputed for a range of eye, head and object positions,on the one hand, and for several transformationsbetweenhalf-imagesof a stereogram,on the other hand. Through-out the text and figures, disparity is calculated inoculocentriccoordinatesfor a field of 80 x 80 deg whichis centered around the fixation point. This field isprovided with a (virtual) lattice of 12x 12 evenlydistributeddirections (the angle between adjacent direc-tions is 80/11= 7.3 deg). Disparity is calculated for eachof the 144 directions.Results are plotted as a function oflongitude(~) and latitude (0) which are taken relative tothe head. In our calculationswe use planar surfaces,sinceplanar surfaces have simple computationalproperties. Inaddition, the disparity fields of these surfaces haveseveral symmetricalproperties,as is shown in the figuresthroughout this paper, which makes them easier tointerpret.However, in principleit is not relevantwhat thesource of the disparity field is. We are not primarilyinterested in the disparity fieldper se. We are interestedin how a disparity field transformsas a result of eye andhead movements.In the calculationswe take the centreofhead rotation to be 10 cm behind the eyes and theinteroculardistance to be 6.5 cm. Again, the exact valuesof these quantitiesare not relevant for the purposeof ourstudy. We make the assumptionthat the nodal point andthe centre of eye rotation coincide. Cormack & Fox(1985) found almost no effect of nodal point motion fordifferent fixations, except under the most extremeconditions.

Disparity and the distance of the object

Figure 4 shows how the disparity field depends onviewing distance and direction. In this figure thehorizontal and vertical disparity fields are shown fortwo fronto-parallel(frontal)planes at distancesof 250 cm(the white patch, dl, in Fig. 4) and 50 cm (grey patch,d2). The figure shows that disparity fields of frontalplanes are curved when viewed at a finite distance.Objects that are curved along the horopter have zerodisparity. For stimuli nearer than the horopter thehorizontal disparity is, by definition, positive. Conver-

sely, for stimuli further away than the horopter,horizontal disparity is negative. Since the horizontaldisparity does not depend on latitude (in Fick’s descrip-tion), the horizontalcomponentof the disparityfield [Fig.4(a)] doesnot dependon 9 either.Disparityis zero for thefixation point. When fixation is on the plane in theprimary direction, points of the frontal plane havenegative horizontal disparity because all points arelocated further away than the horopter.

The vertical disparity fields of the frontal planes at250 cm (all) and 50 cm (d2) in front of the eyes are shownin Fig. 4(b). These fields depend both on ~ and 0. Eachpoint located outside the plane of fixation and nearer toone eye than to the other eye has vertical disparity.Sincefixation is chosen to be in the primary direction,verticaldisparity is zero along the directions ~ = Oor 6 = Oandanti-symmetricalwith respect to these axes.

Eye and head movements vs stereograms

It will be demonstrated that disparity fields that arebrought about by eye and head movements can beadequately simulated by stereograms. In the rest of thepaper we compare the disparity field caused by aparticular eye or head movementwith the disparity fieldcaused by the stereogram that corresponds theoreticallyto the eye or head movement.The basic stimulus(beforethe eye or head movement) is always a frontal plane at adistance of 100cm in front of the eyes.

Cyclovergencevs differentialrotation within the stereo-gram

According to Donders’ law (Donders, 1876) andListing’slaw (Listing, 1854)the eyes are slightlyrotatedrelativeto each other aboutthe line of sightwhile fixatinga tertiary position (for a review see Alpern, 1962). Thisimplies that after a change of fixation cyclodisparity isintroduced.The disparityfieldof the frontalplane causedby pure cyclovergence is depicted in Fig. 5. Themagnitude of cyclovergence is chosen to be 1.26 deg(each eye 0.63 deg). Figure 5(a and b) shows thehorizontal disparity and vertical disparity, respectively,of the plane after such cyclovergence.

Figure 5(c and d) shows the horizontal and verticaldisparity field of a stereogramwith differentiallyrotatedhalf-images.Each part of the stereogram is rotated over0.63 deg in oppositedirections.The disparity field (both

STABILITYOF BINOCULARDEPTHPERCEPTION

Cyclovergence

3

a)

-20

@[deg]

b)

-20-40

Ven. 100disp.

1[arcmin] -1$

-40-20

Differentialc) El[deg]40

-20/-7 I

rotation of stereogramd) 13[deg]40

Her. 1disp.

1[arcmin] -1::MJ2”W

e)

< 20 -o [deg] 40

-40-20

Difference between cyclovergenceand differential rotation

-4020

Her.disp.

t[arcmin] -z:

-40

<CD[deg] 40

f)

-20-40

20Vett.disp.

t[arcmin] -2:

-40-20

FIGURE5. The top two panels show horizontal(a) and vertical disparity(b) of a frontal plane at a distance of 100cm viewedwith cyclovergenceof 1.26deg. The middlepanels showthe horizontal(c) and vertical disparity(d) after a differential rotationof 1.26deg within the correspondingstereogram.The bottom two panels show the difference in horizontal (e) and vertical (~disparitybetween cyclovergenceof the eyes and differential rotationwithin the stereogram.Note that the dimensionsalongthe

disparity axes of the bottompanels are different from the other figures.

horizontal and vertical) is more curved for negative Obecause the points of intersection of the light rays thatcome from correspondingpoints of both half-images ofthe stereogramare nearer for negative Othan for positive$.

In the case of cyclovergencethe axis of rotation is thevisual axis. In the case of differential rotation within astereogram the axis of rotation of either half-image isperpendicularto the projectionscreen.Therefore, it is nottrivial that the disparity fields induced by cyclovergenceand differentialrotationare equal.The similaritybetweenthe numerical results of Fig. 5(a, c) and the numericalresults of Fig. 5(b, d) implies that the disparity fieldscaused by cyclovergenceand differential rotation of the

entireparts of a stereogramare approximatelyequivalent[Fig. 5(e, f)].

Head translation in the primary direction vs horizontalshift within the stereogram

Generally, the disparitiescaused by a head translationtowardsa frontalplane can be simulatedby the followinglateral shift between the two half-images of the stereo-gram:

TZ.1shift = ————

Z. —Tz‘(5)

where T=denotes the translationof the head towards theplane, Z the interocular distance and Z. the distancebetween the stimulus and the eyes. We can derive this


Frontal screent

,.,, .’ ..

Frontal plane’’~,, -

2

.’ .,,’ ...

‘ .28

,.,,.,

.,”,’ .,; ..”,’ ,..’,’ ,...,,., 112

Left eye

[ hiftLeft eye

is half-image

c;

*,..’

,.,[ ‘:6 ; ““”TZ‘. .’

‘, ...”‘. .....

,...,..” ;

,.,::,.

.. -.

,...,, -- ZO-TZ... ..

,.,, -.... :

,.,,’.’,,.,.,..4a v

Right eye

FIGURE6. A lateral shiftbetweenthe twohalf-imagesof a stereogramleads (within the horizontal plane) to similar disparities as a headtranslation in the primary direction. T, denotes the translation of thehead towards the plane, Zthe interocrdardistance and Z. the distance

between the stimulus and the eyes.

a)Head translation

-20-40

Her.

1

100disp.[arcmin] -lo:

-40-20

relationshipin a straightforwardmanner as is illustratedin Fig. 6. From this figure it immediately follows that:

()

~shift ~1tan :6 = — —

T,=(6)

Z. —TZ“

Figure 7(a, b) shows the horizontaland vertical disparityfields of the plane (which was initially positioned at100cm) after a head translation of 25 cm towards theplane. Effectively these fields are similar to the fieldscaused by a plane at 75 cm, which means that both thehorizontal and vertical disparity fields are more stronglycurved than those of a plane at 100cm. According to thegiven relationshipbetween lateral shift and head transla-tion the disparitiescaused by a head translationof 25 cmtowards the frontal plane (at 100cm in front of the eyes)can be simulatedby a lateral shift of 2.2 cm between-thehalf-images of a stereogram (at 100cm in front of theeyes). Figure7(c),showsthe horizontaldisparityfieldandFig. 7(d) the vertical disparityfield of such a stereogram.

in primary direction

-20-40

Veri. 100disp.

I[arcmin] .lJ

-40-20

01. 20

‘cD [deg] 40

Offset of stereogramc)

-20-40

Her.

1

100diap.[arcmin] -lJ

-40-20

d) -

.--20

Difference between head translation and uffsete) f)

-20 -20-40

20-40

Hor

1

20Veri

dlsp o dlsp[arcmm] -20 t[arcmm] -2;

-40-20

-40-20

<

FIGURE7. The top twopanels showhorizontal(a) andvertical disparity(b) of the frontalplane (initiallyat 100cm) after a headtranslationof 25 cm in the primary direction.The middlepanels showthe horizontal(c) and vertical disparity(d) after a lateralshift of 2.2cm within the theoretically correspondingstereogram. The bottom two panels show the difference in disparity

between head translation and a stereogram-inducedlateral shift.

STABILITYOF BINOCULARDEPTHPERCEPTION

Head rotation about the vertical axis

3835

a)

-40

Her.

t


-40-20

c)

b)

-20-40

Velt.

1


-40-20

Horizontal scale of stereoaram

-20 -20-40 -40

Her. ‘M

1

Vert.

I


disp.[arcmin] .l&

-40 -40-20 -20

Difference between head rotation aboutvertical axis and horizontal scale

e) f)

-20 -20-40 -40

20Her.

20

t

Vert.disp. disp.[arcmin] -,: t[arcmin] -~

-40 -40-20 -20

<

FIGURE8. The top two panels show horizontal (a) and vertical disparity (b) of the initially frontal plane after a head rotationover 20 deg about the vertical axis. The middlepanels showthe horizontal(c) and vertical disparity(d) after horizontalscalingof 2.2%within the theoreticallycorrespondingstereogram.The bottomtwo panels showthe difference in disparity inducedby

head rotation about the vertical axis and the stereogram-inducedhorizontal scale.

The similarity between Fig. 7(a) and (c) as well asbetween Fig. 7(b) and (d) implies that in the disparitydomain head translation in the primary direction andlateral shift of the two entire parts of a stereogram arealmost equivalent. Figure 7(e,f) show the difference indisparity between the Fig. 7(a,b) and (c,d).

The results reported so far can be related to the resultsof Erkelens & Collewijn (1985a,b). They recognized thata change of fixation causes a translation of the retinalimages. They also suggested that an offset in thedisparity domain corresponds to a lateral shift betweenthe two parts of a stereogram. Figure 7 shows that thelatter suggestion is not entirely correct. A lateral shiftwithin a stereogram corresponds to a head translationtowards the stimulus,but not to a vergence movementofthe eyes.

Rotation of the head about the verticalaxis vs horizontalscale within the stereogram

Considera frontalplane at 100cm which is fixatedat alongitudeof 20 deg. Next, consider a clockwise rotationof the head about the vertical axis over an angle of 20deg. Figure 8(a) shows the horizontal disparity of theplane after the rotation. The distance between the planeand the head is shorter for negative than for positive ~.As a result the disparityfield is more curved for negative~, which is in agreement with the results shown in Fig.4(a). Figure 8(b) showsthe vertical disparityof the planeunder the sameviewingconditions.The vertical disparityis anti-symmetricalwith respect to the line # =20 andwith respectto the line 0 = O.The fact that there is a largerdifference in distance between the plane and either eyefor negative than for positive ~ results in larger verticaldisparitiesfor negative ~.


In the introductory section we explained why thedisparity fields of horizontal scale can be expected tocorrespondto the disparityfieldscaused by head rotationabout the vertical axis. Now we will calculate thedisparity fields caused by horizontal scale between thetwo parts of the theoretically correspondingstereogram.In order to know which stereogramcorrespondsbest weuse a relationship between slant and the amount ofhorizontal scale [see van Ee & Erkelens (1996a) for aderivation]:

‘ l a(7)

where M is the magnificationfactor of horizontalscale,Zthe interocular distance and Z. the distance from thestimulus.* According to this relation, a frontal plane at adistance of 100cm viewed after a head rotation over 20deg about the vertical axis correspondstheoreticallyto astereogram at a distancet of 106cm with a horizontalscale of 2.2$Z0.

Considera stereogramat a distanceof 106cm in frontof the eyes. Figure 8(c) depicts the horizontal disparitycaused by a horizontalscale of 2.270.Since the points ofintersection of the light rays which come from corre-spondingpointsof both half-imagesof the stereogramarenearer for negative q5than for positive ~, the disparityfield is more curved [as in Fig. 8(a)]. Figure 8(d) showsthe verticaldisparitythat is causedby this transformation.Note that the horizontal scale transformationalso has aninfluenceon vertical disparitybecause the left-handsidesof the half-images of the stereogram are translated inopposite directions, which alters the distance of theseparts from the eyes (the same holds for the right-handsides).As shownin Fig. 8(b), the verticaldisparityis anti-symmetrical with respect to the line O= O. The bottompanels of Fig. 8 show the differencein horizontal(e) andvertical (~ disparity induced by head rotation about thevertical axisand by the correspondinghorizontallyscaledstereogram.

Rotation of the head about the horizontal axis vshorizontalshear within the stereogram

Figure 9(a) shows the horizontal disparity when the

*We adopt Ogle’s notation. Ogle (1950), who related slant to hori-zontal magnificationusing an aniseikonic lens, found:

‘ 1The relationship between slant and magnificationin the case of astereogram is slightly different from the relationshipin the case ofaniseikonic lenses.

~FromFig. 2 it can be inferredthat the geometryof a frontal plane at adistance of z. viewed after a head rotation over Mdeg is similar tothat of a plane slanted over a deg but at a distance of z@os ct.

&4fter completing this paper, Prof. Collewijn remarked that Ogle &Ellerbrock (1946) derived a similar equation in order to describethe relationship between the slant of one, in the medial plane-positioned,vertical line and the retinal orientationdisparityof thisline.

initially frontal plane at a distanceof 100cm and fixatedat a latitudeof 20 deg is viewed after a forwardrotationofthe head (about the horizontal axis) over an angle of 20deg. Since the distancebetween the plane and the head isshorterfor negativethan for positive(3,the disparityfieldis more curved for negative 0. Figure 9(b) shows thevertical disparity for the same viewing conditions. Thevertical disparity is anti-symmetricalwith respect to the@= O directionbut is not anti-symmetricalwith respect tothe 0 = O direction. This time, vertical disparities fornegative 0 are larger than those for positive 0.

The relationship between slant and horizontal shear(angle P) is (see van Ee & Erkelens (1996a) for aderivation)~:

‘lant=arctan(tan~ ”?)(8)

This means that, theoretically, the disparity field of afrontal plane at a distance of 100cm viewed with thehead forwardly rotated over 20 deg corresponds to ahorizontallyshearedstereogramwith magnitude1.26degat a distance of 106cm. Figure 9(c, d) shows thehorizontaldisparityand vertical disparity induced by thecorrespondinghorizontallysheared stereogram. [Figures9(c) and 5(c) are similar to each other because thehorizontal component of disparity induced by a hori-zontal shear of 1.26 deg is similar to the horizontaldisparity caused by differential rotation of 1.26 degwithin the stereogram.]The bottompanels of Fig. 9 showthat the horizontal(e) and vertical (f) disparitycaused bya forwardhead rotationresemblesthe horizontaldisparityinducedby horizontalshear. The equivalenceis not verygood. A possible reason is that the horizontal shear of astereogram affects the x-component of the perceivedplane, which is not the case with perceivedfrontalplanesafter a rotation of the head. A slanted plane, induced byhorizontalshear of a rectangularstereogram,is perceivedas a trapezoid with the small side nearer than the largeside.

In this paper we have studied the influenceof eye andhead movements on the binocular disparity field. Wehave also investigatedwhat sortof disparityis inducedbydifferential rotation, horizontal lateral shift, horizontalscale and horizontal shear between half-images of astereogram.We have found that in the disparity domain:

1. Cyclovergence resembles differential rotation be-tween the half-imagesof the stereogram;

2. Head translationin the primary direction resembleshorizontal lateral shift between the half-images ofthe stereogram;

3. Rotationof,the head about the vertical axis (side-to-side rotation) resembles horizontal scale betweenthe half-imagesof the stereogram;

4. Rotation of the head about the horizontal axis(forward rotation) resembles horizontal shear be-tween the half-imagesof the stereogram.

—

STABILITYOF BINOCULARDEPTH PERCEPTION

Head rotation about the horizontal axis

3837

a)

-20-40

Her. 100disp.

1[arcmin] .lo~

-40-20

b) ~d;9J40

m

-20-40

Vert. 100disp.

1[arcmin] .lo~

-.--20

Horizontal shear of stereogramc) 13[deg] 40

-20-40

Her. 100disp.

t[arcmin] -108

-40-20

d)

-20-40

Vert. 100disp.

I[arcmin] -108

-40-20

e)

Her.disp.[arcmin]1

Difference between head rotation abouthorizontal axis and horizontal shear

13[deg] 4CI f) EI[deg] AnI

-20 -20-40

20-40

20Verl

o dlspt

-20 [arcmm] -2:

-40-20

-40-20

FIGURE9. The top two panels show horizontal (a) and vertical disparity(b) of the initially frontal plane after a head rotationover 20 deg about the horizontal axis. The middle panels show the horizontal (c) and vertical disparity (d) after horizontalshearingover 1.26degwithin the correspondingstereogram.The bottomtwo panelsshowthe differencein disparityinducedby

head rotation about the horizontal axis and stereogram-inducedhorizontalshear.

These numerical results lead to new interpretationsofthe results of earlier experiments.

Sensitivity of stereopsis

In order to interpretthe disparitiesof the bottompanelsof Figs 5 and 7, 8 and 9 (that is, the differencesbetweeneye and head movement-induceddisparitiesand theore-tically corresponding stereogram-induceddisparities) itis importantto realize that sensitivityof humanstereopsisvaries with eccentricity.The relevantquestionis: Are thedisparities of the bottom panels small enough for thevisual system not to perceive the difference between adisparity field induced by an eye or head movement andthe disparity field induced by the above-mentionedstereograms,correspondingto the eye or head movement.Not much is known about the sensitivity of peripheralbinocular vision. Several studies showed that stereo-

acuity strongly degrades outside the foveal area (e.g.Fendick & Westheimer, 1983; Badcock & Schor, 1985;McKee et al., 1990a).

Fendick & Westheimer (1983) found that stereoacuityis about 1 arcmin at an eccentricity of 10 deg periph-erally. According to Drasdo (1991) the stereoacuity (V)found by Fendick & Westheimer (1983) can be extra-polated with eccentricity < (in deg) by a linear function:V= 0.1 + 0.12 L (in minarc). In this equation there is nodifferencebetween horizontaland vertical eccentricities.The idea of extrapolation is based on similar linearextrapolation functions for several monocular domains(vernier,LandoltC acuity, etc.) but has not been verifiedfor binocularvision.

However, it should be realized that the experimentaldata concerning stereoacuity have been obtained underideal and controlled laboratory conditions,usually with


experienced subjects. The primary aim of such studieswas to investigatethe limits of the visual system and notthe. sensitivity of binocular vision during naturalbehaviour. Moreover, in view of the fact that instereoacuitymeasurementsdepth judgments are alwaysrelative and invariant under whole-field transformationsdue to eye and head movements, stereoacuity is not anindicator of the precision of all disparity information.

A useful experiment has been done by Schumer &Julesz (1984). They showed by using random-dotpatterns that at a pedestral disparity of 0.5 deg, thethreshold for detecting a corrugated plane from a flatplane was 33 minarc (theirFig. 12)almost irrespectiveofthe corrugationfrequenciesthey used. However, as far aswe know, the only paper on stereosensitivity forrelatively realistic stimuli is the study by McKee et al.(1990a). They showed that in comparison to lateraljudgments of distance, stereoscopicjudgments are notprecise. In addition, their study mentioned variousexamples.and provides several references to demonstratethe insensitivityof stereopsis.They argue that in the firstplace stereopsis is for performing tasks at an arm’sdistance or for breaking camouflage.

We still have to answer the question of whether thedisparitiesof the bottompanelsof Figs5,7,8 and 9 are sosmall that the visual system cannot perceive thedifference between a disparity field induced by an eyeor head movement and the disparity field induced by theabove-mentioned stereograms corresponding to the eyeor head movement. Although stereoacuity is too preciseto be an appropriate indicator for the sensitivity ofstereopsis, we are more or less obliged to use it as anindicator, since no other indicators have been investi-gated in the literature on peripheral vision. We useDrasdo’s theoretical linear stereoacuity-thresholdfunc-tion in order to interpret the detectabilityof the disparityfield of the bottompanels of Figs 5,7,8 and 9. Most (butnot all) of these disparity fields fall below Drasdo’sthresholds. A tolerance analysis which we conductedrevealed that even for planes at a distance of only 40 cm(where the horizontaldisparityfield is stronglycurved,ascan be inferred from Fig. 4) the disparity differences inmost situationsfall below Drasdo’s (Drasdo, 1977,1991)thresholds.

Figure 8(e) shows the difference between horizontaldisparityinduced by head rotation about the vertical axisand horizontaldisparityinducedby a horizontallyscaledstereogram. Figure 9(e) shows the difference betweenhorizontal disparity induced by head rotation about thehorizontal axis and horizontal disparity induced by ahorizontally sheared stereogram. The latter differencedoes not fall below stereoacuity thresholds for largeeccentricities. The results of Figs 8(e) and 9(e) can be

*Thereport of Rogers& Graham(1983)showedthat subjectsare moresensitive to horizontal shear than to horizontal scale; perceivedslants induced by horizontal shear demonstrate lower detectionthresholds and lower latencies than slant induced by horizontalscale.

related to the well-knownhorizontal/verticalanisotropy*in depthperceptionwhich has been reportedby Rogers&Graham (1983). One might argue that the anisotropy iscaused by the fact that the resemblance betweenhorizontal shear and forward rotation is less than theresemblance between horizontal scale and side-to-siderotation. Therefore, one could further argue that hori-zontalshear is lessambiguousthan horizontalscaleand isconsequentlyperceived better.

Eye movements and the stability of depthperception

Vergence movementsof the eyes lead to a translationof the images over the retinae. A commonlyused way toinduce translation of a stimulus over the retinae in anartificialway is to generate the images by a haploscope.In a haploscope the displays of the stimuli for the twoeyes can be independentlyrotated about the centre of theeyeball. However, except in the case of one uniquecombination of a fixation point and a location of thescreens of the haploscopethere is in principle a conflictbetween oculomotor cues and disparity. Cyclovergenceleads to a rotation of the images over the retinae.Cyclovergencecan be mimicked by differential rotationof the half-images of a stereogram. However, withoutcompensationalrotation of the eyes, differential rotationcannot mimic a real world stimulus which means that,again, there is a conflict between disparity cues andoculomotorinformation.Thus, (cyclo)vergencegenerallycannotbe mimickedby a stereogramwithout introducingthis conflict.However, several reports mentioned in theIntroductionshow that this conflict is not dominantwithrespect to depth perception. In situationswhere conflict-ing eye muscle information is present, overall retinaldisplacements of a stimulus do not lead to changingbinocularperceptionof depth in the case of large stimulior they lead to only weak perceptionof depth in the caseof small stimuli.

Head translation in the primary direction and thestabili~ of depthperception

Erkelens & Collewijn(1985a) and Regan et al. (1986)showed that differential lateral translation of the entiredichoptically presented half-images does not elicitperception of depth, even when the eyes pursue thelateral motion with a gain unequal to one (Erkelens &Collewijn, 1985b).They found that in the presence of avisual reference (which moved with a translationalvelocity different from that of the stimulus) perceptionof depth was elicited vividly.

In this report we show that disparity induced by atranslationof the head towards the stimuluscorrespondsto a disparity field caused by a lateral shift between theentire half-images of a stereogram. On the basis of thisinsight we conclude that the experimental results ofErkelens & Collewijn (1985a) and Regan et al. (1986)imply that the classof disparityinducedby translationsofthe head in the primary direction does not elicit depthperception. Of course we realize that cues other thandisparity are modifiedwhen the head is translated in the

STABILITYOF BINOCULARDEP1’HPERCEPTION 3

TABLE 1. Relationship between depth perception and the disparity caused by an ego-movementor its correspondingstereogram

Ego-motion Stereogram Relation to depth perception

Change of fixation Haploscopicrotation Poor*Cyclovergence Differential rotation Poort

HeadForward translation Horizontal translation No$Sidewardrotation Horizontalscale Poor~Forward rotation Horizontalshear Poor~

Relevanteye and head movementsare presentedin the first column.The secondcolumnshowsthe transformationsbetween(the entire) stereogramhalf-imageswhich give rise to about the same disparityfield as the movementslisted in column1. Column3 gives psychophysicaldepthperceptionresults relating to experimentswhere disparityfieldscaused by thestereograms of column 2 are presented in isolation.

*See text of the subsection “Eye movements and the stability of depth perception” and also the Introduction. By“haploscopicrotation” we mean a rotation of the displays about the centre of the eyeball.

~Howard& Zacher (1991), see also Introduction.$Erkelens & Collewijn (1985a).~vsn Ee & Erkelens (1996a).

primary direction. (When the distancebetween the headand the stimulusis so large that the stimulusis effectivelyat infinity, both retinal images are identical and thusdisparityhas vanished.During the translationtowardstheobject, disparity developsbecause the retinal projectionsof the object become different and the retinal imagesbecome larger.) However, althoughchanging-sizestimu-lation and changing-disparity stimulation can bothproduce a sensation of motion in depth (Regan &Beverley, 1979)and eye movements(Erkelens& Regan,1986),they act largely independently.Both the motion indepth sensation and the eye movements produced bychanging-size stimulation can be cancelled by antag-onisticchanging-disparitystimulation.In our analysisweconcentrate on the disparity domain.

Rotation of the head and the stabilityof depthperception

Steinman & Collewijn (1980) measured eye move-ments of subjects while they actively rotated their headabout a vertical axis. They found that vergence velocityerrors of the order of 1 deglsec occurred. They alsoobtained the impression that vision remained fused,stable and clear. In their 1985 study (Steinman et al.,1985) they examined their impressionpsychophysically.The study resulted in the conclusion that stereoacuity isnot disturbed by large fixation disparities or highvergence velocities.Patterson & Fox (1984) showed thatthe recognitionof a stereoscopicallypresentedLandoltCwas not impaired by active head rotations either.Concerning tasks which require stereoscopic slantperception, the stability of these tasks during headrotationsor in situationswhere neck muscle informationand disparity information are decoupled, has still to bededuced.

van Ee & Erkelens (1996a) studied the temporalaspects of slant perception with large-field stimuli (novisual reference was present).They found that horizontalscale and horizontalshearbetween the entire half-imagesof a stereogram elicit poor perception of depth for

observationperiods lasting only a few seconds (see alsoGillam et al., 1988). Their experimental result impliesthat the class of disparitywhich is inducedby rotation ofthe head elicits only poor depth perception.

Jones & Lee (1981) reported on experimentsin whichhuman binocular performance and monocular perfor-mance were compared in a variety of visuomotor tasks.They found that stereopsis was not important in theperformance of visuomotor skills in three dimensionswhen the subjects were free to move their heads. Theyconcluded that an important benefit of binocular frontalvision with movinghead is binocularconcordanceratherthan (changing)binocular disparity. In other words, thebenefitof stereopsismay in fact be limited to situationsinwhich the head is stationary (Jones & Lee, 1981).

The relationshipbetween ego-movement-induceddispar-ity and the stabili~ of depthperception

From the previous three subsectionswe conclude thatclassesof disparitywhich can be inducedby eye and headmovements do not appear to be very relevant forstereopsis, at least if presented in isolation. In Table 1the resultsare summarized.We suggestthat the classesofdisparitywhich can be inducedby ego-movementpoorlyelicit depth perceptionbecause they are ambiguous.Theclasses of disparity which correspond to haploscopicrotation or to differential rotation of the entire half-images of a stereogram are ambiguous because thesedisparities could also be induced by vergence orcyclovergence,respectively.Disparitycausedby a lateralshift between the entire half-images of a stereogram isambiguous because instead of being caused by thestimulus it could be caused by head translation in theprimary direction. The classes of disparity associatedwith horizontal scale and horizontal shear between theentire half-images of a stereogram are ambiguousbecause they could also be induced by head rotation.

As explained in the Introduction, tasks based ondisparityprocessingcan be distinguishedinto relief tasks


and metrical tasks (Girding et al., 1995).Insensitivityofstereopsis to disparity fields which result from eye andhead movements (for short observation periods) wouldmean that stereoscopicvision is, in principle,not able toperformmetrical tasks(for shortobservationperiods).Onthe other hand, relief characteristicsare preserved undereye and head movements.From the literature it is knownthat relief tasks can be done reliably in a couple ofmilliseconds (e.g. Kumar & Glaser, 1993; Uttal et al.,1994). The result of performing metrical tasks based onstereopsis alone is not veridical (Gogel, 1960; Foley,1980; Gillam et al., 1984; Mitchison & McKee, 1990;Johnston, 1991;van Ee & Erkelens, 1996a).Visual tasksthat require a metric reconstruction of the three-dimensionalvisual world are not very common.

The insensitivityof stereopsisto disparityfieldswhichresult from eye and head movements(such as thosegivenin Table 1) means that stereopsis is insensitiveto globalzero and first order modifications between the half-images of a stereogram. Stevens & Brookes (1988)reported that binocular 3D information is best acquiredwhere a stereogram contains curvature features. Theirresultsaboutcurvaturefeaturesare related to stereogramsper se and not to the (retinal) disparity fields caused bythese stereograms, since even a stereogram without adifferencebetween the half-imagesgives rise to a curveddisparity field (see for instance Fig. 4). In this study weshow why the zero and first order characteristicsof thestereogram form a special class for which the visualsystem is relatively insensitive.

In our view there are other relevant distinctions inaddition to the distinctionof stereopsisinto metrical andrelief tasks. One of them is the distinctioninto short andlong observationperiods (Gillam a 1988;van Ee &Erkelens, 1996a). A second one is the distinction intoconditions with and without a visual reference (e.g.Gogel, 1963;Shipley& Hyson, 1972;Gillamet al., 1984,1988;Regan a 1986;Howard & Kaneko, 1994;vanEe & Erkelens, 1995). In the case of long observationperiods there is not much of a difference between slantestimation (metrical task) with a visual reference andslant estimationwithout a visual reference. In both casesthere is a large underestimation of slant. However, forshort observation periods, slant estimation without avisual reference is generally extremely poor (van Ee &Erkelens, 1996a).A third relevant distinctionof stereop-sis is a distinction into small and large stimuli.A coupleof reports (Rogers & Bradshaw, 1993, 1995;Howard &Kaneko, 1994) show that vertical disparities have asmaller influence on depth perception for small stimulithan for large stimuli. Oculomotorcues have a consider-able influence for small stimuli but hardly any for largestimuli (Rogers & Bradshaw, 1995; Regan et al., 1986).

*The slant of a surface is not only determinedby horizontal scale orshear between its own half-images, but also by the scale or shearbetween the half-imagesof a visual reference. A positive slant of avisual reference causes a negative slant of the object at hand (andvice versa).

Temporalaspects

Taken together, we suggest that depth perception isinvariant under eye and head movement-induced dis-parity. This formulation is probably too general becausemany authors have found that subjects are able toperceive slant caused by whole-field transformations(inprolongedviewing). Generally, slant estimation inducedby whole-field transformations between the two half-images of a stereogram depends on observation time. Ifsubjects are allowed to view the stereogram for morethan, say, 10 see, then slant estimation is far moreveridical than if they view it for, say, 1 sec (Gillam1988; van Ee & Erkelens, 1996a). That subjects canperceivewhole-fieldslant after prolongedviewing couldbe caused by the integrationof either non-binocularcuesor extra disparity signals (for instancevertical disparity).Inspection of the stimulus by actively making eyemovements might also contribute to the enhancementof the slant perception over time (Enright, 1991).

Recently van Ee & Erkelens (1996b) have suggestedthat Werner’s illusory depth contrast effect* (Werner,1938) may be explained by the idea that stereopsis isrelatively insensitiveto whole-fieldhorizontal scale andshear.This insensitivity,in turn, resultsfrom the fact thatthese transformations induce disparity fields similar tothose inducedby head rotationsas we have shown in thispaper. The fact that slant estimation becomes moreveridical over time, makes their explanation consistentwith the fact that the illusoryslantof a stimuluscausedbyWerner’s depth contrast effect decreases over time (e.g.Kumar & Glaser, 1993).

Robot vision

Three-dimensional imaging has various applications.A possibleapplicationis the designof a binocularsystem(robot) which can produce information about places towhich human beings cannot go or do not wish to go.However, in practice during movementsof the robot theinstability of camera images is such that disparityprocessingunder practical circumstancesfails (Eklundh,1993). The idea that “ego-movement-induceddisparity”is irrelevant for stereopsismay have interesting implica-tions for the future of robotics. Shape perception bymeans of two cameras could be greatly improved if thetypes of disparities brought about by the robot’s ownmovements(which are classifiedin this report) could befiltered out or ignored.

We have calculated the binocular disparity field for awide range of possibleeye, head and stimuluspositions.From the literature it is known that certain classes ofdisparity (such as whole-field horizontal lateral shift,differentialrotation,horizontalscale and horizontalshearbetween the half-images of the stereogram) inducerelatively poor perception of depth, at least if presentedin isolation. These classes of disparity turn out to besimilar to those caused by eye and head movements.Our

STABILITYOF BINOCULARDEPTH PERCEPTION 3841

numerical calculations support the suggestion thatbinocular 3D vision is based primarily on the classes ofdisparity that are invariant under ego-movement.

R

Alpern,M. (1962).Kinematicsof the eye. In Davson,H. (Ed.),Theeye(Vol. 3, pp. 15-27). New York: Academic Press.

Backus,B. T., Banks,M. S. & Crowell,J. A. (1996).Eye positionandvertical disparity in stereoscopic surface perception. IO pV S S

Badcock, D. R. & Schor, C. M. (1985). Depth-increment detectionfunction for individual spatial channels. J OS o 1

Berry, R. N. (1948). Quantitative relations amongvernier, real depthand stereoscopic acuities. J E P7

Carpenter, R. H. S. (1988).M e 2nd edn. London:Pion.

Collett, T. S., Schwarz, U. & Sobel, E. C. (1991). The interaction ofoculomotorcues and stimulus size in stereoscopic depth constancy.P 7

Collewijn,H. & Erkelens, C. J. (1990).Binoculareye movementsandperception of depth. In Kowler, E. (Ed.), Eye m tr v c p (pp. 216-261). Amsterdam:Elsevier.

Collewijn,H., van der Steen,J. & van Rijn,L. J. (1991).Binoculareyemovements and depth perception. In Gorea, A. (Ed.), Rt v t t a v r (pp.165–183).Cambridge,U. K.: CambridgeUniversityPress.

Cormack, R. & Fox, R. (1985). The computationof retinal disparity.P & P s1

Cumming, B. G., Johnston, E. B. & Parker, A. J. (1991). Verticaldisparities and perception of three-dimensionalshape.N 34

Donders, F. C. (1876). Versuch einer genetischen Erklarung derAugenbewegmrgen,P A pp. 373421. (Cited in vonKries, 1911).

Drasdo, N. (1977).The neural representationof visual space. N2 6

Drasdo, N. (1991). Neural substrates and threshold gradients ofperipheral vision. In Kulikowski, J. J., Walsh, V. & Murray, I. J.(Eds),L o (pp. 251-265). London:Macmillan Press.

van Ee, R. & Erkelens, C. J. (1995). Binocular perception of slantabout oblique axes relative to a visual frame of reference.P 2

van Ee, R. & Erkelens, C. J. (1996a) Temporal aspects of binocularslant perception. V R 4

van Ee, R. & Erkelens,C. J. (1996b).Arrisotropyin Werner’sbinocrrlardepth contrast effect, V R 2

Ekhrndh, J. O. (1993). Vergence control in machine vision. Oralpresentation at the International Conference and Nato AdvancedWorkshop on Binocular Stereopsis and Optic Flow, Toronto,Canada.

Enright, J. T. (1990). Stereopsis, cyclotorsional “noise” and theapparent vertical. V R 1

Enright, J. T. (1991). Exploring the third dimension with eyemovements:Better than stereopsis.V R 1

Erkelens, C. J. & Collewijn, H. (1985a). Motion perception duringdichoptic viewing of moving random-dot stereograms. VR 5

Erkelens, C. J. & Collewijn, H. (1985b). Eye movements andstereopsis during dichoptic viewing of moving random-dotstereo-grams. V R 1

Erkelens, C. J. & Regan, D. (1986). Human ocular vergencemovements induced by changing size and disparity. JP 3 1

Fendick, M. & Westheimer, G. (1983). Effects of practice and theseparation of test targets on foveal and peripheral stereoacuity.VisionR 1

Ferman,F., Collewijn,H., Jansen, T. C. & van den Berg,A. V. (1987).

Human gaze stability in the horizontal, vertical and torsionaldirectionduringvoluntaryhead movements,evaluatedwith a three-dimensionalscleral inductioncoil technique. V R8

Fick, A. (1854). Die Bewegungen des menschlichen Augapfels.Z r M 1

Foley, J. M. (1980). Binocular distance perception. PR 4

Friedman,R. B., Kaye,M. G. & Richards,W. (1978).Effect of verticaldisparity upon stereoscopicdepth. V R 3

Frisby, J. P., Buckley, D., Wishart, K. A., Porrill, J., G&ding, J. &Mayhew,J. E. W. (1995). Interaction of stereo and texture cues inthe perception of three-dimensional steps. V R1

G&ding, J., Porrill, J., Mayhew, J. E. W. & Frisby, J. P. (1995).Stereopsis, vertical disparity and relief transformations. VR 7

Gillam, B., Chambers,D. & Rus$o,B. (1988).Postfusionallatency instereoscopic slant perception and the primitives of stereopsis.J E P H PP 1

Gillam, B., Flagg, T. & Finlay, D. (1984). Evidence for disparitychange as the primary stimulus for stereoscopic processing.P & P 5

Gogel,W. C. (1960).The perceptionof shapefrombinoculardisparitycues. J P 1

Gogel,W. C. (1963).The visualperceptionof size anddistance. VR 1

von Helmholtz, H. (1911). H t po Translated(1925)from the tliird Germaneditionof Southall,J. P. C. (Ed.),H P O York:TheOptical Society of America.

Howard, I. P. (1982).H v o Toronto:John Wiley& Sons.

Howard,L P. & Kaneko,H. (1994).Relative shear disparities and theperception of surface inclination. V R 2

Howard, L P., Ohmi, M. & Sun, L. (1993). Cyclovergence: Acomparisonof objective andpsychophysicalmeasurements.Em B R 3

Howard, 1. P. & Zacher, J. E. (1991). Human cyclovergence as afunction of stimulus frequency and amplitude.E BR 4

Johnston, E. B. (1991). Systematic distortions of shape fromstereopsis. V R 1

Jones, R. K. & Lee, D. N. (1981).Why two eyes are better than one:The two views of binocular vision. J EP H P P 3

Julesz, B. (1971). F c p Chicago:Universityof ChicagoPress.

von Kries, J. (1911). Notes on $27 on the ocular movement. InH t o o Translated(1925)fromthe third German edition of Southall, J. P. C. (Ed.), HP O (Vol. 3, pp. 127–154). New York: TheOptical Society of America.

Kumar,T. & Glaser, D. A. (1993).Temporalaspects of depthcontrast.V R 9

Listing, Unpublished.(Cited in Helmholtz, 1911,Vol 3,1854, p. 121).Logvinenko,A. D. & Belopolskii,V. I. (1994).Convergenceas a cue

for distance.P 2Mayhew,J. & L.onguet-Higgins,H. C. (1982).A computationalmodel

of binoculardepth perception.N 2 3McKee, S. P. (1983). The spatial requirements for fine stereoacuity.

V R 1McKee, S. P., Levi, D. M. & Bowne, S. F. (1990a) The imprecision

stereopsis. V R 1McKee, S. P., Welch, L., Taylor, D. G. & Bowne, S. F. (1990b)

Findingthe commonbond:Stereoacuityand the other hyperacuities.V R 8

Mitchison, G. J. & McKee, S. P. (1990). Mechanismsunderlyingtheanisotropy of stereoscopic tilt perception. V R1


Mitchison,G. J. & Westheimer, G. (1984).The perceptionof depth insimple figures. V R 1

Mitchison, G. J. & Westheimer, G. (1990). Viewing geometry andgradients of horizontal disparity. In Blakemore, C., (Ed.), VC e (pp. 302–309).Cambridge,U. K.: CambridgeUniversityPress.

Nelson, J. (1977). The plasticity of correspondence: After-effects,illusions and horopter shifts in depth perception. JT B 2

Ogle, K. N. (1950). R b v Philadelphia:Saunders.

Ogle, K. N. & Ellerbrock, V. J. (1946). Cyclofusional movements.A O p3

Patterson, R. & Fox, R. (1984). Stereopsis during continuous headmotion. V R 2

Regan, D. & Beverley, K. I. (1979). Binocularand monocularstimulifor motion in depth: Changing-disparityand changing-sizefeed thesame motion-in-depthstage. V R 1

Regan, D., Erkelens, C. J. & Collewijn, H. (1986). Necessaryconditions for the perception of motion in depth. IO pV S 2

van Rijn, L. J., van der Steen, J. & Collewijn,H. (1994).Instabilityofocular torsion during fixation: Cyclovergence is more stable thancycloversion. V R 1

Rogers, B. J. & Bradshaw, M. F. (1993). Vertical disparities,differential perspective and binocular stereopsis. N 32

Rogers, B. J. & Bradshaw, M. F. (1995). Disparity scaling and theperception of frontoparallel surfaces. P 1

Rogers, B. J. & Graham, M. E. (1983).Anisotropiesin the perceptionof three-dimensionalsurfaces. Science, 221, 1409–1411.

Schor, R. H., Kearney, R. E. & Dieringer, N. (1988). Reflexstabilization of the head. In Peterson, B. W. & Richmond, F. J.(Eds), C h m (pp. 141-166). Oxford: OxfordUniversityPress.

Schumer, R. A. & Julesz, B. (1984). Binocular disparity modulationsensitivity to disparities offset from the plane of fixation. VR 5

Shipley, T. & Hyson, M. (1972). The stereoscopic sense of order-aclassification of stereograms.A J OA A A O 8

Steinman, R. M. & Collewijn, H. (1980). Binocular retinal imagemotion during active head rotation. V R 4

Steinman, R. M., Levinson,J. Z., Collewijn, H. & van der Steen, J.(1985). Vision in the presence of known natural retinal imagemotion.J O S A 2

Stevens, K. A. & Brookes, A. (1988). Integrating stereopsis withmonocular interpretations of planar surfaces. V R3

Uttal, W. R., Davis, S. D. & Welke, C. (1994).Stereoscopicperceptionwith brief exposures.P & P 5

Werner, H. (1938).Binoculardepth contrast and the conditionsof thebinocular field.A J P 4

Westheimer, G. (1979a) The spatial sense of the eye. IO V S 8

Westheimer, G. (1979b) Cooperative neural processes involved instereoscopic acuity.E B R 8

Westheimer, G. & McKee, S. P. (1978). Stereoscopic acuity formoving retinal images.J O S A6

Westheimer, G. & McKee, S. P. (1979). What prior uniocularprocessingis necessary for stereopsis.I O

V S 6Wheatstone,C. (1838).On some remarkable and hitherto unobserved

phenomenaof binocular vision. P TR S 1 3

A are indebted to Bert van den Berg andHendrik-Jan van Veen for many inspiring discussions. We thankAndrea van Doom and Jan Koenderinkfor the use of their computerand various helpful remarks. We thank S. M. McNab for linguisticadvice. This work was supportedby the NetherlandsOrganizationforScientific Research (NWO),grant no. 805-33-702.

,., , !

,,,

Documents

Stability of Binocular Depth Perception with Moving Head and Eyes