Sharkey Et Al. - 1998 - A Local Perception Filter for Distributed Virtual

8/13/2019 Sharkey Et Al. - 1998 - A Local Perception Filter for Distributed Virtual

1/8

A Local Perception Filter for Distributed Virtual EnvironmentsPaul M. Sharkey Matthew D. Ryan David J. Roberts

P.M [email protected] M [email protected] cybdj [email protected] Systems Research Group: www .cyber.readi ng.ac.uk/r esearcWI SRG/home.html

Department of Cybernetics, The University of Reading, Whiteknights, Reading RG6 6AY, UKAbstract

Dynamic multi-user interactions within distributedcollaborative virtual environment CVE) suffer f romabrupt stare transitions due fo communication delays -anacrion by one user only becoming apparent to anotheruser after the delay. This results in a divergence of theenvironmentfor the duration of the delay, followed by anabrupt jump to resynchronise. so that the currenf state ofthe virrual world is displayed. Such discontinuiries do notoccur in the real world and rhus appear unnatural anddisconcerting to the, users. This paper develops theconcept of a 3+-D Perception Model, as an alternarive foprediction, which locully filters the underlying model,ensuring that each user views a continuous version of heenvironmenr, such that no jumps occur despite delaysarixiq from wmofc user interaction. Each users 3/-Dfilter is specific ro their own circumsrances, whence eachusers perception of the environment will be slighrl)different from that of other users.1: Introduction

Distrihuted virtual reality systems generally usedatabase replication to reduce bandwidth consumption.Early systems replicated the entire environment at eachparticipating computer. As environments have grown,newer systems [5][8][161 employ partial replication - acomputer will only hold copies of objects in which thelocal user (or application) is potentially interested.

Bandwidth consumption is reduced in most systems byonly communicating changes in object state or hehaviour,hereafter called an event. Some systems use discreteevents [4], for example, the movement of an object maybe communicated as a continuous stream of positionevents. This approach is bandwidth intensive and onlysuitable for communication of movement between a lownumber of participants. Other systems support continuousevents that define the evolution of an objects state overtime. For example, the movement of an object may herepresented by a start time and parametric description of

velocity and acceleration [13],[14].In a distributed system, the commencement of a

remotely generated event will normally he delayed by thecommunication system. With discrete events this has dieeffect of observing the action of remote users behindcurrent time. With continuous events, remote users actionsare perceived at current time hctween the time an event isreceived and a superseding event is generated. The periodof communication delay, however, induces an erroneouscondition where an objects behaviour still follows asuperseded event. This has the effect of inducing suddenjumps in the normally continuous evolution of state. Forexample, in a game of baseball, a hall may be seen to havegone through a remote users bat and then suddenly jumpinto its true position and path.1.1: Deterministic vs. non-deterministic events

Bandwidth consumption may be further reducedthrough replicating the generation of deterministic events.For example, a hall bouncing from a wall can hecalculated at each replication without the need forcommunicating the bounce event. This approach has theadded advantage of overcoming network inducedinconsistencies for those events that are deterministic. Anevent is, however, only deterministic when no other eventcan negate it. This implies some synchronisation of thereplications that ensures all potential conflicts are knownbefore an event is generated. Ensuring that all events aredeterministic is fundamentally impossible in human in theloop simulations supporting real time interaction [I I].

Non-deterministic events may arise from user input.partial replication and network delays. These events mayalso he replicated, provided that some form of roll hackstrategy is employed to negate the effect if the event isfound to he false. Returning to the basehall example, auser might he seen to hit a hall when in fact they slipped atthe last moment and missed it.

To explain the relationship between deterministic andnon-deterministic events let us consider collision detectionand collision prediction of objects [ 141. As path graphs

242O-8186-8362-7/98 $10.00 0 1998 IEEE


2/8

define the future states of objects, collisions can bepredicted by scanning through object path graphs, e.g. bypairwise comparison of the movement of two objects andfinding the time when such collisions occur.Consequently, the appropriate response at the moment ofcollision may be calculated locally and in advance and thepath graphs may be suitably modified to reflect thisresponse at the collision time, thereby avoiding the need tocommunicate new state changes for each collision. Givena (distributed) deterministic model of the virtualenvironment, no communications are required until a non-deterministic event occws.

The evolution of this model is only affected by theunpredictability of (non-deterministic) user input whichcan cause unexpected changes in the path graph of anobject, which in turn may affect other object paths,resulting in modified path graphs for some or all objects.These changes must subsequently be broadcast to all sites,effectively generating a new deterministic model for allusers to model until the next non-deterministic input. It isnoted that local users will only require a time parametricdescription of the specific non-deterministic input - theconsequences of the input may be locally calculated.1.2: The effect of communications delays

A non-trivial problem associated with multi-userdistributed environments lies in the communications delaybetween the physical sites - an action by one userbecomes apparent to a second user only after thecommunication delay, resulting in a required jump in thelocal model to display the current state. Specifically, if astationary object starts to move at a velocity v, and thedelay in receiving that information is At, the resultingpositional error on receipt is Ap = Y x At. The positionalerror is proportional to both the delay and velocity of theassociated object.

Perhaps, the most widely known example of wheredelays cause inconsistency in the model, is that ofbattlefield simulation [8]. Here, tanks move relativelyslowly and are reasonably spaced, so the effects of delaysare not readily apparent. The introduction of aircraft intothe environment (travelling at high velocities) illustratesthe difficulty in visualising combat. Missiles which werecalculated to hit aircraft would be seen to explode beforereaching the aircraft. One solution to this is to synchroniseclocks on all machines, and to timestamp messages sent.Here, the aircraft receives a message giving the time offiring as well as the trajectory. However, to implement thistrajectory requires that the aircrafts local modelinstantaneously update the missile position from thelaunch site to its current position, an instantaneouspositional jump (inconsistency) of Ap= v,,,i,,i,ex At.

1.3: Smooth visualisationDistributed CVEs use many different techniques to

maintain consistency and synchronisation of the world[IO]. A crucial difference in implementation depends onwhether or not the CVE is synchronised to a global clock:Asynchronous model. Here, no concept of global time isimplemented. Messages are not timestamped and areimplemented as they are received. Thus, remote users areshown delayed by A to remove discontinuities. Objectscoincident with the remote user are also delayed by A inthe local users view. Therefore, objects being interactedwith by a remote user are displayed at the remote userstime and interactions are viewed without discontinuity.However, were the object to be passed to a different user,at a different delay, then inconsistencies arise.Synchronous model. Using a global clock, all federatesare synchronised and all messages timestamped. Onreceipt of messages the correct current trajectories areimplemented. Thus, objects are displayed in real time and,close to local user, allow for interaction in real-time.However, were the object to be passed to a remote user, ata some delay, then, as before, inconsistencies arise.Hybri d synchr onous/asynchr onous model. The techniquedescribed in this PaPer proposes a hybridsynchronous/asynchronous model, which combines theadvantages of each method. A global time must bemaintained between all participants to ensure consistencyand allow interactions between users. As in theasynchronous model, remote users are displayed at adelayed time to ensure no discontinuities arise, facilitatingreal-time interaction with objects (albeit delayed by theappropriate delay). As in the synchronous model, objectsclose to the local user are implemented in real time, thusfacilitating real-time local user/object interaction. Thetechnique is realised though the 3X-D Perception Filter.1.4: The 3 D perception filter

The 3%D filter extends the 3-D spatial model of thesimulated enwronment to include viewer-centredinformation on the temporal separation (which embodiesthe communications delays) of all other remote users fromthe local user. The 3%-D model places a local user at alocalised co-ordinate frame (x, y. z, A) = (0, 0, 0, 0) andeach remote user at (xi. yi. zj, AJ. where Ai represents thecommunications delay between user i and the local user.Furthermore, the 3SD model provides a smooth andcontinuously differentiable contour that connects all pointsdefined by the users. The result is that: all objects andusers evolve continuously; no discontinuities areperceived; the effects of delays are ameliorated; and thelocal model is seen to be consistent at all times.

It is important to note that the perception model acts

243


3/8

as a local filter on the underlying local model. The modelis conservative in that no prediction is carried out onperceived object trajectories. Yet the system is optimisticin that the underlying model may implement predictivestrategies on actual trajectories for collision detection, andinclude rollback. As the communications delays arelargely dependent on the physical network, different userswill perceive different delays. Thus it is accepted that eachuser will have a different perception of the environment -the difference being explicitly dependent on the delays.With small delays this difference is small.2: Multi-user virtual environments

It is assumed that the multi-user CVE utilises a timeparameterised model which is distributed to all users, suchthat each model defines exactly the motions of aildeterministic objects within. Any non-deterministic eventis initiated at a user site, is modelled exactly at that siteand then distributed, either exactly or as anapproximation, to all other users via the network. Eachuser (which includes the platform upon which the localmodel is calculated and displayed) is connected to otherusers via a network, associated with which arecommunications delays. The delay between any two usersmay be variable and indeed direction dependent. Tomanage a distributed model synchronisation of each localmodel is required.2.1: Clock synchronisation for parametric mo els

Synchronisation of all sites on the network whichobserve the same virtual world is required to maintaindatabase consistency. Recent approaches use a real-timeglobal clock to synchronise (and maintain synchronisationof) machines (such as NTP). More expensive methods,including GPS and radio clocks, can increase accuracywithout relying on network communications. Oncesynchronised, all machines clocks are set to the sametime, thus all local path graphs will bc synchronised.Incremental changes of position take place with respect totime, so users on fast machines will observe smootherobject changes than users on slower machines as theirmachines can apply higher frame rates and perform moreiterations in a given time interval. The advantages of usingreal-time synchronisation with parametric paths are:all models will be in sync irrespective of frame rates; ifany one site fails the remaining sites will not be forced tostop (robustness); message passing, and hence networkload, is reduced; and the past or future (with the exceptionof non-deterministic user input) can be observed.

Each user has a local real-time system clock which,with proper synchronisation, will ideally be equal to theglobal real-time clock. In practice, howcvcr, the local real-

time clocks will tend to drift slightly but can beguaranteed to be within a certain threshold, Ft. Withcontinuous resynchronisation this threshold may be keptwithin 30 milliseconds of the Network Time Protocol[ I][31 and is acceptable for real-time interaction.2.2: Characteristics of communications delays

While each user accessing a multi-user system will bephysically separated by the users location in the network,users are also separated spatially and temporally withinthe virtual environment. Spatial separation is character&dby the distance within the virtual world between the localuser and each remote users, denoted DR, while temporalseparation derives from the communications delay, LAK.The delay can cause inconsistencies between sites - if onesite broadcasts a new parametric description for a movin,gobject, then within the time taken for the communicationto be received, the transmitting site and all receiving siteswill have differing models of the environment.The immediate effect of this is to make objects appearto behave in an unnatural manner. In the example above,the remote sites can only apply the new parametric modelafter receipt of message. Until that instant all objects willcontinue on their previous paths for the duration of thenetwork delay. On receipt of the information the newenvironment model will cause all affected objects to jumpto their new parametric paths to keep in sync with thecorrect new model of the sending site. These abrupt statetransitions will appear disconcerting and diminishconsiderably the illusion of realism of the virtualenvironment (such jumps do not occur in the real world).

Where a user is simply an observer to the environment,user 0, i.e. one who does not interact with theenvironment, this problem can be eliminated by updatingthe observers environment using time or = -A,,,,,,where Ahax is the maximum delay. Thus, observer 0will have received all relevant changes before the localenvironment has caught up with the current state of themodel - model 0 will be always be delayed by Ahaxseconds. Clearly this creates a problem if 0 wishes tointeract with the environment as any interaction between0 and the model must occur in real time with ideally zero(but practically small) local latency.

From this it is clear that two users will not be able todirectly and dwumically interact with each other if thereis any perceptible communication delay between them - acommon example of this occurs during long distancetelephone conversations where the delay uwses unnaturalpauses and interruptions when one caller erroneouslyanticipates that the other is not responding. A secondexample occurs in t&operation where there is aconsiderable distance between the master and slaveenvironment (e.g. space t&robotics control from an earth

244


4/8

station). To successfully nnplement smooth user-userinteraction the delay must ideally he eliminated.

Practically, the delay cannot he eliminated (evenbetween two users using the same computer platform).However, if the nature of the interaction is of sufficientlylow bandwidth then the interaction may seem reasonablysmooth, eve if a bit laboured. For many virtualenvironments this restriction may be overly limiting - amore dynamic form of interaction may be required.

For the implementation considered in this paper it isassumed that users are spatially separated, i.e. directdynamic user-user interaction is not allowed (indirectinteraction can occur. for example. passing an object). Wenote, however, that direct collaborarive interaction ispossible, for example two users moving a table, thoughspatial separation is still required.2.3: Measuring communications delays

Estimation of communication delays in wide areanetworks [lo] has shown that the median of the delaybetween any two networked sites is constant except foroccasional peak periods which occur at certain times ofthe day when the delay rises considerably. It was shownthat network load has little effect on the mediorr thoughsome effect on the mean delay. The report recommendscalculating the delay using a robust line-fitting algorithm[IO] which, when given a set of points, returns the roundtrip time. The algorithm describes the best-tit line, isinsensitive to outlying values, and also returns the absolutestandard deviation indicating the accuracy of thecalculation. Thus, when identifying communication delaysin wide area networks, the delay obtained by the robustline-fitting algorithm may be considered accurate most ofthe time.

For this paper, it can be reasonably assumed thatnetwork delays, though variable, are measurableaccurately (e.g. via radio clock (re-)synchronisation). Weassume that they are normally constant and within certainmaximum bounds. In the case of exceptional delays, thestability of the system developed here, in terms ofcommunicating over the WANs, becomes no worse thanthat of current systems.3: Virtual time

Virtual time was proposed as a new paradigm fororganising and synchronising distributed systems andwhich could he applied to distributed discrete eventsimulation and distributed database concurrency [7]. Thevirtual time paradigm defines a virtual time system as onewhich executes in co-ordination with an imaginary virtualclock which ticks virtual time, which may or may not herelated to real time. In the past, work on virtual tim has

heen concerned with varying the rate of Ilow of timeacross the entire virtual environment. One example hasshown that users can achieve accelerated learning throughtraining using a virtual environment in which the virtualtime evolves faster than normal time [6].

Another use of virtual time is to vary the tlow of timeacross different regions in a virtual world, originallysuggested in [IS], by providing a time alteration as a filterupon the underlying explicit model of the system. Eachremote user is surrounded by an aura (similar in concept tothose proposed in 121) within which time evolvesnormally, yet is delayed by the extent of the delay, i.e.time evolves as rRj = f - Ani, where f is real time, rRi is theperceived time within the aura of Rj, and AKi is the delay.Here, all objects within the neighbourhood of the remoteuser are seen to evolve with the delayed time of the remoteuser. Different delays can be incorporated for differentusers and, importantly, the local user can interact with hislocal environment in real-time.It is important to note that this is merely a localinterface - the underlying explicit model of the system isunaffected, so there are no additional temporalinconsistencies. Any delaying of time within each of theseauras affects only the local users perception of events.

Clearly, having regions in which time evolves withdifferent offsets presents problems when objects movefrom one region to another. Discrete steps in the offset, forexample, from fRI = f - AR, to fRZ= f - An?, will result inobjects undergoing discontinuous jumps in position asthey move from one region to the next. This presents thepossibility of the objects either disappearing or appearingin duplicate for a short length of time, given by An,-AnI.What is required is a formal mapping which blendsregions with different time offsets in a continuous way.This mapping is provided by the 3/rD Perception Filter.4: The 3GD perception filter

Consider, as an illustrative example, four users, onelocal (L) and three remote (R,, RzS Rs), operating within atwo dimensional distributed virtual environment, oroperating in a plane within a 3D environment, see Fig. I.(This reduction to 2-D is solely used to aid thevisualization of the model, the extension of the model to3-D is straightforward.). Without any loss of generality,the spatial coordinates are assigned relative to the positionof the local user. No restriction is placed on the mobilityof any of the users, though all users must maintain aspatial separation (a minimum distance, independent ofposition or delay). The spatial separation constraint can bereduced in certain circumstances by limiting thebandwidth/dynamics of the interaction.

245


5/8

Figure 1. The 3/-Dposirions of f our s er sz . 0).The communications delays for each remote user, AK,,

An?, and An, respectively, may be plotted against thespatial position of each user in the virtual world, in a thirddimension, placing remote users at the relative position(x,, )i, Ani). As with the spatial coordinates, the delay axiscan be viewed as a relative time axis - thus the local userwill always be at their current time and AL = r, - f = 0.

This representation is termed the 3&D PerceptionModel as it gives a viewer centered representation of thevirtual environment, in that the local user has explicitinformation about the interactions of each remote user upto time f - Ani, but cannot gain any information on anyremote user interactions within the interval (f - ARir 11.

It would be trivial to encompass each remote user in acircle (or sphere for 3-D) where the value of the delay isthe same within each circle. However, the delay betweeneach discrete point should also be smoothly interpolated.User requirements and system constraints will indicate theform of this interpolation, the most relevant of these are:I.2.3.4.

5.

The local user must be able to interact in real-time inthe neighbourhood of the local user;The local user must similarly be able to view remoteuser/object interactions in real-time, albeit delayed;Objects must traverse the environment smoothly;Object-object interaction must appear realistic andconsistent; consequently, they must occur at the sameplace at same time;Objects moving from one US~T o another must appearto leave and arrive at the respective users smoothly.Given two users, L and R, separated by a distance, D,

and a delay, A, then it is clear from point 5 above that anobject moving from R to L at an actual speed of Y = D/A,will appear to move instantaneously. Objects approachingthis critical velocity will appear to move at very highvelocities. Thus, object radial velocities should be limitedto a proportion of this critical velocity. This will, ofcourse, apply to each user, leading to a global constrainton the maximum velocity based on the lowest ratio ofD/Ai. This critical velocity is a fundamental constraint ofdistributed interactive simulations and not specific to thediscussion presented in this paper.

From the specifications above it is clear that the delayassociated with every point in space must together form acontinuously differentiable contour. Blending two regionsat different delays implies that an object moving at aconstant velocity between those regions will be perceivedto accelerate (from a region of greater delay to a region oflesser delay) or decelerate. This acceleration must beminimized-the contour should be as flat as possible.

A contour which satisfies these constraints can beenfound using radial basis type functions of the form

2A = S(x,y,z) =AR, e* (1)

where A is the value of the delay at any point in space, Aniis the value of the delay at the remote user location. 5 isthe radial distance from remote user i, and vi is thevariance of the function for that user. The delay in theneighbourhood of the remote user is reasonably constantand the effect of the delay some distance away from theuser is negligible. This is confirmed by considering therate of change of the delay, representing the perceivedacceleration of objects with constant velocities, given by

(2)This function approaches zero when r, + 0 and r, + =

and the larger vi is, the flatter the surface will be. Formultiple users the delay at each point in the plane issimply as a summation of the functions (I), as

S(x,y,z)=@(rL)&ARi ,g (3)i=,The function Q(rL) is a prescalar function, based on

the radial distance from L, and is used to force to zero thedelay in the neighbourhood of the local user. It graduatessmoothly from 0 at rL = 0, to a maximum value of I, atand beyond a predefined distance. To improve robustnessin the face of variable delays a gain can be applied in Eqs(l)-(3) to ensure that the value of S(x,y,z) at each remoteuser includes a margin of error, e.g. 150% of Ani.

It is clear that the delay from one usa will add to thedelay at another users location if two users are close toeach other (this will not be a problem if the contribution issmall). The only necessary condition on the contour is thatthe value of the delay at each remote user site is af leastthat of the delay at that site. Calculation of the variance foreach user, vi, can reduce the contribution to other userdelays. Heuristically, the variance can be maximized whilerestricting the contribution of any one users delay to itsnearest neighbour to, say 5%.

As a final modification, a plateau may he introducedaround each remote user, ensuring that remote userinteractions in a wider neighbourhood of that user areperceived by the local user to be as close to real-time as

246


6/8

possible, albeit at a given delay, see Fig. 2. An approximation to (6) is given by

* .,,.~ ,,., a~= ~_ -.,*- I. _ 1,.o.,1Figure 2. The 3 -D ca usa l confour ma p and (r igh t ) thecm four incorpora t ing p lareaus of cm umnt de lay. User spo sit ions are not orde red in any tense by their d elay s.

The resulting contour need not be calculated in itsentirety, but only at specific points occupied by objectsand users. It is also dynamic in that it changes as usersmove. The contour represents a surface , S, boundingcausal volumes within space-time defining when events byone user can causally effect events at any other point inspace [ 121. The contour can also be viewed as a surface onwhich events are perceived to occur simultaneously.5: Implementation

To implement the Perception Filter, consider a 3-Denvironment with user, one local and one remote. and asingle moving point object, 0. Let the exact motion of 0,x,(t), be given by the kth order polynomial

X,,(T) =a, +a,f+a,r* + +a/ (4)where the vectors ai represent the [x, y, z] coefficients. If0 approaches the remote user. then the local user willneed to delay their view of the object to ensure that anypotential in te ract ion with the object by a remote user canbe viewed in real-time. From (4), the position of the objectat any time can be calculated. From (3), the value of thedelay can be calculated at all positions the object may beat. The local visual&rim of th object, r(f), is

For this function, the delay is represented by[-,/ 2 )A=S=+,)xA, e 6)

where the radial distances are rL = pxl, and I = IDR - ILxII,and D, is the distance from the local user to remote userR. It is evident that this produces a highly non-linearequation in %. However, the scheme may be implementedin a recursive way, by noting that the contour representingthe delay is smooth, reasonably flat, and continuous. Thus,where objects are moving at velocities much less than thecritical velocity, the difference in the delay from onesample to the next will be small (and negligible ifprocessed using multi-sample rates). Also, most CVEs useapproximations, whence the constants a, = 0, for i 2 2 fordead-reckoned paths (or i 2 3 for inertial approximations).

AI =Q(r,)xA, (7)where k is the current and k-l the previous samplesrespectively. Some general observations may be inferredfrom equations (4)~(5)- close to the local user, A = 0, and x(t) =x(t);- close to the remote user, A = An. and %(t) = x(t-A,);- static objects are unaffected, L~(t) = x(t) = a&5.1: Objects with constant velocity

Consider an object moving at constant velocity, x(t) =a0 + a,& and .x(t) = a0 + a,(t-A), where, assuming 0 = 1,

L*(t) = a,, +0, , _ AR c~-~~~-L~Jixi I

8)Differentiating (8) yields a closed form differentialequation, which may be solved explicitly for the perceivedvelocitv. as

~1,. for ID, -x(r)1 >> - 46 anda,. for[D,-Lx(r)l+O

Thus the perceived velocity is not constant, beingexplicitly dependent on position in space, though it tendsto the actual value, a,, when the object is close to, or faraway from the remote user. Differentiating again, a similarclosed form expression may be found for the perceivedacceleration, which is also position dependent. However,both perceived velocity and acceleration deform smoothly.Deviations from the true velocity only become apparentwith large delays and/or large velocities (i.e. velocitieswhich approach the critical velocity).6: Example

A feature of the filter for systems with small delays (ox,(f)= (),


8/8

Documents

Sharkey Et Al. - 1998 - A Local Perception Filter for Distributed Virtual