6
Smooth Object State Updates in Distributed Haptic Virtual Environments Clemens Schuwerk and Eckehard Steinbach Institute for Media Technology Technische Universit¨ at M¨ unchen, Germany Email: {clemens.schuwerk, eckehard.steinbach}@tum.de Abstract—In a multi-user distributed haptic virtual environ- ment (DHVE), object state update messages are necessary to maintain a consistent virtual environment at every entity involved. Traffic control schemes applied to reduce the usually high number of update packets lead to inconsistencies among the participants. Whenever a new state update is received, the discrepancy between the old and new object state has to be resolved. For this purpose, various interpolation methods can be used to smoothly correct the state error. In this paper, we investigate and compare three such methods using subjective experiments to determine a haptically preferred method that leads to a convincing force feedback. Besides basic linear interpolation, we test two other methods from literature which are based on cubic polynomials. Our exper- imental evaluation, based on a pursuit-tracking task and pairwise comparisons, shows that linear interpolation performs better than the two cubic approaches and is subjectively preferred. I. I NTRODUCTION Enhancing human-machine or human-human interaction mediated by technical systems by incorporating the haptic modality is a growing research area. As humans rely heavily on their kinesthetic and tactile perception during interaction with their environment, the realism of interaction via technical sys- tems is increased substantially if haptic feedback is provided to the user [1]. Additionally, the sense of immersion into distant real or virtual environments is improved [2]. Also, it has been shown that the integration of the haptic modality into technical systems leads to an increased task performance [3]. Advances in existing communication infrastructure like the Internet enable physical (haptic) interaction or manipulation of objects in distant real or virtual environments. The trans- mission of haptic information over today’s communication networks, however, faces several challenges [4]. Firstly, the high temporal resolution of the human haptic perception sys- tem requires a stimulus refresh rate of 1kHz, resulting in the need for high-rate information exchange on the communication channel. Additionally, the communication network closes a control loop between the local human and the remote entity being controlled, e.g. a remote robot or a virtual avatar in a remote virtual environment. The stability of this control loop necessities low-delay communication of haptic data samples. To address these issues in telepresence and teleaction (TPTA) systems, lossy perceptual haptic data reduction schemes have been introduced [5]. Similar challenges also apply for so-called Shared Haptic Virtual Environments (SHVEs), wherein several users, possibly spread around the globe, jointly manipulate objects within a shared virtual world. Different communication architectures based on either the client-server or the peer-to-peer paradigms can be conceived for building SHVEs. As more than one user is involved in the interaction, also scalability and consistency issues arise. The haptic rendering algorithms need to run at a minimum rate of 1kHz to display high-fidelity force feedback to the user. To allow true collaboration between users, the virtual scene perceived by all of them should be consistent. Therefore, low-delay and high-rate information exchange between collaborating users is necessary causing the communication network to quickly become a bottleneck. Contrary to a common real world TPTA system, where contact forces with the remote environment are measured by a force sensor and sent back to the operator, in virtual applications, parts of the rendering pipeline can be distributed and calculated either on a centralized server or locally at the user. A commonly used architecture, proposed by Hikichi et al. in 2001 [6], is shown in Fig. 1a and is referred to as the Consistency Client-Server Architecture throughout this paper. Here, the centralized server is the only entity in the system running a physics engine which simulates the current object state (position, velocity, rotation and rotational velocity) at a rate of 1kHz. This state is multicasted to the clients, where the local object state database is updated and interaction forces are rendered locally. By shifting the force rendering to the clients, the transmission of force samples is avoided and stable local force rendering is achieved. However, perceptual transparency is lost with increasing delay [7]. In the so-called Fully-Distributed P2P Architecture shown in Figure 1b, a local physics engine is running at each par- ticipating peer. Users only exchange their current device state which is then used as input to the local physics simulations. Obviously, the locally calculated object states will diverge in presence of communication delay, as the physics simulations are all based on delayed information. Thus, consistency control mechanisms are needed to correct this inconsistency. Both architectures have drawbacks in terms of either de- lay, scalability or consistency. Additionally, the load on the communication network increases with the number of partici- pating users. Traffic control mechanisms, shortly reviewed in Section II, are applied to reduce the communication load and to avoid congestion in the network. For the Consistency Client- Server Architecture, such traffic control schemes lead to fewer object state updates sent from the server to the clients. For the P2P architecture, inconsistency resolution mechanisms need to resolve discrepancies between the currently rendered object state and the corrected state. If the local object state database is immediately updated, sudden jumps in rendered force feedback 978-1-4799-0849-3/13/$31.00 ©2013 IEEE

[IEEE 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE) - Istanbul, Turkey (2013.10.26-2013.10.27)] 2013 IEEE International Symposium on Haptic

Embed Size (px)

Citation preview

Page 1: [IEEE 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE) - Istanbul, Turkey (2013.10.26-2013.10.27)] 2013 IEEE International Symposium on Haptic

Smooth Object State Updates in Distributed HapticVirtual Environments

Clemens Schuwerk and Eckehard SteinbachInstitute for Media Technology

Technische Universitat Munchen, GermanyEmail: {clemens.schuwerk, eckehard.steinbach}@tum.de

Abstract—In a multi-user distributed haptic virtual environ-ment (DHVE), object state update messages are necessary tomaintain a consistent virtual environment at every entity involved.Traffic control schemes applied to reduce the usually high numberof update packets lead to inconsistencies among the participants.Whenever a new state update is received, the discrepancy betweenthe old and new object state has to be resolved. For this purpose,various interpolation methods can be used to smoothly correct thestate error. In this paper, we investigate and compare three suchmethods using subjective experiments to determine a hapticallypreferred method that leads to a convincing force feedback.Besides basic linear interpolation, we test two other methodsfrom literature which are based on cubic polynomials. Our exper-imental evaluation, based on a pursuit-tracking task and pairwisecomparisons, shows that linear interpolation performs better thanthe two cubic approaches and is subjectively preferred.

I. INTRODUCTION

Enhancing human-machine or human-human interactionmediated by technical systems by incorporating the hapticmodality is a growing research area. As humans rely heavily ontheir kinesthetic and tactile perception during interaction withtheir environment, the realism of interaction via technical sys-tems is increased substantially if haptic feedback is provided tothe user [1]. Additionally, the sense of immersion into distantreal or virtual environments is improved [2]. Also, it has beenshown that the integration of the haptic modality into technicalsystems leads to an increased task performance [3].

Advances in existing communication infrastructure like theInternet enable physical (haptic) interaction or manipulationof objects in distant real or virtual environments. The trans-mission of haptic information over today’s communicationnetworks, however, faces several challenges [4]. Firstly, thehigh temporal resolution of the human haptic perception sys-tem requires a stimulus refresh rate of 1kHz, resulting in theneed for high-rate information exchange on the communicationchannel. Additionally, the communication network closes acontrol loop between the local human and the remote entitybeing controlled, e.g. a remote robot or a virtual avatar in aremote virtual environment. The stability of this control loopnecessities low-delay communication of haptic data samples.To address these issues in telepresence and teleaction (TPTA)systems, lossy perceptual haptic data reduction schemes havebeen introduced [5].

Similar challenges also apply for so-called Shared HapticVirtual Environments (SHVEs), wherein several users, possiblyspread around the globe, jointly manipulate objects within ashared virtual world. Different communication architectures

based on either the client-server or the peer-to-peer paradigmscan be conceived for building SHVEs. As more than one useris involved in the interaction, also scalability and consistencyissues arise. The haptic rendering algorithms need to runat a minimum rate of 1kHz to display high-fidelity forcefeedback to the user. To allow true collaboration betweenusers, the virtual scene perceived by all of them shouldbe consistent. Therefore, low-delay and high-rate informationexchange between collaborating users is necessary causing thecommunication network to quickly become a bottleneck.

Contrary to a common real world TPTA system, wherecontact forces with the remote environment are measuredby a force sensor and sent back to the operator, in virtualapplications, parts of the rendering pipeline can be distributedand calculated either on a centralized server or locally at theuser. A commonly used architecture, proposed by Hikichi etal. in 2001 [6], is shown in Fig. 1a and is referred to as theConsistency Client-Server Architecture throughout this paper.Here, the centralized server is the only entity in the systemrunning a physics engine which simulates the current objectstate (position, velocity, rotation and rotational velocity) at arate of 1kHz. This state is multicasted to the clients, where thelocal object state database is updated and interaction forces arerendered locally. By shifting the force rendering to the clients,the transmission of force samples is avoided and stable localforce rendering is achieved. However, perceptual transparencyis lost with increasing delay [7].

In the so-called Fully-Distributed P2P Architecture shownin Figure 1b, a local physics engine is running at each par-ticipating peer. Users only exchange their current device statewhich is then used as input to the local physics simulations.Obviously, the locally calculated object states will diverge inpresence of communication delay, as the physics simulationsare all based on delayed information. Thus, consistency controlmechanisms are needed to correct this inconsistency.

Both architectures have drawbacks in terms of either de-lay, scalability or consistency. Additionally, the load on thecommunication network increases with the number of partici-pating users. Traffic control mechanisms, shortly reviewed inSection II, are applied to reduce the communication load andto avoid congestion in the network. For the Consistency Client-Server Architecture, such traffic control schemes lead to fewerobject state updates sent from the server to the clients. For theP2P architecture, inconsistency resolution mechanisms needto resolve discrepancies between the currently rendered objectstate and the corrected state. If the local object state database isimmediately updated, sudden jumps in rendered force feedback

978-1-4799-0849-3/13/$31.00 ©2013 IEEE

Page 2: [IEEE 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE) - Istanbul, Turkey (2013.10.26-2013.10.27)] 2013 IEEE International Symposium on Haptic

User 2

Haptic Device

Collision Detection

Force Rendering

Physics Engine

Object Database

F

p1, v1

-F

S

S

Screen

1000 Hz

Communication Network

di

Consistency Server

User 1

Object Database

Collision Detection

Force Rendering

S Visual Rendering

S

F11000 Hz

(a) (b)

Legend:pi = position of user i di = penetration depth of user ivi = velocity of user i Fi = force for user iS = object state Si = object state at client iF = ∑(Fi)

1000 Hz

Haptic Device

F

Screen

User 1

Object Database

Collision Detection

Force Rendering

S1 Visual Rendering

F1-FPhysics Engine

S1

1000 Hz

Haptic Device

F

p2, v2

p1, v1

Screen

Communication Network

User 2

Object Database

Collision Detection

Force Rendering

S2 Visual Rendering

F2

To Object Database

-FPhysics Engine

S2

Fig. 1. Distributed architectures for SHVEs: (a) Consistency Client-Server Architecture with centralized physics engine and local force rendering (adopted forour experiments). (b) fully-distributed P2P architecture with physics engine running at every peer.

will occur. Such jumps can be avoided by using appropriateinterpolation methods to smoothly converge from the old tothe new state. Different methods, based on linear interpo-lation or higher-order polynomials [8], [9], [10], have beenproposed for distributed applications without haptic feedback.For DHVEs, there is a lack of knowledge about which methodsshould be applied to achieve perceptually convincing forcefeedback. Thus, this paper investigates and compares differentinterpolations techniques, known from computer graphics andnetworked games in a distributed haptic virtual environment.The Consistency Client-Server Architecture depicted in Fig. 1ais deployed for this purpose. Subjective experiments are con-ducted to determine the method that leads to the perceptuallymost convincing force feedback. Thus, the comparison is basedsolely on user preference, not considering other factors as, e.g.,computational complexity.

The remainder of this paper is structured as follows.Relevant traffic control schemes are reviewed in Section II,followed by an introduction of the three investigated stateinterpolation methods in Section III. The experimental design,the analyses as well as the gathered results are described inSections IV-VI. Finally, in Section VII we summarize ourresults and give an outlook on future work.

II. TRAFFIC CONTROL IN SHARED HAPTIC VES

The concept of dead reckoning (DR), supported by theIEEE Standard for Distributed Simulations [8], is a popu-lar technique used in distributed simulations to reduce thenumber of transmitted state updates. It is based on localprediction of object states using the last received update. Theentity issuing the state updates, e.g. the centralized serverin the afore-mentioned client-server architecture, implementsthe same prediction method and triggers new updates onlyif the error between the predicted and the current real stateexceeds certain thresholds. Here, the globally valid real objectstate is calculated by the physics engine on the centralizedserver. Differences in position and rotation of the objectsare compared to the corresponding thresholds and updatesare triggered accordingly. Both thresholds have to be tunedaccording to the simulation’s characteristics to find a trade-offbetween the fidelity of the rendered objects and packet ratereduction.

In virtual environments enhanced with the haptic modality,the choice of these thresholds heavily influences the fidelityof the force feedback displayed to the human. If a user is in

contact with an object, the displayed force is rendered locally,commonly based on Hooke’s Law F = k ∗ d, where k is therendered stiffness and d the device’s penetration depth into theobject [11]. Any object state update leads to a sudden changein rendered force, as the penetration depth changes abruptlydue to the new object position. A dead reckoning approachfor DHVEs which takes the perceptibility of the resultingforce changes into account and adjusts the error thresholdaccordingly is presented in [12]. According to Weber’s law,the just noticeable difference (JND) in a comparison of twoforce magnitudes is proportional to the magnitude of the firststimulus [13]. By keeping the force changes caused by suddenobject state updates below the human perception limits, aperceptually transparent force rendering at the clients can beachieved.

An interpolation scheme might be used at the clients tosmoothly correct the state of an object, rather than updating itabruptly. Approaches based on first-order [8] or higher-orderpolynomials [9] are used in the literature for VEs withouthaptic feedback. There is, however, no well-accepted modelfor a perceptually convincing method, even for visual-onlyapplications [10]. By smoothly correcting state inconsistenciesin DHVEs, higher error thresholds can be chosen, leading to ahigher packet rate reduction, while satisfactory force feedbackquality is still maintained. In [6], the packet rate on thechannel from the centralized consistency server to the clientsis adjusted according to the current network load to preventcongestion. Missing updates are absorbed by predicting andinterpolating the objects’ positions and rotations. The usedinterpolation techniques, however, are not described.

III. OBJECT STATE INTERPOLATION

The three interpolation techniques used in this paper tosmoothly correct the inconsistency after a state update isreceived at the clients are briefly introduced in the following.The given formulas correspond to the interpolation of anobject’s position. Similarly, they can be used with Euler anglesto correct rotation errors [8], [9]. Let S∗ = {p∗, v∗} denote anobject state, where p∗ ∈ <3 is the position and v∗ ∈ <3 thevelocity in global coordinates. The current object state at timet = t0 when a new update is received at a client is denotedby S0, and the newly received object state by S1. The timeused for the convergence from the old to the new position isdenoted as tn and is pre-decided before the interpolation starts.With this, i(t) = (t − t0)/tn where t0 ≤ t ≤ t0 + tn can

Page 3: [IEEE 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE) - Istanbul, Turkey (2013.10.26-2013.10.27)] 2013 IEEE International Symposium on Haptic

be defined. Assuming a linear prediction at the client-side, thetarget position p1 can be calculated with S1 as p1 = p1+v1tn.

A. Linear interpolation

The most straightforward interpolation between the currentposition p0 and the target position p1 is the linear interpola-tion [8], which is given for the object position p(t) at time tas follows:

p(t) = p0 + (p1 − p0)i(t)

= p0(1− i(t)) + p1i(t) + v1tni(t)(1)

Obviously linear interpolation leads to abrupt changes inthe object velocity at the times t = t0 and t = t0+tn. Thus theobject’s trajectory in the virtual world is not C1-continuous.

B. Hermite interpolation

Alternatively, the trajectory can be smoothed with a hermitecurve segment, which takes also the old and new velocity,v0 respectively v1, into account. Thereby C1 continuity ismaintained. With the common hermite curve segment [14] andthe predicted target position p1, the object position p(t) isgiven as:

p(t) = p0(2i(t)3 − 3i(t)2 + 1) + p1(−2i(t)3 + 3i(t)2)

+ v0tn(i(t)3 − 2i(t)2 + i(t))

+ v1tn(−i(t)3 + 2i(t)2)

(2)

The drawback of hermite interpolation is the tendencyto show an oscillatory behaviour [9]. Although the objecttrajectory is smooth, higher acceleration is needed duringinterpolation to correct the inconsistency within the giveninterpolation time tn.

C. Projective velocity blending

The so-called projective velocity blending for dead reckon-ing in virtual environments proposed in [9] uses two combinedlinear interpolations. According to the authors, this methodleads to a visually convincing interpolation in networkedgames that tends to show less oscillations than the hermiteinterpolation. The resulting curve segment, following the equa-tions given in [9], can be denoted as follows:

p(t) = p0(1− i(t)) + p1i(t)

+ v0tn(i(t)3 − 2i(t)2 + i(t))

+ v1tn(−i(t)3 + 2i(t)2)

(3)

The first part of Eq. 3 consists of a linear interpolationbetween the old and new object position, equal to the first twoterms in Eq. 1. The prediction is incorporated into the lattertwo terms as in Eq. 2. Obviously, it is a mixture of the twoafore-mentioned approaches and C1-continuity is not achieved.However, the velocity used for extrapolation is blended fromv0 to v1 compared to Eq. 1 where only v1 is used in theextrapolation term.

Haptic Interface Point (HIP)

start pointend point

velocity

user cubereference cube

Fig. 2. Pursuit tracking task with user cube (pink, haptically active) andreference cube (green, only visual).

In the following we describe two experiments conducted toevaluate the impact of the interpolation scheme on the hapticfeedback quality.

IV. EXPERIMENTAL SETUP

Both subjective experiments are based on the same appa-ratus and task described in the following.

A. Apparatus

The PHANToM Omni haptic device (Sensable Technolo-gies Corp, Woburn, MA) is used for the experiments, togetherwith Chai3D [15], an open source library for haptic render-ing. As the target of both experiments is to investigate theperceptual quality of the haptic feedback for the differentinterpolation techniques, only a single user is connected tothe server and his haptic device position is transmitted to theserver at the full rate of 1kHz. On the return channel fromthe server to the client, the traffic control scheme proposedin [12] is used to reduce the number of object state updates.In both experiments, a new state update is only triggered if theexpected force change due to a sudden object position changeexceeds 25% of the current force displayed to the user. Thisparameter was determined in pilot tests, where we found it tobe a reasonable trade-off between operability and perceptionof artefacts. According to the known human perceptual limi-tations, this force change is easily perceivable [13]. However,due to the applied object state interpolation, the abrupt jumpin object position is smoothed out before being displayed tothe user. For the task described below and the applied trafficcontrol scheme, only 4.1 packets per second on average areneeded from the server to the client. Additionally, the serverand the client are running on the same machine to avoid anyeffect of delay in the communication between them, as theimpact of delay is not the subject of this study. During theexperiments, the subjects wore headphones with active noisecancellation to prevent any auditory feedback from the hapticdevice or distractions from the environment.

B. Task description

It is important to have a simple to accomplish task whichavoids as much variability between runs and subjects as possi-ble to compare artefacts introduced by different interpolationschemes. Therefore we choose a pursuit tracking task, whereinthe subject has to push a cube from a start location to an endlocation following a reference cube as closely as possible (seeFigure 2). The reference cube’s movement is experimentallycontrolled to show constant acceleration in the first half of thetrajectory and constant deceleration afterwards. Additionally,the movement of both the device and user cube is limited

Page 4: [IEEE 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE) - Istanbul, Turkey (2013.10.26-2013.10.27)] 2013 IEEE International Symposium on Haptic

TABLE I. USED RATING SCHEME

5 4 3 2 1no difference perceptible, slightly disturbing strongly

but not disturbing disturbing disturbing

to one degree-of-freedom. This simplifies task execution andsubjects can pay more attention to the perception of arte-facts. Without this constraint task and input behaviour thecomparison between different parameter settings is difficult assubjective ratings might be influenced by individual interactionpatterns of the subjects.

V. INTERPOLATION INTERVAL EXPERIMENT

The interpolation interval experiment evaluates the timeparameter tn that is used for the interpolation. The positioninconsistency can be corrected more smoothly with largertn. However, the interpolation should be finished before thenext state update arrives. The inter-arrival time between twoconsecutive updates is not constant as the server applies trafficcontrol which non-uniformly triggers new update packets. Incase of an update arriving during interpolation, we immediatelystop the current interpolation and apply the newly receivedstate. Otherwise the client falls behind the intended movementand the inconsistency between the server and the clientsincreases.

A. Procedure

We test a range of interpolation times from 10ms to 250msin our experiment. Additionally, we test an adaptive method,wherein tn is chosen based on the latest packet arrival times.For this, we use the mean of the last five packet inter-arrivaltimes and, to be more conservative, choose 75% of this meanas the new tn. Moreover, whenever an interpolation still needsto be aborted due to the arrival of a new update, our heuristichalves tn.

The linear interpolation is fixed for all subjects as weonly want to study the impact of the interpolation time inthis first experiment. We use a within-subjects experimentaldesign with all subjects performing the same task under thesame parameter settings. Each setting is tested in a singlerun, which is presented randomly. The subjects are asked torate their perception of force feedback on the rating scaleshown in Table I. For this purpose we conduct a trainingphase to familiarize the subjects with the task, the device,the experimental procedure and two references correspondingto a rating of 5 and 1. One reference is a run without anytraffic control scheme applied and, thus, 1000 packets persecond transmitted to the client. The second reference has thesame traffic control scheme enabled as all the other runs, butno interpolation is used to smoothly correct the object state.During the experiment, the subjects are allowed to performboth reference runs whenever they desire to calibrate theirratings.

B. Participants

Fourteen subjects participated in the experiment, elevenmale and three female with an age between 24 and 30. Fivesubjects are daily users of haptic devices. Four subjects have

0ms 10ms 50ms 100ms 150ms 200ms 250ms Adaptive Ref.1

2

3

4

5

Mea

n O

pini

on S

core

(MO

S)

00.10.20.30.40.50.60.70.80.91

Frac

tion

of a

borte

d sm

ooth

ings

Interpolation time tn

Fig. 3. User ratings (green), their standard deviation (black) and fraction ofaborted interpolations (blue) for the interpolation interval experiment.

participated in earlier subjective experiments in this researcharea and were familiar with the device. The remaining fivesubjects are novice users and although a training phase wasconducted with all of the subjects prior to the experiment,they had a more extensive training.

C. Results and analysis

The results of the interpolation interval experiment areshown in Fig. 3. As expected, the perceived force feedbackquality improves with increasing interpolation time tn. A bigincrease to a MOS of 3.5 can be observed for tn = 50ms.From tn = 50ms to tn = 150ms the MOS remains nearlyconstant and increases again for higher tn. The adaptivemethod performs best compared to all fixed interpolation timeswith a MOS of 4.5. In general, the standard deviation in theuser ratings is high. This lies in the nature of such subjectiveexperiments, but might also come from the difficulty to adjustthe rating scale consistently between subjects in the trainingphase.

Statistical significance of the differences in user ratingsfor the different interpolation times is evaluated with a non-parametric Kruskal-Wallis test as normal distribution in thepopulation could not be assumed due to the small samplesize (χ2(8) = 81.42, p < .001)). Tukey’s honestly significantdifference (HSD) test indicated that all differences compared tothe reference without interpolation (tn = 0ms) are significant(p < .05), except for tn = 10ms. For tn = 200ms,tn = 250ms and the adaptive method there is no significantdifference to the 1000 packets/s reference, however there is adifference compared to tn = 10ms. No significant differencewas found between tn = 50ms and the adaptive method.

The fraction of aborted interpolations increases linearlyfrom around 10% at tn = 10ms to nearly 40% at tn =250ms. The adaptive method reduces this fraction as expectedand leads to 20%, which is about the same fraction as fortn = 50ms. The mean interpolation time used in the adaptivemethod is tn = 148ms. This clearly shows the usefulness ofthe proposed heuristic to choose the interpolation time, as ahigh tn with a low interpolation abortion ratio is achieved.

From these results we conclude that for our specific setting(task and traffic control) we achieve a substantial improvementin force feedback quality if we use at least 50ms as inter-polation time tn. On average, the adaptive method is ratedhigher while keeping the fraction of aborted interpolations ata similar level. The adaptive method performs best compared

Page 5: [IEEE 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE) - Istanbul, Turkey (2013.10.26-2013.10.27)] 2013 IEEE International Symposium on Haptic

1−1 2−2 3−3 1−2 1−3 2−3 1−4 2−4 3−4 1−0 2−0 3−00

102030405060708090

100

Comparison (A−B)

Giv

en A

nsw

ers

(%)

A preferedB preferedNo difference

Fig. 4. User preference in the preference experiment (0: no interpolation, 1:linear interpolation, 2: cubic hermite, 3: velocity blending, 4: reference).

to the reference case and, thus, will be used in the secondexperiment described in the following.

VI. PREFERENCE EXPERIMENT

The preference experiment uses several pairwise compar-isons of the methods described in Section III to determinea haptically preferred approach, following the methodologydescribed in [16].

A. Procedure

Two runs (denoted as run A and run B) as explained inSection IV-B are presented sequentially in each comparison.Afterwards, the subject is asked to select his preferred run.Additionally, the subject has the option to state that there isno perceived difference between both runs. The comparisonsinclude all possible combinations between the three interpo-lation methods, as well as comparisons to the two referencesexplained in Section V-A. Additionally, there are comparisonswith both runs A and B set to be identical to determine anidentity norm. Each comparison is evaluated twice, leading toan overall number of 24 comparisons per subject. The orderof comparisons and runs within a comparison is selected ran-domly to avoid any systematic error in the results. Repetition ofruns within a comparison or the repetition of the comparisonitself on behalf of the subject is not allowed. Following theresults of the interpolation interval experiment, the adaptivemethod is used to decide on the interpolation time tn.

B. Participants

Twelve subjects participated in the second experiment. Twoof them are novice users of haptic devices. Ten subjects alreadyparticipated in the interpolation interval experiment, whichwas conducted several weeks before, and thus, were familiarwith the device and the task. Nevertheless, a training sessionwas conducted with every subject before the actual experimentstarted. One subject was excluded from the later analyses asthe two repeated comparisons were judged differently by morethan 50%, indicating random answers.

C. Results and analysis

The user preference for all comparisons is shown in Fig. 4.The first three bars show the comparisons for the case whenthe runs A and B use the same interpolation mode. Thesubjects correctly choose the no difference answer only in

50% to 60% of the cases, showing the subjects’ difficulties ofcomparing two separate runs. Besides the uncertainty of everysubjective experiment, we think the following characteristiccould be a reason for this. Although we carefully built ourexperiment around a special and simplified task to reducethe variation between different experimental runs, a humanis not able to exactly repeat a run and to perfectly followthe reference cube. Occasionally, a subject realizes duringtask execution that the error between the user cube and thereference cube is growing too big and quickly adapts hismovement accordingly. Due to the nature of the applied trafficcontrol scheme, new state updates are only triggered if thelinear client-side prediction fails, meaning whenever suddenaccelerations in movement occur. New updates lead to newinterpolations on the client-side and as a result perceivedartefacts in the force feedback displayed to the user. These firstthree comparisons are explicitly included in the experiment tocounteract this phenomena. We use them as identity norms assuggested in [16] and investigate if other comparisons differfrom them by applying Pearson’s chi-squared test. With thiswe can conclude if any interpolation mode performs differentlycompared to the corresponding norm. We further investigateif there is a significant preference by applying the statisticalmethod described in [16].

The comparisons 1-2, 1-3 and 2-3 shown in Fig. 4 arerespective direct pairwise comparisons between linear inter-polation (1), cubic hermite interpolation (2) and velocityblending (3). Linear interpolation seems to be clearly preferredcompared to the other two interpolation modes. Comparedto cubic hermite interpolation (1-2), it is preferred in 68%of the comparisons, while in 9% the cubic approach waspreferred and in 23% of the runs the subjects perceived nodifference (in short 68%:9%:23%). Pearson’s chi-squared testindicates a significant difference compared to the identitynorm 1-1 (χ2(2, N = 22) = 37, p < .001), indicatingthat method 1 and 2 do not perform equally. The follow upanalysis shows a significant preference for linear interpolation(χ2 = 10.89, p = .0008 with α = .05). Similar results arereported for the comparison to velocity blending (1-3) withsignificant difference to the identity norm (59%:14%:27%,χ2(2, N = 22) = 24.75, p < .001) and a preference for linearinterpolation (χ2 = 6.70, p = .009). The comparison betweencubic hermite interpolation and velocity blending (2-3) is sig-nificantly different from the identity norm 2-2 (32%:50%:18%,χ2(2, N = 22) = 15.73, p = .0003). However, the followup analysis shows no clear preference for one of these twocandidates (χ2 = 0.90, p = .34).

The comparisons of all methods to the two references areshown in the last six bars in Fig. 4. Clearly, the reference casewith 1000 packet/s is preferred in nearly all of the runs (see1-4, 2-4, 3-4 bars). Similarly, every interpolation scheme ispreferred to the reference without interpolation (compare 1-0, 2-0, 3-0). These results are not surprising, as we choose arelatively large perceptual threshold for the traffic control inour experiment as explained Section IV-A. It is the subject offuture work to adopt the preferred linear interpolation and totune the deadband parameter in the traffic control scheme tofind an optimal trade-off between packet rate reduction andsatisfying force feedback displayed to the user.

Page 6: [IEEE 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE) - Istanbul, Turkey (2013.10.26-2013.10.27)] 2013 IEEE International Symposium on Haptic

VII. CONCLUSION

This work investigates object state interpolation in dis-tributed haptic virtual environments. Two experiments, dealingwith the interpolation time and the interpolation method arepresented. The first experiment indicates that for the giventask, traffic control scheme and communication architecture,at least 50ms are needed for a significant improvement indisplayed haptic force feedback in comparison to suddenobject state updates. However, this time cannot be seen asa general boundary, as it heavily depends on the task andsystem parameters, for example the error thresholds in thetraffic control scheme. With more update packets sent to theclient, the object state error will obviously decrease. But, therewill be less time available for the interpolation as it should befinished before the next update arrives. The best result wasachieved with the proposed heuristic to adaptively determinethe interpolation time based on the latest packet inter-arrivaltimes.

The three different interpolation methods presented in Sec-tion III are compared to each other subjectively in the secondexperiment. The subjects show a clear preference for the linearinterpolation. We think the reason for this is the accelerationneeded in both cubic methods to correct the inconsistency.During linear interpolation there is a sudden change in velocityonly at the beginning and in the end of the interpolation.The other two schemes adapt more severely to the old (new)velocities at these time instances. Thus, they need to catchup during the interpolation itself. This acceleration was alsoreported after the experiment to be a driving factor in the sub-jective preference decisions. Although e.g. projective velocityblending might lead to a convincing visual object movementas stated in [9], in terms of displayed force feedback, theusers prefer a trajectory with less acceleration. These resultsdo not depend on the underlying communication architecturenor any consistency maintenance scheme, as in any case, theinterpolation needs to be performed locally and the goal isalways to smoothly converge from an old to a new objectstate. Thus, in summary, if inconsistency in a distributed hapticvirtual environment has to be corrected, we conclude that alinear interpolation is haptically preferred. The time used forthe interpolation should be chosen adaptively according to thelatest update arrivals to ensure a timely end of the interpolation.

Although the preference was evaluated with a simplifiedone degree-of-freedom task, we hypothesise that these insightsalso hold for more realistic three dimensional tasks. The cubicinterpolation methods tend to show an oscillatory behaviour,e.g., if the movement direction changes sharply. We assumethat the linear interpolation is haptically preferred as it out-performs cubic methods even in the evaluated most basic task,where we actually expected the more smooth cubic methodsto be superior. This hypothesis, however, needs to be validatedwith further experiments.

By using object state interpolation at the client we needto trigger fewer state updates in distributed haptic virtualenvironments, while still achieving convincing force feedback,by avoiding sudden jumps in the rendered force displayedto the human. We use the limitations of the human hapticperception to hide these force changes within the interpolationtime without explicitly knowing these limits. Their determina-tion is an interesting direction for future work. Similar to the

perception of increased weight of virtual objects due to delayin the communication network [7], a large reduction of objectstate updates in our adopted client-server architecture leads toa loss of transparency. Thus, the influence of the traffic controlscheme including state interpolation on perceived transparencyshould also be investigated in detail.

ACKNOWLEDGMENT

This work has been supported by the European ResearchCouncil under the European Unions Seventh Framework Pro-gramme (FP7/2007-2013) / ERC Grant agreement no. 258941.

REFERENCES

[1] M. Srinivasan and C. Basdogan, “Haptics in virtual environments:Taxonomy, research status, and challenges,” Computers & Graphics,vol. 21, no. 4, pp. 393 – 404, April 1997.

[2] C. Basdogan, C.-H. Ho, M. A. Srinivasan, and M. Slater, “An experi-mental study on the role of touch in shared virtual environments,” ACMTransactions on Computer-Human Interaction (TOCHI) - Special issueon human-computer interaction and collaborative virtual environments,vol. 7, pp. 443–460, December 2000.

[3] J. T. Dennerlein, D. B. Martin, and C. Hasser, “Force-feedback improvesperformance for steering and combined steering-targeting tasks,” inProceedings of the SIGCHI conference on Human factors in computingsystems, April 2000, pp. 423–429.

[4] E. Steinbach, S. Hirche, M. Ernst, F. Brandi, R. Chaudhari, J. Kammerl,and I. Vittorias, “Haptic communications,” Proceedings of the IEEE,vol. 100, no. 4, pp. 937–956, April 2012.

[5] P. Hinterseer, S. Hirche, S. Chaudhuri, E. Steinbach, and M. Buss,“Perception-based data reduction and transmission of haptic data intelepresence and teleaction systems,” IEEE Trans. on Signal Processing,vol. 56, no. 2, pp. 588–597, February 2008.

[6] K. Hikichi, H. Morino, I. Fukuda, S. Matsumoto, Y. Yasuda, I. Arimoto,M. Iijima, and K. Sezaki, “Architecture of haptics communicationsystem for adaptation to network environments,” in IEEE InternationalConference on Multimedia and Expo, ICME, August 2001, pp. 563–566.

[7] S. Lee and J. Kim, “Transparency analysis and delay compensationscheme for haptic-based networked virtual environments,” ComputerCommunications, vol. 32, no. 5, pp. 992–999, March 2009.

[8] IEEE Computer Society, “IEEE Standard for Distributed InteractiveSimulation–Application Protocols,” IEEE Std 1278.1-2012 (Revision ofIEEE Std 1278.1-1995), pp. 1 – 747, December 2012.

[9] C. Murphy, “Believable Dead Reckoning for Networked Games,” inGame Engine Gems 2, E. Lengyel, Ed. A K Peters/CRC Press, 2011,pp. 307–328.

[10] X. Zhang, T. E. Ward, and S. Mcloone, “An information-based dynamicextrapolation model for networked virtual environments,” ACM Trans-actions on Multimedia Computing, Communications, and Applications,vol. 8, no. 3, pp. 1–19, July 2012.

[11] C. Basdogan and M. A. Srinivasan, Haptic Rendering in Virtual Envi-ronments, Handbook of Virtual Environments: Design, implementation,and applications. Lawrence Erlbaum Associates Publishers, 2002.

[12] C. Schuwerk, R. Chaudhari, and E. Steinbach, “Perceptually robusttraffic control in distributed haptic virtual environments,” in Haptics:Perception, Devices, Mobility, and Communication, ser. Lecture Notesin Computer Science. Springer Berlin Heidelberg, June 2012, vol.7282, pp. 469–480.

[13] E. Weber, Die Lehre vom Tastsinn und Gemeingefuehl, auf Versuchegegruendet. Friedrich Vieweg und Sohn Verlag, 1978.

[14] D. Salomon, Curves and surfaces for computer graphics. SpringerNew York, 2006.

[15] F. Conti, F. Barbagli, D. Morris, and C. Sewell. (2005) Chai 3d:An open-source library for the rapid development of haptic scenes.[Online]. Available: http://www.chai3d.org

[16] D. M. Ennis and J. M. Ennis, “Accounting for no difference/preferenceresponses or ties in choice experiments,” Food Quality and Preference,vol. 23, no. 1, pp. 13–17, January 2012.