RealME: The Influence of Body and Hand Representations on ... › wp-content › uploads › 2018 › 06 › 2017_Jung_RealMe.pdfRealME: The Influence of Body and Hand Representations

RealME: The Influence of Body and Hand Representationson Body Ownership and Presence

Sungchul JungUniversity of Central Florida

4000 Central Florida BlvdOrlando, Florida, USA 32816

[email protected]

Christian SandorNara Institute of Science and Technology

8916-5 TakayamaIkoma, Nara, Japan 630-0192

[email protected]

Pamela J. WisniewskiUniversity of Central Florida


[email protected]

Charles E. HughesUniversity of Central Florida


[email protected]

Figure 1: Experimental setup: We provided two types of body reection through a virtual mirror while the participants lookedat a virtual hand and arm from a rst person perspective. (a) Visually personalized body reection where the clothes andshape are identical to those of the participant seen in (c). (b) Generic Avatar body representation. (d,e): Two levels of handrepresentation. (d) Fully modeled limb from shoulder to hand. (e) Arm removed resulting in disconnected hand.

ABSTRACTe study presented in this paper extends earlier research involvingbody continuity by investigating if the presence of real body cues(legs that look like and move like one’s own) alters one’s senseof immersion in a virtual environment. e main hypothesis isthat real body cues increase one’s sense of body ownership and

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor prot or commercial advantage and that copies bear this notice and the full citationon the rst page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permied. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specic permission and/or afee. Request permissions from [email protected] ’17, Brighton, United Kingdom© 2017 ACM. 978-1-4503-5486-8/17/10. . .$15.00DOI: 10.1145/3131277.3132186

spatial presence, even when those body parts are not essential tothe activity on which one is focused. To test this hypothesis, wedeveloped an experiment that uses a virtual human hand and armthat are directly observable but clearly synthetic, and a lower bodyseen through a virtual mirror, where the legs are sometimes visuallyaccurate and personalized, and other times accurate in movementbut not in appearance. e virtual right hand and arm are thefocus of our scenario; the lower body, only visible in the mirror, islargely irrelevant to the task, providing only perceptually contextualinformation. By looking at combinations of arm-hand continuity (2conditions), freedom or lack of it to move the hand (2 conditions),and realism or lack of it of the virtually reected lower body (2conditions), we are able to study the eects of each combination onthe perceptions of body ownership and presence, critical featuresin virtual environments involving a virtual surrogate.

3

SUI ’17, October 16–17, 2017, Brighton, United Kingdom S. Jung et al.

CCS CONCEPTS•Human-centered computing → User studies; Virtual real-ity; Mixed / augmented reality;

KEYWORDSUser Study; Virtual Reality, Virtual Body Ownership, Body Conti-nuity, Presence; Human Computer InteractionACM Reference format:Sungchul Jung, Christian Sandor, Pamela J. Wisniewski, and Charles E.Hughes. 2017. RealME: e Inuence of Body and Hand Representationson Body Ownership and Presence. In Proceedings of SUI ’17, Brighton, UnitedKingdom, October 16–17, 2017, 9 pages.DOI: 10.1145/3131277.3132186

1 INTRODUCTIONA person’s perception of their body, called body ownership [18],and their recognition of the surrounding environment, called (phys-ical or spatial) presence [15], are known to be factors that are criticalto one’s senses of identity and experience. ese concepts, initiallystudied in psychology and neuroscience, have been extensively in-vestigated by the Virtual Reality (VR) community, taking advantageof its ability to create realistic illusions, enabled by high resolutionhead mounted displays (HMDs) and accurate tracking technologies.Using VR technologies, virtual body ownership (the illusion thata virtual body is one’s own) and (spatial) presence (the illusionthat we are in a synthetic environment) can provide a strong senseof immersion by stimulating associations between physical andvirtual body parts. However, most published studies focus on theassociation of a physical body part to a visually virtual counterpartwithout considering person-specic visual features, even thoughour perception is closely related to visual stimulations in the humanbrain [28]. In fact, humans can notice realistic body features (color,texture, etc.) because our brain forms the connection based onexplicit and implicit memory associated with the actual body [7].Based on this insight, Jung and Hughes [10] began to measure theeect of real body cues for virtual body ownership.

In the study reported here we extend those earlier experiments toinvestigate the interplay between arm-hand continuity, freedom ofhand movement in the presence of a threat, and realism of a lowerbody reection (from middle torso to legs in a siing position) thatis personalized but not directly relevant to the user’s central focus.Our goal is to investigate how combinations of each of these factorsinuence the illusions of virtual body ownership (primary) andpresence (secondary).

Recently, researchers have shown that the visually unbroken con-nection of body parts from shoulder to hand, called body continuity,provides a supporting factor to elicit virtual body ownership [17].Earlier research [10] has also shown a tendency of personalizedvisual cues to elicit the psychological illusion of body continuitybetween a virtual hand and forearm corresponding to a user’s realbody. In our research, we investigate the eect of personalizedbody cues on body continuity, testing two levels of detail. Alsowe examine agency [27], which is the sensation of controlling thevirtual body, because the coordination of movement and visualperception, visuomotor, has been shown to be a signicant factorfor virtual body ownership [20].

In our experiment our primary focus is on virtual body owner-ship; however, we are also interested in spatial presence, the senseof ”being there” [23], since the sense of presence in a virtual en-vironment is closely related to one’s virtual body representation[22]. We designed our experiment to provide either a visually per-sonalized body cue or a generic avatar body cue, always seen asa reection of one’s lower body in the absence of articial tactilesensory stimulation.

To investigate the eect of a visually personalized body cue, weplaced a virtual mirror in front of the participants so they couldsee their lower body reection (See Figure 1 (a)). A virtual mirrorwas also used in previous research [10, 11]. ose studies showedthat seeing a reected avatar body from the rst person perspectivehelps to elicit a greater illusion of body ownership than if there isno visual representation. e study reported here builds on thoseprevious experiments by comparing the inuence of a personalizedvisual body cue versus that of a generic avatar body cue.

e virtual mirror was positioned so participants could observetheir reected lower body, mainly their legs, while performing aspecied task with a virtual hand. To prevent a bias from renderingartifacts as described in [10], we used the RGB pixel values andthe depth information from an RGBD camera to render the par-ticipant’s lower body. Because of the low resolution of our RGBDcamera, the reected image on the virtual mirror seemed relativelyfuzzy, but most participants easily recognized the personalizedbody rendering as their own body. While participants looked at thevirtual environment involving the mirror reection, we providedtwo levels of virtual arm/hand representation – fully rendered fromshoulder to hand, and arm removed resulting in a disconnectedhand (See Figure 1 (d) and (e)) with two types of motor action – amovement-enabled hand and a movement-disabled hand.

Each participant experienced one of two body reection typeswith both hand levels and both motor action conditions, so the totalcombinations of conditions experienced by a single participantwere four. We clearly asked each participant to occasionally lookat the body reection while they were performing the given task,which means that, except for the visual dierence, all conditionswere identical for all participants.

To our knowledge, there is no previous experiment that comparesthe relative eect of a visually personalized body cue to that of avirtual body cue on the illusions of ownership and presence. eresults of our experiment, which we will explain in detail in theanalysis section, provide statistical support for our hypothesis thata personalized body cue enhances the sense of body ownership andpresence more than does a generic one, even when the body partbeing displayed is irrelevant to the required task.

2 RELATEDWORKAs virtual reality technology evolves, researchers are beer ableto investigate conditions that support a human’s perception ofa virtual body in a computer generated environment. Existingresearch has shown that an avatar’s resemblance to human appear-ance, synchronous visuo-tactile cues, synchronous visuo-motorcues, positional congruence, and anatomical plausibility [12, 17]are all major factors for virtual body ownership illusion. In addi-tion, the existence of visually connected body parts, called body

4

RealME: The Influence of Body and Hand Representationson Body Ownership and Presence SUI ’17, October 16–17, 2017, Brighton, United Kingdom

continuity, has been shown to be a supportive factor for the virtualbody ownership illusion. While the virtual body ownership illusionrepresents perception regarding a synthetic body, presence indi-cates perception regarding existence in a virtual or remote space. Inthis section we will present an overview of some existing researchrelated to the illusions of virtual body ownership and presence.(Note: We will oen abbreviate the term virtual body ownershipillusion as VBOI and body continuity as BC.)

2.1 Virtual Body OwnershipBecause the hand is the most frequently used human part during theperformance of tasks, hand ownership has been studied widely inboth real world and virtual reality research. Botvinick and Coheninvestigated body ownership using a fake rubber hand [5]. Anextended version of the rubber hand experiment was studied invirtual reality by Ye and Steed [30]. Similar to the rubber handexperiment, Petkova et al. conducted a body ownership study usinga mannequin [18]. Argelaguet et al. conducted a study for virtualarm ownership to discover correlations between the human visualand motor sensory systems in a purely virtual environment [1].ey demonstrated that the morphologically realistic resemblanceof the virtual hand is a signicant factor for one’s sense of virtualhand ownership. Hoyet et al. [9] studied body ownership issuesusing unnatural hand shapes with a similar setup to that of [1].ey used a virtual hand with six-ngers and showed that thesix-nger hand still elicited body ownership despite the explicitstructural dierence from a user’s real hand. Recently, Lugrin et al.investigated virtual body ownership with anthropomorphic modelsthat included a robot avatar, a generic avatar and a human avatar,each appearing in a purely virtual environment [14].

To investigate the correlation between visual real body cues onthe visual illusion of a virtual hand, Jung and Hughes [10] studiedthe impact of a virtual mirror reection of a subject’s lower torso.at study suggested a tendency of a trunk-centered real body cueto increase virtual hand ownership. Using a virtual mirror in avirtual body ownership experiment is not a new idea, and is onethat has been addressed from a variety of research perspectives[3, 8, 11]. Using the mirror reection, González et al. observed arelation between motor actions and virtual body ownership thatsuggested a synchrony of the mirror-reected avatar with a partici-pant’s movement was an important factor in one’s sense of bodyownership [8]. Kilteni et al. studied the relationship of the appropri-ate appearance of context on virtual body ownership [11]. In theirstudy, participants played a drum with dierent costumes, seeingtheir appearance through mirror reection. eir study demon-strated the cognitive consequence of proper consistency betweenvisual appearance and task context.

2.2 Virtual Body Ownership and BodyContinuity

Body continuity refers to visually connected body parts, as in theconnection of a hand to its shoulder through a wrist and arm. Pérezet al. experimented with a fully represented hand but no arm toconnect it to the rest of the body [17]. e goal was to nd therelationship of body connectivity to virtual body ownership. eirresults suggest that body continuity is a supporting factor for the

illusion of virtual body ownership. To further investigate bodycontinuity, Tieri et al. studied various types of hands – full limb,wire-connected hand, removed wrist, and missing wrist replaced bya plexiglass hand to arm connection [25, 26]. ey demonstratedthat, while the full limb case elicited the strongest sense of bodyownership, even an articial wire connection between hand andforearm elicited an autonomic reaction, e.g., involuntary protectivemovement, as a virtual body ownership indicator. Also Blenke at al.studied body ownership in the context of face, hand and trunk, andargued that the multisensory signals in the space immediately sur-rounding our trunks is of particular relevance to self-consciousness[4].

2.3 PresencePresence indicates the sensation of behaving and feeling as if one isin the computer generated world [19]. Presence is categorized intothree classications: social presence – the sense of not only sharingspace but also sharing an experience with another entity [23], co-presence – the sense of being in a shared space with another entity[2], and physical presence or spatial presence – the perception ofexisting in another space [15]. In this paper, we focus on spatialpresence because our study investigates the eect of visual percep-tion to participants feeling since spatial presence is closely relatedto a mental feedback from unconscious human cognitive processto given environments [21]. In general, presence is measured byusing questionnaires and by observing a participant’s reaction tostimuli, most oen in the form of threats [23].

3 EXPERIMENTTo investigate the eect of a personalized visual body represen-tation, we developed a virtual oce space that includes a virtualmirror to reect a personal body or avatar body as a visual cue. Inthis experiment, we examined virtual body ownership – includingbody continuity and agency – and presence as dependent variables.For independent variables, we chose varying body representations,levels of hand representation, and motor action capabilities. Weused subjective measures based on a questionnaire with a 7-pointLikert scale. Our experiment is a 2x2x2 mixed Within-Betweenfactorial design intended to show the eect of a personalized bodyrepresentation. We divided the participants into two groups, one forpersonalized visual body representation, and one for generic avatarbody representation (Between factor with two levels). Each groupexperienced both hand representations (Within factor with twolevels) and motor actions (Within factor with two levels). To pre-vent an ordering eect, we used a counter-balanced ordering. Ourexperiment was approved by our organization’s Internal ReviewBoard Oce.

3.1 Research HypothesesStarting with results from previous research [10], we conducted ourexperiment to nd answers for the following research questions:(1) ”Do personalized visual body cues inuence the sense of bodyownership of one’s virtual hand and of one’s sense of presence?”If yes, (2) ”Do personalized visual body cues create psychologicalcontinuity between a participant’s real body and their purely virtualhand?” e following hypotheses are based on prior results from

5


[10] and our beliefs concerning the eect of a personalized visualbody representation.

• Body Representation Using a personalized visual bodyreection will be more immersive than having a genericavatar body reection.

• Body Continuity A virtual body with a continuous, fullarm will be more immersive than a virtual arm where thelimbs between hand and shoulder have been removed.

• Body Motion Allowing users to move their hand will bemore immersive than requiring them to keep their hand ina static position.

• Combination e combination of personalized body rep-resentation with a full hand, enabled with dynamic motoraction, will give the highest levels of VBOI, BC, agency andpresence.

3.2 ParticipantsFor this experiment, we conducted a priori power analysis to deter-mine our sample size before recruiting participants. Using G*Power,to detect a medium eect size with a power of 0.80, we needed aminimum of 24 participants [6]. We recruited participants withnormal to corrected-to-normal vision using on-campus iers. Mostparticipants had higher education backgrounds and were studyingin diverse majors, but mainly in computer science. We conductedour experiment with 41 participants. We divided the participantsinto two groups, one with 21 participants (15 male, 6 female, MeanAge = 21.1, SD=2.92) for personalized visual body representationand the other with 20 participants (15 male, 5 female, Mean Age =21.65, SD=2.50) for avatar body representation. Because of a datalogging problem, we omied one person’s data (male) from thepersonalized body cue group. erefore we conducted the exper-iment with 40 participants, evenly divided into two groups of 20each. Most of the participants had a small number (under 5 times)of experiences wearing an HMD. We gave each a $10 gi card fortheir participation.

3.3 Experimental PlatformWe designed a physical experiment space isolated from any visualinterference. To reduce fatigue for the participants during theexperiment, we had them sit on a stool and rest their right hand ona stand. We used an HTC Vive to provide the virtual environment,and the HMD was tracked using the Vive’s tracking system. Torender each participant’s lower body, we placed an RGBD camera infront of the stool so we could capture their lower body. We created avirtual oce model similar to the physical experiment space exceptfor the presence of a table and small foot occluder in the virtualspace (See Figure 2). Note that, due to physical stand’ narrow width,its placement and the focus of the RGBD camera, the stand is notseen by the camera, and so only its synthetic counterpart appearsin the virtual world.

In the virtual oce, we included a table in front of the partici-pants and a prop to their right side that mimics the stand presentin the real space. To represent the personalized visual lower bodypart seen on the virtual mirror, we rendered the RGB pixel valueswith matched depth values on a plane and reected the image ontothe virtual mirror. Because of limited delity of the depth value for

Figure 2: Experiment environments. (a) Virtual oce. (b)Physical setup

body parts close to the oor, the feet were not rendered correctly,so we hid that part with a block cube occluder located on the oor.

3.4 estionnaireTo operationalize our dependent measures for Virtual Body Own-ership [1], Body Continuity [25], Agency [1], and Presence [13],we drew items from pre-validated survey instruments based onrelevant literature. However, we modied some of the wordingand chose the items that were most relevant to our current study.For example, an item selected for Virtual Body Ownership was, “Ifelt that my real body was endangered during the experiment.” Wealtered this item to be more specic to the current experiment: “Youfelt that your hand was endangered by the falling rock.” Similarly,we modied a question from previous research, “How much doyou have a feeling of being in underwater? [13]” to “You felt asif you were physically present in the oce room.” for measuringspatial presence. To avoid distractions while immersed within thevirtual environment, we chose to ask only one direct question forpresence that participants heard through the headset during theexperiment. We did this in lieu of adopting the more extensivepresence questionnaire developed by [29]. is methodological de-cision was inuenced by prior work also published in the domain ofvirtual body ownership and spatial presence [13, 30]. All items weremeasured on a 7-point Likert scale from 1 = Strongly Disagree to 7

6


Table 1: estionnaire

Item estion SourceVBOIα=0.73

You felt as if the virtual hand was yourown.

I felt as if the virtual representation of thehand was part of my body. [1]

You felt that your hand was endangeredby the falling rock.

I felt that my real body was endangeredduring the experiment. [1]

BCα=0.85

You felt as if the virtual hand started tolook like your own.

I felt as if I were looking at my own hand[25]

You felt as if the virtual hand was a partof your body.

I felt as if the virtual hand were part ofmy body. [25]

Agencyα=N/A

You felt as if you could control the virtualhand.

I felt like I controlled the virtual represen-tation of the hand as if it was part of myown body. [1]

Presenceα=N/A

You felt as if you were physically presentin the oce room.

How much do you have a feeling of beingin underwater? [13]

= Strongly Agree. Items for each construct were averaged to createcomposite indices for analysis. Because we modied questions frompre-validated survey instruments, we calculated Cronbach’s alphaas a metric for construct validity [16, 24] for Virtual Body Owner-ship and Body Continuity. e reliability of these measures wasabove the acceptable threshold of 0.7 [16, 24]. Since Agency andPresence were both measured with a single item, we treated thesemeasures as interval data (instead of as a continuous variable) inour analyses. Table 1 presents the wording of all measures used inthe current study compared to the original items that were drawnfrom the literature. It also reports Cronbach’s alpha metrics for themulti-item constructs.

3.5 ProtocolPrior to starting the experiment, we asked each participant to readour informed consent and ll out their demographic information.Aer they had lled out demographic data, we asked them to sit ona stool in the experiment room and gave them information aboutour study related to their experiencing the system. We were in-sistent that their initial pose have them siing on the chair in anormal forward facing position toward an RGBD camera and thatthey place their right hand on the physical stand. We then placedthe Vive HMD and headphones on the participant and asked themto look at their right arm, from the shoulder to hand, at least oncewhile in the VR environment, and to look at the virtual mirror toobserve their lower body representation as well. e participantlistened to an announcement of instructions for the study in our vir-tual oce. at announcement was delivered through headphonesusing a recorded native American speaker’s voice. Each participanthad two kinds of hand representations with two motor action ca-pabilities and one of two body reections. To avoid sequencingbiases, all conditions were combined with counter balanced order.Because body representation was a between factor, we conductedjust four sessions with each participant. To mitigate the inuenceof changing motor conditions, we divided the study procedure intotwo phases, one with the hand representation under static motorcondition and the other one with the hand representation underdynamic motor condition. For each session, we gave the participantone minute to look at the environment, including the arm and hand,and the mirror reection. Aer the participant had observed thevirtual seing, we began the process of dropping a photorealistic

rock onto the virtual hand ve times, randomly distributed over aone-minute interval with a corresponding sound eect (See Figure3). 15 seconds aer nishing the rock dropping event, we asked par-ticipants questions through the headphones using a recorded nativeAmerican speaker’s voice, and participants answered these verballyin each of the four conditions while we recorded their answers. Wethen provided an additional 30-second break to help participantsbe prepared for the new hand context under an identical motorcondition. During this transition time, participants continued towear the HMD but with the HMD displaying a neutral VR scene.Aer nishing the rst phase with both hand representations, wegave the participant a three-minute break siing on a stool withoutthe HMD and headphones. Aer the break, we resumed by redoingthe two hand conditions but now in the alternative motor actioncondition.

Figure 3: We dropped a photorealistic rock onto partici-pant’s virtual right hand ve times. (a) Fully representedhand, wrist and arm condition. (b) Arm and wrist removedcondition.

In the static motor action condition, we ignored the tracked posi-tional data from the participant’s arm and hand movement, askingthem to keep their hand on the stand without movement. eycould, however, still move their legs and head. In this static con-dition, they passively observed the rock dropping onto the virtualright hand. In the dynamic motor action condition, participantswere allowed to move their real hand, resulting in a correspondingmovement of the virtual right hand. us, in the dynamic condition,they could actively carry out certain behaviors such as avoiding,touching or hiing the dropping rock. We address details of ourobservations in the analysis section.

Aer completing all the sessions, we asked the participant toprovide feedback concerning their recognition of the hand sta-tus and mirror reected body representations. All participantsdistinguished the connectivity status between the two hand repre-sentations that they had experienced. Similarly most participantsperceived the body reection in the personalized representation astheir own body, even though the mirror reected image had rela-tively low resolution. ey even reported on how the body had thespecic paerns or colors of their own clothes. In contrast, most

7


participants could recognize the generic virtual avatar as beingpurely virtual, and so not their real body.

4 ANALYSISIn this section, we present our study results for the eect of thepersonalized visual body cue as the dominant factor for virtualillusion. As we described in the experiment section, we ran ourstudy as a 2x2x2 mixed Within-Between factorial design. Beforewe analyzed the data, we clustered the measured data into iden-tical categories. To analyze the subjective measurement, generalMultivariate Analysis of Variance (MANOVA) was used for VBOIand BC, and Kruskal Wallis test was used for presence and agencysince they were non-parametric data. We conrmed that VBOIand BC measures were normally distributed before analyzing thesedata. We provide the interquartile range box with outlier and me-dian symbol in all box-plot graphs along with numerical statisticalresults (See Table 2).

4.1 Virtual Body OwnershipFrom the main eect result (See Figure 4), we observed the inter-esting outcome that body representation and hand type are moreinuential on VBOI, while motor conditions did not show a signi-cant eect. We found a signicant dierence between personalizedbody representations (p


Figure 7: Personalized visual body representation showshigher sense of body continuity than avatar body represen-tation in all identical conditions.

and motor conditions, and between body representations and handrepresentations, but we found an interaction eect between thehand representations and motor conditions.

Figure 8: Only motor status shows explicit dierence insense of agency.

We noticed that the dynamic motor condition showed a strongtendency for agency in comparison to the static motor condition,with all other conditions equal. Similar to previous results, thepersonalized visual body representation with fully represented armshowed the strongest eect on agency (mean(sd)=5.50(1.05)).

Figure 9: Dynamic motor condition shows a higher senseof agency than does the static motor condition in identicalsituations.

4.2 PresenceWe found that body representation had a positive inuence onthe main eect results regarding sense of presence (See Figure 10).e personalized visual body representation shows a signicantdierence for presence compared to the avatar body representation(p


Table 2: Descriptive Statistics with mean value(SD)

Item Real AvatarFull NoArm Full NoArm Motor

VBOI 4.48(1.68) 3.63(1.66) 3.43(1.82) 2.75(1.46) Dynamic3.95(1.81) 3.18(1.68) 3.40(1.95) 2.80(1.68) Static

BC 4.90(1.61) 3.75(1.48) 3.28(1.72) 2.55(1.58) Dynamic4.75(1.45) 3.65(1.76) 3.45(1.85) 2.53(1.52) Static

Agency 5.50(1.05) 5.10(0.91) 4.57(1.50) 4.90(1.56) Dynamic3.75(1.20) 3.05(1.82) 3.25(1.97) 2.55(1.36) Static

Presence 5.65(1.23) 5.40(1.23) 4.95(1.32) 4.55(1.50) Dynamic5.6(1.19) 5.15(1.63) 5(1.38) 5.15(1.63) Static

Interestingly, we found a statistical dierence between person-alized visual body representation and avatar body representationregarding presence, a result that was not shown in [10]. We believethis is because artifacts were produced in the earlier experimentfrom the point cloud that rendered a participant’s mirror reection.ese artifacts distracted participants, resulting in a decreased senseof presence. In the experiment reported here, we did not use thepoint cloud data for rendering the participant’s mirror reection;rather we used a 2D image based on the RGB and depth values fromthe RGBD camera. is approach removed the unexpected artifactsaround the participant’s siing location. With this more precise ex-perimental environment, participants reported that a personalizedvisual body representation gave them a stronger sense of presencethan with a generic virtual avatar.

A limitation of our results is that the number of items used tooperationalize our dependent measures was constrained by theexperimental design of our study. Future studies should build uponthese early results, which suggest signicant eects for VBOI, BC,Agency, and Presence, by using more extensive and pre-validatedmeasures to assess these constructs. However, while we cannotsay that personalized body representation is the primary factorin the sense of presence, we can say it is a contributing factor tosuch an illusion as validated by VBOI and BC. Further study will berequired to support our hypothesis that this is, in fact, the dominantcontributor to the increased sense of presence reported to us byparticipants.

6 CONCLUSIONS AND FUTUREWORKIn this paper, we investigated the eect of a visually personalizedlower body representation on dominant illusions such as virtualbody ownership and presence, even when we have a purely virtualhand. To measure this from a subjective point of view, we extendedand revised the experimental design from earlier research studies,adding two hand representations and two motor action capabilities.From our experiment, we found statistical support for a signicantdierence in virtual illusion between a personalized visual bodyrepresentation and an avatar body representation. Specically,we showed that a personalized visual body representation had animportant role in eliciting a high sense of virtual body ownership,body continuity and presence in comparison to an avatar bodyrepresentation. Additionally, motor action capabilities had a criticalrole for agency, an expected result since agency is indicative ofthe sense of controlling a virtual body. In our setup, even thoughthe rendering quality of a participant’s mirror reection lackedvisual artifacts, the image was not particularly sharp. Despite the

lack of sharpness, most participants noticed their own body basedon the color of their clothes and the shape of their legs. Overall,personalized virtual body representation showed positive inuencefor VBOI, BC, presence and even agency.

In summary, we believe that our experiment makes three contri-butions to the VR community: 1) We found a personalized visualbody representation is a signicant factor in eliciting desired visualillusions of those tested and we provided a best combination toarouse such illusions. Specically, we demonstrated that a per-sonalized visual body representation with a fully represented armand hand, combined with a dynamic motor capability elicits thestrongest sense of desired visual illusions. 2) By investigating com-binations of conditions that aect VBOI, BC, presence and agency,we showed how a developer can compensate for unavailable op-tions when there are design trade-os. 3) We showed that removingvisual artifacts improves a participant’s sense of presence.

In future work, we will develop a system to measure human per-ception when participants have a virtual hand that seems identicalto their own. As the hand is the most frequently used body partwhen carrying out tasks, creating a person-specic virtual handthat has features visually similar to the participant’s real hand, in-cluding skin color and wrinkles, and, where worn, rings, braceletsor a watch should have a positive eect on virtual illusions. Becausewe demonstrated the eectiveness of personalized visual cues, evenwhen seen indirectly through mirror reection, we believe a per-sonalized visual cue of one’s own hand will dramatically increasetheir senses of illusion in a synthetic reality environment.

ACKNOWLEDGMENTSis research was, in part, supported by the Bill & Melinda GatesFoundation .ny opinions, ndings, conclusions, or recommenda-tions expressed in this material are those of the authors and do notnecessarily reect the views of the Bill & Melinda Gates Founda-tion. Additionally, the authors wish to extend their appreciation toall members of the University of Central Florida Synthetic RealityLaboratory team.

REFERENCES[1] Ferran Argelaguet, Ludovic Hoyet, Michael Trico, and Anatole Lécuyer. 2016.

e Role of Interaction in Virtual Embodiment: Eects of e Virtual HandRepresentation. In In IEEE Conference on Virtual Reality. 3–10.

[2] Wilma A. Bainbridge, Justin Hart, Elizabeth S. Kim, and Brian Scassellati. 2008.e Eect of Presence on Human-Robot Interaction. In Proceedings of the 17thIEEE International Symposium on Robot and Human Interactive Communication.

[3] Ilias Bergstrm, Konstantina Kilteni, and Mel Slater. 2016. First-Person PerspectiveVirtual Body Posture Inuences Stress: A Virtual Reality Body Ownership Study.PLOS ONE 11, 2 (FEBRUARY 2016), 1–21.

[4] Olaf Blanke, Mel Slater, and Andrea Serino. 2015. Behavioral, Neural, andComputational Principles of Bodily Self-Consciousness. Neuron 88 (JULY 2015),145–166.

[5] Mahew Botvinick and Jonathan Cohen. 1998. Rubber Hands ’Feel’Touch atEyes See. Nature 391, 6669 (FEBRUARY 1998), 756–756.

[6] Faul Franz, Erdfelder Edgar, Lang Albert-Georg, and Buchner Axel. 2007.G*Power 3: A Flexible Statistical Power Analysis Program for e Social, Behav-ioral, and Biomedical Sciences. Behavior Research Methods 39 (2007), 175–191.

[7] omas Fuchs. 2012. e Phenomenology of Body Memory. Amsterdam; hiladel-phia:John Benjamins Publishing Company.

[8] Mar González-Franco, Daniel Pérez-Marcos, Bernhard Spanlang, and Mel Slater.2010. e Contribution of Real-Time Mirror Reections of Motor Actions onVirtual Body Ownership in An Immersive Virtual Environment. In In IEEE Con-ference on Virtual Reality. 111–114.

[9] Ludovic Hoyet, Ferran Argelaguet, Corentin Nicole, and Anatole Lécuyer. 2016.‘Wow! I Have Six Fingers!’: Would You Accept Structural Changes of Your Hand

10


in VR? Frontiers in Robotics and AI 3 (MAY 2016), 1–12.[10] Sungchul Jung and Charles E. Hughes. 2016. e Eects of Indirect Real Body

Cues of Irrelevant Parts on Virtual Body Ownership and Presence. In ICAT-EGVE 2016 - International Conference on Articial Reality and Telexistence andEurographics Symposium on Virtual Environments. 107–114.

[11] Konstantina Kilteni, Ilias Bergstrom, and Mel Slater. 2013. Drumming in Immer-sive Virtual Reality: e Body Shapes e Way We Play. IEEE Transactions onVisualization and Computer Graphics 19, 4 (APRIL 2013), 597–605.

[12] Konstantina Kilteni, Antonella Maselli, Konrad P Kording, and Mel Slater. 2015.Over My Fake Body: Body Ownership Illusions for Studying e MultisensoryBasis of Own-Body Perception. Frontiers in Human Neuroscience 9, 4 (MARCH2015).

[13] Sungkil Lee and Gerard Jounghyun Kim. 2008. Eects of visual cues and sustainedaention on spatial presence in virtual environments based on spatial and objectdistinction. Interacting with Computers 20 (September 2008), 491–502.

[14] Jean-Luc Lugrin, Johanna La, and Marc Erich Latoschik. 2015. Avatar Anthro-pomorphism and Illusion of Body Ownership in VR. In In IEEE Conference onVirtual Reality. 229–230.

[15] Michael Meehan, Brent Insko, Mary Whion, and Frederick P. Brooks Jr. 2002.Physiological Measures of Presence in Stressful Virtual Environments. ACMTransactions on Graphics (TOG) (JULY 2002), 645–652.

[16] Jum C. Nunnally. 1978. Psychometric eory (2nd ed.). McGraw-Hill, New York.[17] Pérez-Marcos, Daniel, Maria V. Sanchez-Vives, and Mel Slater. 2012. Is My Hand

Connected to My Body? e Impact of Body Continuity and Arm Alignment one Virtual Hand Illusion. Cognitive Neurodynamics 6 (JULY 2012), 295–305.

[18] Valeria I Petkova, Mehrnoush Khoshnevis, and H. Henrik Ehrsson. 2011. ePerspective Maers! Multisensory Integration in Egocentric Reference FramesDetermines Full-Body Ownership. Frontiers in Psychology 2, 2 (MARCH 2011),1–7.

[19] Maria V Sanchez-Vives and Mel Slater. 2005. From Presence to Consciousnessthrough Virtual Reality. Nature Reviews. Neuroscience (APRIL 2005), 332–339.

[20] Maria V. Sanchez-Vives, Bernhard Spanlang, Antonio Frisoli, Massimo Berga-masco, and Mel Slater. 2010. Virtual Hand Illusion Induced by VisuomotorCorrelations. PLOS ONE 5, 4 (APRIL 2010), 1–6.

[21] omas W. Schubert. 2009. A New Conception of Spatial Presence: Once Again,with Feeling. Communication eory 19, 2 (2009), 161–187.

[22] Martijn J. Schuemie, Peter van der Straaten, Merel Krijn, and Charles A.P.G.van der Mast. 2004. Research on Presence in Virtual Reality: A Survey. CyberPsy-chology and Behavior 4 (JULY 2004), 183–201.

[23] Mel Slater, Anthony Steed, and Martin Usoh. 2013. Being ere Together: Experi-ments on Presence in Virtual Environments (1990s). Technical Report, Departmentof Computer Science, University College London, UK.

[24] Mohsen Tavakol and Reg Dennick. 2011. Making sense of Cronbach’s alpha.International Journal of Medical Education 2 (2011), 53–55.

[25] Gaetano Tieri, Emmanuele Tidoni, Enea Francesco Pavone, and Salvatore MariaAglioti. 2015. Body Visual Discontinuity Aects Feeling of Ownership and SkinConductance Responses. Scientic Reports 5, 17139 (November 2015).

[26] Gaetano Tieri, Emmanuele Tidoni, Enea Francesco Pavone, and Salvatore MariaAglioti. 2015. Mere Observation of Body Discontinuity Aects Perceived Own-ership and Vicarious Agency over A Virtual Hand. Experimental Brain Research233 (APRIL 2015), 1247–1259.

[27] Manos Tsakiris, Simone Schtz-Bosbach, and Shaun Gallagher. 2007. On Agencyand Body-Ownership: Phenomenological and Neurocognitive Reections. Con-sciousness and Cognition 16 (SEPTEMBER 2007), 661–666.

[28] Max Velmans. 1998. Physical, Psychological and Virtual Realities. e VirtualEmbodied (1998), 45–60.

[29] Bob G. Witmer and Michael J. Singer. 1998. Measuring Presence in VirtualEnvironments: A Presence estionnaire. Presence: Teleoperators and VirtualEnvironments 7, 3 (JUNE 1998), 225–240.

[30] Yuan Ye and Anthony Steed. 2010. Is the Rubber Hand Illusion Induced byImmersive Virtual Reality?. In In IEEE Conference on Virtual Reality. 95–102.

11

Abstract1 Introduction2 Related Work2.1 Virtual Body Ownership2.2 Virtual Body Ownership and Body Continuity2.3 Presence

3 Experiment3.1 Research Hypotheses3.2 Participants3.3 Experimental Platform3.4 Questionnaire3.5 Protocol

4 Analysis4.1 Virtual Body Ownership4.2 Presence

5 Discussion and Limitation6 Conclusions and Future WorkAcknowledgmentsReferences

Documents

RealME: The Influence of Body and Hand Representations on ... › wp-content › uploads › 2018 › 06 › 2017_Jung_RealMe.pdfRealME: The Influence of Body and Hand Representations