CyberPsychology & BehaviorVolume 1, Number 2, 1998Mary Ann Liebert, Inc.

Challenges of Recreating Reality in Virtual Environments

MILTON P. HUANG, M.D.,1 JOSEPH HIMLE, Ph.D.,1 KLAUS-PETER BEIER, Dr.-Ing.,2and NORMAN E. ALESSI, M.D.1

ABSTRACT

Virtual environments (VE), also known as virtual reality (VR), have been used in exposuretreatment of phobias by simulating a situation that the patient fears. Initial studies of thesetreatments have demonstrated effectiveness of treatment, but have not compared it to the"gold standard" of in vivo exposure. We are in the process of comparing VE exposure treat¬ment for acrophobia to in vivo exposure treatment by replicating an actual in vivo exposureenvironment in a virtual model. Our design of both experimental system and experimentalprotocol aims to extract the essential, unique aspects of the VE experience that make it dif¬ferent from traditional treatment, and to increase our understanding of how these relate tothe psychological history that people bring to such encounters. Besides the challenges of pro¬tocol design, this process also provides an illustration of the challenges of working withrapidly changing hardware and software standards, as we are attempting to use state-of-the-art equipment and software, such as the CAVE (CAVE Automatic Virtual Environment) andVRML (Virtual Reality Modeling Language).

INTRODUCTION

THE USE OF VIRTUAL ENVIRONMENTS (VE), alsoknown as virtual reality (VR), is growing

rapidly with continuously advancing tech¬nologies and related standards. Many potentialapplications exist for the use of VE in the treat¬ment of psychiatric disorders,1 the most stud¬ied being that of the treatment of phobias. Inthis case, VE has been generally used in expo¬sure treatment, by creating a reared environ¬ment in virtual reality, then leading a patientthrough that environment using traditionaltechniques of graded exposure. Many case re¬

ports suggest the effectiveness of VE treatmentfor various phobias including fear of flying,2'3fear of spiders,4 and acrophobia.5 Controlled

'University of Michigan Department of Psychiatry.2University of Michigan Virtual Reality Laboratory.

studies have been conducted primarily foracrophobia.6'7 These studies have shown the ef¬fectiveness of VR exposure using comparisonsto wait-list controls and cognitive therapy, buthave failed to compare it to in vivo exposure,which is the current state-of-the-art treatmentfor acrophobia. Our work is aimed at explor¬ing such a comparison, so that we can betterunderstand how the psychological effects ofvirtual reality differ from the normal psycho¬logical interaction with reality itself. This ex¬

ploration has lead us to face three primary chal¬lenges. The first is in the design of an

appropriate experimental system that takes ad¬vantage of current technology to permit a com¬

parison between the modality of VE therapyand traditional therapy. The second challengeis to design an experimental protocol for thissystem that cannot only measure the effective¬ness of this new treatment modality, but alsoallow insight into what specific factors produce

163

164 HUANG ET AL.

differences between it and traditional treat¬ment. Our final challenge has been to imple¬ment such these designs in an environmentwhere computer hardware and software stan¬dards change on a constantly accelerating ba¬sis.

EXPERIMENTAL SYSTEM

As we wish to compare how people react tovirtual environments and how they react to re¬

ality, our primary challenge in design of theexperimental system has been controlling forthe differences between virtual and in vivo ex¬

posure. Our two-pronged approach includescreating a virtual model to duplicate the actualenvironment as closely as possible and mini¬mizing the intrusiveness of the VR equipment.The Anxiety Disorders program of the Univer¬sity of Michigan has traditionally treated pa¬tients suffering from acrophobia by exposingthem to progressively greater heights lookingout the windows of the East Elevator shaft ofthe University of Michigan main hospitalbuilding. We created a virtual model of the dif¬ferent elements of this experience, includingwalking in the lobby space, using the elevator,and examining the views out the window atdifferent floors. The three-dimensional (3D)geometry was developed through the use ofgeometric modeling software8 and enhancedwith various multimedia software.9 It includesthe building's interior elements (floor, walls,doors, benches, plants, elevator, etc.), as well as

the exterior visible from the window (court¬yard, trees, other buildings, walkways, citybackground).

Virtual environments often have a "cartoon¬like" feel, because the appearance of geometryis only modeled using color and reflective qual¬ities to reduce the computational load. A tech¬nique called "texture mapping" provides amore realistic appearance, but is computationalintensive. Textures are digital images and pho¬tographs that are pasted onto the geometry andsimulate the complex appearance of real objectsin a more natural way. Using digital cameras,we captured elements of the in vivo environ¬ment and transferred these images into the vir¬tual model. In addition, libraries of textures

were used for appearance enhancement (e.g.,fabric on elevator walls, wall textures, and floortiling).

Use of state-of-the-art technology helps tominimize differences between VR exposureand in vivo exposure. Our model is experi¬enced in a CAVE™ (CAVE Automatic VirtualEnvironment) as developed by the Universityof Illinois at Chicago and is now commerciallysold by Pyramid Systems. The CAVE is a pro¬jection-based VR system that surrounds theviewer with four 10' X 10' screens, arranged toform three walls and the floor of a cube (Fig¬ure 1). The viewer wears liquid crystal shutterglasses and a six-degrees-of-freedom head-tracking device. As the viewer moves inside theCAVE, a Silicon Graphics Onyx computer cal¬culates the correct perspective projections foreach wall, based on head-tracking measure¬ments that describe the viewers location in thevirtual model. The images for the left and theright eye are projected in a rapid, alternatingsequence. The shutter glasses alternately blockthe left and the right eye in synchronizationwith the projection sequence, giving the ap¬propriate 3D image. The CAVE environment isa superior system when compared with the tra¬

ditionally used head-mounted display (HMD)that is worn like a helmet. An HMD is ex¬

tremely intrusive, uncomfortable to wear, andprovides an unnatural restricted field of view(FOV). In contrast, the lightweight shutterglasses and the surrounding walls of the CAVEprovide a very wide FOV that properly stimu¬lates the human peripheral vision and facili-

BackWäll

Projector

FIG. 1. CAVE projection system.

RECREATING REALITY IN VIRTUAL ENVIRONMENTS 165

tates significantly orientation and navigation inthe virtual world. The stereo projection on thefloor enables the viewer to perceive objectabove and below the floor. Looking down froma window or approaching a cliff can become a

very convincing experience. The viewer can

also see real objects in the CAVE like his/herown body or that of others, more closely sim¬ulating the effect of therapist presence in vivo.The use of these technologies allows our sys¬tem to make the experience of our virtualenvironment as close to reality as is currentlyfeasible. To control for the remaining short¬comings of the CAVE experience in the com¬

parison with in vivo exposure, we will placecomparable restrictions on the in vivo partici¬pants by asking them to wear shutter glassesand not to touch physical objects like walls,since subjects in the virtual environment are

unable to feel any objects of their surroundingvirtual world.

provement among all conditions, as well as ex¬

amine the ability of virtual exposure to gener¬alize to real life.

Many other variables may show differencesbetween the virtual and in vivo groups. Phys¬iologic measurements and subjective measure¬ments of the impact of these exposures will al¬low us a sense of the effect on anxiety. We willalso measure demographic variables, prior ex¬

posure to different types of technology, andother measures of the impact of "virtuality."The literature contains several attempts tomeasure the extent to which a subject feels true

"presence" in a virtual environment.17-19 Wewill examine how such variables influence bothmeasures of anxiety and measures of treatmentefficacy.

IMPLEMENTATION IN HARDWAREAND SOFTWARE

EXPERIMENTAL PROTOCOL

Our experimental protocol is designed to

clarify variables that separate the treatment inthe CAVE to treatment in the actual environ¬ment.10 This requires us to not only examinethe subjects using traditional assessments thatare used in the psychiatric literature, but to alsouse other assessment tools to attempt to mea¬

sure other variables that are not usually evalu¬ated. Subjects are selected as having diagnosisof specific phobia11 using the Structured Clin¬ical Interview for DSM-IV (SCID),12 then givena battery of questionnaires to assess their acro¬

phobia.13"16 A behavioral approach test in theelevator lobby establishes a uniform measure

of their ability to tolerate the in vivo situation.They are then randomized to a treatment con¬

dition of either in vivo exposure, CAVE expo¬sure, or a control condition of relaxation train¬ing. In each condition, a therapist takes thesubject through a standardized training for 90min, accompanied by physiologic monitoringof heart rate, respiration rate, and galvanic skinresponse. After completion of exposure train¬ing, each subject is returned to the in vivo sit¬uation for a repeat behavioral approach test.This will allow a direct comparison of im-

Our process of modeling and experimentaldesign has been instructive, teaching us lessonsabout the interactions of hardware, software,and psychiatric knowledge and the challengesof integrating them together in producing prac¬tical virtual reality. The interaction is complex.If we extrapolate from other models of tech¬nology, such as that of the Internet,20 we rec¬

ognize that each of these three elements are de¬pendent upon the others, yet they each havetheir own trajectory of development with hard¬ware moving the fastest, software quickly fol¬lowing, and psychological models lagging be¬hind; the difficulties we have faced in our

modeling process give concrete example ofthis.

Computer hardware changes continuouslyas industry competes to create better andcheaper products. Processing speed doublesevery 18 months,21 allowing faster rendering of3D models and more effective inclusion of tech¬niques like texture mapping. Our aim of usinga state-of-the-art system like the CAVE is an

example of an attempt to take advantage ofthese trends. Unfortunately, this rapidity of de¬velopment tends to make each hardware solu¬tion isolated, not easily integrated with otherdevices and outstripping the development ofsupporting software. Although the CAVE sys-

166 HUANG ET AL.

tern is commercial, each installation requiresunique adjustments. In our case, setup of theCAVE has been a long process with frequenttesting for realigning projectors, connecting thecomputer to the projectors, or calibrating thetracking devices in the 3D space. We are ex¬

ploring the interferences between the electronicfields of the tracking system and of our physi¬ologic monitoring equipment as the CAVE wasnot designed for the use of such systems inside.We are also continuing to get additions to theCAVE as other hardware is added to increasethe speed of information flow. Unfortunately,we have already found that such changes haveinvalidated some of our current software, ne¬

cessitating rewrites. We are always attemptingto adjust for ongoing changes in hardware.

We have seen in other ways how hardwaredevelopment outpaces the development ofsupporting software. The CAVE is pro¬grammed in either C or C + + using specific li¬brary calls that control different hardware ele¬ments. Higher level libraries (EVL, Performer,Open GL, and others) allow easier program¬ming for the CAVE environment. Recently, astandard for the development of virtual envi¬ronments has been developed for the WorldWide Web (WWW). This standard, calledVRML (Virtual Reality Modeling Language), isnot restricted to the WWW, but can be utilizedfor any VR application. VRML22'23 defines a

virtual environment by geometry, appearance(e.g., color, textures), and illumination, and in¬cludes functions for animations, interactions,and behavior scripting. Translators that con¬vert a VRML application into a CAVE applica¬tion are under development and will, in the fu¬ture, make VRML an excellent standard for thedefinition and exchange of virtual environ¬ments. In the meanwhile, we are coding the in¬teractions of our model in both Performer andVRML, requiring some reduplication of work.

The challenges of keeping up with advanc¬ing standards of hardware and software andinterconnecting them offer a contrast to deal¬ing with a lack of standards in how we orga¬nize our understanding of the relevant psy¬chological and social factors that shouldinfluence the creation of a virtual environment.Clearly, the potential number of such factors is

endless. In designing our environment fortreatment of acrophobia, our primary consid¬eration has been trying to make the virtual en¬vironment as similar to the real environment as

possible. Our choices have been made based on

subjective experience, reiteratively entering thevirtual environment, "feeling" what seems

"right," and then making appropriate changesin system programming. This design process issubject to errors from the fact that many per¬ceptions of the environment are unconscious intheir influence and, therefore, ignored. In ad¬dition, we intentionally left out some parts ofthe experience because of the difficulty in ac¬

curately replicating them. We do not simulatereflections off of the window, window tinting,or dust, which all commonly serve as cues thata barrier is present. Thus far, there is little re¬

search on how important choices like "windowtinting" or the presence of reflections are inmaking virtual reality simulate reality.

CONCLUSIONS

Our experimental procedure is aimed at de¬termining where the differences exist betweenvirtual reality and reality in the context of acro¬

phobia treatment. It compares the treatment ofsubjects in a real location to treatment in a VRsimulation of that real location. Use of mini¬mally intrusive technology and appropriateprotocol design allow us to make treatment inthese two settings as similar as possible, per¬mitting us to examine VR itself as an indepen¬dent variable. This requires much work as we

have seen in our attempts to use the CAVE andVRML.

The design of this project and the difficultiesof realizing it suggest some of the challengeswe face in using virtual reality in mental health,as well as important considerations for futurework and research. Keeping up with continu¬ously changing hardware and software re¬

quires time and dedication. We need educationin how such learning can be integrated into our

usual work, as well as how to define our own

expertise and interface it with that of techno¬logical experts.24 As mental health experts, weneed to focus on the design of experimental

RECREATING REALITY IN VIRTUAL ENVIRONMENTS 167

systems and protocols that can be used to in¬vestigate questions of interest, independent yetcognizant of the underlying technology used torealize these designs. Perhaps the greatest chal¬lenge before us is how we can organize our ex¬

pertise in a way in which other researchers invirtual reality and computer interface designcan access it to incorporate it into their workand research. We need to work on models thatdefine the mental attributes that should be con¬sidered in creating virtual environments. Al¬though work has been done on the effect of en¬vironmental variables on factors related to

perception, little exists on their impact on otherpsychological factors and how they relate toour current knowledge of mental illness, per¬sonality attributes, and normal mental func¬tion.

Computer technologies now face the mentalhealth field with new challenges and opportu¬nities.25 We hope that a thoughtful explorationof the design process of mental health applica¬tions in virtual reality will encourage the ex¬

perts in this area to work together to furtherdevelop ideas about other standards we need.Such work will be essential to encourage re¬search and increase our understanding and useof virtual environments in psychiatric and psy¬chological applications.

REFERENCES

1. Glantz, K., Durlach, N.I., and Aviles, W.A. (February1997). Virtual reality (VR) and psychotherapy: Op¬portunities and challenges. Presence: Teleoperators andVirtual Environments, 6(1), 87-105.

2. Rothbaum B.O., Hodges, L.F., Watson, B.A., Kessler,G.D., and Opdyke, D. (1996). Virtual reality exposuretherapy in the treatment of fear of flying: A case re¬

port. Behaviour Research and Therapy, 34(5/6), 477^181.3. North, M.M., North, S.M., and Coble, J.R. (February

1997). Virtual environments psychotherapy: A case

study of fear of flying disorder. Presence: Teleoperatorsand Virtual Environments, 6(1), 127-132.

4. Carlin, A.S., Hoffman, H.G., Weghorst, S. (February1997). Virtual reality and tactile augmentation in thetreatment of spider phobia: A case study. BheaviourResearch and Therapy, 35(2), 153-158.

5. Rothbaum, B.O., Hodges, L.F., Kooper, R., Opdyke,D., Williford, J., and North, M.M. (1995 Summer). Vir¬tual reality graded exposure in the treatment of acro¬

phobia: A case report. Behavior Therapy, 26(3), 547-554.

6. Rothbaum, B.O., Hodges, L.F., Kooper R., Opdyke, D.,Williford, J., and North, M.M. (April 1995). Effective¬ness of computer-generated (virtual reality) gradedexposure in the treatment of acrophobia. AmericanJournal of Psychiatry, 152(4), 626-628.

7. Lamson, R.J. (1997). Virtual Therapy: Prevention andTreatment of Psychiatric Conditions by Immersion in Vir¬tual Reality Environments. Montreal: Editions de l'ɬcole Polytechnique de Montréal.

8. Mortenson, M.E. Geometrie modeling (2nd ed.). NewYork: Wiley, 1997.

9. Huang, M.P., and Alessi, . (1996). Tools for devel¬oping multimedia in psychiatry. In M. Miller, K. Ha-mond, & M. Hile (eds.), mental health computing. NewYork: Springer-Verlag, pp. 322-341.

10. Huang, M.P., Himle, J., Beier, K., and Alessi, N.E. (inpress). Comparing virtual and real worlds for acro¬

phobia treatment. In Medicine Meets Virtual Reality 6.Amsterdam: IOS Press.

11. American Psychiatric Association. (1994). Diagnosticand Statistical Manual of Mental Disorders (4th ed.).Washington DC: APA Press.

12. First, M.B., Spitzer, R.L., Gibbon, M., and Williams,J.B.W. (1997). Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I), Clinician Version. Wash¬ington, DC: American Psychiatric Press.

13. Cohen, D.C. (1977). Comparison of self-report andovert-behavioral procedures for assessing acropho¬bia. Behavior Therapy, 8, 17-23.

14. Marks, I.M., and Mathews, A.M. (1979). Brief stan¬dard self-rating for phobic patients. Behaviour Researchand Therapy, 17(3), 263-267.

15. Michelson, L. (1986). Treatment consonance and re¬

sponse profiles in agoraphobia: The role of individ¬ual differences in cognitive, behavioral and physio¬logical treatments. Behaviour Research and Therapy,24(3), 263-275.

16. Menzies, R.G., and Clarke, J.C. (May 1993). The etiol¬ogy of fear of heights and its relationship to severityand individual response patterns. Behaviour Researchand Therapy, 31(4), 355-365.

17. Sheridan, T.B. (Spring 1996). Further Musings on thePsychophysics of Presence. Presence: Teleoperators andVirtual Environments, 5(2), 241-246.

18. Welch, R.B., Blackmon, T.T., Liu, ., Meilers, B.A.,and Stark, L.W. (Summer 1996). The effects of picto¬rial realism, delay of visual feedback, and observerinteractivity on the subjective sense of presence. Pres¬ence: Teleoperators and Virtual Environments, 5(3),263-273.

19. Hendrix, C, and Barfield, W. (Summer 1996). Pres¬ence within virtual environments as a function of vi¬sual display parameters. Presence: Teleoperators andVirtual Environments, 5(3), 274-289.

20. Huang, M.P., and Alessi, N.E. duly 1996). The Inter¬net and the future of psychiatry. American Journal ofPsychiatry, 153, 861-869.

21. Schaller, R.R. (June 1997). Moore's Law: Past, present,and future. IEEE Spectrum, 34, 52-59.

168 HUANG ET AL.

22. Virtual Reality Modeling Language ISO/IEC Draft Inter¬national Standard 14772-1 (December 1997).

23. Wagner, M. (July 7, 1997). Real-world uses for 3-DWeb spec. Computerworld, 31, 45 +

.24. Huang, M.P., and Alessi, N.E. (in pess). An infor¬matics curriculum for psychiatry. Academic Psychiatry.

25. Alessi, ., Huang, M., and Quinlan, P. (1997). 2005:Information Techology impacts psychiatry. In L.J.Dickstein, M.B. Riba, and J.M. Oldham (Eds.), Amer¬ican Psychiatric Press Review of Psychiatry (Vol. 16).

Washington DC: American Psychiatric Press, pp.VI69-87.

Address reprint requests to:Dr. Milton Huang

1500 E. Medical Center Dr., TC 3502/Box 0390Ann Arbor, MI 48109-0390, USA

Email: mhuang@umich.edu