CyberPsychology & BehaviorVolume 1, Number 2, 1998Mary Ann Liebert, Inc.
Challenges of Recreating Reality in Virtual Environments
MILTON P. HUANG, M.D.,1 JOSEPH HIMLE, Ph.D.,1 KLAUS-PETER BEIER, Dr.-Ing.,2and NORMAN E. ALESSI, M.D.1
Virtual environments (VE), also known as virtual reality (VR), have been used in exposuretreatment of phobias by simulating a situation that the patient fears. Initial studies of thesetreatments have demonstrated effectiveness of treatment, but have not compared it to the"gold standard" of in vivo exposure. We are in the process of comparing VE exposure treatment for acrophobia to in vivo exposure treatment by replicating an actual in vivo exposureenvironment in a virtual model. Our design of both experimental system and experimentalprotocol aims to extract the essential, unique aspects of the VE experience that make it different from traditional treatment, and to increase our understanding of how these relate tothe psychological history that people bring to such encounters. Besides the challenges of protocol design, this process also provides an illustration of the challenges of working withrapidly changing hardware and software standards, as we are attempting to use state-of-the-art equipment and software, such as the CAVE (CAVE Automatic Virtual Environment) andVRML (Virtual Reality Modeling Language).
THE USE OF VIRTUAL ENVIRONMENTS (VE), alsoknown as virtual reality (VR), is growingrapidly with continuously advancing technologies and related standards. Many potentialapplications exist for the use of VE in the treatment of psychiatric disorders,1 the most studied being that of the treatment of phobias. Inthis case, VE has been generally used in exposure treatment, by creating a reared environment in virtual reality, then leading a patientthrough that environment using traditionaltechniques of graded exposure. Many case reports suggest the effectiveness of VE treatmentfor various phobias including fear of flying,2'3fear of spiders,4 and acrophobia.5 Controlled
'University of Michigan Department of Psychiatry.2University of Michigan Virtual Reality Laboratory.
studies have been conducted primarily foracrophobia.6'7 These studies have shown the effectiveness of VR exposure using comparisonsto wait-list controls and cognitive therapy, buthave failed to compare it to in vivo exposure,which is the current state-of-the-art treatmentfor acrophobia. Our work is aimed at exploring such a comparison, so that we can betterunderstand how the psychological effects ofvirtual reality differ from the normal psychological interaction with reality itself. This exploration has lead us to face three primary challenges. The first is in the design of anappropriate experimental system that takes advantage of current technology to permit a comparison between the modality of VE therapyand traditional therapy. The second challengeis to design an experimental protocol for thissystem that cannot only measure the effectiveness of this new treatment modality, but alsoallow insight into what specific factors produce
164 HUANG ET AL.
differences between it and traditional treatment. Our final challenge has been to implement such these designs in an environmentwhere computer hardware and software standards change on a constantly accelerating basis.
As we wish to compare how people react tovirtual environments and how they react to reality, our primary challenge in design of theexperimental system has been controlling forthe differences between virtual and in vivo exposure. Our two-pronged approach includescreating a virtual model to duplicate the actualenvironment as closely as possible and minimizing the intrusiveness of the VR equipment.The Anxiety Disorders program of the University of Michigan has traditionally treated patients suffering from acrophobia by exposingthem to progressively greater heights lookingout the windows of the East Elevator shaft ofthe University of Michigan main hospitalbuilding. We created a virtual model of the different elements of this experience, includingwalking in the lobby space, using the elevator,and examining the views out the window atdifferent floors. The three-dimensional (3D)geometry was developed through the use ofgeometric modeling software8 and enhancedwith various multimedia software.9 It includesthe building's interior elements (floor, walls,doors, benches, plants, elevator, etc.), as well asthe exterior visible from the window (courtyard, trees, other buildings, walkways, citybackground).
Virtual environments often have a "cartoonlike" feel, because the appearance of geometryis only modeled using color and reflective qualities to reduce the computational load. A technique called "texture mapping" provides amore realistic appearance, but is computationalintensive. Textures are digital images and photographs that are pasted onto the geometry andsimulate the complex appearance of real objectsin a more natural way. Using digital cameras,we captured elements of the in vivo environment and transferred these images into the virtual model. In addition, libraries of textures
were used for appearance enhancement (e.g.,fabric on elevator walls, wall textures, and floortiling).
Use of state-of-the-art technology helps tominimize differences between VR exposureand in vivo exposure. Our model is experienced in a CAVE (CAVE Automatic VirtualEnvironment) as developed by the Universityof Illinois at Chicago and is now commerciallysold by Pyramid Systems. The CAVE is a projection-based VR system that surrounds theviewer with four 10' X 10' screens, arranged toform three walls and the floor of a cube (Figure 1). The viewer wears liquid crystal shutterglasses and a six-degrees-of-freedom head-tracking device. As the viewer moves inside theCAVE, a Silicon Graphics Onyx computer calculates the correct perspective projections foreach wall, based on head-tracking measurements that describe the viewers location in thevirtual model. The images for the left and theright eye are projected in a rapid, alternatingsequence. The shutter glasses alternately blockthe left and the right eye in synchronizationwith the projection sequence, giving the appropriate 3D image. The CAVE environment isa superior system when compared with the traditionally used head-mounted display (HMD)that is worn like a helmet. An HMD is extremely intrusive, uncomfortable to wear, andprovides an unnatural restricted field of view(FOV). In contrast, the lightweight shutterglasses and the surrounding walls of the CAVEprovide a very wide FOV that properly stimulates the human peripheral vision and facili-
FIG. 1. CAVE projection system.
RECREATING REALITY IN VIRTUAL ENVIRONMENTS 165
tates significantly orientation and navigation inthe virtual world. The stereo projection on thefloor enables the viewer to perceive objectabove and below the floor. Looking down froma window or approaching a cliff can become avery convincing experience. The viewer canalso see real objects in the CAVE like his/herown body or that of others, more closely simulating the effect of therapist presence in vivo.The use of these technologies allows our system to make the experience of our virtualenvironment as close to reality as is currentlyfeasible. To control for the remaining shortcomings of the CAVE experience in the comparison with in vivo exposure, we will placecomparable restrictions on the in vivo participants by asking them to wear shutter glassesand not to touch physical objects like walls,since subjects in the virtual environment areunable to feel any objects of their surroundingvirtual world.
provement among all conditions, as well as examine the ability of virtual exposure to generalize to real life.
Many other variables may show differencesbetween the virtual and in vivo groups. Physiologic measurements and subjective measurements of the impact of these exposures will allow us a sense of the effect on anxiety. We willalso measure demographic variables, prior exposure to different types of technology, andother measures of the impact of "virtuality."The literature contains several attempts tomeasure the extent to which a subject feels true"presence" in a virtual environment.17-19 Wewill examine how such variables influence bothmeasures of anxiety and measures of treatmentefficacy.
IMPLEMENTATION IN HARDWAREAND SOFTWARE
Our experimental protocol is designed toclarify variables that separate the treatment inthe CAVE to treatment in the actual environment.10 This requires us to not only examinethe subjects using traditional assessments thatare used in the psychiatric literature, but to alsouse other assessment tools to attempt to measure other variables that are not usually evaluated. Subjects are selected as having diagnosisof specific phobia11 using the Structured Clinical Interview for DSM-IV (SCID),12 then givena battery of questionnaires to assess their acrophobia.13"16 A behavioral approach test in theelevator lobby establishes a uniform measureof their ability to tolerate the in vivo situation.They are then randomized to a treatment condition of either in vivo exposure, CAVE exposure, or a control condition of relaxation training. In each condition, a therapist takes thesubject through a standardized training for 90min, accompanied by physiologic monitoringof heart rate, respiration rate, and galvanic skinresponse. After completion of exposure training, each subject is returned to the in vivo situation for a repeat behavioral approach test.This will allow a direct comparison of im-
Our process of modeling and experimentaldesign has been instructive, teaching us lessonsabout the interactions of hardware, software,and psychiatric knowledge and the challengesof integrating them together in producing practical virtual reality. The interaction is complex.If we extrapolate from other models of technology, such as that of the Internet,20 we recognize that each of these three elements are dependent upon the others, yet they each havetheir own trajectory of development with hardware moving the fastest, software quickly following, and psychological models lagging behind; the difficulties we have faced in ourmodeling process give concrete example ofthis.
Computer hardware changes continuouslyas industry competes to create better andcheaper products. Processing speed doublesevery 18 months,21 allowing faster rendering of3D models and more effective inclusion of techniques like texture mapping. Our aim of usinga state-of-the-art system like the CAVE is anexample of an attempt to take advantage ofthese trends. Unfortunately, this rapidity of development tends to make each hardware solution isolated, not easily integrated with otherdevices and outstripping the development ofsupporting software. Although the CAVE sys-
166 HUANG ET AL.
tern is commercial, each installation requiresunique adjustments. In our case, setup of theCAVE has been a long process with frequenttesting for realigning projectors, connecting thecomputer to the projectors, or calibrating thetracking devices in the 3D space. We are exploring the interferences between the electronicfields of the tracking system and of our physiologic monitoring equipment as the CAVE wasnot designed for the use of such systems inside.We are also continuing to get additions to theCAVE as other hardware is added to increasethe speed of information flow. Unfortunately,we have already found that such changes haveinvalidated some of our current software, necessitating rewrites. We are always attemptingto adjust for ongoing changes in hardware.
We have seen in other ways how hardwaredevelopment outpaces the development ofsupporting software. The CAVE is programmed in either C or C + + using specific library calls that control different hardware elements. Higher level libraries (EVL, Performer,Open GL, and others) allow easier programming for the CAVE environment. Recently, astandard for the development of virtual environments has been developed for the WorldWide Web (WWW). This standard, calledVRML (Virtual Reality Modeling Language), isnot restricted to the WWW, but can be utilizedfor any VR application. VRML22'23 defines avirtual environment by geometry, appearance(e.g., color, textures), and illumination, and includes functions for animations, interactions,and behavior scripting. Translators that convert a VRML application into a CAVE application are under development and will, in the future, make VRML an excellent standard for thedefinition and exchange of virtual environments. In the meanwhile, we are coding the interactions of our model in both Performer andVRML, requiring some reduplication of work.
The challenges of keeping up with advancing standards of hardware and software andinterconnecting them offer a contrast to dealing with a lack of standards in how we organize our understanding of the relevant psychological and social factors that shouldinfluence the creation of a virtual environment.Clearly, the potential number of such factors is
endless. In designing our environment fortreatment of acrophobia, our primary consideration has been trying to make the virtual environment as similar to the real environment aspossible. Our choices have been made based onsubjective experience, reiteratively entering thevirtual environment, "feeling" what seems"right," and then making appropriate changesin system programming. This design process issubject to errors from the fact that many perceptions of the environment are unconscious intheir influence and, therefore, ignored. In addition, we intentionally left out some parts ofthe experience because of the difficulty in accurately replicating them. We do not simulatereflections off of the window, window tinting,or dust, which all commonly serve as cues thata barrier is present. Thus far, there is little research on how important choices like "windowtinting" or the presence of reflections are inmaking virtual reality simulate reality.
Our experimental procedure is aimed at determining where the differences exist betweenvirtual reality and reality in the context of acrophobia treatment. It compares the treatment ofsubjects in a real location to treatment in a VRsimulation of that real location. Use of minimally intrusive technology and appropriateprotocol design allow us to make treatment inthese two settings as similar as possible, permitting us to examine VR itself as an independent variable. This requires much work as wehave seen in our attempts to use the CAVE andVRML.
The design of this project and the difficultiesof realizing it suggest some of the challengeswe face in using virtual reality in mental health,as well as important considerations for futurework and research. Keeping up with continuously changing hardware and software requires time and dedication. We need educationin how such learning can be integrated into ourusual work, as well as how to define our ownexpertise and interface it with that of technological experts.24 As mental health experts, weneed to focus on the design of experimental
RECREATING REALITY IN VIRTUAL ENVIRONMENTS 167
systems and protocols that can be used to investigate questions of interest, independent yetcognizant of the underlying technology used torealize these designs. Perhaps the greatest challenge before us is how we can organize our expertise in a way in which other researchers invirtual reality and computer interface designcan access it to incorporate it into their workand research. We need to work on models thatdefine the mental attributes that should be considered in creating virtual environments. Although work has been done on the effect of environmental variables on factors related toperception, little exists on their impact on otherpsychological factors and how they relate toour current knowledge of mental illness, personality attributes, and normal mental function.
Computer technologies now face the mentalhealth field with new challenges and opportunities.25 We hope that a thoughtful explorationof the design process of mental health applications in virtual reality will encourage the experts in this area to work together to furtherdevelop ideas about other standards we need.Such work will be essential to encourage research and increase our understanding and useof virtual environments in psychiatric and psychological applications.
1. Glantz, K., Durlach, N.I., and Aviles, W.A. (February1997). Virtual reality (VR) and psychotherapy: Opportunities and challenges. Presence: Teleoperators andVirtual Environments, 6(1), 87-105.
2. Rothbaum B.O., Hodges, L.F., Watson, B.A., Kessler,G.D., and Opdyke, D. (1996). Virtual reality exposuretherapy in the treatment of fear of flying: A case report. Behaviour Research and Therapy, 34(5/6), 477^181.
3. North, M.M., North, S.M., and Coble, J.R. (February1997). Virtual environments psychotherapy: A casestudy of fear of flying disorder. Presence: Teleoperatorsand Virtual Environments, 6(1), 127-132.
4. Carlin, A.S., Hoffman, H.G., Weghorst, S. (February1997). Virtual reality and tactile augmentation in thetreatment of spider phobia: A case study. BheaviourResearch and Therapy, 35(2), 153-158.
5. Rothbaum, B.O., Hodges, L.F., Kooper, R., Opdyke,D., Williford, J., and North, M.M. (1995 Summer). Virtual reality graded exposure in the treatment of acrophobia: A case report. Behavior Therapy, 26(3), 547-554.
6. Rothbaum, B.O., Hodges, L.F., Kooper R., Opdyke, D.,Williford, J., and North, M.M. (April 1995). Effectiveness of computer-generated (virtual reality) gradedexposure in the treatment of acrophobia. AmericanJournal of Psychiatry, 152(4), 626-628.
7. Lamson, R.J. (1997). Virtual Therapy: Prevention andTreatment of Psychiatric Conditions by Immersion in Virtual Reality Environments. Montreal: Editions de l'cole Polytechnique de Montral.
8. Mortenson, M.E. Geometrie modeling (2nd ed.). NewYork: Wiley, 1997.
9. Huang, M.P., and Alessi, . (1996). Tools for developing multimedia in psychiatry. In M. Miller, K. Ha-mond, & M. Hile (eds.), mental health computing. NewYork: Springer-Verlag, pp. 322-341.
10. Huang, M.P., Himle, J., Beier, K., and Alessi, N.E. (inpress). Comparing virtual and real worlds for acrophobia treatment. In Medicine Meets Virtual Reality 6.Amsterdam: IOS Press.
11. American Psychiatric Association. (1994). Diagnosticand Statistical Manual of Mental Disorders (4th ed.).Washington DC: APA Press.
12. First, M.B., Spitzer, R.L., Gibbon, M., and Williams,J.B.W. (1997). Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I), Clinician Version. Washington, DC: American Psychiatric Press.
13. Cohen, D.C. (1977). Comparison of self-report andovert-behavioral procedures for assessing acrophobia. Behavior Therapy, 8, 17-23.
14. Marks, I.M., and Mathews, A.M. (1979). Brief standard self-rating for phobic patients. Behaviour Researchand Therapy, 17(3), 263-267.
15. Michelson, L. (1986). Treatment consonance and response profiles in agoraphobia: The role of individual differences in cognitive, behavioral and physiological treatments. Behaviour Research and Therapy,24(3), 263-275.
16. Menzies, R.G., and Clarke, J.C. (May 1993). The etiology of fear of heights and its relationship to severityand individual response patterns. Behaviour Researchand Therapy, 31(4), 355-365.
17. Sheridan, T.B. (Spring 1996). Further Musings on thePsychophysics of Presence. Presence: Teleoperators andVirtual Environments, 5(2), 241-246.
18. Welch, R.B., Blackmon, T.T., Liu, ., Meilers, B.A.,and Stark, L.W. (Summer 1996). The effects of pictorial realism, delay of visual feedback, and observerinteractivity on the subjective sense of presence. Presence: Teleoperators and Virtual Environments, 5(3),263-273.
19. Hendrix, C, and Barfield, W. (Summer 1996). Presence within virtual environments as a function of visual display parameters. Presence: Teleoperators andVirtual Environments, 5(3), 274-289.
20. Huang, M.P., and Alessi, N.E. duly 1996). The Internet and the future of psychiatry. American Journal ofPsychiatry, 153, 861-869.
21. Schaller, R.R. (June 1997). Moore's Law: Past, present,and future. IEEE Spectrum, 34, 52-59.
168 HUANG ET AL.
22. Virtual Reality Modeling Language ISO/IEC Draft International Standard 14772-1 (December 1997).
23. Wagner, M. (July 7, 1997). Real-world uses for 3-DWeb spec. Computerworld, 31, 45 +
.24. Huang, M.P., and Alessi, N.E. (in pess). An informatics curriculum for psychiatry. Academic Psychiatry.
25. Alessi, ., Huang, M., and Quinlan, P. (1997). 2005:Information Techology impacts psychiatry. In L.J.Dickstein, M.B. Riba, and J.M. Oldham (Eds.), American Psychiatric Press Review of Psychiatry (Vol. 16).
Washington DC: American Psychiatric Press, pp.VI69-87.
Address reprint requests to:Dr. Milton Huang
1500 E. Medical Center Dr., TC 3502/Box 0390Ann Arbor, MI 48109-0390, USA