*Corresponding author. Fax: #65-779-3457.E-mail address: douglas@mip.nus.edu.sg (D. Gourlay).

Computers & Graphics 24 (2000) 695}699

Dynamic Medical Visualization

Virtual reality and telemedicine for home health care

D. Gourlay*, K.C. Lun, Guan Liya

Medical Informatics Programme, Department of Community, Occupational and Family Medicine, National University of Singapore,Singapore 119260, Singapore

Abstract

The rapid increase in the numbers of old people in society and the corresponding strain on medical resources will resultin greater importance being placed on home health care. Advances in communications technologies will enable the oldand disabled to receive health care in the comfort and safety of their homes, eliminating costly and potentially hazardousjourneys to hospital. We have developed a prototype networked virtual system to aid the cognitive and motorrehabilitation of patients su!ering from conditions such as traumatic brain injury, stroke and Parkinsons disease. Ourvirtual kitchen aids the retraining of patients in important daily living skills such as meal preparation tasks. ( 2000Elsevier Science Ltd. All rights reserved.

Keywords: Virtual reality; Telemedicine; Cognitive rehabilitation; Traumatic brain injury

1. Introduction

A virtual environment (VE) enables a user to navigateand interact with a three-dimensional computer-gener-ated environment in real time. Varying degrees of realismcan be obtained depending on the hardware and softwareused. A desktop system using a standard PC will notgenerate the same level of realism as an immersive systemin which the user dons a headset and becomes `immer-seda in the VE. Although perceived by the general publicas a technology suited mainly to games, recent years hasseen a host of more serious applications being developed.Many recent research papers have focussed on the prom-ise that VEs hold for the treatment of patients withpsychological disorders and cognitive impairments[1}4]. These disorders can be caused by traumatic braininjury (TBI) or through stroke and disease. It has beenestimated that, in the US alone per year, 100,000 peoplesu!er varying degrees of permanent disability from TBI[5]. The most susceptible group are young males agedbetween 14 and 24 years and so the requirement fore!ective rehabilitation to ensure a meaningful future life

becomes very important. Some of the more commondisabilities include memory loss, concentration prob-lems, spatial disorientation, problems with vision, hear-ing, smell, taste, speech, and emotional di$culties such asanxiety and depression.

Many traditional methods of assessment for the cogni-tively impaired use either simple motor tasks or penciland paper psychometric tests such as the Vandenbergmental rotations test. For example, in cases of visualneglect, the patient is asked to indicate the centre ofa straight line or to mark all cases of a speci"c symbol ona sheet of paper, as quickly as possible. One commoncriticism of these tests is that the patient is not beingtested in a real world task. A study by Andrews et al.supports the lack of ecological validity in current assess-ment tests [6]. A VE enables the patient to be assessedand rehabilitated in life-like situation in which there istotal control over all content. Importantly, all move-ments and interactions can be recorded in a database andanalyzed when required. Rizzo et al. describe other ad-vantages which include adding gaming factors to increasemotivation, and a safer learning environment thatreduces risks caused by errors [7].

The recent improvements in communications techno-logy and the explosive growth of the internet haveensured that telemedicine will be a major means ofimproving health care in the next century. We have

0097-8493/00/$ - see front matter ( 2000 Elsevier Science Ltd. All rights reserved.PII: S 0 0 9 7 - 8 4 9 3 ( 0 0 ) 0 0 0 7 2 - 8

Fig. 1. VR glove grasping a virtual ball.

created a prototype system, which combines both virtualreality (VR) and telemedicine, and which enables adoctor/specialist to monitor, from a remote location, therehabilitation of one or more patients.

2. Side e4ects of exposure to VEs

Cybersickness is the common name to describeadverse e!ects of exposure to a VE. Some symptomsinclude motion sickness, visual disturbance [8] andproblems with coordination and balance. Fatigue isanother common e!ect reported by users [9]. A well-documented cause for many of the problems is the delayin updating the image within the head-mounted-display(HMD) upon head rotation by the user. This delay iscaused by a number of factors, most importantly theinherent latency in the position tracking system. Al-though a lot of research in this area involves case studieswith healthy individuals the results are especially rel-evant when VEs are being considered for use by subjectswho already have some kind of medical impairment. Theauthors, from personal experience, "nd it very easy tobecome dizzy simply by watching a rotating viewpointon a computer monitor. For someone who is neurologi-cally impaired in some way the e!ects could be verydistressing resulting in disorientation and greatly increas-ing the risk of falls and injuries. There is very littlepublished literature that speci"cally examines the e!ectson neurologically impaired individuals of exposure toa VE. One study by Pugnetti et al. "nds no di!erence inafter-e!ects between impaired and non-impaired people[10] although clearly the speci"c impairment and thedegree of disability are important.

3. User interaction

An important point to consider for disabled indi-viduals is the means of user navigation and interactionwithin the VE. Commonly a mouse, joystick or the key-board are used to control input i.e. move forwards, back-wards, rotate, etc. Since all these methods involve use ofthe hands this may be unsuitable for certain individuals.Alternatives can be considered such as HMDs, footpedals, voice recognition, and even `china joysticks [11].If there are serious problems with communicating withthe VE then there may be a loss of motivation and alsoreduced e!ectiveness of rehabilitation due to the extrae!ort required.

We have developed a home-brew VR glove to enablethe patient to pick and move objects in our VEs. The VRglove sensory input device consists of a glove, bendsensors, motion-trackers and a motion-translation unit.The variable resistive bend sensors are attached to each"nger of an ordinary glove. Signals are sent from the

bend sensor to the motion translation unit. Inside thetranslation unit are a pair of Analog-to-Digital chips anda small micro-controller. This micro-controller processesthe signals and translates them into a digital signal rangeand sends them to the VR application through the serialport. Position and orientation tracking of the glove isachieved using a commercial electromagnetic trackingsystem. Our glove is basic and su!ers from `jitteryaoutput and a limited resolution. For greater performanceand accuracy it will be necessary to use commercialgloves which use optical "bre based sensors and whichincorporate tactile and force feedback.

4. Spatial skills project

This project required the user to use our VR glove topick and move objects within a VE. Collisions weredetected at the tips of each "nger. A commercial electro-magnetic position tracking system was used to record theposition and orientation of the hand. Fig. 1 shows a vir-tual ball being grasped. A screen shot from a simpleapplication to assess spatial skills is shown in Fig. 2. Thepatient must move each object into the correct holetrying, as far as possible, not to hit the object against thesides of the hole. All collisions and movements are re-corded and can be played back. Additionally the path,or trajectory, of movement can be displayed andanalysed. Currently, there is no immersion with thisapplication and this causes di$culties for the user whenestimating depth. We plan to greatly improve the e!ec-tiveness by using a HMD or 3D glasses to provide theuser with depth cues. Additionally, a glove with forcefeedback is necessary to constrain the user's hand at thepoints of collision.

696 D. Gourlay et al. / Computers & Graphics 24 (2000) 695}699

Fig. 2. Application to assess spatial skills.

Fig. 3. Pouring water using VR glove.

Fig. 4. User is prompted after a certain time.

5. VR kitchen

In order to re-train stroke patients and victims oftraumatic brain injury to re-learn daily living tasks wehave developed a VR kitchen. This VE contains a sink,kettle, cup, co!ee jar, microwave, power sockets, cup-boards and drawers. We are currently focussing on thevarious tasks required to prepare a cup of co!ee and planto implement a whole series of kitchen tasks such aspreparing a microwave meal and using a stove. Ourkitchen is fully interactive. The user can turn the tap and"ll the kettle with water. Then the kettle can be poweredon, and the water boiled, by connecting the cable to thepower socket. The hot water can now be poured intoa cup as shown in Fig. 3. A kitchen environment canpresent considerable dangers to a cognitively impairedindividual such as the danger of being burnt from scald-ing water, switching on a kettle which contains no wateror forgetting to switch o! an oven or boiling kettle. A VEenables a patient to become familiar with importanttasks in a safe learning environment. The patient can usethe VE unsupervised with the program monitoring thepatient's progress and giving prompts as required. This isillustrated in Fig. 4.

Attentional di$culties can be caused by injuries toseveral parts of the brain and are often the chief disabili-ties in stroke or traumatic brain injury victims. There areseveral di!erent types of attention [12] and our VRkitchen o!ers the opportunity for a therapist to monitorseveral of these.

The patient must focus on the kettle boiling and mustbe aware when it automatically switches o!, thus en-abling the water to be poured into the cup. If there isa long delay before the patient reacts then there may bedi$culties with focussed attention. Sustained attention,or concentration, can be tested by creating a scenario

involving the patient boiling an egg. The patient mustconstantly monitor the state of the boiling water overa period of time, ensuring the water does not boil over or,alternatively, go o! the boil. Rizzo et al. suggest applica-tions to test di!erent types of attention [7].

6. A preliminary case study

We have implemented a preliminary case study usingour virtual kitchen involving a few patients from a near-by hospital. The objective of the study was to monitor thepatients' ability to navigate and interact with the VEusing a computer mouse and our home-brew VR glove.The system comprised a high-end PC which displayedour VR kitchen, and a 17 in monitor. A head-mounteddisplay was not used although we will consider its usagein future. We examined nine patients who had mild to

D. Gourlay et al. / Computers & Graphics 24 (2000) 695}699 697

Fig. 5. Networked system using Singapore ONE.

moderate degrees of cognitive impairment and who were,in the opinion of the occupational therapists, good candi-dates. The patients were required to select objects reques-ted by the therapists using both devices. Selection usingthe mouse presented few problems although some prob-lems, were encountered with the glove. These were due toproblems perceiving depth caused by a lack of immer-sion, and therefore depth-cues. The position trackingsystem performed adequately as long as the patient'shand was at least 2 ft away from the monitor. After somepractice most, especially the younger patients, were ableto adapt and use both devices satisfactorily. We havebeen encouraged by the results and plan to soon imple-ment a more comprehensive study whereby the patientsare required to plan and implement the steps required toprepare a cup of co!ee.

7. Networked virtual reality rehabilitation system

NASA de"nes telemedicine as `the integration of tele-communications technologies, information technologies,human-machine interface technologies, and medical caretechnologies for the purpose of enhancing health caredeliverya [13]. The primary purpose of telemedicine is toimprove patient care by providing greater access to medi-cal consultation, diagnostic evaluation and medical in-formation exchange. It enables communication betweenhealthcare providers and their patients regardless of geo-graphic distance and uses such technologies as the world-wide-web, satellite communications and multimedia soft-ware. In particular, it allows health care in areas of lowdemography such as many parts of the African continentor at the site of natural disasters such as earthquakes.

Several remote monitoring systems are being developedwhich take advantage of improved communications tech-nologies. The current condition of asthma patients can betested remotely using a laptop PC, a spirometer andwireless communication to a remote server [14]. Ourproposed system enables a remotely located specialist tomonitor and interact with 1 or more patients as theyperform cognitive rehabilitative tasks in a VE.

We plan to make use of a high-speed network such asSingapore ONE [15], the broadband network that linksall o$ces, homes and institutions in Singapore. The largebandwidth is over 100 times faster than a normal28.8 kbps analogue modem and this high speed shouldensure almost real time interaction between multipleusers with in a VE (Fig. 5). Our VR kitchen will be runover this network. A remotely located doctor can interactwithin the environment and can illustrate the task to beperformed by the patient. A mistake by the patient can beplayed back by the doctor and paused at the criticalpoint. Communication between patient and doctor musttake into account the patient's disabilities although inmany cases a speaker phone could be used. We haveimplemented a chat box to enable textual communication.

A database will be used to store all movements andinteractions performed by each patient to enable acomplete reconstruction of any session. This enables thedoctor to analyze past performance and results, or toplay back perviously recorded sessions which were notviewed in real time. In addition we envisage the databasesummarising data for the patient, creating tables andgraphs, and detailing common mistakes perhaps sugges-ting possible methods of improvement. The rate ofprogress will be evaluated and the doctor can beinformed by a computer-generated e-mail.

698 D. Gourlay et al. / Computers & Graphics 24 (2000) 695}699

There is also the possibility to create multi-user VEswhich would be especially bene"cial to housebound pa-tients who may feel isolated. For example, 4 or 5 patientscould meet together in a virtual pub. Each person wouldbe represented as a realistic 3-D human with a facialrepresentation taken from a photograph.

8. Conclusion

Home health care will be a very important cost-savingmethod of treatment in the new millenium. There will bea huge strain on medical resources such as personnel andhospital beds due to the growing number of elderlypeople who are, by a long way, the largest users ofhealth-care services. Our networked system, in conjunc-tion with virtual reality, is a practical solution to thisproblem by enabling health care to be performed athome. VR technology holds great promise as a next-generation health care tool. The bene"ts are consider-able, including the capability to create VEs in which allcontent is controlled and where the patients' actions aremonitored and recorded. However, although VR seemsas though it has been around a long time, it is still ata very early stage in its development with considerableproblems to be overcome such as the realistic simulationof force, touch and smell. Another major drawback iscost. A high-quality system comprising a PC, a VR glove,a tracking system and a HMD could easily exceed 100thousand US $. Additionally the undesirable e!ects ofcybersickness caused by hardware limitations are prob-lematic. Fortunately, in the future, the technology willimprove, reducing the e!ects of cybersickness, and allow-ing the creation of ever more realistic VEs.

Acknowledgements

The authors would like to thank Dr. David Warner forhis advice and guidance on the telemedicinal aspect of theproject. The study is funded by a Grant (RP 3960348)from the National Science and Technology Board andthe Academic Research Fund of the National Universityof Singapore.

References

[1] Strickland D. Virtual reality for the treatment of Autism.In: Riva G, editor. Virtual reality in neuro-psycho-physi-ology. Amsterdam: IOS Press, 1997. p. 81}6.

[2] Carlin AS, Ho!man HG, Weghorst S. Virtual reality andtactile augmentation in the treatment of spider phobia:a case report. Behaviour Research and Therapy 35, 2, 99.153}8. 1997.

[3] Rose FD, Attree A, Brooks BM, Johnson DA. Virtualenvironments in brain damage rehabilitation: a rationalefrom basic neuroscience. In: Riva G et al., editors. Virtualenvironments in clinical psychology and neuroscience.Amsterdam: IOS Press, 1998.

[4] Grealy MA, Johnson DA, Rushton SK. Improving cogni-tive function after brain injury: the use of exercise andvirtual reality. Archives of Physical Medicine and Reha-bilitation 1999;80(6):661}7.

[5] Kraus J, Sorenson S. Epidemology. In: Silver J, YudofskyS, Hales R, editors. Neuropsychiatry of traumatic braininjury. Washington, DC: American Psychiatric Press, Inc.,1994.

[6] Andrews TK, Rose FD, Leadbetter AG, Attree EA,Painter J. The use of virtual reality in the assessment ofcognitive ability. The European Context for AssistiveTechnology: Proceedings of the second TIDE Congress,Amsterdam, 1995. p. 276}9.

[7] Rizzo AA, Buckwalter JG. Virtual reality and cognitiveassessment and rehabilitation: the state of the art. In: RivaG, editor. Virtual reality in neuro-psycho-physiology: IOSPress, 1997. p. 123}45.

[8] Stanney K, Salvendy G. Aftere!ects and sense of pres-ence in virtual environments: formulation of a researchand development agenda. International Journal ofHuman-Computer Interaction (special issue) 1998;10(2):135}87.

[9] Kennedy RS, Massey CJ, Lilienthal MG. Incidences offatigue and drowsiness reports form 3 dozen simulators:relevance for the sopite syndrome. Workshop on Simula-tion and Interaction in Virtual Environments. Iowa City,July 1995.

[10] Pugnetti L, Mendozzi L, Motta A, Cattaneo A, Barbieri E,Brancotti A, Cazullo CL, Immersive VR for the retrainingof cognitive defects. Medicine Meets Virtual Reality3 Conference, San Diego, CA, 1995.

[11] Warner D. Eyal Sherman Webumentary, 1996. http://www.pulsar.org/eyal/index.html.

[12] Sohlberg MM, Mateer CA. Introduction to cognitiverehabilitation: theory and practice. New York: TheGuildford Press, 1989.

[13] NASA Telemedicine web site, 1997. http://www.hq.nasa.gov/o$ce/olmsa/aeromed/telemed/welcome.html.

[14] Finkelstein J, Hripcsak G, Cabrera M. Telematic systemfor monitoring of asthma severity in patients' homes.MedInfo 1998, Part I, p. 272}6.

[15] Singapore ONE web site. http://www.s-one.gov.sg/html/mainmenu.html.

D. Gourlay et al. / Computers & Graphics 24 (2000) 695}699 699