Immersive Virtual Humans for Immersive Virtual Humans for Educating Medical and Pharmacy Educating Medical and Pharmacy

Communication SkillsCommunication SkillsA. Raij, A. Kotranza, B. Rossen, J. Chuah, L. Cao, B. A. Raij, A. Kotranza, B. Rossen, J. Chuah, L. Cao, B. LokLokComputer and Information Science and EngineeringComputer and Information Science and EngineeringJ. CendanJ. CendanSurgerySurgeryD. Beck, C. KimberlinD. Beck, C. KimberlinPharmacyPharmacyR. Ferdig, J. DePietroR. Ferdig, J. DePietroEducationEducation

H. Shah, A. Gehlot, A. Gucwa, B. Rossen, A. Foster, C. H. Shah, A. Gehlot, A. Gucwa, B. Rossen, A. Foster, C. Kalaria, D. LindKalaria, D. LindSurgical OncologySurgical Oncology

K. JohnsenK. JohnsenEngineeringEngineering

L. Cuty-Ruiz, M. Feldmen L. Cuty-Ruiz, M. Feldmen MedicineMedicine

Project OverviewProject Overview Started Spring ’04Started Spring ’04 n > 2100 health care n > 2100 health care

studentsstudents One of the most One of the most

experienced VH/VR experienced VH/VR experiencesexperiences

Integration at five institutesIntegration at five institutes– University of FloridaUniversity of Florida– University of Central University of Central

FloridaFlorida– Medical College of GeorgiaMedical College of Georgia– University of GeorgiaUniversity of Georgia– Keele University (U.K.)Keele University (U.K.)

21 person team21 person team– Computer ScienceComputer Science– MedicineMedicine– EducationEducation– PsychologyPsychology

Virtual PatientsVirtual Patients

Speak to a life-size Speak to a life-size VHVH

Interview a virtual Interview a virtual patient via the webpatient via the web

Visit a virtual patient Visit a virtual patient in Second Lifein Second Life

Developing Virtual Character Developing Virtual Character Conversation ModelConversation Model

• Virtual People Virtual People FactoryFactory– Web-based interfaceWeb-based interface– End-user creation of End-user creation of

contentcontent– Pharmacy patientPharmacy patient– Create a VP in Create a VP in

two weekstwo weeks• In 6 monthsIn 6 months

– Over 2000 Over 2000 interactionsinteractions

– 12 end-user created 12 end-user created scenariosscenarios

• End-user created End-user created scripts can reach scripts can reach 85% accuracy85% accuracy

After-Action After-Action ReviewReview

• StudentsStudents– Students received email with linkStudents received email with link– Sample student (Brent/45169)Sample student (Brent/45169)– Self-directed review of content, video, Self-directed review of content, video,

and feedbackand feedback– Compare against expertsCompare against experts

• EducatorsEducators– Educators can review student Educators can review student

interactionsinteractions– Data from studyData from study– Filter based on user backgroundFilter based on user background

• Level of expertiseLevel of expertise• GenderGender• Educator-defined metricsEducator-defined metrics

– Identify trends and outliersIdentify trends and outliers

Physical Examinations of Physical Examinations of Virtual Human PatientsVirtual Human Patients

Mixed reality humansMixed reality humans– Passive-haptic interface to life-Passive-haptic interface to life-

sized virtual humansized virtual human– ApplicationsApplications

Clinical breast examClinical breast exam Prostate exam (upcoming)Prostate exam (upcoming)

Medical students canMedical students can– Concurrently practice physical Concurrently practice physical

examination and examination and communication skillscommunication skills

– Get real-time feedback of exam Get real-time feedback of exam performanceperformance CoverageCoverage PressurePressure

Abnormal Findings: Cranial Abnormal Findings: Cranial NerveNerve

Multi-tool interface to Multi-tool interface to VHVH– Wii Remote controls Wii Remote controls

multiple virtual tools multiple virtual tools used in eye examused in eye exam

– Gestural and tool Gestural and tool input, in addition to input, in addition to speechspeech

Medical students canMedical students can– Experience interactions Experience interactions

difficult to curricularly difficult to curricularly plan or are infrequent plan or are infrequent occurrencesoccurrences

– Learn how to use Learn how to use neurological tests to neurological tests to diagnose a patient diagnose a patient with a vision disorderwith a vision disorder

Biases with Virtual HumansBiases with Virtual Humans

“Could this be cancer?”

• Small visual change => large behavioral change in Empathy

• Empathy correlates with real world bias • People treat the VHs below differently

Community Outreach: Community Outreach: Museum of Science and Museum of Science and

IndustryIndustry Science museum in Science museum in

Tampa, FLTampa, FL 55thth largest in US (75,000 largest in US (75,000

sq ft.)sq ft.) 1 million guests1 million guests Integrate a VH Integrate a VH

interactioninteraction– The Amazing You exhibitThe Amazing You exhibit– Lifelong wellness Lifelong wellness

informationinformation– Fall ’09Fall ’09

Inspire, InformInspire, Inform Explore race, VH biasExplore race, VH bias