Special Topic: Virtual Reality, Mixed Reality, Augmented Reality, Augmented Virtuality based Collaborative Virtual Environments Dr. Xiangyu Wang
Agenda Today (important)2:00-3:00 lecture3:00-3:45:Group 1-5 use Secondlife to do design session 1 (design a studio with Taiwan student).Group 6-10 discusses your face-to-face design session results with instructors, OR discuss the workstation design you will do next week.3:45-4:30:Group 6-10 use Secondlife to do design session 1 (design a studio with Taiwan student).Group 1-5 discusses your face-to-face design session results with instructors, OR discuss the workstation design you will do next week.
OutlineMixed Reality (MR)Virtual Reality (VR)Augmented Reality (AR)Augmented Virtuality (AV)
Mixed RealityMixed Reality (MR) is a term that has been used to refer to the entire spectrum of situations that span the continuum between virtual reality and actual reality.
Mixed Reality ContinuumMixed reality includes augmented reality, augmented virtuality, and other mixed configurations.
Mixed Reality (MR)
Mixed Reality (MR)
Reality-Virtuality (RV) Continuum
Virtual RealityWhat is Virtual Reality?Brooks (1999) defines it as: [an] experience .. in which the user is effectively immersed in a responsive virtual world Sherman and Craig (2003) defines it as a medium composed of interactive computer simulations that sense the participants position and actions and replace or augment the feedback to one or more senses, giving the feeling of being mentally immersed or present in the simulation (a virtual world)Immersive, Semi-immersive, Non-immersive
Virtual RealityFour Key Elements in Experiencing Virtual RealityA virtual world an imaginary space, often (but not necessarily) realized through a medium (rendering pipeline, display, etc.)Immersion (physical and mental) having a sense of presence within an environment; this can be purely a mental state, or can be accomplished through physical means.Mental Immersion: a state of being deeply engaged, with suspension of disbelief.Physical Immersion: bodily/physically entering into a projected areaSensory feedback: visual/aural/haptic feedback to a participantInteractivity: in a virtual reality experience, participants are able to move around and change their viewpoint, generally through movements of their head.
Virtual RealityVR Technology: DisplaysHead Mounted Displays: Tiny displays, mounted on the head, fully immersive, needs tracker to track the position and orientation of users head.
Virtual RealityProjection-Based Systems: A front-projected or rear-projected system in which the user is surrounded by images, projected onto large screens, create high immersion sense.
Virtual RealityBlue-c system by ETH institutes: collaborative virtual reality
Virtual RealityWorkbenches: flat, rear-projection screens that display images in stereo, and can be set up in a horizontal or tilted position, create partially immersion and high object presence.
Virtual RealityHot Open Challenges:Interactive more effectively with virtual worlds:ManipulationWayfindingMeasuring the illusion of presence
Video for Virtual RealityVideo: Collaborative Immersive Networked Environments (CINE)http://www.youtube.com/watch?v=mVQrhsjOlwU
Augmented RealityAugmented Reality (AR) is a technology or an environment where the additional information generated by a computer is inserted into the users view of real world scene. The augmenting information may consist of virtual geometric objects placed into the environment, or a display of non-geometric information about existing real objects such as distribution of room temperature.
Augmented RealityOptical see-through: Partly transmissive optical combiners are placed in front of the users eyes, allowing the user to see in them the reflection of virtual images portrayed on miniature displays in their line of sight.
Augmented RealityUses miniature cameras to capture the view of the world that would be seen by each eye. The video images of the real world are then combined with the computer-generated images of the virtual world, to create augmented-reality images that can be displayed on a traditional (non see-through) HMD
AR ApplicationsApplications of Augmented RealityMedical: Display of acquired data from ultrasound, superimposed over the live patientGoal - to facilitate minimally invasive operations such as fine needle biopsies, and laproscopic surgeries
Augmented Reality in Cooperative DesignCollaborative Virtual Environment Mixed Reality view through head-mounted displayMRCVE collaboration scenario
Augmented Reality in Cooperative DesignAR-based Videoconferencing
Video for Augmented RealityVideo: Magicbookhttp://www.hitlabnz.org/wiki/Video_-_MagicBookVideo - AR Tennishttp://www.hitlabnz.org/wiki/Video_-_AR_Tennis Video: AR for Urban Designhttp://www.hitlabnz.org/wiki/Video_-_AR_Urban_DesignVideo: Shared Design Space (tangible interaction)http://www.youtube.com/watch?v=kS_G4WLhwZU
Augmented VirtualityAV describes all the cases where the real object is inserted into a computer-generated environment.A system in which real multi-sensory input is provided, which supplements the visually presented virtual environment.Examples of AVVisual exampleInsert real video stream (e.g., video conferencing) into virtual environmentOlfactory exampleDirecting the smell of coffee grounds to a user when s/he passes near a virtual coffee maker in the virtual environmentHaptic exampleTurning on a heat lamp over the user when s/he approaches a spot in the virtual environment that is exposed to full sunTurning on a fan in front of the user when s/he approaches a spot in the virtual environment that is exposed to wind
Video for Augmented VirtualityOlfactory display: Directing the smell of fruit to a user when s/he selects/grasps the fruit in the virtual environment
Augmented Virtuality in Cooperative DesignAV-based videoconferencing
Video for Augmented VirtualityVideo: CARPE (Tangible Teleconferencing) a good example for distributed Augmented Virtuality environment http://www.hitlabnz.org/wiki/Video_-_CARPE
Case Illustrations ICase 1: Mixed Reality-based Face-to-Face System
Case Illustrations IGroupware IssuesGain in social interaction: The effect is a possible gain from a social aspect and a possible loss in efficiency from time wasted. More distraction issues: Seems to be overly distracting. A fundamental difference between single-user and multi-user interfaces. A simple solution is a protocol to audibly announce users intentions prior to taking action.
Case Illustrations IICase 2: Mixed Reality-based Virtual Space System
Distributed Virtual Space Conferencing
Case Illustrations IIGroupware Issues
Discussion is made more difficultGroup focus is more difficult, requiring greater concentrationSocial interaction is decreased