Embed Size (px)
Citation preview
CREATE YOUR OWN VIRTUAL REALITY
EXPERIENCE
Mark Billinghurst, Gun [email protected]
August 16th 2017
Virtual Reality
Computer generated multi-sensory simulation of an artificial environment that is interactive and immersive.
First VR Experience
• “This is so real..”• https://www.youtube.com/watch?v=pAC5SeNH8jw
What is Virtual Reality?Virtual reality is..
computer technology that replicates an environment, real or imagined, and simulates a user's physical presence and environment to allow for user interaction. (Wikipedia)
• Defining Characteristics• Environment simulation• Presence• Interaction
David Zeltzer’s AIP Cube
nAutonomy – User can to react to events and stimuli.
nInteraction – User can interact with objects and environment.
nPresence – User feels immersed through sensory input and output channels
Interaction
Autonomy
Presence
VR
Zeltzer, D. (1992). Autonomy, interaction, and presence. Presence: Teleoperators& Virtual Environments, 1(1), 127-132.
Key Technologies• Autonomy
• Head tracking, body input• Intelligent systems
• Interaction• User input devices, HCI
• Presence• Graphics/audio/multisensory output• Multisensory displays
• Visual, audio, haptic, olfactory, etc
Typical Virtual Reality System
HMD
Input
Tracking
User Experience
Types of VR
9
HISTORY OF VR
https://immersivelifeblog.files.wordpress.com/2015/04/vr_history.jpg
1800’s – Capturing Reality• Panoramas (1790s)
• Immersive paintings• Photography (1820-30s)
• Oldest surviving photo (Niépce, 1826)• Stereo imagery (1830s)
• Wheatstone (1832)• Brewster (1851)
• Movies (1870s)• Muybridge (1878)• Roundhay Garden Scene (1888)
Stereo Viewers + MoviesBrewster (1860)
1900s – Interactive Experiences• Early Simulators (<1960s)
• Flight simulation• Sensorama (1955)
• Early HMDs (1960s)• Philco, Ivan Sutherland
• Military + University Research (1970-80s)• US Airforce, NASA, MIT, UNC
• First Commercial Wave (1980-90s)• VPL, Virtual i-O, Division, Virtuality• VR Arcades, Virtual Boy
Early VR Experimenters (1950’s – 80’s)
Helig 1956
Sutherland 1965
Furness 1970’s
Sutherland VR Display Prototype
https://www.youtube.com/watch?v=NtwZXGprxag
The First Wave (1980’s – 90’s)
NASA 1989VPL 1990’s
Virtuality 1990’s
Desktop VR - 1995• Expensive - $150,000+• 2 million polys/sec• VGA HMD – 30 Hz• Magnetic tracking
Demo – Dactyl Nightmare (1991)
https://www.youtube.com/watch?v=L60wgPuuDpE
Second Wave (2010 - )• Palmer Luckey
• HMD hacker• Mixed Reality Lab (MxR)
• Oculus Rift (2011 - )• 2012 - $2.4 million kickstarter• 2014 - $2B acquisition FaceBook• $350 USD, 110o FOV
Desktop VR 2016
•Graphics Desktop• $1,500 USD• >4 Billion poly/sec
•$600 HMD • 1080x1200, 90Hz
•Optical tracking• Room scale
Oculus Rift
Sony Morpheus
HTC/Valve Vive
2016 - Rise of Consumer HMDs
HTC Vive
• Room scale tracking• Gesture input devices
Example HTC Vive App – Tilt Brush
https://www.youtube.com/watch?v=ijukZmYFX-0
Computer Based vs. Mobile VR
Mobile VR: Google Cardboard
• Released 2014 (Google 20% project)• >80 million shipped/given away• Easy to use developer tools
+ =
Google Cardboard (V1 and V2)
• Two versions of Google Cardboard• Version 2 suitable for any type of smart phone
Many Mobile VR Viewers Available
• In 2016 – 46m possible desktop VR users vs. 400 m mobile VR users• https://thoughts.ishuman.co/vr-will-be-mobile-11529fabf87c#.vfcjzy1vf
• zxcvz
MOBILE VR
Types of VR Experiences• Immersive Spaces
• 360 Panorama’s/Movies• High visual quality• Limited interactivity
• Changing viewpoint orientation
• Immersive Experiences• 3D graphics
• Lower visual quality• High interactivity
• Movement in space• Interact with objects
Immersive Panorama
• High quality 360 image or video surrounding user• User can turn head to see different views• Fixed position
Example Applications• Within – Storytelling for VR
• https://with.in/ • High quality 360 VR content
• New York Times VR Experience• NYTVR application• Documentary experiences
• Vimeo360• https://join.vimeo.com/360/• Immersive 360 movies
Applications: Virtual Tours
https://www.youtube.com/watch?v=vN16M2RZ8k4
Capturing Panoramas• Stitching photos together
• Image Composite Editor (Microsoft)• AutoPano (Kolor)
• Using 360 camera• Ricoh Theta-S• Fly360
Capturing 360 images
Kodak 360 Fly 360 Gear 360 Theta S Nikon
LG 360 Pointgrey Ladybug Panono 360 Bublcam
Example: Cardboard Camera
• Capture 360 panoramas• Stitch together images on phone• View in VR on Cardboard
Demo: Cardboard Camera
https://www.youtube.com/watch?v=d5lUXZhWaZY
Google Cardboard App• 7 default experiences
• Earth: Fly on Google Earth• Tour Guide: Visit sites with guides• YouTube: Watch popular videos• Exhibit: Examine cultural artifacts• Photo Sphere: Immersive photos• Street View: Drive along a street• Windy Day: Interactive short story
100’s of Google Play Cardboard apps
BUILDING VR EXPERIENCES
What You Need• Cardboard Viewer/VR Viewer
• https://www.google.com/get/cardboard/• Smart phone
• Android/iOS• Authoring Tools/SDK
• Google VR SDK• Unity/Unreal game engine• Non programming tools
• Content• 3D models, video, images, sounds
Software Tools• Low level SDKs
• Need programming ability• Java, C#, C++, etc
• Example: Google VR SDK (iOS, Android)• https://developers.google.com/vr/
• Game Engines• Powerful, need scripting ability• Unity - https://unity3d.com/• Unreal - https://www.unrealengine.com/vr• Combine with VR plugins (HMDs, input devices)
• Google VR Unity plugin
Unity Interface
Tools for Non-Programmers• Focus on Design, ease of use
• Visual Programming, content arrangement• Examples
• Insta-VR – 360 panoramas• http://www.instavr.co/
• Vizor – VR on the Web• http://vizor.io/
• A-frame – HTML based• https://aframe.io/
• ENTiTi – Both AR and VR authoring• http://www.wakingapp.com/
• Eon Creator – Drag and drop tool for AR/VR• http://www.eonreality.com/eon-creator/
INSTAVR
InstaVR
•http://www.instavr.co/•Free, fast panorama VR, deploy to multi platforms
Demo - Using InstaVR
• https://www.youtube.com/watch?v=M2C8vDL0YeA
Results
• https://www.youtube.com/watch?v=JTW86aA1QiM
Development Flow•Collect assets
•360 panoramas, video, images
•Upload to web repository•InstaVR account
•Add content flow•links, hotspots, text content
•Test in browser•Publish to platform:
•Android, IOS, Gear VR, Web, Daydream, Oculus Rift, HTC Vive, Playstation VR
Getting Started• Collect assets
• Images, video, panoramas• Create account
Collecting Assets• Sample Assets
• Under Support > Sample• Tutorial Assets
• https://goo.gl/tqyLxU• Create your own
• 360 Camera• Insta 360, Ricoh Theta, Gear 360• Google Street View Camera• Cardboard Camera
Process• 1. Authoring
• Upload content and create links• 2. Branding
• Create icon, splash image, etc• 3. Create Screens
• Home screen, launch experience• 4. Package into application
• Create Android .apk file, iOS, etc• 5. Download to device
1. Authoring
• Process• Upload content• Load panorama images• Create links between images• Add information hotspots and actions
InstaVR Interface
• Web based interface
Upload Content• Select “Click to Add VR Contents” button• Upload desired content
• File chooser• Panorama images
• Select files you want in app
Create Links
• Select “+Link” button• Position, size link, add icon• Add link destination image
• Image jumped to when link selected
Create Information Hotspot
• Select “+Hotspot”• Position and Size• Add pop-up image
Preview on a Mobile Device
• Preview App download• iOS: https://console.instavr.co/distribution/editor
• Settings > General > Device Management > InstaVR Inc. > Trust
• Android: https://goo.gl/B5Bde9• Settings > Security > Unknown Sources
2. Branding
• Create app icon and splash screens
3. Screens
• Create home screen and start process
4. Package
• Creates executable application• Select platform you want to deploy to
• Currently Android, iOS, GearVR• Click package button and wait
• Note – takes a long time in free version
5. Download to Device
• Select download tab• iOS: https://console.instavr.co/distribution/list• Android: Click the link or scan QR code to download apk
Summary• InstaVR
• Authoring tool for creating 360 panorama based VR apps• Web-based interface, easy to learn and use• Deploy to various platforms
OTHER EDITING TOOLS
Editing Immersive Experiences• Developing interactive 3D experiences
• Creating 3D scene• Adding richer interactivity
• Simple Tools• World Builder
• Authoring VR in VR• Entiti
• AR/VR visual authoring• Vizor
• Web based VR
• Developer Tools• Unity, Unreal
World Builder
• https://www.youtube.com/watch?v=65u3W7wjXs0
Entiti
• https://www.wakingapp.com/• Stand alone application for VR/AR authoring
• Download for Windows/Mac• Works with Entiti mobile application
• Android, iOS versions• Delivers multiple VR experiences
• 360 panorama, 3D environments• Template based VR + Visual Programming
Entiti Overview
https://www.youtube.com/watch?v=SRuYQoT45Tg
Vizor
• http://www.vizor.io/• Web based full featured VR development• 360 panorama, 3D environment, interaction• Visual programming• Deploy to WebGL - just share URL to run, no app needed
Visual Programming
• Select Program Tab • Connect visual elements together
Demo: Getting Started with Vizor
https://www.youtube.com/watch?v=_FvWtFyeNss
VR RESEARCH
Many Areas for VR Research• Display
• Wide field of view, new display technologies• Tracking
• Precise tracking, wide area• Interaction
• Natural gesture interaction, human factors• Authoring Tools
• Content capture, authoring in VR• Applications
• Collaboration, large scale VR
Bare Hands
• Using computer vision to track bare hand input• Creates compelling sense of Presence, natural interaction• Challenges need to be solved
• Not having sense of touch• Line of sight required to sensor• Fatigue from holding hands in front of sensor
Example: Leap Motion
• https://www.youtube.com/watch?v=QD4qQBL0X80
Eye Tracking
• Technology• Shine IR light into eye and look for reflections
• Advantages• Provides natural hands-free input• Gaze provides cues as to user attention• Can be combined with other input technologies
Example: FOVE VR Headset
• Eye tracker integrated into VR HMD• Gaze driven user interface, foveated rendering• https://www.youtube.com/watch?v=8dwdzPaqsDY
Pedestrian Devices• Pedestrian input in VR
• Walking/running in VR• Virtuix Omni
• Special shoes• http://www.virtuix.com
• Cyberith Virtualizer• Socks + slippery surface• http://cyberith.com
Cyberith Virtualizer Demo
• https://www.youtube.com/watch?v=R8lmf3OFrms
Social VR
• Facebook Social Virtual Reality, AltspaceVR• Bringing Avatars into VR space• Natural social interaction
Demo: Facebook Social VR
https://www.youtube.com/watch?v=yxHwWHHg4Vs
CONCLUSION
Conclusion• VR has a long history
• Early prototypes from 1960’s onwards• VR is having second phase of commercial success
• Projected to grow to over $120B market by 2020• Mostly mobile VR in near term
• It is easier than ever before to develop VR apps• Multiple easy to use tools for non-programmers• Powerful developer tools for programmers
• There are many opportunities for VR research• Collaboration, interaction, navigation, etc
THANKS
