Upload
diego-bezares
View
205
Download
1
Embed Size (px)
Citation preview
Virtual Reality Stereoscopy A year of prototypes and conclusions.
Diego Bezares. @diegobez
What if... ?
Accommodation reflex
Convergence
Stereoscopy
IPD. Interpupilar distance
- How does the brain “view” different IPD’s ?
Stereoscopy
Normal Mapping vs. Parallax MappingThe technique known as “normal mapping” provides realistic lighting cues to convey depth and texture
without adding to the vertex detail of a given 3D model. Although widely used in modern games, it is much
less compelling when viewed in stereoscopic 3D. Because normal mapping does not account for binocular
disparity or motion parallax, it produces an image akin to a flat texture painted onto the object model.
“Parallax mapping” builds on the idea of normal mapping, but accounts for depth cues normal mapping
does not. Parallax mapping shifts the texture coordinates of the sampled surface texture by using an
additional height map provided by the content creator. The texture coordinate shift is applied using the per-
pixel or per-vertex view direction calculated at the shader level. Parallax mapping is best utilized on
surfaces with fine detail that would not affect the collision surface, such as brick walls or cobblestone
pathways.
Why stereoscopy ?
Presence : The sense of “being there” .
An illusion that a mediated experience is not mediated.
Pre-render
¿ what if…. ?
Scene Prerendered to equirectangular stereo
360 panorama capture
360 panorama capture
Animated waves + Kite shadow
Virtual cameras movement
Omni Directional Stereo Content
https://developers.google.com/vr/jump/rendering-ods-content.pdf Google
Omni-directional stereo (ODS) is a projection model for stereo 360 degree videos. It’s designed for VR viewing with a head-mounted display (HMD).
ODS uses a special projection format which has the following advantages: ● It is panoramic and stereo everywhere—there are no bad seams or dead zones (except directly above and below the camera, which we will discuss later). ● It's pre-rendered—like film and video, it plays well on all devices. ● It’s encoded as two video streams—it can be stored, edited, and transmitted using conventional tools.
stereo ?
ODS
4096 x 2048 pixels -> 4096 camera positions
ODS : No manual stitching- Stitching : the process of combining multiple images with overlapping fields of view to produce a segmented panorama
Stitching errors ->
Google Jump is using machine learning for stitching
Unity3D ODS camera implementation
- Problemas en los polos
BlackJackVR
Cubemap projection vs equirectangular
20% improvement
Cubemap sizeEjemplo : GearVR.Resolución : 2560 x 1440 pixels => 1280 x 1440 cada ojoFOV : 96 gradosPíxeles por grado : 1280 / 96 = 13.4 horizontal, 1440 / 96 = 15 verticalSi giramos podemos ver 360º horizontales -> 360 x 13.4 = 4824 píxelesSi giramos podemos ver 180º verticales-> 180 x 15 = 2700 píxeles
render target bigger ( for distorsion correction ), 1.4x
Positional Tracking support
¿ What if ... ?
Pedro Fernando Gómez
@pedrofgomez
pedrofe.com
Positional Tracking support
Pedro Fernando Gómez
@pedrofgomez
pedrofe.com
lights, stereo billboard
Full screen prerendered.
Viable :High poly, complex scenes, complex lightning.
Not-realtime rendering engines , VR cameras like : Pedro Fernando Gómez VR Camera for Arnold : http://pedrofe.com/oculus-camera/
Complex, expensive.
What if... ?
Capturing reality
- Arduino + step motor- OpenCV :
video -> imagen ODS
Capturing reality
http://cctoolkit.vectorcult.com/
Capturing realityhttps://docs.google.com/document/d/14CpJQKUeeFDt_P07oStEBbJqbdJikMvgv8tIyHb8F2I/edit?usp=sharinghttp://mistralsocialgames.com/vr/test2.html
1 minute 360 degrees, 2 cameras 60fps 1080p.2 pixel columns per position.60 fps x 60 secs/min x 2 pixels = 7200 pixels
7200 x 1900 pixels per eye.
Arduino one, accelstep library, 28BYJ-48 step motor
Animated, interactive scene
- prerendered animations.- Loops : tree leaves, characters : idle -> action -> idle.- Interactivity- Integration with Realtime 3D objects.
What if ... ?
VR Video
- 360 degrees, just 100 degrees at the screen..- Big video files : resolution, 360º, stereo, etc...- Decoding processing limited : 20mbs high end mobile, 50mbs PC.- Bad stereoscopy ( 16 camera positions, stitching errors … ).- Complex stitching, expensive.
VR Video
- https://t.co/I3BKHJTSIt- https://drive.google.com/file/d/0B39SXMRp_xSTQmZGRl94ZjloUXc/view?usp=sharing
VR VideoArc of alpha, beta degrees
- stitchado bordes
VR VideoStitch
VR Video
Blend
VR Video
- Chroma- OpenCV : splitting static background and foreground.
Diego bezares. @Diegobez
Thanks !GearVR PlaystationVRCardboard SteamVR
XBOX Scorpio
Oculus RIFT
2014June
2015November
2016March
2016April
2016October
2016Fall
2017Late
Daydream
Locomotion
Resolución en R.V
Resolución en R.V
Píxels por grado de visión.
Render Target
Trends