Computational Near-eye Displays!Engineering the Interface between our Visual System and the Digital World!

Gordon Wetzstein!www.computationalimaging.org!

!September 19, 2016!

Frontiers of Engineering Symposium!

image courtesy: vpl research!

A Trip Down the Rabbit Hole!

drawings from “Alice in Wonderland”, Lewis Caroll!

A Brief History of Virtual Reality!

1838! 1968! 2012-2016!

Stereoscopes!Wheatstone, Brewster, …!

VR, AR, !Ivan Sutherland!

VR explosion!Oculus, Sony, HTC, MS, …!

AR Displays!

Nintendo!Virtual Boy!

1995!

Ivan Sutherland’s HMD!•  optical see-through AR, including:!

•  displays (2x 1” CRTs)!

•  rendering!•  head tracking!

•  interaction!

•  model generation!

•  computer graphics!

•  human-computer interaction!

I. Sutherland “A head-mounted three-dimensional display”, Fall Joint Computer Conference 1968!

= ?

low cost, high-res, compute, low-latency!!

f

d’ d

Gaussian thin lens formula: 1d+ 1d '

= 1f

VR Display Optics = Simple Magnifier!

Top View!

Real World:!!

Vergence & Accommodation Match!!

Top View!

Near-eye Displays Today (all stereo displays):!!

Vergence-Accommodation Mismatch!!

virtual image!

Related Work!

Related Work!1. Gaze-contingent 2D Displays! 2. Multi-plane 3D Displays! 3. 4D Light Field Displays!

[Lanman & Luebke 2013]!

[Maimone et al. 2013]!

Age-related Degradation of Accommodation !

Response for Physical Stimulus!Heron & Charman 2004!

presbyopia L !

Bifocals!

Monovision!

Focus-tunable Near-eye Displays = Drive Accommodation!!

f

d’ d

Gaussian thin lens formula: 1d+ 1d '

= 1f

focus-tunable lens à vary f

actuator à vary d’

at ACM SIGGRAPH 2016!

EyeNetra.com!

at ACM SIGGRAPH 2016!

translation stage for !IPD adjustment!

autorefractor for measuring !accommodation state of viewer!

2K LCD display!

2K LCD display!

focus-tunable lens!

focus-tunable lens!

NIR/visible!beam splitters!

152 total!126 male!25 female!

0

5

10

15

20

25

30

35

PresbyopeEmmetropic PresbyopeDistance correctedEmmetrope

65605550454035302520Age

Demographics!

Display Modes – “Normal”!

•  fixed distance to virtual image

•  used by all existing AR/VR displays

Display Modes – “Dynamic”!

•  dynamically adjust virtual image based on fixated object depth

•  newly proposed display mode not used in existing display

Display Modes – “Monovision”!

•  we proposed this mode recently for people who can accommodate

•  hypothesis: viewer could potentially accommodate to two different distances

Normal!

34 viewers!

Accommodative Gain!D

ista

nce

from

cen

ter [

D]!

stimulus!average accommodative response!

non-zero gain = !vergence-driven accommodation!!

Dynamic!Normal!

21 viewers!34 viewers!

Accommodative Gain!D

ista

nce

from

cen

ter [

D]!

Dynamic!Normal! Monovision!

21 viewers!34 viewers! 6 viewers!

Accommodative Gain!D

ista

nce

from

cen

ter [

D]!

Accommodative Gain!

Response for Physical Stimulus!Heron & Charman 2004!

Accommodative Gain!

Accommodative Gain!

0.0

0.5

1.0

1.5Presbyopes

Nonpresbyopes

DynamicMonovisionUncorrectedNormal

Gain

Accommodative Gain!

Nonpresbyopic! Presbyopic!

Ability to Fuse Stereo Images!

Insights so Far!•  if eye tracking available, use gaze-contingent focus!•  without good eye tracking, things can get worse!!•  presbyopes do not want dynamic focus – would see blurry image!•  will see this in next-generation VR/AR displays!!

HMD!

lens!micro

display!

virtual image!

eye tracking!

Near-eye Light Field Displays!

Idea: project multiple different perspectives into different parts of the pupil!!

SIGGRAPH 2015!

Remaining Engineering Challenges for VR/AR!

•  Vergence-accommodation conflict (VAC)!

•  Vestibular-visual conflict (motion sickness)!

•  AR! •  occlusions!•  aesthetics / form factor!

•  battery life!•  heat!

•  wireless operation!

•  low-power computer vision!•  registration of physical /

virtual world and eyes !•  consistent lighting!

•  scanning real world!

•  VAC more important!•  display contrast &

brightness!•  fast, embedded GPUs!

•  …!

Stanford EE 267!

Gordon Wetzstein!Computational Imaging Group!Stanford University!

stanford.edu/~gordonwz!

www.computationalimaging.org!