19
Tim Leland VP, Product Management Qualcomm Technologies, Inc. August 2015 1

Entering a new era of immersive experiences

Embed Size (px)

Citation preview

Page 1: Entering a new era of immersive experiences

Tim Leland

VP, Product Management

Qualcomm Technologies, Inc.

August 2015

1

Page 2: Entering a new era of immersive experiences

2

Immersive experiencesThe experiences worth having, remembering, and reliving

Draw you in…Take you to

another place…

Keep you present

in the moment…

Page 3: Entering a new era of immersive experiences

Intuitive

interactions

Immersive is the intersection of three areas

3

Visual

quality

Sound

quality

Immersion

Intuitive

Interaction

Page 4: Entering a new era of immersive experiences

Virtual reality is the ultimate level of immersion

Visuals so vibrant that they are indistinguishable from

the real world

Sound so accurate that it is true to life

Interactions so intuitive that you forget there is even an

interface

Stimulating our senses with realistic feedback

Page 5: Entering a new era of immersive experiences

Still a few questions to be answered….

5

Sources: CNET: Business Insider, Oct. 30 2014

“There seems to be a lot of excitement around something that, to me, is a

yawn, frankly,” James Cameron said when asked about the rapidly growing

VR space.

“The question that always occurred to me is, when is it going to be mature,

when is it going to be accepted by the public at large, when are people

going to start authoring in VR and what will that be?”

Page 6: Entering a new era of immersive experiences

2/26/2016 Qualcomm Confidential and Proprietary

When will the devices come? Global VR Headset Forecast

>100M Units in 2020

$62B Market Size in 2025

Wireless/Mobile = Most of

the VR Headset volume

Too early to know if this is

an accurate ramp, but there

will be new VR uses for

premium tier Snapdragon

Source - Piper Jaffray, May 2015

VR Headset Sales (In Millions)

Page 7: Entering a new era of immersive experiences

Far more Immersive Gaming

360 Degree Video

− Sports

− Concerts

− Cinema

Social Experiences

− Virtual Tourism, Virtual Shopping

− New Education & Working Environments

So what will the new VR experiences be?

Page 8: Entering a new era of immersive experiences

Visual

qualitySound

quality

Intuitive

interactions

Immersive virtual reality has extreme requirements

8

High resolution audioUp to human hearing capabilities

Extreme pixel quantity and qualityScreen is very close to the eyes

3D surround soundRealistic 3D positional audio that is

accurate to the real world

Precise motion trackingAccurate on-device motion

tracking

Broad field of viewHumans have a 180 degree field

of view

Minimal latencyMinimized system latency to

remove perceptible lag

Visual

quality

Sound

quality

Intuitive

interactions

Immersion

Stereoscopic displayHumans see in 3D

Natural user interfacesSeamlessly interact with VR using

natural movements, free from wires

Page 9: Entering a new era of immersive experiences

An end-to-end approach is required for virtual realityOptimizing for motion to display latency

Gyro, accelerometer, camera, or other

sensors detect movement

Motion detectionCalculate movement and create

appropriate image through multi-stage

visual processing

Visual processingImmediately update display with image to minimize latency

Screen update

9

End-to-end approach

Page 10: Entering a new era of immersive experiences

A heterogeneous computing approach is needed for VRSpecialized engines to meet processing requirements at low power and thermals

10

Location

GPU

DPU (Display)

Camera ISP ModemDSP

(audio, sensor, &camera co-processor)

CPUs

Memorysubsystem

VPU (Video)

Virtual realityComputer vision, image processing, sensor processing, graphics, video processing, location, and cloud interaction

* Not to scale

Entire SoC is used!

High-utilization

Page 11: Entering a new era of immersive experiences

Still photos & captured videos that are:

− Never blurry, always focused on what the user wants to see

− Clear and sharp regardless of motion & lighting conditions, with leading performance

Color as nature & artists intended it to be

− Accuracy & color gamut matching all the way through the pipeline, from camera, video,

graphics out to display

Highest quality mobile movies and games

− Mobile performance that rivals what users currently get in the living room

Responsive, smooth interaction with home screen UIs, App UIs & Browsers:

− Free from “jank” and low frame rate that can detract from user experience

Other user experience goals for Snapdragon Visual Processing

Page 12: Entering a new era of immersive experiences

Taking an end-to-end approach to improve visual qualityThe journey of light from the mobile camera to a correct pixel on the display is challenging

End-to-end solutionsColor management, artifact removal, optimized click-to-shoot time

Component tuning | Image processing | System optimization

Mobile size constraints create challenges:

• Poor light gathering

• Increased noise

Lens and sensorAddresses camera and display

challenges while enhancing images

SoCPhysical limitations of LCDs and LEDs:

• Partial color gamut

• Limited contrast and brightness

Display

Page 13: Entering a new era of immersive experiences

Qualcomm SpectraTM Camera ISP

• Improved image quality, with more natural skin tones

• Pre & Post processing on HVX for new UX

• Improved power efficiency and higher throughput

Benefits

What’s New in Snapdagon 820

• Better Image Quality

• Enhanced de-mosaicing for new ~1.0u image sensors

• New hardware for radial noise reduction & color artifact

correction

• Camera sensor pre-processing streaming to DSP/HVX

• Improved hybrid autofocus framework

(PDAF/Contrast/Laser)

• GPGPU Optical Zoom

• Universal Bandwidth Compression (UBWC)

• Serial C-PHY interface

Without

SpectraTM ISP

Using

SpectraTM ISP

Page 14: Entering a new era of immersive experiences

The new Adreno 530 GPU for Snapdragon 820

Adreno 530 is the top tier in a new family GPUs

− Designed for mobile UIs, Games, Web and VR

− Up to 40+% better performance than Adreno 430

Will include next generation capabilities

− Supports latest in Graphics and Compute APIs

− Support for 64-bit Virtual Addresses

Will deliver superior performance per mW

− New sorting and compression techniques

− A new GPU Power Manager that provides finer

granularity of power control

Improved compute and quality of service

“Paris Apartment”

Snapdragon 820 +

Epic Unreal Engine4Photorealistic 3D Graphics

Page 15: Entering a new era of immersive experiences

A heterogeneous computing approach is neededSpecialized engines to meet visual processing requirements at low power and thermals

15

Location

GPU

Display Processor

Camera ISP ModemDSP

(audio, sensor, &camera co-processor)

CPUs

Memorysubsystem

Video Processor

Camera image captureLens correction, noise removal, and 3A

Image processingColor correction, tone mapping, and computational photography

High-end gamingRendering, tessellation, and geometry shading

Video compressionVideo encode and decode

Display imageCompositing, gamut mapping, and calibration to displays * Not to scale

High-utilization

Low-utilization

Page 16: Entering a new era of immersive experiences

Kryo™

CPUs

Location

Adreno™ GPU

Adreno™ DPU

Snapdragon

“X” LTE

Modem

Memory

Subsystem

Spectra™

ISP

Adreno™ VPU

Using an efficient heterogeneous computing solutionSpecialized engines to meet processing requirements at low power and thermals

• TruPalette™ color enhancement

• ecoPix™ power savings

• Composition

• Superior performance per mW

• GPU compute for image processing

• Graphics rendering with HW tessellation

• Universal bandwidth compression

• Memory management for heterogeneous

computing

• Custom instructions for image, audio, and

sensor processing

• High-quality voice with Fluence™ Pro

• 3D immersive audio

• Ultra-low power sensor processing and fusion

• 14-bit twin ISP

• Ultra-fast laser & phase detect autofocus

• Triple noise reduction: Bayer, wavelet, and

temporal

• Dual camera support for depth, sensor fusion,

and ultra low light

• Efficient, high quality 4K encode/decode

• Industry leading Wi-Fi Display

• Efficient custom CPU architecture

• Image and audio processing

* Not to scale

16

• High bandwidth at low power and latency

• Multiple radio technologies and bands

Snapdragon processors

Hexagon™ DSP (audio, sensor, &

camera co-

processor)

Some features are only available on specific Snapdragon processors. Consult processor specifications for feature availability. TruPalette, ecoPix, Adreno, Spectra, Kryo, Krait, Fluence, and Hexagon are products of QTI.

Page 17: Entering a new era of immersive experiences

Offering superior development and optimization toolsEnabling content creation and tuned devices

17

Content creation tools

Specialized solutions for development

• Adreno SDK

• Vuforia™ SDK

• Hexagon SDK

Debugger for solving

• Snapdragon Debugger

Profiler for optimization

• Snapdragon Profiler

Third-party middleware support

• Unity & Epic Unreal Engine

Device optimization tools

Calibration and tuning

• Chromatix™ color optimization tool

• Qualcomm Display Color

Management

• Qualcomm Audio Calibration Tool

Analysis and debugging

• Qualcomm Commercial Analysis

Toolkit

• Qualcomm eXtensible Diagnostic

Monitor

Other ecosystem enablement

Development devices

• Mobile Development Platform

• DragonBoard™

Customer support

• Customer engineers

Vuforia and Chromatix are products of QTI. DraonBoard is distributed by Intrinsyc.

Page 18: Entering a new era of immersive experiences

Efficient heterogeneous

computing architecture

Within device constraints

Development time / Sleek form factor /

Power and thermal efficiency / Cost

Delivering superior immersive experiencesProviding efficient, comprehensive solutions

18

Visual quality

Immersive experiences

Sound quality

Intuitive interactions

Commercialization

Consistent, accurate color

In-focus images

Low-light video & photography

3D surround sound

Noise removal

Dynamic sweet spot

Seamless, responsive,

smooth user interfaces

Intelligent, contextual

interactions

Via Snapdragon™

solutions

Via ecosystem

enablement

Comprehensive solutions

across tiers

Custom designed

processing engines

Snapdragon development

platforms

Ecosystem collaboration

App developer tools

Snapdragon is a product of Qualcomm Technologies, Inc.

Page 19: Entering a new era of immersive experiences

For more information on Qualcomm, visit us at:

www.qualcomm.com & www.qualcomm.com/blog

Qualcomm is a trademark of Qualcomm Incorporated, registered in the United States and other countries. All trademarks of

Qualcomm Incorporated are used with permission. Other products and brand names may be trademarks or registered

trademarks of their respective owners

Thank youFollow us on:

© 2015 Qualcomm Technologies, Inc. 19