Using Vuforia to Build Breakthrough Mixed Reality
ExperiencesDavid BeardDeveloper Evangelist, Product ManagerPTC
VuforiaJune 2nd 2016
#Hello everyone. My name is David Beard, Im a senior product
manager for the Vuforia platform and our developer evangelist. My
role is to help developers learn how to create AR/VR app using
Vuforia, and on the best practices for designing mixed reality
experiences.Id started with Vuforia over 5 years ago as an engineer
and am currently on the product management team, responsible for
tools workflows.1
AgendaVuforia IntroductionMixed Reality ExperiencesVuforia 5.5
for Mixed Reality
#
#Today Im going to be discussing how Vuforia can be used to
create mixed reality apps in UnityIll explain what mixed reality is
and present some examples of mixed reality use casesThen Ill
describe the features of Vuforia 5.5 that support mixed realityBut
first, let me give you a brief introduction to Vuforia
2
#Vuforia is PTCs Mobile AR platform.It is an SDK that enables
your apps to see things in the real world and then interact with
them. Weve done this by taking very sophisticated computer vision
technologies, building them ground-up for mobile platforms, and
making these technologies accessible through an easy to use API and
developer workflows. so that ANY app developer can quickly and very
easily enable their apps with vision functionality.
3
A simple API describes what is in the cameras FOV, and
whereImagesObjectsText Meshes
Vuforia The leading AR platform
#
#Vuforia provides a very simple API. It tells developers what it
sees and where it sees these objects precisely in the cameras
FoVallowing you as developers to then render content accurately on
or relative to the objectToday with Vuforia you can see a number of
thingsImages these are every day images like magazine covers,
product packaging, brochuresObjects rigid like toys and appliances
objects with little or no articulating or moving partsText we have
text recognition for roman characters, such as English language
wordsAnd finally with our Smart Terrain feature .. meshes of an
unknown environmentVuforia can look at a table and will return the
table surface and the props on the table in a mesh on top of which
, you can overlay contentso you can drive a virtual car on a table
that avoids a vase or a stack of booksor falls of the table. With
Vuforia smart Terrain, we provide apps the awareness and
intelligence to interact with their immediate environment
SDKsWe offer the flexibility to program with Java, C++/Objective
C or with UnityAbout 85% of our developers use Unity because it
provides an easy to use visual authoring workflow4
Platform ComponentsVuforia EngineCloud ServicesTools
Supported DevicesPhones & Tablets - Android & iOSEyewear
- Samsung Gear VR, Epson-BT 200, ODG R-6/7SDK support for Unity,
Xcode, Android StudioTarget ManagerCreate & Manage databases
for use on device or cloudVuforia Object ScannerScan objects such
as toys to create Object TargetsEyewear Calibration
Vuforia Cloud Recognition ServiceAllows use of cloud reco
database with up to 1 million targetsVuforia Web ServicesRESTful
APIs to manage cloud-based target databases
#
#Platform ReviewThere are 3 main components to the platformThe
client side EngineAndroid and iOS devicesSmart phones and
tabletsDigital EyewearTools to create and manage targets The Target
manager is an online tool for creating, evaluating and managing
targetsYou can create both device databases, which are embedded in
your app, or cloud databases that are accessed using a web
serviceThe Object scanner tool is used to create Object
TargetsEyewear calibration tool is for generating custom eye
calibration profiles for users of optical see-through eyewearCloud
services enable you to deploy targets to your apps from a web
service online. This service can support up to 1 million target
images and delivers the same kind of trackable data as a device
database.Cloud Recognition is on online image recognition service
you can query this service from your app Cloud Recognition also
supports the Vuforia Web Service REST API which enables you to
interface with the service from your own Content Management
System5
AR + VR = Mixed Reality
AR on your headAR in your hand
AR experiences today
#
#Wed originally developed Vuforia for hand held devices smart
phones and tabletsThen as wearable displays became more readily
available, we adapted our SDK to support optical and video see
through eyewearOptical see through eyewear has a transparent lens,
where digital content is rendered into the field of view of the
wearerExamples include the Epson BT-200 and ODG R-6 and R-7Video
see-through eyewear relies on a device camera to see the world,
which is presented to the user on the device screenExamples include
Gear VR and Google CardboardThis evolution has enabled the
development of a new class of AR/VR applications that support Mixed
Reality experiences6
Mixed RealitySimilar technology stack Different level of
immersionComplementaryVR and AR 2 sides of the same coin
#Let me talk about Vuforias vision for mixed reality. People
often describe AR and VR as distinct technologies, and there are
some key differences between pure AR and VRBut there are also
significant similarities that make these complementary
technologiesThey share a similar technology stack for computing,
rendering, applications and sensor inputsThey differ in their
levels of immersion, between viewing the real world and
experiencing a fully virtual worldAnd these difference are actually
complementary they enable a new range of use cases, and more
creative freedom7
Mattel View-Master
#This is the new Mattel View MasterYou may remember this toy it
was originally a stereoscope. You would insert a disk of images and
it enabled you to view them in 3DMattel has updated this classic
toy as an AR/VR viewer. Now the disks are AR targets that launch
panoramic VR experiencesView Master demonstrates how you can use AR
to intuitively select and transition into VR experiences8
QC399_View-Master_Jays_VersionAUTOMATIC ADVANCE(DO NOT
DELETE)
#
#
9
#
#Germbusters is a game developed by Realiteer, an independent
developerHes developed a set of VR controllers using AR targets in
this game the target represents a bubble gunThis game shows that
you can use an AR target as an interactive device in ARNotice that
this is a fully immersive world AR is simply used to recognize and
track the game target10
#
#
#
#Cordy Orly is a app that enables you to view a proposed
commercial development near the Cordy Orly airport in FranceYou can
view the plans in AR and then select a point of interest to view in
VRThis is a good example of a dual modalilty experience it starts
as handheld AR and transitions of VR in a Viewer12
AUTOMATIC ADVANCE(DO NOT
DELETE)QC399_Coeur_dOrly_Jays_Version
#
#
13
Vuforia 5.5Stereo Rendering with Distortion CorrectionRotational
device trackingMixed Reality Controller API
#
#We introduced several new features to better support VR apps
with Vuforia 5.5 which were released a few weeks agoStereo
Rendering with Distortion CorrectionRotational device trackingMixed
Reality Controller API
Let me give you some details on each of these14
Stereo scene rendering for digital eyewear appsHigh fidelity
rendering with viewer and device specific lens distortion
correctionCustom viewers can be defined by developers
Copyright 2016 PTC Inc. All Rights Reserved. Confidential and
Proprietary Protected under copyright and other laws.
Stereo rendering with distortion correction
#
#Stereo scene rendering for digital eyewear apps automatically
corrects for pin cushion distortionHigh fidelity rendering with
viewer and device specific lens distortion correctionCustom viewers
can be defined by developers15
Rotational head tracking for VR appsRotational device tracking
for handheld VRSupports custom head and hand rotation models
Rotational device tracking
Copyright 2016 PTC Inc. All Rights Reserved. Confidential and
Proprietary Protected under copyright and other laws.
#
#Provides a device independent, stand alone, solution for head
tracking. You can use device tracking to track both the users head
or their hand when they are holding the deviceThe device tracker
returns the rotation of the device around a central pivot pointThe
pivot point is defined by a rotational model we provide head and
hand models that match typical human anthropometry ( body
measurements )We also enable you to change these models to define a
custom rotational offset16
Mixed Reality Controller APIEasily transition from AR to VR and
back againSupports mixed reality for both handheld and viewer
experiencesEnables dual modality apps - from hand to head
Copyright 2016 PTC Inc. All Rights Reserved. Confidential and
Proprietary Protected under copyright and other laws.
#
#The mixed reality controller is a utility API that handles AR /
VR app lifecycle automaticallyIt enables you to transition from AR
to VR and also from handheld to viewer based experiences with a
single method callThis enables the AR/VR use cases Ive shown and
dual modality appsThe nice thing is that you can develop a standard
AR app for handheld mobile devices and easily enabled it for VR by
simply implementing this API and a single method call.17
#
#
18
developer.vuforia.com
#
