Upload
emohacker402
View
778
Download
5
Embed Size (px)
DESCRIPTION
This Presentation explains about Augmented Reality on Android using Vuforia SDK that Developed by Qualcomm
Citation preview
Augmented Reality on Android
What is Augmented Reality (AR)?
Virtual 2D and 3D content superimposed onto a live image
GPS and Compass-based Augmented Reality Experience
Limited to outdoor
experiences
Real World ViewLive camera view
Virtual Content
Information about points of interest
Good for rough location of items outside our field of view
Real World ViewLive camera view
Virtual ContentVision-based AR recognizes objects in field of view
and aligns graphics tightly to target object
Vision-based Augmented
Reality Experience
Enables a more immersive and interactive
experience
Pull Video Frame from Camera,
Scan for Known Objects or Features
Compare to Database of Known Images
Position and Orientation Determined
Z
X
Y
Graphics are Rendered
SCANS COMPARES POSITIONS RENDERS
times persecond30+
OYXGEN
HYDROGEN
HYDROGEN
OYXGEN
HYDROGEN
OYXGEN
HYDROGEN
Visualizing instructions
e.g. furniture assembly
Instructional
Applications
Gaming & Play
3D games in real world environments vs. virtual worlds
Turn Left on Main St.
Navigation & Discovery
Visualizing points of interest
Making print , outdoor, TV media, and product packaging “come alive”
Media / Advertising
Text
Translate words to multiple languages
Visual Search
Retrieving related informationfrom web for the object in view
Today
In the future…
AR is gaining momentum
16
�“According to Visiongain, in 2012, 25% of mobile apps will
feature augmented reality.”
�The firm forecasts the mobile AR industry will see $3 billion in
global revenue by 2016, up from $87 million (in 2011) and $21
million in 2010.”
�“Consumer engagement is the main reason that (augmented
reality) technology has been so successful. QR codes have
proven to be effective engagement tools, which has lead many
retailers, such as Macy’s, to incorporate them into their
marketing campaigns
17
Brands are using AR to engage consumers
AR deepens consumer interactions
with brands
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |18
�Point of Advertising – campaign has added AR element across media channels
�Point of Sale – product comes alive on the shelves
�Point of Use – added AR dimension when the product comes home
10/18/2012Qualcomm Vuforia | Vuforia –
Augmented Reality Platform |19
Vuforia – Qualcomm’s AR Platform
Vuforia Platform Traction
21,000+Registered
Developers from 130 Countries
400+ Models of
Smartphones and Tablets Supported
400+ Apps in Android
Market and iOS App Store
Accolades, Testimonials, and HonorsRECOGNITION FOR OUR AWARD WINNING AR TECHNOLOGY
“…your AR library is by far the best AR Library I’ve ever
used. It’s amazing, works incredibly fast, and really easy
to work with. Awesome job QCAR team!!”
2011 Future Mobile
Entertainment Award
for Mobile Augmented
Reality
Nicholas Rudolfsky, iOS Developer, Smule
“When it comes to mobile Augmented Reality technology,
Qualcomm is the top dog.”
Pocket-lint
“The opportunities to do amazing things—which we quite
literally couldn’t live without—are unparalleled with
augmented reality. This is Qualcomm’s secret weapon,
and it may assure the company thrives, rather than
simply survives the coming conflict.”
Rob Enderle, Analyst, TG Daily
“Qualcomm’s AR SDK has made it extremely easy
for us to prototype, design, and develop our ideas and
concepts.”
Morgan Jaffit, Co-Founder, Defiant Development
100s of articles in mainstream
technology and business press
Near Field Usage Environments
Retail ShelfRetail ShelfTable or FloorTable or Floor WallWall
Representations of real world elements that can be detected and tracked
“Augmentable” Objects
Image Target Frame Markers Simple 3D Objects
� Game boards
� Product packaging
� Posters
� Signs
� Greeting cards
� Business cards
� Books / magazine pages
� Game pieces
� Control cards
� Boxes
� Bottles
� Cans
Enables multiple cards to simultaneously trigger AR experiences
Frame Markers
Allows users to press “buttons” by touching certain areas on an image or object
Virtual Buttons
Features and Performance
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |27
Vision
• Ordinary Images
• Simple 3D Images
• Marker
• Virtual Buttons
Performance
• Simultaneous Object
Recognition
• Optimized for
Mobile
Flexibility
• Unity
• Eclipse
• Xcode
• 3rd Party
Android Development Options• Download Qualcomm Vuforia SDKs http://developer.qualcomm.com/ar
Eclipse Unity 3
Advantages • Free
• Low level APIs provide
enhanced flexibility and
performance
• Fully integrated game
engine reduces
development time and
cost
• Single app supports both
iOS and Android
Availability Now Now
29
Vuforia AR Applications Structure
30
Example
Target Management
Creating Image Targets
Targets are Trackable
DataSets
Camera start
Camera stop
Pixel Format
Conversion
The pixel format converter converts between the camera
format (e.g. YUV12) to a format suitable for OpenGL ES
rendering (e.g. RGB565) and for tracking (e.g. luminance)
Tracker
• The tracker detects and track real
world objects in camera video frames.
• Different algorithms take care of
detecting new targets or markers, and
evaluating virtual buttons.
• The tracker can load multiple datasets,
but only one can be active at a time
• The results are stored in a state object
that is used by the video background
renderer and can be accessed from
application code.
State Object Structure
Dataset Targets“Teapot on target A”
Target A
Target B
3D Assets
The State Object
Member Description Type of Data
Camera Frame Current image
Trackables List of active trackable objects
• Planar Image
• Frame Marker
• Multi Target
Events List of events
• Virtual Button 1
• Virtual Button 2, etc.
RGB
YUV
GRAYSCALE
Name
Button Press
Name
4x4
Matrix
• The video background renderer renders the camera image stored in the state object.
• The performance of the background video rendering is optimized for specific devices.
Video BackgroundRenderer
10/18/2012
Application Code
• Query the state object for newly detected targets, markers or updated states of these elements
• Update the application logic with the new input data
• Render the augmented graphics overlay
Application States
onCreate()
initApplication ()
loadTexture()
InitApplication ()
onResume()
Init Tracker()
initApplicationAR ()
loadTrackerData ()
StartCamera ()
initRenddering()
updaterRendering()
onPause ()
StopCamera ()
onDestroy()
deinitApplication()
destroyTrackerData()
deinitTracker()
Native Java
Native C/C++
Creating an Android AR App
GLRenderer implementsGLSurfaceView.renderer {. . .onDrawFrame(..) {
myNativeRenderer()}
}
myNativeCameraStartup (){. . .QCAR::CameraDevice.init(..);QCAR::CameraDevice.start(..);
}
myNativeTrackerStartup{. . .
QCAR::Tracker.setActive(QCAR::PLANAR_IMAGE);
}
myNativeRenderer{. . .// Render video background// Update app logic using State
Object// Render app assets
}
myApp extends Activity {. . .void onCreate() {
QCAR.init(..);}. . .
void onResume() {. . .setupOpenGLViews();myNativeCameraStartup();myNativeTrackerStartup();
}. . .
void onDestroy() {QCAR.deinit();
}}
Native C/C++ Java
Initialize SDK
Start Trackers
Handle Tracker
Updates
Close SDK
QCAR::ImageTracker
Frame Markers
QCAR::MarkerTracker
Virtual Buttons
Buttons coordinates<?xml version="1.0" encoding="UTF-8"?>
<QCARConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="qcar_config.xsd">
<Tracking> <ImageTarget size="247 173" name="wood">
<VirtualButton name="red" rectangle="-108.68 -53.52 -75.75 -65.87" enabled="true" />
<VirtualButton name="blue" rectangle="-45.28 -53.52 -12.35 -65.87" enabled="true" />
<VirtualButton name="yellow" rectangle="14.82 -53.52 47.75 -65.87" enabled="true" />
<VirtualButton name="green" rectangle="76.57 -53.52 109.50 -65.87" enabled="true" />
</ImageTarget> </Tracking> </QCARConfig>
Simple 3D Objects
• Boxes
• Bottles
• Cans
Enables products to trigger AR experiences, and interactive content to reflect geometry of
product packaging
Creating Simple 3D targets
Multi Image Targets
Swappable Datasets
Swappable Datasets contd..
• The script exposes a list of datasets (that are part of the project) to be loaded at scene startup.
• A single script can be used across different scenes
• Mainly used to load/unload and activate/deactivate datasets at runtime.
• Can ONLY be used at scene startup but doesn’t do anything when its properties are changed later on in the application lifecycle.
Dataset Script
Background Texture Access
Background Texture Access contd..
• Video background access allows you to use the
camera video as a texture through the use of
shaders
• Two shaders are used
– vertex shader --- converts video into grayscale and
inverts the black and white
– fragment shader--- listens for a touch on the screen ad
then distort the video around that touch.
Occlusion Management
Occlusion Management contd..
• Illustrates how shaders(four in this case) can be used to
make objects appear semi-transparent.
• Pointing your camera at the FlakesBox target allows you
see the teapot object inside the partially transparent
looking box.
• Four shaders, one per object, are rendered in correct
order (the video background, the checkerboard-box, the
teapot inside and the occlusion box).
Image overlay
Change code in
teapot.h
Images to
be replaced
Cube Structure• -1.00f, -1.00f, 1.00f, // front
• 1.00f, -1.00f, 1.00f,
• 1.00f, 1.00f, 1.00f,
• -1.00f, 1.00f, 1.00f,
• -1.00f, -1.00f, -1.00f, // back
• 1.00f, -1.00f, -1.00f,
• 1.00f, 1.00f, -1.00f,
• -1.00f, 1.00f, -1.00f,
• -1.00f, -1.00f, -1.00f, // left
• -1.00f, -1.00f, 1.00f,
• -1.00f, 1.00f, 1.00f,
• -1.00f, 1.00f, -1.00f,
• 1.00f, -1.00f, -1.00f, // right
• 1.00f, -1.00f, 1.00f,
• 1.00f, 1.00f, 1.00f,
• 1.00f, 1.00f, -1.00f,
• -1.00f, 1.00f, 1.00f, // top
• 1.00f, 1.00f, 1.00f,
• 1.00f, 1.00f, -1.00f,
• -1.00f, 1.00f, -1.00f,
• -1.00f, -1.00f, 1.00f, // bottom
• 1.00f, -1.00f, 1.00f,
• 1.00f, -1.00f, -1.00f,
• -1.00f, -1.00f, -1.00f
x
y
Top
Bottom
BackFrontRight
Left
z
Set of four vertices
defining a plane
One vertex/corner
of a plane defined
by x,y and z
Image overlay contd..
10/18/2012 61
• Step 1: Replace values of
teapot.h with cube.h (find cube.h in dominos app )
• Step2:Remove all coordinates
expect Top .
• Step3:Set the zero vale of z
coordinate of Top.
• Step4:Replace the images in
assets folder with required Image.
• Top
x
y
z
Top
Bottom
BackFrontRight
Left
Top
Video Playback
Video Playback
• Play video on an image or texture on
compatible devices
• Gracefully fall back to an alternate video-
viewing experience on incompatible devices
(android video play triggered on target)
Java
URL Example
Native C/C++Initialize SDK
Start Trackers
Handle Tracker
Updates
Close SDK
onDrawFrame(..) {public static Handler
mainActivityHandler;
public void displayMessage(String text)
{Message msg = new
Message();msg.obj = text;
mainActivityHandler.sendMessage(msg);} }
renderFrame() {for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++){
jstring js = env>NewStringUTF(trackable>getName());
jclass javaClass = env->GetObjectClass(obj);
jmethodID method = env->GetMethodID(javaClass,
“displayMessage", "(Ljava/lang/String;)V");
env->CallObjectMethod(obj, method, js);
}
protected void onResume(){
super.onResume();
ImageTargetsRenderer.mainActivityHandler = new Handler() {
@Overridepublic void
handleMessage(Message msg) {
Intent intent = new Intent(Intent.ACTION_VIEW);
intent.setData(Uri.parse("http://www.qualcomm.com/partials/video/34133"));
startActivity(intent);}
Unity3d
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |65
I. Installing the Extension into your unity project
II. Compiling a simple AR app, and
III. Mastering some of the more advanced AR topics.
Installation on Windows
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |66
I. Download installer EXE-file, from the download page.
II. Run installer
III. Select a location for the package installation convenient to
your development environment. The extension-only
package will also be copied to the Standard Packages
folder of your Unity installation
Compiling a Simple Project
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |67
I. Create Project
II. AR Assets and Prefabs to Scene
III. 3D Objects to Scene and Attach to
Trackables
Create Project
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |68
Step
1
Step
2
Step
3
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |69
I. Step 1: Click On File-> New Project
II. Step 2: Select Project path and name
III. Step 3: Select Project package name
•
Structure inside Unity
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |70
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |71
� Editor
• Contains the scripts required to dynamically interact with Trackable data in the Unity editor.
� Plugins
• Contains Java and native binaries that integrate the QCAR SDK with the Unity Android application.
� Qualcomm Augmented Reality
• Contains the prefabs and scripts required to bring augmented reality to your Unity application.
� Streaming Assets
• Contains the swappable target datasets downloaded from the online Target Management System.
AR Assets and Prefabs to Scene
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |72
� ARCamera prefab:The ARCamera is responsible for rendering the camera
image in the background and manipulating scene objects to react to tracking
data
� ImageTarget prefab :This prefab represents a single Image Target
Trackable object
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |73
Delete
Main
Camera
Adding Prefabs
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |74
I. Delete main camera from hierarchy
II. Add ARCamera and ImageTarget in hierachy
from project window
ARCamera prefab
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |75
ImageTarget prefab
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |76
3D Objects to Scene and Attach to
Trackables
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |77
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |78
I. Cube object (GameObject > Create Other >
Cube)
II. Lighting (GameObject > Create Other >
Directional Light)
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |79
Directional
Light
Animation
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |80
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |81
I. Select Object from hierarchy window
II. In menu bar select( Windows-> Animation)
•
Animation Window
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |82
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |83
Step
1
Step
2
Step
3 Step
5
Step
4
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |84
I. Step 1:Press the recording button and save animation .
II. Step 2:Press it for creating break points in animation.
III. Step 3:Drage time line according to required length.
IV. Step 4:Allows required type of movement (circular/Hor/Ver etc)
V. Step 5:It give options about to play(once/loop etc)
Android Deployment Process
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |85
Platform
Player
Settings
Bundle
Identifier
and API
Level
Android Deployment Process
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |86
• Unity provides a number of settings when building for Android devices – you
will need to select from the menu (File > Build Settings… > Player Settings…) to
see the current settings. Also, choose your platform now – Android or iOS.
• Click on Resolution and Presentation to select the required Default Orientation.
Note: the Vuforia AR Extension now supports Auto Rotation.
• Click on Icon to set your application icon.
• Click on Other Settings. Set the Minimum API Level to Android 2.2 'Froyo' (API
level 8) or higher. Set Bundle Identifier to a valid name
(e.g. com.mycompany.firstARapp).
• Next save your scene (File > Save Scene).
• Then open the build menu (File > Build Settings…). Make sure that your scene is
part of Scenes in Build. If this is not the case either use Add Current to add the
currently active scene or drag and drop your saved AR scene from the project
view into the Window.
Scripting
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |87
ImageTargetB
ehaviour
ScriptsScript, Data Set
and
Image Target
Virtual Button Case Statements
Loading Default Object w.r.t Trackable
Physics Engine
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |90
Physics
Properties
Physics Engine contd..
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |91
I. Rigidbodies: These are physically simulated objects.Use Rigidbodies for things that the
player can push around, eg. crates or loose objects, or move Rigidbodies around directly by adding forces to it by scripting.
II. Character Controllers: use to make a humanoid character
III. Collider: GameObject that has a Collider but not a Rigidbody. Static Colliders are used for
level geometry which always stays at the same place and never moves around� Box Collider: primitive shape of a cube
� Sphere Collider : primitive shape of a sphere
� Capsule Collider : primitive shape of a capsule
� Mesh Collider : creates a collider from the object's mesh, cannot collide with another Mesh Collider
� Wheel Collider : specifically for creating cars or other moving vehicles
VirtualButtons Example
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |92
Scripts
VirtualButtons Example contd..
10/18/2012Qualcomm Vuforia | Vuforia –
Augmented Reality Platform |93
Buttons
3D Object
Creating and deleting Virtual Buttons
at run-time
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |94
I. create a new Virtual Button for a given Image Target at run-time by
calling the CreateVirtualButton member function on the corresponding
instance of your ImageTargetBehaviour.
II. destroy a Virtual Button by calling DestroyVirtualButton, again defined
in ImageTargetBehaviour
Soccerball Example
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |95
Scripts
Physics
Engine
10/18/2012Qualcomm Vuforia | Vuforia – Augmented
Reality Platform |96
• Regional Ecosystem Strategy
• Brigitte Alexander 1/17/12
Thank you!