An Introduction to Augmented Reality - unict.itfurnari/slides/augmented_reality_10_05_2014.pdfAn Introduction to Augmented Reality [email protected]. Computer Vision AA. 2013/2014

Image Processing LabDipartimento di Matematica e Informatica

Università degli Studi di Catania

Antonino Furnari

10/05/2014

An Introduction to Augmented Reality

[email protected]

Computer Vision AA. 2013/2014 - Augmented Reality

Augmented Reality

“a live, copy, view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory

input such as sound, video, graphics or GPS data”

“it is related to a more general concept called mediated reality, in which a view of reality is

modified (possibly even diminished rather than augmented) by a computer”

“as a result, the technology functions by enhancing one’s current perception of reality.”

wikipedia definition


Applications


Applications


Some History

● The term “augmented reality” appears since the 1940s;

● The first augmented head mounted display is invented by Ivan Sutherland in 1968;

● First systems using mobile devices, internet and geolocalization appear in the 90s;

● Advances in the 2000s;● Augmented Reality diffusion in the 2010s.


Hardware

Some technologies which make AR interesting:● Head-mounted (with 6 DoF monitoring);● Handheld:

– Mobile phones;– Tablets;

● Wearable devices:

– Google glass;– Orcam (video - http://www.orcam.com/);– Epson moverio.

http://www.orcam.com/


Computer Vision based Augmented Reality

● Computer Vision allows to create augmented reality applications by superimposing 2D or 3D contents on the scene;

● In order to do so we need to:

1) detect and track the area where to show the content;

2) estimate its 3D position in the real world;

3) render the 2D/3D content according to the estimated position;

● Two main technologies:

– fiduciary markers;– markerless (i.e., object detection).


Fiduciary Markers Augmented Reality: ARToolkit

● ARToolkit is an Open Source toolkit for marker-based augmented reality;

● It is quite old (last update in 2007) but still a good starting point for understanding the AR concepts (open & well documented);

● It offers functions for detecting and tracking single or multiple markers while relaying on OpenGL/glut for 2D/3D rendering;

● http://www.hitl.washington.edu/artoolkit.

http://www.hitl.washington.edu/artoolkit


ARToolkitdemo


ARToolit

● For additional informations about the next topics, the reader is referred to the very well written ARToolkit documentation and tutorials:

– http://www.hitl.washington.edu/artoolkit/documentation/

● Some other useful information can be found in the examples provided with the toolkit.

documentation

http://www.hitl.washington.edu/artoolkit/documentation/

http://www.hitl.washington.edu/artoolkit/documentation/


ARToolkit

● Follow the instructions at:

– http://www.hitl.washington.edu/artoolkit/documentation/usersetup.htm

● For Linux/GStreamer configuration, include the following line (check your video device) in your .bashrc file:

– export ARTOOLKIT_CONFIG="v4l2src device=/dev/video0 usefixedfps=false ! ffmpegcolorspace ! capsfilter caps=video/xrawrgb,bpp=24 ! identity name=artoolkit ! fakesink"

setup

http://www.hitl.washington.edu/artoolkit/documentation/usersetup.htm

http://www.hitl.washington.edu/artoolkit/documentation/usersetup.htm


ARToolkitbasic principles


ARToolkitthe marker

● The marker plays the role of an object which geometry is known;

● In particular:

– we chose markers which are easily detectable (thick black borders);

– we know the real world size of the marker;– we chose the inner symbol which is neither

horizontally nor vertically symmetric in order to estimate its rotation.


ARToolkitmarker detection

original image thresholded image connected components

contours edges and corners fitted square


ARToolkitmaker candidates

● This method actually allows to find just candidates: any pattern with thick black borders;

● The actual marker is found normalizing the candidates and comparing them with the searched pattern using template matching;

● The candidate giving the highest confidence is selected.


ARToolkitmarker matching

0.3

0.92

0.2

0.4

normalization

...

...

...

searched pattern

normalized candidates

found candidates


ARToolkit3D position and orientation estimation

● Now that we have an object which geometry, size, position and orientation are known, we can estimate its 3D position with respect to the camera;

● It can be done computing the extrinsic parameters as seen for camera calibration;

● Intrinsic parameters which are good for most cameras are part of the toolkit. Specific parameters can be obtained calibrating the camera.


ARToolkitcoordinate systems


ARToolkitcamera and marker relationships (demo - exview)


ARToolkitarchitecture


ARToolkitdevelopment principles


ARToolkitinto the code!


ARToolkit

● Facilities are included in the toolkit to enable multi-marker tracking;

● A multi-marker pattern is used where the relative position of the markers is known;

● The transformation matrices between the different markers must be supplied.

multiple markers (demo)


ARToolkittrain a new pattern

● In order to use your own pattern, the blank pattern has to be modified and printed;

● The provided mk_patt application allows to generate a patt.yourpatt file containing all the information needed for the pattern matching.


ARToolkitcamera calibration

● Intrinsic parameters which are enough general to work with most of the cameras are available in the toolkit;

● Although, in order to improve the detection and tracking performances, a utility for camera calibration is provided in order to calibrate your own camera.


Markerless Augmented Reality?

Tracking a number of feature points (e.g., SIFT) in order to detect a marker object (e.g., a photo) and to estimate its position and orientation.

Some mobile frameworks available (e.g., Vuforia).


Question Time


Contacts

● For any doubts please contact me:

– [email protected];

– Room 30;● Slides available at:

– Studium course page:● http://studium.unict.it/dokeos/2014/course

s/73072C2/

Documents

An Introduction to Augmented Reality - unict.itfurnari/slides/augmented_reality_10_05_2014.pdfAn Introduction to Augmented Reality [email protected]. Computer Vision AA. 2013/2014