Chapter 1 Getting Started with Augmented Reality Augmented Reality Augmented reality (AR), in which

  • View

  • Download

Embed Size (px)

Text of Chapter 1 Getting Started with Augmented Reality Augmented Reality Augmented reality (AR), in which

  • C hap te r 1

    Getting Started with Augmented Reality

    Augmented reality (AR), in which virtual content is seamlessly integrated with displays of real-world scenes, is an exciting area of interactive design.

    With the rise of personal mobile devices capable of producing interesting AR environ-

    ments, the vast potential of AR has begun to be explored. The goal of this chapter is to

    help you become familiar with the terminology, tools, and technology necessary to begin

    your own AR explorations and experiments.

    In this chapter, you’ll learn about the following topics:

    What is augmented reality?■■

    Tools and technologies■■

    AR necessities■■

    036631c01.indd 1 9/1/11 5:33:25 PM

    CO PY

    RI GH

    TE D

    M AT

    ER IA


  • 2■ ■ Chapter 1 : Getting Started with Augmented Reality

    What Is Augmented Reality? The term augmented reality (AR) is used to describe a combination of technologies

    that enable real-time mixing of computer-generated content with live video displays.

    Traditionally, it is distinguished from virtual reality (VR) in that VR involves creating

    complete, immersive 3D environments, and AR uses various hardware technologies to

    create an annotated, or “augmented,” composite based on the real world.

    There are several ways that the virtual components and real content can be made to

    interact. Techniques from image processing and computer vision can be used to make the

    computer-generated elements interact with the content of the video in a convincing way.

    Most current computer vision–based methods rely on predefined physical markers to

    enable the computer vision system to get its

    bearings in the visible 3D space. In Figure 1.1,

    you can see an example of AR in which two

    3D models are manipulated by use of printed

    markers. AR systems that do not require

    specially made markers, known as marker-

    less systems, are also possible. Markerless

    AR is a steadily progressing area of research,

    and in the near future, robust visual AR sys-

    tems that do not require markers will surely

    become widely available.

    Nonvision-based AR methods are gain-

    ing in popularity on smartphone platforms.

    These methods use a combination of the

    device’s global positioning system (GPS)

    or other location-tracking data and accelerometer data to determine where the device

    is located and in what direction it is pointing. Based on this information, tags and

    annotations are superimposed over the scene. These methods are the basis of several

    geographical annotation services such as Layar, which is annotated for locations in the

    Netherlands; mixare, an open source mix augmented reality engine, which currently

    has data for Switzerland; and the Wikitude World Browser, which enables users from

    around the world to contribute localized data.

    This book is primarily concerned with computer vision–based AR. You’ll learn how

    to incorporate computer-generated 3D content into live video using physical markers.

    Getting into the details of location- and accelerometer-based AR is beyond the scope of

    this book. However, I will mention relevant links and references to these technologies

    where they are pertinent to the topic, such as in Chapter 10, “Setting Up NyARToolkit

    for Android.”

    Figure 1.1

    Live video augmented with

    3D models

    036631c01.indd 2 9/1/11 5:33:25 PM

  • What Is Augmented Reality? ■ 3

    A Brief History of AR AR technology has its roots in computer interface research in the early days of computer

    science. Many of the main concepts of AR have been familiar in movies and science fic-

    tion at least as far back as movies like The Terminator (1984) and RoboCop (1987). Both

    movies feature cyborg characters whose views of the world are augmented by a steady

    stream of annotations and graphical overlays in their vision systems.

    Practical systems using AR as it’s currently viewed began to be developed in the next

    decade. The term augmented reality was coined in 1990 by Tom Caudell, a researcher at

    The Boeing Company. Throughout the early and mid 1990s, Caudell and his colleagues

    at Boeing worked on developing head-mounted display systems to enable engineers to

    assemble complex wire bundles using digital, AR diagrams superimposed on a board over

    which the wiring would be arranged. Because they made the wiring diagrams virtual, the

    previous system of using numerous unwieldy printed boards was greatly simplified.

    Throughout the 1990s, industrial and military AR applications continued to be devel-

    oped. But the technical requirements for useful AR displays, such as bulky, expensive

    head-mounted display devices, kept the technology out of reach for most users.

    There were experiments incorporating AR technologies with the arts. Julie Martin,

    wife and collaborator of art technology pioneer Billy Klüver, is credited with producing

    the first AR theater production in 1994. The work, titled Dancing in Cyberspace, uses

    dancers who interact with virtual content projected onto the stage.

    At the end of the 1990s, another major development in AR came about when Hirokazu

    Kato created ARToolKit, a powerful library of tools for creating AR applications.

    ARToolKit helped to make AR accessible to a much wider audience of designers and

    developers, and it provided the basis of much of the technology addressed in this book.

    AR in Practice In spite of the availability of ARToolKit, potential applications of AR have been slow to be

    explored fully. One big reason for this has been the demanding hardware requirements

    for achieving the benefits of an AR interface. The user must be looking at a display that

    has access both to camera data and to data from a computer processor capable of running

    the AR application. This setup was not standard even for consumer desktop environments

    just a few years ago, and it was almost unheard of in consumer mobile environments until

    the rise of smartphones, such as the iPhone and the Android family of phones. These

    smartphones, and the burgeoning tablet computer market that has followed them, have

    helped usher in a new era of interest in AR programming. With smartphones and tablets,

    users have an integrated camera and computer at their fingertips at all times, opening up

    many interesting new possibilities for AR applications.

    036631c01.indd 3 9/1/11 5:33:25 PM

  • 4■ ■ Chapter 1 : Getting Started with Augmented Reality

    Whether by using mobile apps or through browser interfaces on computers with web-

    cams, it has become easier than ever to give users an engaging AR experience on their

    own devices, and the possibilities don’t end there. AR marketing campaigns have been

    used by Hallmark, Adidas, and many other companies. In 2010, toymaker Lego created

    a pioneering point-of-sale AR marketing campaign. Computer screens were set up at toy

    stores carrying Lego toys and shoppers could hold up a box in front of the screen to see a

    3D virtual representation of the completed Lego model in the video. In addition to being

    an incredibly effective point-of-sale campaign, the campaign generated a great deal of

    attention online, as shoppers posted their own videos of the experience. As discussed in

    Chapter 7, “Interacting with the Physical World,” exploring alternate interfaces through

    physical computing can expand the possibilities even further.

    Several trends are leading to an explosion of interest in AR. One is the steady improve-

    ment of computer vision technology, which will enable developers to work with more sub-

    tle, less obtrusive, and more robust markers and even completely markerless AR. Another

    trend is the rapid advance in display technologies for enabling AR. These include head-

    mounted displays and projector-based displays, as well as handheld mobile devices.

    Head-mounted displays are essentially a combination of a camera and display screens

    that are worn like goggles or glasses. The viewing area may be a fully digital, opaque

    video screen that displays video from the camera, or it may be transparent in a way that

    allows the user to see the world directly with data laid over it. Head-mounted displays are

    ideally suited to AR applications and are becoming lighter, less obtrusive, and less expen-

    sive. In the long term, as wearable computers grow in popularity, head-mounted displays

    are likely to become increasingly commonplace.

    Perhaps even more interesting are recent experiments in projector-based AR displays.

    Projector-based displays use projectors to project images or text directly onto surfaces

    in the real world. This approach can be extremely versatile in suitable environments.

    Projector-based displays can be used to present AR environments to large groups of

    people at once in ways that head-mounted displays or mobile devices cannot. Small,

    wearable projectors can also be used to create personal AR environments.

    Other display methods may be available depending on specific applicati