Building Usable AR Interfaces

Mark Billinghurst HIT Lab NZ

University of Canterbury

The Vision of AR

To Make the Vision Real..   Hardware/software requirements

 Contact lens displays   Free space hand/body tracking   Speech/gesture recognition   Etc..

  Most importantly  Usability

Usability Issues   Perception

  How to make images appear part of real world?

  Interaction   How to pick content in mid air?

  Social   Do I look stupid?

  Cultural   Wave up or down?

  Cognitive   How do I remember where everything is?

“The product is no longer the basis of value. The

experience is.”

Venkat Ramaswamy The Future of Competition.

experiences

services

products/tools

components

Valu

e Gilmore + Pine: Experience Economy

Function

Emotion

Build Experiences NOT Applications!

experiences

applications

tools

components

Sony CSL © 2004

Building Compelling AR Experiences

Tracking, Display

Authoring

Interaction

Usability ??

The Interaction Design Process

Understand Your Users   Workshops or focus groups

  Group interviews/activities

  Observations   Spending time with users in day to day tasks

  Questionnaires   Looking for specific information

  Interviews   Good for exploring issues, using props

  Documentation   Procedures and rules written down in manuals

Consider the Whole User

AR as Perception Problem   Goal of AR to fool human senses – create

illusion that real and virtual are merged   Depth

  Size  Occlusion   Shadows   Relative motion   Etc..

TAT Augmented ID

AR Rapid Development

Prototyping and User Testing  Low Fidelity Prototyping  Sketches  Paper Prototyping  Post-It Prototyping  PowerPoint Prototyping

 High Fidelity Prototyping  Wikitude, Junaio, Layar, BuildAR etc

POST IT PROTOTYPING

First  Dra)  

Camera  View  with  3D  

Second  Dra)   Third  Dra)  

•   Selec8on  highlighted  in  blue  

•   Home  bu=on  added  for  easy  naviga8on  to  main  menu  

POWERPOINT PROTOTYPING Benefits    •   Used  for  User  Tes8ng  •   Interac8ve  •   Func8onali8es  work  when  following  the  story  of  Scenario  1  •   Quick  •   Easy  arrangement  of  slides  

User  Tes8ng  •   Par8cipants  found  •   15  minute  sessions  screen  captured  

•   ‘Talk  Allowed’  technique  used    •   Notes  taken  •   Post-­‐Interview  

BuildAR

  http://www.buildar.org/   Stand alone application   Visual interface for AR model viewing application   Enables non-programmers to build AR scenes

  Interface Components   Physical components  Display elements

-  Visual/audio

  Interaction metaphor

Physical Elements

Display Elements

Interaction Metaphor Input Output

AR Design Space

Reality Virtual Reality

Augmented Reality

Physical Design Virtual Design

AR Lens Design Principles   Physical Components

  Lens handle -  Virtual lens attached to real object

  Display Elements   Lens view

-  Reveal layers in dataset

  Interaction Metaphor   Physically holding lens

AR Chemistry (Fjeld 2002)

  Tangible AR chemistry education

Case Study: LevelHead

  Block based game

Case Study: LevelHead

  Physical Components   Real blocks

  Display Elements   Virtual person and rooms

  Interaction Metaphor   Blocks are rooms

Goal: An AR application to test molecular structure in chemistry

  Physical Components   Real book, rotation cube, scoop, tracking markers

  Display Elements   AR atoms and molecules

  Interaction Metaphor   Build your own molecule

AR Chemistry Input Devices

Natural Hand Interaction

  Using bare hands to interact with AR content  MS Kinect depth sensing   Real time hand tracking   Physics based simulation model

Evaluation   Need for more evaluation

  2008 -10% AR papers in IEEE, ACM had any evaluation

  Informal   Pilot, ‘quick and dirty’

  Formal   Lab studies, field studies, heuristic

2D vs. AR Navigation?

VS

HIT Lab NZ Test Platform – AR View

HIT Lab NZ Platform – Map View

Distance and Time

No significant differences

Paths Travelled

  Red – AR   Blue – AR + Map   Yellow - Map

Navigation Behaviour   Depends on interface

 Map doesn’t show short cuts

Survey Responses

User Comments   AR

  “you don't know exactly where you are all of the time.”   “using AR I found it difficult to see where I was going”

  Map   “you were able to get a sense of where you were”   “you are actually able to see the physical objects around you”

  AR+MAP   “I used the map at the beginning to understand where the

buildings were and the AR between each point”   “You can choose a direction with AR and find the shortest way

using the map.”

Building Usable AR Interfaces   Understand user needs

 Consider whole user needs -  Physical, emotional, cognitive, social, cultural

  Perceptual issues

  Design for those needs   Rapid prototyping   Virtual, physical elements, interaction metaphor

  Test your design   Formal, informal testing

More Information

•  Mark Billinghurst – mark.billinghurst@hitlabnz.org

•  Website – www.hitlabnz.org – www.buildar.org