User Interfaces and User Centered Design Techniques for Augmented Reality and Virtual Reality

Embed Size (px)

Text of User Interfaces and User Centered Design Techniques for Augmented Reality and Virtual Reality

UIs and User Centred Design Techniques for AR + VR

Whats AR and VR?!Augmented Reality and Virtual reality.AR: technology that layers over our everyday VR: technology that transports us to a different world

But They Must be Years Away?!

How do we design applications for AR and VR?

IxD for System vs IxD for FrameworkDesigners are even confronted with the obstacle of a methodology or framework to design the system itself and quickly iterate through prototypesSteps are being taken to address these concerns Few applications / frameworks although commercial + open source are helping

Evaluating VR UIs

Goals FormationNavigate the WorldLocate ObjectsPosition for InteractionDeciding on ActionManipulating ObjectsRecognise FeedbackInterpret FeedbackDecide on New Action

VR Related Issues with EvaluationObjects can obscure and may break interaction cycleDifferent modes of design for navigation and for environment driven VR systemsExpert users see the modes blend together

Feedback should be multisensoryCant see things off screen or behind a wall etc. Environmental cues are key

Evaluation TechniquesWalkthrough each phase step by stepAsk necessary questions along the way Aim to uncover breaks in affordancesQuestions guided to create generic design principles (GDPs)Collect design issues and virtual environment featuresPrioritise them for development

UCD Tips for AR Install on a familiar device

Choose a design scenarioRun in an appropriate setting

Build for two hands if on mobileChoose the right audience

Challenge users with mental flow not with physical strain

Issues OverallInformation is often weakly exhibited that link design problems with design solutions in VR and AR.Even some of Normans interaction evaluation techniques can break down when dealt with navigating 3D space.Need a tailored methodology for dealing with emerging issues in hardware and interaction.

Virtual RealityUses, Current Tech and Design

Uses of VREducationWith the leaps in technology, virtual reality can be used to transport people to other planets, tourist destinations and the many jungles and oceans on earth.Video GamesVirtual Reality allows players to be transported to other worlds and puts them in the middle of the action! MedicalVirtual reality can allow surgeons to move throughout the body and diagnose problems that patients have. Virtual reality is also being used for therapy for PTSD veterans and phantom limb syndrome.




VR Technology available today

Designing for VR

User Interface design in VRWhen designing interfaces in a virtual reality, there are some considerations which must be taken into account.

Is there a motion controller used in conjunction with the VR system?What is the nature of the experience? Who is the interface being designed for? What are the perceptual limitations of the user?

Perceptual LimitationsPerceptual limitations occur in the areas that users find interaction difficult in their field of view.3055








Perceptual LimitationsPerceptual limitations occur in the areas that users find interaction difficult in their field of view.3055







Peripheral0.5mRegion where convergence can occur

i love this graph :) although does the person on the right not have their nose in the wrong spot or am i reading the graph wrong?

Theyre nose is in the right position! its a side view

How do we design UIs with these limitations in mind?

How do we design UIs with these limitations in mind?

Is this a good UI design or bad UI design in VR?

Augmented RealityUses, Current Tech and Design

Uses of ARNotificationAR headsets can notify you of social media, texts or email as you go about your daily life.Video GamesAR allows players to have their world transformed in front of their very own eyes!NavigationAR allows people to find their destination in a non- obtrusive way. source: google.comsource:

AR Technology available today

Designing for AR

User Interface design in ARWhen designing interfaces in a augmented reality, the considerations taken into account are similar to VR.

But there is one thing that is the utmost importance!

Obscurity / Opacity.

User Interface design in VR

Continue 10m

User Interface design in VR

User Interface design in VR

Case Studies1.Design and Evaluation of Menu Systems for Immersive Virtual Environments - Bowman & Wingrave 2001

2.Experimental Evaluation of User Interfaces for Visual Indoor Navigation - Moller et al. 2014

Key Interaction Tasks VRNavigation



System Control



Floating Menu


Pen and Tablet Menu

Pinch Gloves

Hardware for TULIP Menu

Pinch Gloves consist of a flexible cloth gloves augmented with conductive cloth sewn into the tips of each of the fingers.

When two or more pieces of conductive cloth come into contact with one another, a signal is sent back to the host computer indicating which fingers are being pinched

Virtual Research V8 head-mounted display (HMD) and the head and both hands are tracked using a Polhemus Fastrak tracking system

Evolution of TULIP

Scrolling Menu

Three Up Menu

Scrolling Menu

Three Up Menu

Pilot StudyEvaluating these two menu designs, users had to change a virtual object to match a target object.

3 parameters could be controlled: the objects shape, color, and texture. Each of these corresponded to a top-level menu.

There were 3 shapes to choose from, 8 colors, and 6 textures these corresponded to second-level menu items.

Test 4 users - Think Aloud - Informal Results

Pilot ResultsNeither design satisfied the desired requirements.

Users did prefer the Scrolling Menu BUT realized tasks could be completed with less steps using the Three Up Menu.

Three Up Menu hides options if there is more than 3.

Scrolling Menus prompt users to incorrectly attempt to push palms.

Lack of feedback when items selected and fatigue from hands being raised.

RE-DESIGNT hreeU pL abelsI nP alm

Summative Evaluation

Compare Ease Of Use

Compare Ease Of Learning


Comfort of all 3 Menus

Summative Evaluation26 users participated

Repeated object matching task

Completed a questionnaire containing demographic information and information about their experience with computers and VEs

Same equipment used again with added stylus for floating menus and pen and tablet menus

30 trials of each menu and no help provided after initial briefing

Floating MenusPen and Tablet Menus

ResultsAppears that the gloves were the hardest to learn initially, but performance was at reasonable levels for all three types within five trials.

Reason times for the pen and tablet menu are initially poor is that users were not told they needed to look at the tablet in their hand

Comfort Levels

The main drawback of the pen and tablet system is the discomfort it causes users, which might be alleviated by adding an ergonomic handle.

Reflection On StudyCombining the efficiency, comfort, and preference information, it appears that both the pen and tablet menu and the TULIP menus performed well in the evaluation

Fifteen users expressed a preference for the TULIP interface, while nine preferred the pen and tablet, and only two preferred the floating menus

This evaluation reiterated some important heuristics from the traditional human-computer interaction literature. Menu systems for VEs need to have good feedback, affordances, and constraints, and items and their actions should be visible.

Evaluation of User Interfaces for Visual Indoor Navigation

Andreas Moller et al.Implemented a novel UI for visual localization, consisting of Virtual Reality (VR) and Augmented Reality (AR) views that actively communicate and ensure localization accuracy.

Beneficial for large buildings and navigating your way around.

The concept consists of a panorama-based view as a complement to Augmented Reality and proposes different visualizations for motivating users to record good query images.

Good query images are important

Comparing AR vs VRThe users would hold the phone up as seen in the figure and look at the phone in order to see the augmentation i.e. items superimposed onto their real life surroundings.

The virtual reality displays pre-recorded images of the environment (downloaded from a server) that are arranged to a 360 panorama on the mobile device

Enhancing Visual LocalizationVisual localization can be very dependent on how the device is being held.

Four indicator types were proposed which were:Text HintBlurColour ScaleSpirit Level

Initial TestingA questionnaire-based survey with mockup videos were used for early testing. Users did not actually travel through a building.

From initial user evaluations there was an inconsistency. Majority of people stated that the VR mode helped them orient themselves even if the location estimate of the system was incorrect yet the subjects still claimed they prefer the AR mode.

Users preferred the T


View more >