Computers & Graphics 35 (2011) 841–853

Contents lists available at ScienceDirect

Computers & Graphics

0097-84

doi:10.1

n Corr

E-m

shineall

journal homepage: www.elsevier.com/locate/cag

Mobile Augmented Reality

Aroundplot: Focusþcontext interface for off-screen objectsin 3D environments

Hyungeun Jo, Sungjae Hwang, Hyunwoo Park, Jung-hee Ryu n

Graduate School of Culture Technology, KAIST, 335 Gwakak-ro, Yuseong-gu, Daejeon, Republic of Korea

a r t i c l e i n f o

Article history:

Received 11 February 2011

Received in revised form

26 April 2011

Accepted 26 April 2011Available online 6 May 2011

Keywords:

Off-screen object visualization

Orientation

Augmented reality

Virtual environment

Mobile device

93/$ - see front matter & 2011 Elsevier Ltd. A

016/j.cag.2011.04.005

esponding author. Tel.: þ82 42 350 2902; fax

ail addresses: acid@kaist.ac.kr (H. Jo), best@ka

@kaist.ac.kr (H. Park), junghee.ryu@kaist.ac.k

a b s t r a c t

In exploring 3D environments from a first-person viewpoint, the narrow field-of-view makes it difficult

to search for an off-screen object, a task that becomes even harder if the user is looking through the

small screen of a mobile phone. This paper presents Aroundplot, a novel focusþcontext interface for

providing multiple location cues for off-screen objects in an immersive 3D environment. One part of

this technique is a mapping method from 3D spherical coordinates to 2D orthogonal fisheye, which

tackles the problems of existing 3D location cue displays such as occlusion among the cues and

discordance with the human frame of reference. The other part is a dynamic magnification method that

magnifies the context in the direction the view is moving to alleviate the distortion of the orthogonal

fisheye and thus to support precise movement. In an evaluation, the participants could find the target

object for a given location cue faster and more accurately with Aroundplot than with a top-down 2D

radar. They were more accurate with Aroundplot than with a 3D arrow cluster when the number of

objects was large; however, accuracy with a small number of objects and the search speed with any

number of objects were not significantly different.

& 2011 Elsevier Ltd. All rights reserved.

1. Introduction

While performing tasks in augmented reality (AR) and virtualenvironments (VEs), users are often required to search for specificnearby objects. In information-rich virtual environments (IRVEs) [7],the user is expected not only to walk around within the environ-ment but also to examine information associated with 3D locationsand to plan navigation based on that information. AR combined withpervasive computing [27], which has thus far been used mostly foroutdoor 2D locations on the ground, is now rapidly expanding to 3Dand indoor locations as a result of advances in 3D localization formobile phones [3] and in real-time 3D map construction [10,25].

However, searching in AR and VE often becomes difficult becauseof the limited field-of-view (FOV) of the (real or virtual) camera, aswell as the first-person viewpoint in which the user must move theview around every spherical angle to notice the existence of a nearbyobject. This challenge is particularly difficult in AR implementationson mobile phones, which have narrow FOVs (38.71 horizontally and50.11 vertically in the case of the iPhone4 camera) and very smallscreens. This problem leads to the need for useful overviewvisualization of the locations of objects external to the FOV.

The most common overview for this purpose is the top-down2D radar visualization. When applied in 3D environments,

ll rights reserved.

: þ82 42 350 2980.

ist.ac.kr (S. Hwang),

r (J.-h. Ryu).

however, the disparity between the overview and the camera viewcauses a number of problems. The difficulty of mental rotation [2]to compare between the views increases with the number ofobjects and becomes almost impossible when the objects are atdifferent heights. A lack of height indication also means that theuser cannot be informed of whether to move the camera up ordown to access the desired object, even with a single object.

To overcome the limitations of a top-down view, overviewsbased on 3D rendering and pointing, such as a 3D arrow cluster[11], have been actively proposed. However, one common pro-blem is the occlusion that occurs when the number of objects islarge; the rear arrows occlude the more important forwardarrows. Reducing the arrow size is also difficult because theshape of the arrow must be clearly identified if it is to supplydirectional information. Another problem is that many of such 3Doverviews are not based on the human frame of reference [9],which sometimes makes the 3D rotation of the overview hard totrack and the 3D direction confusing; for example, if a target isbehind the head, the user could acquire the misconception that itcould be accessed by raising the view.

In response to the above problems, we propose Aroundplot, anoff-screen object interface for 3D environments based on afocusþcontext approach.

The contributions of this paper are as follows:

�

A mapping method from spherical coordinates to a 2D orthogonalfisheye [31] to display the off-screen objects with little occlusion,

H. Jo et al. / Computers & Graphics 35 (2011) 841–853842

which provides an improved capability to maintain the humanframe of reference compared to existing mapping methods.

� A dynamic magnification technique that complements the

distortion of the orthogonal fisheye by magnifying part ofthe context in the direction of the view movement and thussupports closer examination and precise movement.

� An evaluation of the proposed interface, the top-down 2D

radar, and the 3D arrow cluster, analyzed in terms of thehighlighting condition, the number of objects, and the anglesof the device and target.

The remainder of this paper introduces related work, definesthe problem in more detail, and then describes the two techni-ques comprising Aroundplot: the mapping method and thedynamic magnification. We then present the results from anevaluation of Aroundplot compared to the top-down 2D radarand the 3D arrow cluster.

v v2

v1

Fig. 1. Naıve mapping methods: (a) Lat/Lon mapping and (b) screen projection.

2. Related work

2.1. Overviewþdetail and focusþcontext

Overviewþdetail (OþD) and focusþcontext (FþC), of which acomprehensive review is given in [12], are two representativeapproaches to navigating in a workspace that is greater than thescreen size while maintaining the original resolution. OþD tech-niques are characterized by an overview given in a separatewindow from the detail view. In most cases, the overview is athumbnail of the entire workspace and is located over or near thedetail view. In contrast, FþC techniques such as fisheye views [31]set the detail view as a focus and provide the overview as a contextthat encompasses, seamlessly in most cases, the focus area.

It is difficult to judge which of the two approaches outper-forms the other because the answer varies considerably with thedetails of the implementation and with the specific task [22,29].A drawback of FþC is that remembering the object location isdifficult because of the persistent change in the overview layoutbased on the focus movement [34]. However, this problem can beinherently complemented in AR or VE because the objects can beremembered by their locations in the environment, in contrast toabstract data visualization, where the environment is merely anempty space. On the other hand, the ability of FþC to examine theoverview and detail view in the same viewpoint is known tocontribute to the comparison between views [4] and to steeringtasks [20]. These merits provide a good reason to adopt FþC forthe current study, particularly because orientation with recentmobile devices is often performed by steering the device ratherthan touching it.

2.2. Off-screen object visualization in 2D

In some cases, the user’s interest might be focused on specificobjects rather than the entire image of the workspace: forexample, venues on maps and characters in games. For suchcases, although both OþD and FþC would work, a particular classof FþC has been developed through a number of studies. Thetechniques of this class use screen space more efficiently bydisplaying only the cues for the objects instead of the wholecontext and, moreover, by locating the cues inside the borders ofthe focus area.

One such technique is City Lights [41]. This technique,described by the authors as a ‘‘discrete fisheye,’’ uses a minimalarea by displaying only a line segment or a point to indicate thedirection of an object on the edge of the screen. Halo [5] and

Wedge [17] can be seen as variants of City Lights, using partialobjects instead of lines or points. These techniques let the usersinfer the exact location of the object by mentally completing thepartial objects such as circles (Halo) and triangles (Wedge).EdgeRadar [18] employs an orthogonal fisheye view overlaidabove the edges of the screen to support visual tracking of movingtargets. In addition to these visualization techniques, there is alsoan efficient selection technique for locating off-screen targets byhopping [23].

Although these techniques have proven useful for locations in2D space, utilizing them for 3D locations is not straightforwardbecause developing 3D visualizations extended from them willintroduce new challenges, such as depth perception and front–rearrepresentation; conversely, choosing to exploit their as-is 2Dvisualizations will require a new mapping method from 3D to 2D.Examples of the 3D rendering approach are the two kinds of 3DHalo that were proposed in a recent study [40]. These 3D Halos,however, convey only abstract information about the objectsbehind the user. Moreover, a 3D perspective view reduces animportant advantage of Halo, its ability to represent exact distances.

Our approach is to employ an existing 2D visualization as thefinal representation and to develop a mapping method from 3Dlocations around the user to this 2D visualization. Such a mappingmethod needs to be newly developed because the aforemen-tioned off-screen visualizations are mapping functions betweenspaces of the same dimensionality and do not describe how toproject onto lower-dimensional spaces. As the target visualiza-tion, we choose an orthogonal fisheye, into which EdgeRadar andCityLights with points can be suitably generalized.

However, an orthogonal fisheye and the visualizations inspiredfrom it have common limitations: a high density at the cornersand a scale difference between the corners and sides. In responseto these limitations, we propose a new technique that dynami-cally magnifies part of the context area where the view move-ment is heading so that the high density and the densitydifference can be reduced. This technique has relevance toSCF (speed-coupled flattening) [19], which reduces the distortionlevel of a fisheye view based on the speed of a mouse pointer.However, one of the major differences of our approach is that SCFmagnifies the entire context regardless of the direction ofmovement.

2.3. Naıve methods for 3D–2D mapping

While our approach to employ an existing 2D visualizationneeds an appropriate spherical-to-Cartesian mapping for off-screen objects, finding such a mapping method is a challengingtask. The examples of naıve mapping methods in Fig. 1 illustratesome of the difficulties. Lat/Lon mapping, i.e., mapping elevationand azimuth to x and y directly, is the most easily conceivablemethod. This method is fairly usable near the reference plane, but

H. Jo et al. / Computers & Graphics 35 (2011) 841–853 843

vertically rotating the view creates severe distortion, as shown bythe trail of a point v that is fixed on the camera view.

Screen projection can be seen in some first-person shootergames, often signifying the direction but not the distance fromthe camera. Though usable with one or two objects, a majordrawback of this method is that it guides awkward rotation thatignores the human frame of reference, as in Fig. 3b. In addition,projecting to the front and rear surfaces often causes location cuesto cross each other because the cues for front and rear objectsmove in opposite directions. The user also cannot know whetherthe object is in front of or behind their reference point becausethe cues share the same space (points v1 and v2 in Fig. 1b).

2.4. Off-screen object visualization in 3D

In the early development of AR, Feiner et al. [15] guided theuser’s attention to an off-screen object with a rubber band-likevisualization that was combined with highlighting the objectitself. The attention funnel [6] is a more explicit visualizationmethod that is similar to a flight tunnel, employing a series ofrectangles starting from the user and ending at the off-screenobject. Both of these visualizations provide spatial cues for off-screen objects from the same perspective as the user’s view,corresponding with the aim of our study; however, theseapproaches are not appropriate for displaying multiple objectsbecause the cues for each object are too large and can overlapwith each other, which leads to screen clutter.

World-In-Miniature (WIM) [35] is a 3D OþD approach fornavigation in VE. This technique provides the user with a 3Dminiature of the surrounding space and displays the locations ofobjects on it. However, WIM can only be used in a specific placefor which the 3D model is given. Furthermore, occlusion by otherobjects also occurs.

Using 3D arrows is another approach for providing spatial cuesfor off-screen objects. For a single target object, a 3D arrow hasbeen iteratively tested for alerting car drivers to the direction ofhazards [38,39]. The 3D arrow has also been tested for warehouseorder picking [33], and it performed similarly to the attentionfunnel. For multiple objects, a 3D arrow cluster anchored at asingle pivot was proposed [11]. It has also been shown that fornavigating in VE, the 3D arrow cluster is more useful thanconventional top-down 2D radar for targets in abstract 3D space;however, it performed similarly to the 2D radar for targets on theground. Other researchers studied the efficiency of using 3Darrows and 2D radar for orientation toward ground objects inAR [8,32]. The performances were similar, as in VE navigation. Inaddition, one study [8] has shown that the orientation time isproportional to the angle and that these two interfaces outper-form other interfaces, including vibration or sound feedback.A context compass [36] has been another well-known solutionfor ground objects, although it is less common than top-down2D radar.

There has also been a line of studies related to zooming,panning, or distorting the camera view itself to change the FOVon demand. Animated transition from the camera view to apanorama, similar to the Lat/Lon map, or to a top-down maphas been explored [1]. The radial distortion technique [30]expands the FOV in AR by distorting the whole scene into afisheye view using a 3D-reconstructed model that even includesthe off-screen area. In contrast, the EXMAR technique [21]receives a video stream via an optical fisheye lens and removesthe fisheye distortion in real time, enabling the user to panthrough the scene via touch gestures or small motions ratherthan by moving the camera. Radial distortion and EXMAR sharethe use of fisheye techniques with Aroundplot, but they differ inthe prerequisites (off-line 3D reconstruction or an optical fisheye

lens), the fisheye style (radial), FOV (o1801), etc. More impor-tantly, all these techniques can be considered to have a comple-mentary relationship with Aroundplot rather than a substitutiveone because they do not provide the means to examine both theundistorted video stream and the off-screen information at thesame time.

The 3D off-screen visualization problem was discussed notonly for AR and VE but also for abstract 3D space, particularlyfocused on multi-scale objects and occlusion management [26].The authors presented various designs that seem to stem fromLat/Lon mapping, screen projection, and the 3D arrow cluster. Aninformal test in the study reported that the Wedge Ring, which issimilar to screen projection, was the least preferred because ofconfusion; the Mirror Ball, similar to 3D arrows, was the mostpreferred. The Mirror Ball, however, does not directly apply to ourproblem because it shows only the half-space behind the camerato display information for multi-scale navigation instead.

In summary, the 3D arrow cluster seems to be the most viablevisualization method for multiple off-screen objects registered in3D environments. As explained in the Introduction, however, theproblems with occlusion and the human frame of referenceremain unsolved. These problems have not been clarified in theaforementioned studies [8,11,32] because they used a smallnumber of target objects (between one and four), and theexperiments in AR or realistic VE only involved ground locations.

3. System design

In this section, we will begin by describing the problem spacein more detail. Then, we will illustrate how our method forspherical coordinates to 2D rectangular fisheye mapping solvesthe problems of occlusion and the human frame of reference.Finally, we will introduce the dynamic magnification techniquefor alleviating fisheye distortion and possible variations in theimplementation of Aroundplot.

3.1. Problem definition

The scope of this study involves applications in AR or VE thatare navigated in the first-person viewpoint. The task scenariocovers the following variables: place (indoor/outdoor), number ofobjects, device (PC/mobile phone/HMD), camera (real/virtual) andthe type of target (area or space, physical object, virtual object,and person). Data visualization in an abstract 3D space or a third-person viewpoint VE is outside the scope of this work becausethey do not need to be bound to a single viewpoint. Variousvirtual camerawork and projection techniques, as reviewed pre-viously [14], could be more effective for such applications.

Our scenario of the information search process in 3D environ-ments is shown in Fig. 2. In this diagram, we will concentrate onprocedures (3)–(5), the orientation of the camera view to find thetarget object with the given cue. This process is very similar towayfinding navigation [28] in that it is not composed of a singleorientation but rather of an iterative sequence as follows: gettinga sense of where the target is, orientation, perceiving changes inthe cue, and then modifying the orientation. The difference is thatthe resulting action is orientation instead of locomotion. Theimportance of this orientation should be emphasized, as the fullsequence is often repeated in performing a single real-world taskuntil a proper target is found and a proper decision is made.

The number of simultaneously presented objects shouldbe regarded with considerable concern here. When the userbrowses venues in an urban area or messages in a popular place,the query results can number in the dozens or even hundreds.A commercial service may adopt filtering or clustering processes

H. Jo et al. / Computers & Graphics 35 (2011) 841–853844

or show only a portion of the results. However, because suchmethods can introduce other problems in usability, the visualiza-tion should itself still be able to handle a certain number ofobjects. For example, the evaluations in this paper set the numberof objects at 5 and 50. If 50 objects were uniformly distributed inevery direction, then 2–3 objects would appear in the FOV of amobile phone camera.

3.2. Spherical coordinates to a 2D orthogonal fisheye

3.2.1. Results of mapping

Among the various possible visualization formats, we adoptedthe orthogonal fisheye as the basis for the results of the mapping.Given the shape of the camera view as a rectangle and matchingthe rectangle to the focus area, the orthogonal fisheye provides an

Fig. 3. (a) Which rotation does the ‘‘right’’ sign indicate

1) Perceive the existence of any location cue

5) Identify the target by comparing with the cue

7) Decision & Action: Navigation, Communication, Transaction, etc.

Decision Making Possible?

N

Y

For a

noth

er o

bjec

t

3) Recognize the path to rotate the viewto see the object to which the cue is pointing

4) Rotate the view while tracking the cue

2) Choose a target cue

6) Examine the position in the environment andthe characteristics of the target

Feels the target isin the view?

Y

N

Fig. 2. The information search process in a 3D environment with location cues for

off-screen objects.

intuitive sense of the relative direction from the camera view tothe target and the switching of the target from off-screen to on-screen. Moreover, if we do not count the empty space in thecontext area, a cue on the orthogonal fisheye can indicate boththe direction and distance to a target with minimal screen space:a dot. The variation of this basis through the implementationdetails will be illustrated in Section 4.3. For one variation, Fig. 10shows an EdgeRadar [18] style, which was used in our evaluation.

We also explored radial visualizations, which seemed to bemore advantageous for displaying an arbitrary angle, but wediscarded the idea because a linear movement of the view wastransformed into circular movements of the cues that are unin-tuitive and difficult to track, as the authors of City Lightsdescribed [41]. The number of objects we aimed to track alsoprevented us from using the partial-object approach, whichrequires much larger cues than the fisheye with simple dots.Note that these visualizations may be useful when there are onlya small number of objects.

3.2.2. Frame of reference based on gravity

Here, we solve the problem of a frame of reference by setting aproper rotation axis and rotation bound. If the user is looking atthe screen in an inclined posture and the guidance suggests ‘‘right’’(as shown in Fig. 3a), what will be the recognized axis of rotation?Although humans can use three kinds of reference frames torecognize the spatial position of an object—view-centric (deictic),object-centric (intrinsic), and environment-centric (extrinsic)[37]—the most dominant frame is the environment-centric per-spective because of the effect of gravity [9,16]. This means that theleft and right directions should be the rotation in azimuth aroundthe gravity axis (Fig. 3c). The rotation around the device axis(Fig. 3b) is not only physically difficult to follow but also causesconfusion from the unexpected movement of cues.

The second problem is the rotation boundary shown in Fig. 4.Simply put, the human ability to look up and down is limited toabout 7901 in pitch. This coincides with the gravity axis because

? (b) Device axis rotation. (c) Gravity axis rotation.

v

Fig. 4. (a) Unbounded rotation in pitch and (b) bounded rotation in accordance

with the gravity axis.

H. Jo et al. / Computers & Graphics 35 (2011) 841–853 845

standing upright is the basic posture of humans. It is noteworthythat the boundary of rotation does not mean the boundary of theviewing area. Accessing objects that are slightly behind the user(point v in Fig. 4b) can be more efficiently accomplished by liftingthe view without rotating in azimuth.

3.2.3. Mapping process

This section describes how we generate the 2D rectangularfisheye that satisfies the human reference frame and guidesefficient movement within the limitation of human physical ability.Most of all, the result should not only look but also behave like anormal orthogonal fisheye on a 2D plane. That is, given yaw andpitch rotations matched to x- and y-axis movement, the objects forthe cues above/below the viewport can be reached with only pitchrotation, and the x-coordinates of all the cues undergo little changeduring the rotation (similar to the left/right cues of the viewport).

yaw axis

�

pitch axis

�

Fig. 5. Basic scheme for the mapping. (For interpretation of the references to c

v

cv

�v

�v

�max

v

cv�v

�max

upper/lower area

diagonal area

example of wro

�max

�max

�v = 0

v

cv

�v

left/right ar

Fig. 6. (a) Acquiring yv, jv, ymax, jmax and cv, which are needed to define the position of

the view frustum. (b) Definition of y0 and j0, which are needed in the calculation of t

This requirement helps preserve the relative layouts among thecues, thus making their movement more predictable.

Fig. 5 shows the mapping scheme used to implement therequirements. Given a target object v and a distance from therotation pivot Dv, Aroundplot creates an imaginary sphere ofradius Dv; the intersection between the sphere’s surface and theview frustum then becomes the orange area in the figure. Forconvenience, we will call the orange area an intersection patch.The white areas in Fig. 5 show the areas that can be reached byeither yaw or pitch rotation only. The upper and lower whiteareas are bounded, more specifically, by the arcs around the yawaxis whose diameters equal the distance between the two upper(or lower) corners of the intersection patch.

Next, depending on which of the three kinds of areas the object v

populates, we can calculate the exact rotation angles yv and jv, themaximum possible rotation angles ymax and jmax, and cv, a point on

Rotate Up Up-RightUp-Left

Down-Left Rotate Down

Rot

ate

Lef

t

Rot

ate

Rig

ht

Down-Right

olor in this figure, the reader is referred to the web version of this article.)

�v

�v = 0

ng �v

v

�0

�0–�

�0

�0

dd

d

�0+��0

d

when � = 0,ea

v in Aroundplot. There exist three cases according to the relative position of v from

he upper and lower margin. d is a configurable parameter.

dynamicdevice halts for a while

H. Jo et al. / Computers & Graphics 35 (2011) 841–853846

the border of the intersection patch that will meet v when therotation has been performed as guided (Fig. 6a). To be more specific:

devicemoves

devicemovesdtop

�0–�

(1)

static restoring�0

If v is in the upper/lower area:yv is the pitch by which the intersection patch should rotatefrom the current position to meet v, and ymax is the pitch tomeet the end of the upper/lower area, while jv is 0, and jmax

does not need to be calculated.

(2)

drightdleft

Else if v is on the left/right area:Similarly, jv is the yaw by which the intersection patchshould rotate to meet v, but note that jmax is the yaw for cv

rather than for the intersection patch to meet the end of theright/left area.

margin sizes

(3) returnedto the defaultdbottom

�0+��0

Fig. 7. State diagram of the dynamic magnification of Aroundplot.

Else (v is in the diagonal area):yv is the pitch by which the intersection patch should rotatefrom the current position to reach the elevation of v, and jv isthe yaw by which the intersection patch should rotate after the

pitch rotation to meet v. cv is always at the corner of theintersection patch in this case. Note that the pitch calculationshould be performed first to avoid generating a jv thatproduces excessive yaw rotation for v at a high or lowelevation. An example of such an incorrect calculation isillustrated in the box in Fig. 6a.

The signs of the angles follow the rotation directions shownin Fig. 5.

We also calculate the margin widths at the four sides aroundthe inner rectangle, as defined in Fig. 6b. The values y0 and j0

are the absolute values of the maximum possible rotation anglesof the intersection patch when the pitch is 0, which do not changethroughout use. Note that the margin size at each side isproportional to the maximum angle in each direction. Becausethe upper and lower maximum angles vary with the current pitch,the upper and lower margins are functions of y.

With these variables, we can finally acquire v0, the resultingscreen coordinate of v. This calculation is performed by projectingcv on the screen plane and adding the offset ov to it as in thefollowing equation:

ov ¼

d fv

9fmax9, d y0�y

f0

yv

9ymax9

� �, yZ0

d fv

9fmax9, d y0þy

f0

yv

9ymax9

� �, yo0

¼ dfv

9fmax9, d

yv

9y09

!8><>:

v0 ¼ projðcvÞþov

3.3. Dynamic magnification of the context

The major weak point of any kind of orthogonal fisheye is thedifficulty in recognizing the amount of rotation as a result of thecompressed context area. In particular, the density of the diagonalareas is much greater than in other areas.

To overcome this problem, we propose an interaction techni-que that dynamically magnifies the context in the direction of theview movement. This magnification is logical because the move-ment of the view means that the user is interested in the off-screen area in that direction, rather than the current camera view.Thus, we suggest that providing more detail and enabling precisemanipulation in the direction of movement can greatly enhanceusability.

The state diagram for this process is given in Fig. 7, and theimplemented example is shown in Fig. 8. In the static state, themagnification begins when the view is displaced from the startingorientation by a threshold angle. During the dynamic state, dtop,dbottom, dleft, and dright separately change from d to dmax accordingto the movement in the corresponding direction so that themaximum magnification scale becomes dmax/d. Note that when

the margin in one direction is magnified and the user changes therotation to the opposite direction, the magnification in the newdirection and the shrinking in the old direction are simulta-neously executed. If the device halts for a while (thresholdvariance4variance of displacements from the current orientationfor a pre-defined duration), then it enters a restoring state, andthe orientation at that time becomes the new starting orientation.

3.4. Variations in implementation

Because some of the 2D FþC and off-screen object visualiza-tion techniques share many characteristics, the visual presenta-tion of Aroundplot can be switched from one of them to anotherby simply altering the implementation parameters. Two numericparameters that can be easily changed are d, which determinesthe width of the context area, and dmax, which determines thedynamic magnification scale. For example, setting dmax¼d¼0turns the appearance of Aroundplot into that of City Lights points[41] (Fig. 9a). Another Boolean parameter is overlay, whichdesignates whether the camera view and the overview are over-laid. With dmax¼d40 and overlay¼ false, Aroundplot becomes anormal orthogonal fisheye view (Fig. 9b). Setting dmax¼d40 andoverlay¼true turns its visual form into EdgeRadar [18] (Fig. 9c).These variables can be chosen according to the kind of device orapplication.

Setting d40 and overlay¼true requires particular attentionbecause of a discontinuity problem between the camera view andthe overview when an object comes into the camera view; the cuedisappears at the inner rectangle, but the object appears at thescreen border. The interface might still be usable because one canpredict where in the camera view the target will appear byobserving where on the edge the cue disappears and can dis-cretely move the visual attention [13], but it is clear that thisprediction process would demand more cognitive load. However,the advantage from this setting is that the size of the camera viewcan be maximized. This can be more important when consideringthe small screen of a mobile device.

There is also a point to consider with dynamic magnification(dmax4d) in relation to the overlay setting. While dynamicmagnification could enhance closer examination and precisemovement, it also forces the focus area to contract and floataround. This fact raises a tradeoff with the overlay value. Thelarger level of magnification will increase the discontinuitybetween the overview and camera view with overlay¼true ordecrease the stability of the camera view with overlay¼ false.Fig. 10 illustrates dmax4d40 and overlay¼true, the setting wechose in our evaluation, in an animation sequence.

Fig. 9. Variations in implementation: (a) dmax¼d¼0, (b) dmax¼d (overlay¼ false)

and (c) dmax¼d (overlay¼true).

Fig. 10. Animation sequence when dmax4d40, overlay¼true. The device is

rotating to the left.

Fig. 8. (a) Static state with 50 objects and (b) dynamic state.

H. Jo et al. / Computers & Graphics 35 (2011) 841–853 847

4. Evaluation

In the evaluation, we compared Aroundplot to two existinginterfaces, the top-down 2D radar and the 3D arrow cluster.

4.1. Participants

A total of 16 participants (11 males and 5 females, agesM¼26.9, SD¼5.1) were recruited on campus. For the prerequisite

survey, questions were posed about their experience using 3Dapplications (e.g., games) and AR applications.

4.2. Experiment environment

The three interfaces were implemented on an iPhone3GS asshown in Fig. 11. The elevation and azimuth of the device werecalculated using the built-in accelerometer and magnetometer ofthe iPhone3GS. The experiment was conducted in a laboratoryenvironment with accompanying instructors, and no problems insensing the elevation or azimuth were detected.

The highlight on the cues or the objects (Fig. 11b and c) wasdesigned to easily catch user attention and to be visible under severeocclusion by other cues. This design choice was intended to minimizethe interference of the cue detection process in the experiment andthus concentrate on the orientation as mentioned in Section 3.1.

The top-down 2D radar was implemented so that the viewdirection is always upward, which has been shown to be better incomparison between the map and forward view [2] and morecommon in AR applications than the north-up. In addition,because distance is an important factor for target identificationwith 2D radar, we adjusted, for all of the three interfaces, the sizesof the object icons in the camera view according to their distance.

The implementation of the 3D arrow cluster followed thedesign described in [11], with the exception of the feature ofauto-orientation by clicking an arrow, which is hard to achieve inAR. For a fair comparison, we enabled the arrow to inform with itscolor whether the object indicated by the arrow had come intothe FOV, as the common 2D radar and Aroundplot do with a fanshape and a rectangle, respectively. Note that we put more weighton the clear perception of directions than on occlusion minimiza-tion. Therefore, the arrows were made opaque and not too slimfor displaying the edges and shades.

4.3. Tasks and materials

4.3.1. T1: normal search

This task was designed to test our base scenario, shown inFig. 2. The user is presented with multiple cues and chooses a cueto examine; the user then rotates the camera view toward theobject that the cue indicates. Cues may indicate facilities to work

Fig. 11. (a) Aroundplot, the 3D arrow cluster, and the top-down 2D radar, all with five objects. (b) Highlight indication of a cue. Cropped from the screenshots with 50

objects. (c) Highlight indication of an object.

H. Jo et al. / Computers & Graphics 35 (2011) 841–853848

on, nearby messages, venues, or people, for example. This sup-posed service would not prioritize any cue above any other, soeach cue is rendered with a similar shape. In other words, T1 is anorientation task with similarly shaped cues.

However, allowing the participant to select the target cue canconfound the results because biases in selection are likely tooccur, and the time to make a choice varies amongst participants.To solve this problem, (1) the system randomly selected one ofthe off-screen cues and then visually highlighted it for twoseconds, and (2) the participants were instructed to start search-ing for the target only after the highlight had been turned off.After two seconds, with the cues in similar shapes and theparticipant bearing a target cue in mind, the task becomesequivalent to the orientation process that we described inprocedures (3)–(5) in Fig. 2. Note that the actual user scenariowill never have this temporary highlight.

Once the search started, the participant’s touching of anyobject in the camera view that was regarded as the target endeda single search, and the system then suggested the next target cueby another two-second highlight. The participants were alsoinstructed to give up and proceed to the next target when theyfelt that finding the target was impossible because of a failure intracking the cue or any other reason.

Both touching the wrong object and giving up were logged as afailure. The task time measured the time that elapsed from whenthe highlight was turned off until the participant touched an object.

4.3.2. T2: highlighted search

This task was to test our sub-scenario: among the multiplelocation cues, the system selects a cue to examine. The target cuemay indicate a facility for the next operation in a fixed workflowor an object with an alert that it has a recent update. Thesupposed service would give explicit priority to the target, andeventually, both the target and the cue for it will have a differentshape with others. T2 is an orientation task with a uniquely shaped

target and cue.This task is more straightforward because the scenario itself

does not involve participant bias. It is equivalent to procedures

(1) and (3)–(5) in Fig. 2. The system randomly selected a cue foran off-screen target. Both the target object and the cue for itremained highlighted throughout the search, and the participantswere instructed to start searching without waiting.

Contrary to T1, a single search did not end until the participanttouched the right target because mistaking another object for thetarget or failing to track the cue was impossible due to theconstant highlight. For this reason, error or giving up was notcounted, and the task did not end until the participant touchedthe right target.

The task time measured the total elapsed time until theparticipant touched the correct target.

4.3.3. Materials

All of the objects were virtual objects without any physicalreferences in the real world. This represents a number of ARscenarios where the existence and location of objects can beknown only through AR vision. The scenarios include not onlyvirtual objects such as floating twitter messages but also physicalobjects that are occluded by other artifacts. For example, if afacility to work on is over the wall or underground, and theworker has no previous knowledge of its location, then theworker would rely entirely on the virtual representations andthe overview to locate the facility. To observe the effect ofphysical reference in the search, more domain-specific experi-mentation would be required.

Considering that target distribution can vary substantially fordifferent applications, we used a random distribution with thesame probability for every angle of the spherical coordinates andthen analyzed the results for a range of angles. The distance wasalso randomly distributed in a range of 1–100 m.

4.4. Hypotheses

We established the following hypotheses for T1.

H1. Compared to the 2D radar, searching with Aroundplot wouldbe more accurate (reduced failure).

H. Jo et al. / Computers & Graphics 35 (2011) 841–853 849

H2. Compared to the 3D arrows, searching with Aroundplotwould be more accurate when there were 50 objects.

H3. The participants would experience an increase in accuracywithout a decrease in task speed when using Aroundplot.

On the other hand, because the consistent highlighting in T2would largely alleviate the occlusion problem with the 3D arrows,our hypothesis for T2 was as follows.

H4. Compared to the 2D radar, searching with Aroundplot wouldbe faster.

4.5. Experimental design and procedure

We employed a 2 (Task: Normal, Highlighted)�2 (Number-of-objects: 5, 50)�3 (Interface: Aroundplot, 3D arrow, 2D radar)within-subjects design (12 conditions total). A trial consisted offinding a target. One session consisted of five trials, finding fivedifferent targets in the same dataset under the same condition.For training, we briefly introduced all three interfaces and theneach participant performed six training sessions per eachtask� interface (30 training trials). Then each participant per-formed a total of 180 trials in 36 sessions, where the order of thesessions for the 16 participants was balanced by Task, Number-of-objects, and Interface as follows (let P1¼participant #1,A¼Aroundplot, B¼3D arrow and C¼2D radar):

�

P1–8: Task 1 first; P9–16: Task 2 first. � Within each Task: P1–4, 13–16: 5 objects first; P5–12: 50

objects first.

�

Fig. 12. (a) Fail rate and task time in T1. The error bar represents the standard

error. n denotes a value with a significant difference (po0.05) (normal search

results). (b) Task time (s) in T2 (highlighted search results).

Within each Task�Number-of-objects: The participant com-pleted 9 sessions where the 3 interface conditions weregrouped as follows and each group was repeated 3 times: P1,7, 13: ABC; P2, 8, 14: BCA; P3, 9, 15: CAB; P4, 10, 16: ACB; P5,11: BAC; and P6, 12: CBA.

So, for example, all the sessions for participants 1 and 11 were:

P1: T1 [N5 (ABC ABC ABC), N50 (ABC ABC ABC)], T2 [N5 (ABCABC ABC), N50 (ABC ABC ABC)].P11: T2 [N50 (BAC BAC BAC), N5 (BAC BAC BAC)], T1 [N50 (BACBAC BAC), N5 (BAC BAC BAC)].

After completing the first task, each participant reported his/her perceived task load for three interfaces and then proceeded tothe second task.

4.6. Results

The task times, fail rates, and subjective task load data wereanalyzed using a repeated measured analysis of variance(RM-ANOVA) with the Greenhouse–Geisser adjustment. Pairwisecomparisons between each level of interface used the Bonferronicorrection for multiple testing.

4.6.1. T1: normal search

The results from T1 are shown in Fig. 12a. We analyzed boththe fail rate and the task time. Note, however, that the task timein T1 has less meaning than the fail rate because it includes casesof giving up in which we think that the latent increase in the tasktime was switched to the increase in the fail rate.

For the fail rate, the main effect of the interface (F1.508, 22.624¼

49.193, po0.001), the number-of-objects (F1.000, 15.000¼133.334,po0.001) and the interaction between the number-of-objectsand the interface (F1.368, 20.524¼7.829, p¼0.006) were significant.The simple main effect of the interface was significant with

50 objects (F1.824, 27.365¼38.580, po0.001), and the post-hoccomparison revealed that Aroundplot had a significantly lowerfail rate than the other two interfaces (po0.001). With 5 objects(F1.674, 25.112¼14.577, po0.001), Aroundplot had a lower fail ratethan the 2D radar (p¼0.003), but the difference against the 3Darrows was not significant (p¼1.0).

For the task time, the main effect of the interface wassignificant (F1.755, 26.324¼21.503, po0.001), but the main effectof the number-of-objects (F1.000, 15.000¼0.103, p¼0.753) was not.Because the interaction between the interface and the number-of-objects was significant (F1.582, 23.734¼9.733, p¼0.002), we alsoanalyzed the simple main effect of the interface in each number-of-objects block. With 50 objects, the difference among theinterfaces was not significant (F1.951, 29.271¼1.942, p¼0.162),but with 5 objects (F1.274, 19.173¼34.535, po0.001), the post-hoc comparison showed that the task time with Aroundplot wasfaster than with the 2D radar (po0.001). However, the differenceagainst the 3D arrows was not significant (p¼0.780).

4.6.2. T2: highlighted search

The results from T2 are shown in Fig. 12b. The main effects ofboth the interface (F1.207, 18.104¼128.076, po0.001) and the number-of-objects (F1.000, 15.000¼4.872, p¼0.043) were significant, but the

H. Jo et al. / Computers & Graphics 35 (2011) 841–853850

interaction between the number-of-objects and the interface was not(F1.164, 17.457¼1.331, p¼0.271). The post-hoc comparison showedthat the difference between Aroundplot and the 3D arrows was notsignificant (p¼0.159), but finding an object using the 2D radar wassignificantly slower compared to the other two interfaces (po0.001).

4.6.3. Effect of the device and the target angles

During the tasks, we logged four kinds of angles as the anglescan more clearly account for the results from the regular mea-surements (task time and the fail rate). The first kind is Closest

Object, the angle between a target object and the closest non-target object. Device–Target is the angle between the device orienta-tion and the target object at the start of the search. 9Device y9and 9Target y9, the absolute values of the device pitch at the startand the target elevation, were also logged. The yaw of the device andazimuth of the target were not topics of interest because they areonly relative values in human sensation, whereas the pitch has anabsolute reference—the ground plane.

We then performed multivariate regressions within eachtask�number-of-objects� interface condition to examine howthese angles affected the task results. For T1, logistic and linearregressions were performed with failure/success and task time asthe dependent variables, respectively. For T2, only linear regres-sions were performed with the task time as a dependent variable.All of the regressions had the four kinds of angles, Closest Object,Device–Target, 9Device y9 and 9Target y9, as their independentvariables. Table 1 shows the coefficients and constants from theregressions. Fig. 13, on the other hand, illustrates the fail rates andtask times divided according to Device y and Target y. Aroundplotyields satisfactory results in a stable manner across most condi-tions. The differences within or between the angle segments inFig. 13 were not statistically analyzed because of the small samplesize in each segment.

4.6.4. Subjective task load

The subjective task load for each task� interface was gatheredusing the NASA TLX method. Table 2 shows the results. The

Table 1Coefficients and constants from multivariate regressions in which four kinds of angles

time was the dependent variable.

Kind of angles (rad) 5 objects

Around Arrow Ra

T1: logistic regression with failure (1) or success (0)

Closest Object �2.164nn�1.890nn

�

Device–Target 0.385 0.678n�

9Device y9 1.342n 0.138

9Target y9 �0.275 �0.011

Constant �2.017 �1.978

T1: linear regression with task time (s)

Closest Object �0.290 �0.164

Device–Target 0.768nn 0.753nn

9Device y9 0.493 �0.031 �

9Target y9 �0.364 0.149

Constant 2.003 1.830

T2: linear regression with task time (s)

Closest Object �0.444 �0.306n�

Device–Target 1.277nn 0.943nn

9Device y9 0.468 �0.045 �

9Target y9 �0.078 0.241

Constant 1.912 2.070

Bold indicates significant coefficients.

n po0.05.nn po0.01.

main effects of the interface were found in both the normalsearch (F1.898, 28.468¼6.305, p¼0.006) and the highlighted search(F1.711, 25.664¼19.153, po0.001). In the post-hoc comparison, thedifferences between Aroundplot and the 2D radar were significantin both the normal (p¼0.009) and highlighted search (p¼0.003),but the differences between Aroundplot and the 3D arrows werenot significant (p¼0.930 and 0.885).

4.7. Discussion

4.7.1. T1: normal search

The results from ANOVA confirmed all of our hypotheses. Theparticipants completed the tasks with a significantly lower failurerate and task time with Aroundplot than they did with the 2Dradar. Compared to the 3D arrows, the failure rate with Around-plot was much lower with 50 objects; however, we could find noother significant difference. Aroundplot also marked the lowesttask load, although its difference from the 3D arrows was notsignificant for this measurement.

In the detailed analysis, regression showed that the anglesClosest Object and Device–Target were the basic factors for the failrate and the task time, respectively. This result was commonacross the interfaces and the number-of-objects. These relation-ships are easily understood by noting that the failures occur dueto confusing the target with another close object and that thetime for rotating the device would increase in proportion to therotation angle.

The main reason for the low performance of the 2D radar wasthe lack of height information (4 participants commented), whichmakes the comparison between the overview and the cameraview, as well as recognizing where to move the view, verydifficult. In addition, the effect of 9Target y9 on the task time alsoshows that finding the targets at high or low elevations is par-ticularly more difficult with this interface. The effect of 9Target y9on the task time was not significant when 50 objects were given,but instead the fail rate was affected, implying that the partici-pants were more likely to give up with targets at high or lowelevations.

describing the trial conditions were independent variables and fail/success or task

50 objects

dar Around Arrow Radar

1.105nn�2.249n

�2.270n�1.743

0.236 0.310 0.471n 0.291

0.049 0.961nn 0.587 0.291

0.460 0.305 0.821n 1.158n

0.298 �1.294 �0.288 0.423

1.106 �0.340 1.286 0.806

0.469 1.402nn 0.817nn 1.087nn

0.688 0.331 �0.474 �0.739

3.544nn�0.412 0.747 �0.608

2.096 1.507 2.150 3.155

2.106 �0.370 0.306 �2.947

1.155 0.975nn 1.243nn 2.165

1.780 0.209 0.335 0.229

2.429 �0.173 0.573n�0.043

6.350 2.530 1.407 5.241

Fig. 13. Task results divided according to the device pitch and the target elevation. Dividing by 271 yielded the most uniform number divisions because the objects were

spherically distributed.

Table 2Subjective task load results (0¼no load, 100¼maximum load).

Task Around Arrow Radar

M (SD) M (SD) M (SD)

Normal search 47.9 (21.7) 54.3 (27.5) 70.5 (20.1)

Highlighted search 23.6 (17.5) 18.8 (14.1) 52.3 (22.7)

H. Jo et al. / Computers & Graphics 35 (2011) 841–853 851

As we predicted, occlusion was the most prominent problemfor the 3D arrows (4 participants commented), which yieldedfrequent confusion and giving up when 50 objects were given.Another problem was the difficulty of 3D recognition (2 partici-pants commented). The regression results reveal two types of 3Drecognition problems. The first is the difficulty of tracking 3Drotation, which is shown as the effect of Device–Target on the failrate. Another is misunderstanding the object position when theobject is at a high or low elevation, and the effect of 9Target y9 onthe fail rate manifests this problem.

While Aroundplot yielded the best results among the threeinterfaces, the problem of high density at the corners wasindicated (2 participants commented). The effect of 9Device y9on the fail rate confirms this problem because a high or lowdevice pitch causes the cues to gather in the lower or upper area,respectively, and thus, the density at the corner increases twice.Although the dynamic magnification helped to resolve the cornerdensity problem so that the task performance was superior ascompared to the other interfaces, the comments and regressionshow that there is still room for improvement.

It is interesting to note that when 50 objects were given withthe 3D arrows or the 2D radar, a considerable number of thefailures came from giving up the search (distinguished by failureafter a very short task time). Each of the two interfaces causeddifferent reasons for giving up. With the 3D arrows, the partici-pants mainly gave up when they failed in visually tracking thetarget cue and could no longer discern it from the other cues. Thisoccurred when the target cue underwent occlusion or the over-view layout looked different after a 3D rotation. In contrast, with

the 2D radar, they gave up when they felt that 1:1 matchingbetween a cue in the overview and an object in the camera viewwas simply impossible because of the viewpoint difference, i.e.,many objects at different elevations were shown as mingled dotsat one location on the radar.

4.7.2. T2: highlighted search

The results confirmed our hypothesis. With Aroundplot, theparticipants completed the task much faster than they did withthe 2D radar. We could find no significant difference betweenAroundplot and the 3D arrows. The subjective task load corre-sponded to these results.

As in T1, the angle Device–Target commonly affected the tasktime across the interfaces and the object numbers.

The main reason for the low performance of the 2D radar wasthe lack of height information (5 participants commented), whichresulted in the users being unable to pre-determine whether thetarget was above or below the current camera view. Cues near thecenter of the radar were also a problem (2 participants commen-ted) because with such cues not only the elevation but also theazimuth is difficult to recognize.

Regarding the 3D arrows, the effect of occlusion was muchalleviated (2 participants commented) when the highlight wasconstantly turned on. However, confusion in the 3D recognitionwas still problematic (2 participants commented). When thetarget is at a high or low elevation, mistaking an object behindfor an object in front sometimes led the participants into physi-cally impossible movements, as shown by the effect of 9Target y9on task time.

For Aroundplot, the participants remarked on the discontinuity(2 participants commented) and the speed difference (2 partici-pants commented) between the overview and camera view.Although we intentionally chose to sacrifice the continuitybetween the cue and object positions to fix the camera view size,this choice seems to be worth reconsidering for efficiency ina search task. In addition, we expect to achieve continuity inspeed by elaborating on the dynamic magnification function.

H. Jo et al. / Computers & Graphics 35 (2011) 841–853852

We initially expected that the effect of the angle Device–Target

on the task time would be larger with Aroundplot than with 3Darrows because the 3D arrows provide the exact orientation of theobjects, while Aroundplot provides only relative angular dis-tances. However, the regression reveals that the effects of theangle Device–Target on the task time are not largely different. Thisimplies that the user does not directly jump to the target butrather approaches it gradually, watching the cue movement, evenwhen the exact angular distance is given.

4.7.3. Limitation of the evaluation

Most of all, it should be noted that this evaluation is alaboratory study to explore the general aspects of orientationwith the proposed technique. Specific applications may have theirown representative user groups, distributions for objects, iconrepresentations, links to physical artifacts, or even 3D reconstruc-tions of the environment. All of these are factors that can affectthe results and demand some deliberate adaptation.

Additionally, this evaluation should be seen as a proof ofconcept for the use of 2D rectangular fisheye to present 3D off-screen locations. This concept still has much to test including thevariations described in Section 3.4 and other tasks such asestimating direction or tracking moving targets, as well as muchroom for elaboration, as identified from the evaluation. In parti-cular, further study is needed to examine the positive and negativeeffects of dynamic magnification separately from other compo-nents of Aroundplot. 3D arrows also need further study. Althoughwe have employed the arrows with opaque bodies for ourevaluation, the arrows without bodies, such as Wedge Sphere [26],must also be tested. A test with the separation of wrong answersfrom giving up in T1 will also be needed in the future. Moreover,one could also consider dynamically transforming the layout ofthe arrows, similar to the dynamic magnification in Aroundplot.Explosion Diagrams [24] might be relevant for such purpose.

5. Conclusion

In this paper, we proposed Aroundplot, an overview interfacefor off-screen object searching in a 3D environment. Aroundplotprovides intuitive direction cues with minimized occlusionamong the cues by a novel mapping method from sphericalcoordinates to a 2D orthogonal fisheye. A dynamic magnificationfeature also complements the high density in the context area,which is the inherent shortcoming of the orthogonal fisheye, andaids in precise movement. The evaluation proved that the inte-gration of these two techniques is adequately useful. Comparedwith a top-down 2D radar, the participants with Aroundplotfound an off-screen object for a given cue faster and with lessfailure in all of the conditions except for the task time in thenormal search with 50 objects, in which no significant differencewas observed because many participants gave up the search tooearly with the 2D radar. In contrast, when compared with a 3Darrow cluster, we could not find any significant difference in anyof the conditions except for the fail rate in the normal search taskwith 50 objects, for which Aroundplot was significantly lower.

The strength of Aroundplot in 3D locations and with a largenumber of objects broadens its usefulness into diverse applica-tions, including AR for professional purposes that might deal withobjects over the ceiling or below the ground and common indoorAR and VEs for gaming or training. Conversely, we also expectthat near-the-viewport exploration could benefit from the screen-oriented guide of Aroundplot.

Considering that the FþC approach to displaying 3D locationcues is still in early development, the evaluation results suggestthat further research in this direction could be fruitful. The

performance could be further increased by more enhanced solu-tions for the corner-density problem and continuity in positionand speed between the overview and the camera view. Combin-ing Aroundplot and the 3D arrows also appears promising. Theymight be transited to each other in a zoom-like manner tomaximize the advantages of each.

Acknowledgments

The authors would like to thank Kwangyun Wohn for hissupport and useful discussions. They also thank the editor andanonymous reviewers who helped improve this paper throughtheir valuable comments.

Appendix A. Supplementary data

Supplementary data associated with this article can be foundin the online version at doi:10.1016/j.cag.2011.04.005.

References

[1] Alessandro M, Duenser A, Schmalstieg D. Zooming interfaces for augmentedreality browsers. In: Proceedings of the MobileHCI; 2010. p. 161–70.

[2] Aretz AJ, Wickens CD. The mental rotation of map displays. Human Perfor-mance 1992;5(4):303–28.

[3] Arth C, Wagner D, Klopschitz M, Irschara A, Schmalstieg D. Wide arealocalization on mobile phones. In: Proceedings of the ISMAR; 2009. p. 73–82.

[4] Baudisch P, Good N, Bellotti V, Schraedley P. Keeping things in context: acomparative evaluation of focus plus context screens, overviews, and zoom-ing. In: Proceedings of the CHI; 2002. p. 259–66.

[5] Baudisch P, Rosenholtz R. Halo: a technique for visualizing off-screenlocations. In: Proceedings of the CHI; 2003. p. 481–8.

[6] Biocca F, Tang A, Owen C, Xiao F. Attention funnel: omnidirectional 3d cursorfor mobile augmented reality platforms. In: Proceedings of the CHI; 2006.p. 1115–22.

[7] Bowman DA, North C, Chen J, Polys NF, Pyla PS, Yilmaz U. Information-richvirtual environments: theory, tools, and research agenda. In: Proceedings ofthe VRST; 2003. p. 81–90.

[8] Buchmann V, Billinghurst M, Cockburn A. Directional interfaces for wearableaugmented reality. In: Proceedings of the CHINZ; 2008. p. 47–54.

[9] Carlson-Radvansky LA, Irwin DE. Frames of reference in vision and language:where is above? Cognition 1993;46(3):223–44.

[10] Castle R, Klein G, Murray D. Video-rate localization in multiple maps forwearable augmented reality. In: Proceedings of the ISWC; 2008. p. 15–22.

[11] Chittaro L, Burigat S. 3D location-pointing as a navigation aid in virtualenvironments. In: Proceedings of the AVI; 2004. p. 267–74.

[12] Cockburn A, Karlson A, Bederson BB. A review of overviewþdetail, zooming,and focusþcontext interfaces. ACM Computing Surveys 2008;41(1):1–42[Article 2].

[13] Egly R, Driver J, Rafal R. Shifting visual attention between objects andlocations: evidence for normal and parietal lesion subjects. Journal ofExperimental Psychology: General 1994;123(2):161–77.

[14] Elmqvist N, Tsigas P. A taxonomy of 3D occlusion management for visualiza-tion. IEEE Transactions on Visualization and Computer Graphics 2008;14(5):1095–109.

[15] Feiner S, MacIntyre B, Seligmann D. Knowledge-based augmented reality.Communications of the ACM 1993;36(7):53–62.

[16] Friederici AD, Levelt WJM. Spatial reference in weightlessness: perceptualfactors and mental representations. Perception & Psychophysics 1990;47:253–66.

[17] Gustafson S, Baudisch P, Gutwin C, Irani P. Wedge: clutter-free visualizationof off-screen locations. In: Proceedings of the CHI; 2008. p. 787–96.

[18] Gustafson S, Irani P. Comparing visualizations for tracking off-screen movingtargets. In: Proceedings of the CHI; 2007. p. 2399–404.

[19] Gutwin C. Improving focus targeting in interactive fisheye views. In:Proceedings of the CHI; 2002. p. 267–74.

[20] Gutwin C, Skopik A. Fisheyes are good for large steering task. In: Proceedingsof the CHI; 2003. p. 201–8.

[21] Hwang S, Jo H, Ryu J. EXMAR: expanded view of mobile augmented reality.In: Proceedings of the ISMAR; 2010. p. 235–6.

[22] Hornbæk K, Hertzum M. Untangling the usability of fisheye menus. ACMTOCHI 2006;14(2):1–39 [Article 6].

[23] Irani P, Gutwin C, Yang XD. Improving selection of off-screen targets withhopping. In: Proceedings of the CHI; 2006. p. 299–308.

[24] Kalkofen D, Tatzgern M, Schmalstieg D. Explosion diagrams in augmentedreality. In: Proceedings of the VR; 2009. p. 71–8.

H. Jo et al. / Computers & Graphics 35 (2011) 841–853 853

[25] Klein G, Murray D. Parallel tracking and mapping on a camera phone. In:Proceedings of the ISMAR; 2009. p. 83–6.

[26] McCrae J, Glueck M, Grossman T, Khan A, Singh K. Exploring the design

space of multiscale 3D orientation. In: Proceedings of the AVI; 2010.p. 81–8.

[27] Moran TP, Dourish P. Introduction to this special issue on context-awarecomputing. Human–Computer Interaction 2001;16(2):87–95.

[28] Nurminen A, Oulasvirta A. Designing interactions for navigation in 3D mobilemaps. Map-based mobile services. Berlin: Springer; 2008. p. 198–227.

[29] Pietriga E, Appert C, Beaudouin-Lafon M. Pointing and beyond: an operatio-nalization and preliminary evaluation of multi-scale searching. In: Proceed-

ings of the CHI; 2007. p. 1215–24.[30] Sandor C, Dey A, Cunningham A, Barbier S, Eck U, Urquhart D, et al. Egocentric

space-distorting visualizations for rapid environment exploration in mobilemixed reality. In: Proceedings of the VR; 2010. p. 47–50.

[31] Sarkar M, Snibbe SS, Tversky OJ, Reiss SP. Stretching the rubber sheet: a

metaphor for viewing large layouts on small screens. In: Proceedings of theUIST; 1993. p. 81–91.

[32] Schinke T, Henze N, Boll S. Visualization of off-screen objects in mobileaugmented reality. In: Proceedings of the MobileHCI; 2010. p. 313–6.

[33] Schwerdtfeger B, Klinker G. Supporting order picking with augmentedreality. In: Proceedings of the ISMAR; 2008. p. 91–4.

[34] Skopik A, Gutwin C. Improving revisitation in fisheye views with visit wear.In: Proceedings of the CHI; 2005. p. 771–80.

[35] Stoakley R, Conway M, Pausch R. Virtual reality on a WIM: interactive worldsin miniature. In: Proceedings of the CHI; 1995. p. 265–72.

[36] Suomela R, Lehikoinen J. Context compass. In: Proceedings of the ISWC; 2000.p. 147–54.

[37] Taylor HA, Tversky B. Perspective in spatial descriptions. Journal of Memoryand Language 1996;35(3):371–91.

[38] Tonnis M, Klinker G. Effective control of a car driver’s attention for visual andacoustic guidance towards the direction of imminent dangers. In: Proceed-ings of the ISMAR; 2006. p. 13–22.

[39] Tonnis M, Sandor C, Lange C, Klinker G, Bubb H. Experimental evaluation ofan augmented reality visualization for directing a car driver’s attention. In:Proceedings of the ISMAR; 2005. p. 56–9.

[40] Trapp M, Schneider L, Holz N, Dollner J. Strategies for visualizing points-of-interest of 3D virtual environments on mobile devices. In: Proceedings of thesixth international symposium on LBS & TeleCartography; 2009.

[41] Zellweger PT, Mackinlay JD, Good PT, Stefik M, Baudisch P. City Lights:contextual views in minimal space. In: Proceedings of the CHI; 2003. p. 838–9.