tapraid4/z92-psen/z92-psen/z9200509/z922113d09g colesona S ... · DSD method that can be used by designers. Finally, we present the results of several studies indicating the effec-tiveness

Jian Chen*Department of Computer ScienceBrown UniversityProvidence, RI 02912

Doug A. BowmanDepartment of Computer ScienceCenter for Human-ComputerInteractionVirginia TechBlacksburg, VA 24061-0002

Presence, Vol. 18, No. 5, October 2009, 000–000

© 2009 by the Massachusetts Institute of Technology

Domain Specific Design of 3DInteraction Techniques: AnApproach for Designing UsefulVirtual Environment Applications

Abstract

Few production virtual environment (VE) applications involve complex three-dimensional (3D) interaction. Our long-term collaboration with architects and engi-neers in designing 3D user interfaces (3D UIs) has revealed some of the causes:existing interaction tasks and/or techniques are either too generic when isolatedfrom the application context, or too specific to be reusable. We propose a newdesign approach called domain specific design (DSD) that sits between the genericand specific design approaches, with an emphasis on using domain knowledge in3D interaction techniques. We also describe an interaction design framework en-compassing generic, domain specific, and application specific interaction tasks andtechniques. This framework can be used by designers to think of ways to producedomain specific interaction techniques. We present a particular DSD method, anddemonstrate its use for the design of cloning techniques in a structural engineeringapplication. Results from empirical studies demonstrate that interaction techniquesproduced with domain knowledge in mind outperformed other techniques by im-proving task efficiency, work flow, and usefulness of the 3D UI.

1 Introduction

A great deal of work has been done, especially in the past decade, on thedesign of effective three-dimensional (3D) interaction techniques and user in-terfaces (UIs) for virtual environments (VEs). In the mid-1990s, many interac-tion techniques were invented for the so-called “universal tasks” of travel, ma-nipulation, selection, system control, and symbolic input (Bowman, Kruijff,LaViola, & Poupyrev, 2004; Foley, Wallace, & Chan, 1984; Hand, 1997).This led to most of today’s interaction research efforts on interaction styles,metaphors, and cognition for these tasks (Bowman et al., Plumlee & Ware,2006; Zhai, Buxton, & Milgram, 1994).

Despite this research on 3D interaction, few production VE applicationsinvolve complex 3D interaction. Our experience suggests that at least two bar-riers exist. First, existing interaction tasks are generic and thus do not directlyaddress application scenarios; second, the generic nature of tasks leads to the

*Correspondence to [email protected].

tapraid4/z92-psen/z92-psen/z9200509/z922113d09g colesona S!12 9/22/09 0:40 Art: z92-2113 Input-kll(kll)

Chen and Bowman 1

design of generic interaction techniques that may beimpractical. For example, Go-Go (Poupyrev, Billing-hurst, Weghorst, & Ichikawa, 1996) is a manipulationtechnique that is generic and should be applicable toany 3D UI. When used for modeling real-world build-ings in a structural engineering application, however,Go-Go is not practical when the number of buildingelements exceeds a few dozen, such as the structureshown in Figure 1.

We claim that one reason for the lack of broad useful-ness of 3D interaction techniques is that these tech-niques have failed to exploit domain knowledge. Mosttechniques designed for universal tasks are too genericto be practical. Design methods, such as those related tousability engineering (UE), can be used to improve thework flow of the 3D UI, but do not address the designof 3D interaction techniques directly. In addition, de-

veloping 3D interaction techniques is complex and ex-pensive, as directly migrating techniques from 2D to 3Ddoes not always produce usable and useful techniquesand the 3D design space is too large to investigate with-out any guiding principles.

In this paper we propose a domain specific design(DSD) approach that directly addresses the design of3D interaction techniques and circumvents the limita-tions of generic design. DSD uses domain knowledge tostructure design tasks and techniques. We also propose anew framework that classifies interaction design intothree levels based on domain specificity, aiding in theassessment of a technique’s reusability and design ofdomain specific techniques. We describe a three stepDSD method that can be used by designers. Finally, wepresent the results of several studies indicating the effec-tiveness of the DSD approach.

2 Related Work

3D interaction is a type of human-computer inter-action (HCI) in which tasks are performed directly in a3D spatial context (Bowman et al., 2004). The value of3D interactivity has been supported by much prior re-search. For example, effective 3D interaction has beenfound to improve task performance by an order of mag-nitude over conventional 2D input for some scientificvisualization tasks (van Dam, Forsberg, Laidlaw,LaViola, & Simpson, 2000); interactivity is also claimedto provide better spatial understanding for training andmodeling, and to improve the user experience (Allison,Wills, Bowman, Wineman, & Hodges, 1997).

The principal way to study 3D interaction is to designtechniques for the “universal 3D tasks” of travel, manip-ulation, selection, system control, and symbolic input(Bowman, Gabbard, & Hix, 2002). Research in thisarea has addressed such issues as the empirical designand evaluation of displays (Lin, Duh, Parker, Abi-Rached, & Furness, 2002), design and comparison ofnovel interaction techniques (Bowman et al., 2004),evaluation in a specific application setting (Lin &Loftin, 2000), and the use of constraints and various

Figure 1. A structure built by an architect using the Virtual-SAPapplication. Modeling such a large structure would be time-consumingif not impossible using conventional 3D interaction techniques in VEs;with our domain-specific interaction techniques, it takes less than 2min.

2 PRESENCE: VOLUME 18, NUMBER 5

F1


aids (e.g., navigation aids, Darken & Cevik, 1999; andphysical props, Ullmer & Ishii, 1997).

Another approach to studying 3D interaction is todesign a 3D UI for a specific application. For example,Hix and her co-authors (1999) used an iterative designmethod to develop a 3D UI for a military training appli-cation. In this design method, designers uncover usabil-ity problems through empirical studies and then tweak(often existing) techniques to work in that situation.Such “situated” design locates the design in a familiarworld with specific practices, artifacts, goals, and plans.The goal, however, is to improve the overall usefulnessof a 3D UI rather than to design interaction techniques.In fact, it has been argued that the UE approach doesnot suggest design solutions (Myers, 1994; Wolf, Rode,Sussman, & Kellogg, 2006) and designers do not alwaysintend to create novel interaction techniques (Beau-douin-Lafon, 2000; Wolf et al., 2006). To our knowl-edge, few studies have attempted to advance designmethods for 3D interaction techniques.

Understanding of 3D interaction design is still lim-ited. Design guidelines exist, but they are targetedmostly at the standard universal tasks rather than real-world task scenarios (Bowman et al., 2004; Gabbard,1997; Kalawsky, 1999; Sutcliffe, 2003). Our work ad-dresses the design of domain specific 3D interactiontechniques that apply to real-world tasks.

Several attempts have been made to guide the designprocess. Kaur, Maiden, and Sutcliffe (1999) extendedNorman’s seven stage HCI model to VEs to measureand improve the user’s action sequences related to thework flow of the target use and developed a model inte-grating interaction and design guidance for VEs. Millsand Noyes (1999) proposed generic design issues forVEs, and Drettakis, Roussou, Reche, and Tasigos(2007) proposed improving work flow in order to in-crease the use of VEs in highly interactive applications.Hix and her collaborators extended conventional UEdesign practices to 3D UI design (Hix & Hartson,1993; Hix et al., 1999). Our work focuses instead onimproving the design of 3D interaction techniques byusing domain knowledge to help us understand what todesign.

Domain knowledge has proved its value in 3D mod-

eling tools. For example, Wonka, Wimmer, Sillioin, andRibarsky (2003) define semantic knowledge to trans-form, scale, and extrude texture when modeling largestructures. Artifact knowledge, such as constraints, hasbeen applied to define physically correct object behav-ior, for example, that objects sit on top of each otherrather than floating in space (Bukowski & Sequin,1995; Oh & Stuerzlinger, 2005; Wonka et al.). Domainknowledge has also been used in designing intelligentinteraction techniques to improve work flow. For exam-ple, Feiner and MacIntyre (1993) designed a rule-basedsystem for maintenance tasks in which a pointer auto-matically indicates the user’s intended target. However,domain knowledge has not been used extensively in de-signing 3D interaction techniques and framing interac-tion tasks. Our work establishes a domain specific ap-proach to 3D interaction design and introduces theconcepts of domain specificity and domain specific de-sign to guide designers’ work and thinking. Thus, ouraim is not to design intelligent interaction or improvework flow per se, but rather to use general domainknowledge to frame more meaningful 3D tasks and toproduce more effective and efficient 3D interactiontechniques for real-world use.

3 Motivating Example: VE for StructuralDesign

Our long-term collaboration with architects led tothe design of Virtual-SAP, a head mounted display(HMD) based immersive VE for structural design, anal-ysis, and visualization (Bowman et al., 2003). Architectsusing this program expect to construct realistic build-ings, run stability tests using finite element methods,and modify the structure until a satisfactory design isachieved.

The first implementation of Virtual-SAP followed ausability engineering process for iterative design, proto-typing, and evaluation (Hix et al., 1999). Task analyseswere conducted with domain experts actively involvedin collecting designer needs and making usability studies(Setareh, Bowman, & Kalita, 2005). Results from thetask analyses were used for functional decomposition

Chen and Bowman 3


and composition, a common method in design disci-plines (Schon, 1983; Simon, 1981; see Figure 2). First,a top-down approach breaks the high-level activity (suchas “edit structure”) into small executable steps (such as“create objects” and “delete objects”) based on workflow analyses. This decomposition continues until atechnique is reached that can be used for a lower-leveltask. For example, “create objects” was decomposedinto “clone objects” and further decomposed to “selec-tion” and “manipulation.” Finally, a bottom-up ap-proach was used to adapt subsolutions (techniques suchas ray-casting and Go-Go, Poupyrev et al., 1996) toobtain a group of interaction techniques supporting theapplication goal.

The initial design resulted in a pen-and-tablet-metaphor user interface for architectural modeling. Theuser held a tablet and used a stylus to operate widgets inorder to load structural elements (beams, columns, and

slabs). Having grabbed an object, the user could place itwith Go-Go (Poupyrev et al., 1996). Finally, the usercould run and observe simulations and edit the structureall within the same environment.

The results of several user studies suggested that theinterface was relatively effective and had few usabilityproblems (Bowman et al., 2003). When we gave theapplication to students and engineers for real-worldstructural design, however, we found that the UI wasnot very useful for complex structure modeling: it couldtake users hours to create and move the hundreds ofbeams and columns in a moderately complex structure.

This example demonstrates that decomposition andcomposition can result in less than optimal 3D UI de-sign when only generic interaction techniques are avail-able. We realized that the design issue was not to tweakexisting interaction techniques but rather to define newtasks and techniques that meet the application require-

Figure 2. Task decomposition in the original Virtual-SAP using the task composition and decomposition approach.


F2


ments. Existing categories of interaction tasks, such asobject selection and placement, were too generic to de-scribe the needs of the application. But designing newtasks and techniques for each new application would notbe practical or efficient. This led to our overall researchquestion: how can we design useful and effective 3Dinteraction tasks and techniques without customizingthem for each application?

It might seem that this problem could be addressedwith further iterations of the usability engineering (UE)method, focusing specifically on the task of cloning, asin most conventional 2D interface design approaches oruser centered design practice. We would argue, how-ever, that UE is of limited value in producing effectiveinteraction without a reasonably rich set of domain spe-cific interaction techniques from which to draw.

Ultimately, all 3D UIs must be to some degree appli-cation specific in order to ensure usability and usefulnessfor real-world work. But the design of 3D interactiontechniques need not always be application specific. Forinteraction techniques to be reusable and useful in de-signing 3D user interfaces, three design requirementsmust be met:

! Interaction tasks must ensure adequate representa-tion of the domain in use.

! Interaction techniques must be useful in other ap-plications without significant modification.

! A design method must allow designers to create acollection of interaction tasks and techniques thatare supported by the usage guidelines.

4 Existing Approaches to 3D InteractionDesign

Before introducing domain specific design, let usconsider the characteristics of existing design ap-proaches for 3D interaction. The approaches can beclassified into two categories: generic and applicationspecific design (see Figure 3). This classification makesuse of the concept of specificity, which indicates theamount of knowledge used from the application domainin the interaction design. We describe interaction using

tasks (what to design for) and techniques (how to ac-complish the tasks).

4.1 Generic Design Approach

We call a technique generic if it is designed for ageneric (universal) task, with no specific applicationcontext. Thus, the specificity for this type of task ortechnique is the lowest. The pros and cons of thisdesign approach include a well defined design space,widely available evaluation results, and low specificity/high reuse.

1. Well Defined Design Space Various taxono-mies are available to describe the structure, ac-tion sequences, and parameter space of a generictask, and to classify generic techniques into use-ful categories (Bowman & Hodges, 1999;Poupyrev, Weghorst, Billinghurst, & Ichikawa,1997).

2. Widely Available Evaluation Results Test bedshave been designed to measure task performanceand user behavior. Designers have plentiful re-sources from which to chose (Bowman, Johnson,& Hodges, 1999; Poupyrev, Weghorst, Billing-hurst, & Ichikawa, 1998; Tan, Robertson, &Czerwinski, 2001).

3. Low Specificity/High Reuse The techniquesare designed not with any particular application or

Figure 3. Conventional two-level design framework.

Chen and Bowman 5

F3


application domain in mind, but rather to workwith any application. When used in real-worldcases, however, significant modifications mightbe required to meet application requirements.In some cases, generic techniques and tasks willbe completely inadequate to meet applicationneeds.

When the generic design approach is used, high-level user activities are decomposed into simple subtasksto form a hierarchical representation of user activity.Then the designer decides how to integrate solutions tothe generic tasks into a final solution to the high-leveldesign task.

This approach was used in the original Virtual-SAPapplication described in Section 3. The modeling taskwas decomposed into manipulation and selection tasks.We then selected several practical generic interactiontechniques (e.g., Go-Go) and considered cross tech-nique consistency issues to generate the final applica-tion.

4.2 Application Specific DesignApproach

The clearest way to improve usability in any partic-ular application is to design the interaction tasks andtechniques specifically for that application. A techniquedesigned for an application specific task, called an appli-cation specific technique, often uses specific applicationdomain knowledge (see Section 5.1) that might not bereusable. This approach typically requires starting fromscratch with each application and using a complete us-ability engineering process to produce a usable 3D UI.Design with this approach has the properties of (1) lesswell-defined design space, (2) techniques designed withcontext and activity in mind, (3) high specificity/lowreuse, and (4) high cost.

1. Less Well-Defined Design Space Because thereis no interaction technique for a given applicationspecific task, designers must perform task analysisto determine design goals. Well-known methodsinclude field studies, interviews, and ethnometh-odological studies.

2. Techniques Designed with Context and Activ-ity in Mind Because the tasks are framed withina specific application context, the techniques aregenerally usable and useful for that context only.

3. High Specificity/Low Reuse There are manydefinitions of reuse, such as the reuse of conceptsolutions, and best practice software modules andpatterns. In this paper, reuse simply means thereuse of interaction techniques, that is, how easy itis to plug in and play a certain technique in an ap-plication. Interaction techniques designed within aspecific application context will be highly usefuland usable in that context. However, it may bedifficult to reuse them in other applications with-out modification. Thus, an interaction techniquedesigned for a particular use is often too specific tobe broadly applicable.

4. High Cost Designing for every application iscostly. Making a range of techniques available thatdesigners simply plug into an application will bemore cost-effective. The design often requires in-tegration of and smooth transitions among severalinteraction techniques that best fit a specific appli-cation. For example, Hix et al. (1999) designedseveral interaction techniques for battlefield visual-ization applications. Some components of this typeof interaction technique can be reused, becausesome fundamental design principles are invariantover many applications, but the technique as awhole is not reusable.

We developed the DSD approach in order to balancethe advantages and disadvantages of generic and appli-cation specific interaction design.

5 Domain Specific Approach to 3DInteraction Design

We define domain specific design (DSD) as aninteraction design approach in which tasks or techniquesare designed with domain knowledge in mind. Our ap-proach is similar in spirit to procedure modeling(Bukowski & Sequin, 1995) regarding the use and clas-



sification of domain knowledge. For example, it is possi-ble to describe the appearance of a building (e.g., theshape of the land, land use, and streets) using semantics.Our work, however, focuses on interactivity rather thanautomatic model generation.

5.1 Domains and Domain Knowledge

Before discussing DSD, it is useful to define whata domain is. We define a domain as a subject matter areaof study that can have subdomains. Often, a domain hasits own processes, data, objects, and artifacts that behaveaccording to the rules of that domain. Thus, the studyof a domain, or domain analysis, can determine the gen-eralized characteristics of a particular domain. In partic-ular, domain knowledge has the following characteris-tics that make it suitable for interaction design: Stability,intuitiveness, feasibility, and reusability.

1. Stability Domain knowledge is stable and reus-able, since multiple applications can share com-mon knowledge. UI designers can benefit from astable collection of tasks and techniques based ondomain knowledge, despite changing applicationrequirements (Chen & Bowman, 2006; Chen,Bowman, Lucas, & Wingrave, 2004; Sutcliffe,Benyon, & van Assche, 1996).

2. Intuitiveness Domain knowledge is generallyintuitive or systematic for uses in the domain. Aninterface designed in line with that knowledge typ-ically provides a better mental model and under-standing of the environment. Users’ familiaritywith the task definition can make the interfacemore intelligible and predictable.

3. Feasibility Domain knowledge can be collectedeffectively by knowledge elicitation techniques(Gordon, 1994; Gordon, Schmierer, & Gill, 1993;Wielinga, Schreiber, & Breuker, 1992).

4. Reusability Since knowledge is shared amongapplications, techniques can be reused within adomain or even in another domain with similarcharacteristics.

Domain knowledge can be general or particular.To illustrate, consider the task of cloning in the sense of

building modeling. The concept of cloning is generaldomain knowledge, since the distinction between a win-dow, a beam, or a chair is irrelevant—they are equallygood for cloning. However, the shape of beams inwelding operations is particular domain knowledge,since differently shaped beams would sustain differentloads. This knowledge would lead to the design of tech-niques that would be difficult to reuse in other situa-tions. The DSD approach targets the general type ofdomain knowledge to make the techniques more reus-able.

5.2 Domain Specific Tasks andInteraction Techniques

Domain specific tasks and techniques are twocomponents of DSD. A domain specific task is an ab-straction of domain activities that can be shared amongapplications in the targeted domain. Activities definewhat users usually do in the real world, while goals andsubgoals of an activity are described as tasks throughtask analysis (Diaper, 1989; Kirwan & Ainsworth, 1992;Moran, 1983). A domain specific interaction techniqueis a way to accomplish an interaction task that makes useof domain knowledge. The goal is to increase the effec-tiveness of interaction.

5.3 Three-Level Design Framework

We extend the two-level framework shown inFigure 3 by adding a third, domain level. This 3Dinteraction design framework (see Figure 4) describesthe design of tasks and techniques at the generic, do-main, and application levels and specifies paths de-signers can take in designing and/or choosing inter-action techniques for an application. This frameworkcan be used to describe all three of the design ap-proaches we have discussed (generic design, applica-tion specific design, and DSD). The goal of the DSDapproach is to reach the “Techniques” box at the do-main level. These domain specific 3D interactiontechniques can be used in real-world applicationswithin the particular domain.

Chen and Bowman 7

F4


5.4 Advantages and Disadvantages

DSD attempts to maximize the advantages andavoid the disadvantages of the generic and the applica-tion specific methods. DSD has the characteristics of aless well defined design space, reusable domain specifictechniques, and high initial cost, low long-term cost.

1. Less well-defined design space: As in applicationlevel design, the design space for domain knowl-edge and domain specific interaction does not ex-ist. However, such knowledge can be collectedand captured using the same methods as in appli-cation level design.

2. Reusable domain specific techniques: The tech-niques are designed with a domain in mind, andthus can be reused in other applications in thesame domain without significant modifications.For example, the cloning operation (see Section 7)can be used in any application requiring multipleobject creation.

3. High initial cost/low long-term cost: The initialdesign can be costly because it requires knowledgespace formation and significant evaluation. How-ever, the long-term benefits are also high becausethe resulting techniques are reusable. DSD be-comes cost-effective once we have a range of do-main specific interaction techniques that designerscan use.

It is important to note that the three design ap-proaches (generic, application specific, and domain-

specific) are not distinct, but rather form a continuum.In fact, the same task can often be described in terms ofany approach. For example, the domain specific task ofcloning might be specialized from the generic task ofnumeric input. One benefit of separating the three de-sign approaches is to assist designers in thinking aboutdesign goals and the utility of each approach, allowingthem to choose an approach that is appropriate to theapplication they are designing.

6 Domain Specific Design Method

The previous section described the DSD approachat a high level. In this section, we present a more de-tailed DSD method—a three-step process that designerscan follow to design domain specific 3D interactiontechniques. In this DSD method, the designer performsthree steps: knowledge elicitation and representation,task identification, and design interaction techniques.

6.1 Knowledge Elicitation andRepresentation

The first step of the DSD method is to describeknowledge representation based on target use. Knowl-edge is first acquired during the design and evaluationprocess, from the conversation with domain experts,and from observation of work and artifacts in the do-main. Designers ask experts to identify the bottleneckswith existing tools (2D or 3D), leading to new designsolutions. Designers explore the range of activities thatexperts conduct, the function of artifacts, and the prosand cons with conventional tools and media. Domainexperts can also be invited to comment on the design ofthe evolving 3D UI on a regular basis. The knowledgethus elicited can be used to help frame tasks and chooseparameters that are critical to improve the usefulness ofthe domain specific interaction techniques.

Such iterative design and testing provides valuableinsights for the generation of domain specific tasks andtechniques and for choosing critical design parametersto form a knowledge base. Getting experts to use the

Figure 4. 3D interaction design framework.



interactive system is also important because people aremore likely to recognize problems while interacting.

We are aware of a growing body of literature that isdirectly concerned with understanding use in context asa basis for design, including ethnographic approaches,scenarios, critical incidents, and personal construct. Wedid not make use of a specific method, since each ofthese techniques is suited to particular situations anddelivers particular kinds of results. There is no guidingprinciple indicating which techniques are suitable orunsuitable for eliciting certain types of knowledge. Weuse a mix of these approaches.

6.2 Task Identification

The second step of the DSD method is to specifyinteraction tasks based on domain knowledge and itsrepresentation. Tasks are formally described by means ofa taxonomy, an analytical description of features mean-ingful in an application domain. The taxonomy typicallydescribes the set of parameters that characterize theidentified task and possible values for those parameters.This description serves as a knowledge base for design-ing interaction tasks and techniques.

6.3 Interaction Technique Design

The last step of the DSD method is to design in-teraction techniques for the domain specific tasks. Weuse test bed design and evaluation (Bowman et al.,1999) to design domain specific interaction techniques.In the test bed, we design a range of interaction tech-niques for the domain specific task by following differ-ent paths (see Figure 5) in our design framework and byintegrating different flavors of domain knowledge.

Specialization, generalization, direct design, andmixed decomposition and composition approaches aresome of the possibilities described by the frameworkshown in Figure 5. In specialization, we start with thedomain specific task, recharacterize it as a generic task,select a generic interaction technique for that task, andthen specialize the generic technique by adding domainknowledge (paths 7, 3, and 8). In generalization, wecast the domain specific task as an application-specific

task, design an application specific technique for thattask, and then remove some specific domain knowledgeto design domain specific interaction techniques fromthe application level (paths 6, 1, and 9). In the directdesign approach, we can design domain specific tech-niques directly by following horizontal path 2. In mixeddecomposition and composition, we recast a domain leveltask to a generic task then reuse generic techniqueswithout modification (paths 7 and 3).

One advantage of using this framework is to make itpossible to reuse existing generic-level or application-level techniques and to cover a broad range of tech-niques. The framework also covers existing interactiontasks and techniques and can be used to describe inter-action design in the overall 3D UI design process. Inour original Virtual-SAP design, we used this approach(starting with application specific tasks, then followingpaths 4, 3, and 5) to design the 3D interaction tech-niques. Following path 4 decomposes an applicationlevel task into the generic level; following path 3 pro-duces a generic level interaction technique; followingpath 5 lets one specify an interaction technique to fit anapplication.

7 Using DSD for Virtual-SAP

We illustrate the design method by our case studyapplication, Virtual-SAP. As described in Section 3, thepurpose of Virtual-SAP is to let the user model buildingstructures while immersed in a VE. We use the domain

Figure 5. Alternative design in the 3D interaction design framework.

Chen and Bowman 9

F5


specific task of cloning (Chen & Bowman, 2006; Chen,Bowman, Lucas, et al., 2004; Chen, Bowman, Win-grave, & Lucas, 2004) as an illustration of the use ofthe DSD method. Choosing the architecture domainand cloning also lets us use the knowledge framed in2D desktop tools, so that we can examine the effective-ness of the framework and the DSD method, ratherthan the effectiveness of knowledge acquisition.

7.1 Knowledge Elicitation andRepresentation

We used interviews with domain experts, observa-tion of experts and students in the architecture andstructural engineering domains, and comments fromusers of our early prototypes to elicit domain knowl-edge. We classified the domain knowledge into threecategories, based on object, environment, and user’scharacteristics (see Table 1). These factors were foundto be important for a structural engineering applicationwhile iterating our design with structural engineers. Themost important knowledge for our application is proba-bly the geometric arrangement of structural elements(e.g., beams, columns, slabs, and walls, etc.) in space.These objects obey certain geometrical, physical, and

mechanical rules that form the knowledge base we usedin designing interaction techniques.

We implemented several geometry constraints basedon this domain knowledge.

Coincidence The system defines point coincidenceand element coincidence. Beams and columns areconnected. Two elements have the same algebraicrepresentation, the same position, and the same ori-entation.

Parallelism Objects follow regular shapes. Beams andcolumns that are parallel remain parallel.

Distance and Size Constraints Distance can be speci-fied between elements and this distance defines thesize of an object. Any distances used in the system areintegers or integers plus 0.5 mm for ease of manufac-ture.

Alignment Objects are snapped to a grid and thus canautomatically change their length when the size ofthe grid is updated.

7.2 Task Identification

By knowledge elicitation with people and artifactsin the architecture domain, we found that many struc-

Table 1. Domain Knowledge Organization

Category Characteristic

Characteristics of domain objectsGeometrical Standard size and shapePhysical Objects have mass, material, texture, etc.Mechanical Objects are rigid

Characteristics of the environmentSpatial Repetitive and symmetrical patterns, large in sizeDynamic Simulation followed objects’ propertiesTopology Object relationship

Characteristics of the usersMental model Workflow/domain of studyAttitude Cognitive and noncognitive abilitiesExpertise Novice, expert


T1

AQ: 1


tures contain spatially repeating elements. Thus thecloning task was identified. We have defined cloning asthe task of generating multiple copies of spatially dis-tributed objects in an environment (Chen, Bowman,Lucas, et al., 2004). This is a similar concept to instanc-ing multiple copies used in desktop computer-aided de-sign tools, such as AutoCAD, SketchUp, and others.Cloning is a higher-level task compared to selection andmanipulation. Interaction techniques for cloning allowusers (here architects) to take the techniques as a wholerather than performing the individual small steps of ma-nipulation to create copies. It might seem obvious todesign for cloning; however, our experiences suggestedthis was not the case. In the initial design, we tended tochoose a set of good existing generic interaction tech-niques rather than designing new techniques.

We further learned that these elements can be de-scribed by a set of parameters, including the distancebetween spatially distributed copies, number of copies,shapes of the cloned objects, and visual attributes suchas color and size. These parameters were placed in a for-

mal taxonomy of the cloning task (Chen, Bowman, Lu-cas, et al., 2004).

7.3 Interaction Technique Design

We designed a series of domain specific interactiontechniques at various levels of specificity by making useof domain knowledge (see Figure 6). Complete descrip-tions of these domain specific interaction techniquesappear in our previous publications (Chen & Bowman,2006; Chen, Bowman, Lucas, et al., 2004). Here, weuse these techniques to illustrate how we have made useof the framework to produce a range of interactiontechniques.

7.3.1 Specializing Generic Techniques. Oneway to design domain specific interaction techniquesis to specialize generic interaction techniques by add-ing domain knowledge. For instance, the copy-by-example technique was a specialization of the genericHOMER technique (Bowman & Hodges, 1997).

Figure 6. Various cloning techniques sorted by domain specificity.

Chen and Bowman 11

F6


Copy by example has relatively greater domain speci-ficity compared to HOMER because it uses repetition, acommon characteristic in building and construction.Instead of placing each object individually, copy by ex-ample only requires the user to place the first copy; therest are placed automatically when the user issues a“paste” command by a button click. The user needs toreposition only if the pattern changes—the programcalculates the next position accordingly. With this tech-nique, users can build stairs, towers, and buildings thatare not necessarily square in shape.

7.3.2 Designing Directly for Domain-LevelTasks. We can also design techniques directly for thecloning task using horizontal path 2 from Figure 5. Forexample, PORT takes into account several constraints ofstructural design, such as the square shape and the re-petitive distribution of architectural elements. The dy-namic slider techniques uses a pen-and-tablet interfacewhile PORT uses a direct pointing technique to specifythe number of copies and the distances in each direc-tion. The user can push a joystick on a handheld wanddevice to change the number of copies and the distancesbetween copies. Both methods support continuous in-put and the axes of the sliders and the directions ofwand pointing indicate the direction of input in an intu-itive manner.

Another example is the space metaphor interactiontechnique, which uses PORT and also allows users toexplicitly define the horizontal span (dimensions in spacethat a building will occupy) before cloning. The span con-straint was suggested by our architect collaborators. Usingthe space metaphor interaction technique, adjusting thedistance between columns or beams changes the numberof copies, so the building is always confined to the pre-specified area. Any objects out of the span are automati-cally erased. This technique had the highest level of do-main specificity among those we designed.

7.3.3 Mixed Decomposition and CompositionApproach. One way to think about the cloning task isto recast it (path 7 in Figure 5) as a numeric input task,which is a symbolic input task at the generic level; there-fore any interaction techniques that allow users to input

numbers should work. The keypad technique we de-signed for cloning used very little domain knowledge;users simply specify which building axis they are work-ing with and input a number.

8 Evaluation of DSD

We conducted a series of user studies on the effec-tiveness of the DSD approach in the case study applica-tion. Details are given elsewhere (Chen & Bowman,2006); here we briefly summarize the evaluationmethod and results. Although these studies do notcompare DSD directly with other design approaches,they provide helpful insights into the relative usabilityand usefulness of 3D interaction techniques producedby DSD and by more traditional design methods.

8.1 The Effect of a Domain SpecificTask

The purpose of the first experiment (Chen, Bow-man, Lucas, et al., 2004) was to study the effects ofconsidering a domain specific task during design. A do-main specific cloning technique (PORT) and a genericobject manipulation approach (HOMER) were com-pared. Users performed three tasks ranging from simple(fewer than 10 elements) to complex (200 or more ele-ments) building modeling. We found that the domain-specific task was suitable for describing users’ activitiesin modeling complex structures. PORT outperformedHOMER for large building modeling.

8.2 The Effect of Domain KnowledgeUse

The second experiment studied the effect of usingdomain knowledge in designing interaction techniques(Chen & Bowman, 2006). Comparing the five domainspecific interaction techniques presented in Section 4,we found that the usefulness of the interaction was asso-ciated with specificity—the greater the domain specific-ity, the better the task performance when modeling alarge number of structural elements. As Figure 7 shows,


F7


the space metaphor, PORT, and dynamic slider tech-niques (those with greater domain specificity) were sig-nificantly faster than the copy-by-example and keypadtechniques for a modeling task where thousands of ele-ments were to be built. On the other hand, techniqueswith less domain specificity support a wider range oftasks. For instance, the copy-by-example technique canbe used to build structures of any shape that has repeti-tive patterns, since it does not require a regular squareshape. The keypad technique is flexible enough to beused in any numeric input task.

Participants commented that they preferred copy byexample for simple tasks and PORT or the space meta-phor for complex modeling work. When participantscreated only one element, they perceived the task as re-quiring a single object copy and placement rather than acloning operation.

8.3 The Effect of DSD on a CompleteApplication

DSD will not be practical if the interaction tech-niques developed cannot be used in real-world appli-cations. We therefore designed a new version of theVirtual-SAP application by integrating several new clon-ing techniques. The keypad, copy-by-example, andPORT cloning techniques were incorporated so as toprovide alternatives to the user in modeling a structure.Figure 8 shows the new design with multiple object se-lection and cloning. A pen and tablet metaphor userinterface is also used in this design. A user study wasconducted to examine whether results produced inDSD increase the overall usefulness of Virtual-SAP.

The empirical evaluation (Chen, 2006) was con-ducted with two groups: a control group using a version

Figure 7. Techniques designed with higher specificity lead to shorter task completion time than lower ones when generating hundreds ofarchitectural elements. Lower specificity increases flexibility in the range of objects that architects can build. Arrows indicate a statisticallysignificant difference for overall task completion time.

Chen and Bowman 13

F8


of Virtual-SAP without support for cloning and an ex-perimental group using an application version that wasidentical except for the inclusion of a cloning technique(PORT). Both user interfaces had rich functionality,such as choosing architectural elements, placing objects,and visualizing earthquake simulations. The VE systemwas HMD-based.

Participants were architectural engineering students andwere grouped into pairs. In each pair, one participant usedthe system to build a structure that he or she had previ-ously drawn on paper, while the other participant watchedthe process on a monitor. They were allowed to talk aboutthe modeling process. We collected participants’ verbalcommunication with their partners, task performance,comments, and the structures they designed.

The results showed that participants from the domainspecific group modeled a wider range of complex build-ings. Many of the conversations in the domain groupwere regarding activity-relevant information (e.g., whatis the horizontal span of a building), as compared toaction-based information (e.g., how to place a beam) inthe generic group. As expected, the domain specificgroup had better task performance.

8.4 Discussion

The results from the experiments favor the userinterfaces produced by DSD. Considering domain spe-cific tasks and using domain knowledge in design led tohigher performance on individual tasks, and integrating

Figure 8. New Virtual-SAP user interface. The semi-transparent box was used for multiple object selection: objects intersecting with the boxwere selected. The wireframe structure was generated using one of our cloning techniques.



domain specific interaction techniques into an existingapplication led to greater overall usefulness.

Of course, increasing domain specificity can alsoreduce the reusability of an interaction technique. Forexample, the keypad can be used for any numeric in-put task, not just in the architecture domain. Copy byexample is flexible enough to be used to form shapessimilar to spiral stairs and frame structures forbridges.

Our evaluation of interaction techniques and theVirtual-SAP application provides empirical supportfor the effectiveness and general principles of DSD,although we cannot claim from these results thatDSD is superior to other design methods for 3D in-teraction. An alternative way to compare designmethods would be to compare the training time andresources needed for different approaches. We didnot do this because we believe that the increased costof DSD arises from the cost of collecting domainknowledge in order to frame domain specific tasksand thus design domain specific interaction tech-niques. Such costs exist, in the form of task analysis,in any effective design method.

9 Conclusions and Future Work

We have described DSD, a design approach for3D interaction techniques with the ultimate goal ofproviding effective and efficient 3D UIs for real-world applications. We have presented a 3D interac-tion design framework that describes the relationshipbetween DSD and the existing generic and applica-tion specific design approaches. The framework canhelp designers understand and think more systemati-cally about 3D interaction tasks and 3D interactiontechniques. We have also introduced a three-stepDSD method that designers can follow for the devel-opment of domain specific 3D interaction techniques.Finally, we illustrated the use of the DSD approachand method through a case study in the architecturedomain, and presented evidence for the effectivenessof DSD through empirical studies.

Major empirical findings from this research include

the following. First, an interaction technique for adomain specific task increased usability over an inter-action technique designed for generic level tasks. Sec-ond, in general, techniques designed with more do-main specificity outperformed techniques with lessdomain specificity for domain specific tasks. Third,integrating a domain specific technique into an appli-cation yielded an effective and efficient interface.

The current research is limited in that we have con-ducted only one case study and did not perform andcompare formal domain knowledge acquisition andorganization methods. In addition, our ability to pro-duce many interaction techniques might be due inpart to our expertise rather than to the DSD methodalone. DSD is a high level method, not a detailedprocess. We leave these topics as future work.

We also plan to further address the reuse of domainspecific interaction techniques. One possibility is to cre-ate a mapping between knowledge representation andinteraction techniques, and build a collection of knowl-edge possessed by domain experts. In this way, knowl-edge and the relevant interaction techniques can be pro-cessed, stored, and improved over time.

We also hope to broaden the impact of our theoriesof domain specific interaction techniques. For example,the domain of visual analytics that includes multivariateand multidimensional data sets is a promising area forfuture research in domain specific interaction.

Finally, we plan to explore other types of specificity toimprove 3D interaction. For example, emerging tech-nologies such as large, tiled, high-resolution displaysmay require display specific 3D interaction techniquesto ensure usability.

Acknowledgments

This work was supported by the National Science Foundationunder grant NSF-IIS-0237412. The authors gratefully ac-knowledge the support of Dr. Mehdi Setareh, who was ac-tively involved in the design and evaluation process. The au-thors also appreciate those who gave their valuable time toparticipate in the empirical evaluations.

Chen and Bowman 15


Documents

tapraid4/z92-psen/z92-psen/z9200509/z922113d09g colesona S ... · DSD method that can be used by designers. Finally, we present the results of several studies indicating the effec-tiveness