Redefining a Contribution forImmersive Visualization Research

Ramik SadanaGeorgia Institute of Technologyramik@gatech.edu

Vidya SetlurTableau Researchvsetlur@tableau.com

John StaskoGeorgia Institute of Technologystasko@cc.gatech.edu

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s).ISS ’16 Companion, November 06-09, 2016, Niagara Falls, ON, CanadaACM 978-1-4503-4530-9/16/11.http://dx.doi.org/10.1145/3009939.3009946

AbstractImmersive computing modalities such as AR, VR, andspeech-based input are regaining prominence as researchthreads in the visualization field due to the advancement intechnology and availability of cheap consumer hardware.This renewed interest is similar to what we observed adecade ago when multitouch technology was gaining main-stream adoption. In this work, we reflect on lessons learnedfrom designing for multitouch, with the goal of highlightingproblems that may also emerge in AR/VR research. Specif-ically, we emphasize the need for the field to rearticulatewhat is expected from research efforts in the area of visual-ization on immersive technologies.

Author KeywordsImmersive technology; AR; VR; speech; multitouch

ACM Classification KeywordsH.5.m [Information interfaces and presentation]: Miscella-neous

IntroductionComputing modalities such as augmented reality, virtualreality, touch, and speech-based input have gained po-tential for mainstream adoption due to the introduction ofadvanced technology in low-cost hardware. The visualiza-tion research community has taken a keen interest in en-

visioning novel ways of using these modalities to supportthe process of sensemaking with data. Although the modal-ities themselves are not new [2, 6, 7], interest in them hasrenewed because there is a greater likelihood of researchcontributions reaching end users now than before.

Each modality unlocks a plethora of novel use cases andopportunities for further exploration. Past research hasclearly demonstrated the potential of these modalities incollaborative data analysis [3] and for envisioning novel in-teraction techniques [8]. The opportunities are, however,accompanied by an equally distinctive set of design chal-lenges. These include representational challenges (e.g.2D vs. 3D visualizations for AR/VR), interaction challenges(e.g. selecting objects in 3D interaction environments [1]),and task ambiguity challenges (e.g. interpreting voice-based questions about data [8]).

Several research endeavors are currently underway thatexamine these challenges across different modalities. How-ever, one problem that often stifles such endeavors is thenovelty fallacy: the argument that an idea is superior exclu-sively because it is new. As a result, an idea that does notdemonstrate novelty is not considered a significant contribu-tion.

This rush for novelty hinders progress by making it difficultfor researchers to address the fundamental problems thatcurrently affect our adoption of the new modalities. In thiswork, we discuss this theme further. Drawing on the anal-ogy between multitouch and the new immersive modalities,we highlight issues from the past that may emerge again.Finally, we discuss the potential ways in which the researchcommunity can address these issues.

A lesson from the pastThe emergence of mainstream affordable AR/VR technol-ogy has surely piqued the interest of the visualization field.Yet, such a boost is not unique. In the past, we witnessedsimilar enthusiasm when multitouch technology enteredmainstream usage with the introduction of Apple’s iPhoneand iPad. Multitouch equally fascinated the research com-munity as it opened doors to new platforms and designchallenges.

However, while the multitouch revolution was real and far-reaching, our community may not have fully unlocked thepotential of visualization on this technology. In the yearssince, only about twenty research articles have been pub-lished in leading conferences, such as CHI1, VIS2, UIST3,DIS4, and ISS5, that address this research thread.

This relative lack of significant research can be interpretedin several ways. One argument could be that we simplyoverestimated the benefits of multitouch for visualization.Alternatively, one could conclude that multitouch may justnot be an ideal platform for visualization. The same argu-ment can then be extended to other, newer technologiessuch as AR/VR, and speech-based interfaces.

However, we believe that such an interpretation would bepremature and inaccurate. Instead, we speculate that thelack of research is due to a different problem that currentlyburdens research explorations in new domains — the in-consistency between the contribution the visualization com-munity expects and the contribution that is needed.

1Computer-Human Interaction www.sigchi.org2Visualization www.ieeevis.org3User Interface Software and Technology Symposium uist.acm.org4Designing Interactive Systems www.dis2016.org5Interactive Surfaces and Spaces iss.acm.org

Deconstructing contributionMunzner highlights five categories in which the contributionof visualization research can be classified — Technique,Design Study, Systems, Evaluation, and Model [5]. Whilethe categorization is fairly comprehensive, it omits any ref-erence to visualization research for new devices, platforms,and technologies. Such research is typically classified un-der the Techniques and Design Study categories, neitherof which effectively represent the effort required in bringingvisualization to these new platforms. Instead, the categoriesindirectly mandate that research endeavors result in a novelrepresentation technique or interaction behavior.

The challenge in bringing visualization to new technolo-gies is not only of envisioning novel methods of presentingdata. Rather, the true challenge is in adapting the exist-ing techniques, ones we are familiar with and presume tobe fairly robust, to these new platforms. For example, howshould data-dense techniques such as parallel coordinatesor scatterplot-matrix be presented on handheld multitouchdevices? Similarly, how can traditional 2D visualizationtechniques such as barcharts and linecharts be projectedin a multiuser 3D Virtual Reality environment?

A research exercise that explores the answers to thesequestions will always run the disadvantage of seeming fa-miliar to the reviewers. Even though the constraints of inputand usage scenarios change drastically, because the un-derlying representation is not new, the research runs therisk of not being novel enough.

A similar disadvantage is also present for interaction, whichis perhaps more important than representation to get righton new technologies. While decades of research has re-sulted in consistent solutions for most interaction scenarioson desktops, it is understood that these solutions do nottranslate to other platforms [4]. In practice, however, we

observe that reviewers often measure the worthiness of acontribution based on how similar or dissimilar it seems tosolutions designed earlier for desktops.

Getting the interaction right is a considerably difficult prob-lem to solve. For instance, it is unclear how we can effec-tively interact with treemaps (e.g. Map of the Market [9]) ona mobile device - how rich can the feature set be and whichinteractions do we use? Similarly, how should we annotatethe components of a visualization to meaningfully interactwith it using voice? Solving these interaction challenges of-ten requires several iterations, and even then we may notfind ourselves any closer to identifying the correct imple-mentation.

The underlying goal, however, is to make visualization workfor users on a new platform or modality. We firmly believethat the solution is not to envision every single aspect ofthe interface and the interactions. Novelty will only excel atbringing a user to the new platform, not keeping her there.

Redefining ContributionIn light of the above discussion, we feel that there is a needto rearticulate what the goals and contributions of a re-search effort must be. This exercise would require researchersfrom different areas of expertise within our field to deliber-ate and converge to a collective understanding. In doing so,it would also be valuable to reflect on the practices followedwithin the design fraternity and the rich knowledgebase ofprinciples for interface and interaction design that they havegathered.

For example, simplicity is often a goal that expert designersstrive for. The goal postulates that the design should makesimple, common tasks easy, and communicate clearly inthe user’s own language. This is relevant for visualization,whether one is designing a technique, an interaction, or an

entire workflow. But simplicity can also be at odds with nov-elty as it requires one to deconstruct what may already existinstead of rethinking additional or complementary solutions.

It is important to consider this and other design principleswhen constructing a description of a contribution. Below,we present four guidelines that may be beneficial as start-ing points in the exercise of redefining the contribution.

1. Novelty - Do not make novelty of work the ultimategoal. In designing for the end-users of a new plat-form, we often overemphasize the need for novelty ofthe platform within the design of our systems. How-ever, there is considerable value in research thatsolves a well-motivated problem using a combinationof preexisting solutions.

2. Value vs. overhead - Measure the value of a solu-tion against the added effort the user needs to put.Even when a solution solves a relevant problem, itmay do so by introducing significant cognitive andphysical overhead. In evaluating a contribution, it isimportant to weigh the value and overhead equally.

3. Effort to publish - Examine the solution for changesrequired to release it to end-users. It is useful to esti-mate how much the solution would differ if one wereto release it publicly. This is not to acknowledge theengineering challenge that may be needed, but in-stead to understand the practicality of the solution,and whether the needs of the end-user are consid-ered in the design of the solution.

4. Demonstration limitations - Accommodate for thefact that text or video may not be fully highlighting thecomplexity of the problem or the solution. With alter-nate input methods that constitute the larger piece

of immersive technologies, it is difficult to judge thequality of the contribution through text, images, orvideos. It is equally difficult to fully comprehend thetrue complexity of the problem being addressed orthe process that was followed to solve it. This mayrequire us to rethink how we release and share re-search findings, but is fairly important for us to con-sider as reviewers.

ConclusionWith the emergence of newer technologies, there are in-creasing opportunities for visualization research. The op-portunities require us to envision novel visual representa-tions and interactions techniques. However, to make mean-ingful contributions on the new technologies, there is alsoa need for us to revisit solutions that may have been foundearlier, but may be non-trivial to adapt to the new domains.We discuss the challenges that are currently faced by re-search of that nature. Drawing analogy from multitouch, wediscuss how these issues are relevant for newer technolo-gies, and present guidelines to assist us in better definingwhat a contribution on these technologies should be.

REFERENCES1. Ferran Argelaguet and Carlos Andujar. 2013. A survey

of 3D object selection techniques for virtualenvironments. Computers & Graphics 37, 3 (2013), 121– 136. DOI:http://dx.doi.org/10.1016/j.cag.2012.12.003

2. Ronald T Azuma. 1997. A survey of augmented reality.Presence: Teleoperators and virtual environments 6, 4(1997), 355–385.

3. Ciro Donalek, S George Djorgovski, Alex Cioc, AnwellWang, Jerry Zhang, Elizabeth Lawler, Stacy Yeh,Ashish Mahabal, Matthew Graham, Andrew Drake, and

others. 2014. Immersive and collaborative datavisualization using virtual reality platforms. In Big Data(Big Data), 2014 IEEE International Conference on.IEEE, 609–614.

4. Bongshin Lee, P. Isenberg, N.H. Riche, and S.Carpendale. 2012. Beyond Mouse and Keyboard:Expanding Design Considerations for InformationVisualization Interactions. IEEE Transactions onVisualization and Computer Graphics 18, 12 (Dec.2012), 2689–2698. DOI:http://dx.doi.org/10.1109/TVCG.2012.204

5. Tamara Munzner. 2008. Process and Pitfalls in WritingInformation Visualization Research Papers. InInformation Visualization, Andreas Kerren, John T.Stasko, Jean-Daniel Fekete, and Chris North (Eds.).Number 4950 in Lecture Notes in Computer Science.Springer Berlin Heidelberg, 134–153.http://link.springer.com/chapter/10.1007/978-3-540-70956-5_6

6. Lawrence Rabiner and Biing-Hwang Juang. 1993.Fundamentals of speech recognition. (1993).

7. Howard Rheingold. 1991. Virtual Reality: Exploring theBrave New Technologies. Simon & Schuster AdultPublishing Group.

8. Vidya Setlur, Sarah Battersby, Melanie Tory, RichGossweiler, and Angel Change. 2016. Eviza: A NaturalLanguage Interface for Visual Analysis. In Proceedingsof the 29th Annual ACM Symposium on User InterfaceSoftware & Technology (UIST ’16). ACM, New York,NY, USA. DOI:http://dx.doi.org/10.1145/2807442.2807443

9. Martin Wattenberg. 1999. Visualizing the Stock Market.In CHI ’99 Extended Abstracts on Human Factors inComputing Systems (CHI EA ’99). ACM, New York, NY,USA, 188–189. DOI:http://dx.doi.org/10.1145/632716.632834