Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces, pages 71–78,Baltimore, Maryland, USA, June 27, 2014. c�2014 Association for Computational Linguistics

Hierarchie: Interactive Visualization for Hierarchical Topic Models

Alison Smith, Timothy Hawes, and Meredith MyersDECISIVE ANALYTICS Corporation

Arlington, VA{alison.smith, timothy.hawes, meredith.myers}@dac.us

Abstract

Existing algorithms for understandinglarge collections of documents often pro-duce output that is nearly as difficultand time consuming to interpret as read-ing each of the documents themselves.Topic modeling is a text understandingalgorithm that discovers the “topics” orthemes within a collection of documents.Tools based on topic modeling become in-creasingly complex as the number of top-ics required to best represent the collec-tion increases. In this work, we presentHierarchie, an interactive visualizationthat adds structure to large topic models,making them approachable and useful toan end user. Additionally, we demonstrateHierarchie’s ability to analyze a diversedocument set regarding a trending newstopic.

1 Introduction

In computational linguistics and related fields, sig-nificant work has been invested in the developmentof algorithms for gaining insight from large bod-ies of text. The raw output of these techniquescan be so complex that it is just as difficult andtime consuming to understand as reading the text.Therefore, it is an especially challenging problemto develop visualizations that add analytic value,making complex analysis accessible by helping auser to understand and interact with the output ofthese algorithms.

Topic Modeling is a common, data-driven tech-nique for summarizing the content of large textcorpora. This technique models documents as dis-tributions of topics and topics as distributions ofwords. In practice, topic models are used to pro-vide a high-level overview and guided explorationof a corpus. Prior work by others (Chaney and

Blei, 2012) and by the author (Smith et al., 2014)has focused on visualizing the results of topicmodeling to support these two goals, but thesevisualizations do not scale beyond 10 to 20 top-ics1. Topic models with a small number of top-ics may not accurately represent very diverse cor-pora; instead, representative topic models requirea number of topics an order of magnitude higher,for which current visualization methods are notsuitable. We propose a visualization that displayshierarchically arranged topics. As opposed to aflat model, which can be thought of as an un-ordered heap of topics, a hierarchical structure al-lows a user to “drill into” topics of interest, mean-ing this technique supports directed exploration ofa corpus regardless of the number of topics in themodel.

Although methods that use inherently hierarchi-cal generative models do exist, we take a simplerecursive approach that scales to large datasets anddoes not change or depend on the underlying topicmodeling implementation. In principle, this tech-nique could be applied to a range of topic model-ing algorithms. We present this hierarchical modelto the user through an intuitive interactive visual-ization, Hierarchie. Additionally, we demonstratethe capability with a Case Study on analyzing thenews coverage surrounding the Malaysia Airlinesflight that went missing on March 8, 2014.

2 Related Work

Latent Dirichlet Allocation (LDA) (Blei et al.,2003b) is an unsupervised algorithm for perform-ing statistical topic modeling that uses a “bag ofwords” approach, treating each document as a setof unordered words. Each document is repre-sented as a probability distribution over some top-ics, and each topic is a probability distribution over

1Either the visualization becomes too confusing to under-stand or using the visualization to explore the corpus takestoo much time — or both.

71

words. LDA is an effective, scalable approach tomodeling a large text corpus; however, the resultis a flat topic model with no hierarchical structurefor a visualization to exploit.

Approaches exist for learning topic hierarchiesfrom data, such as the Nested Chinese restaurantprocess (Blei et al., 2003a) and Pachinko Alloca-tion (Li and McCallum, 2006). These approachesbuild the intuitions of the hierarchy into the mod-eling algorithm. This adds additional complexityand tightly couples the hierarchical process withthe underlying modeling algorithm.

Our Hierarchical Topic Modeling method usesa simple top-down recursive approach of splittingand re-modeling a corpus to produce a hierarchi-cal topic model that does not require a specific un-derlying topic modeling algorithm. This work ismost similar to Dirichlet Compound MultinomialLatent Dirichlet Allocation, DCM-LDA, whichprocesses the corpus via a bottom-up approach.DCM-LDA first trains unique topic models basedon co-occurrence of words in each document, andthen clusters topics across documents (Mimno andMcCallum, 2007).

Existing visualizations support analysis and ex-ploration of topic models. Topical Guide (Gardneret al., 2010), TopicViz (Eisenstein et al., 2012),and the topic visualization of (Chaney and Blei,2012) provide visualization and interaction withtopic models for corpus exploration and under-standing. These visualizations typically repre-sent topics as word clouds, where the topic modelas a whole is presented as an unordered set oftopics. This approach is not optimal for effi-cient exploration and understanding, and the seaof word clouds quickly becomes overwhelming asthe number of topics grows. Termite (Chuang etal., 2012) uses a tabular layout to represent a topicmodel and supports easy comparison of wordswithin and across topics. The Termite visualiza-tion organizes the model into clusters of relatedtopics based on word overlap. This visualizationtechnique is space saving and the clustering speedscorpus understanding. Our approach clusters top-ics by document overlap instead of word overlapand is hierarchical, providing multiple levels of re-lated topics for intuitive corpus exploration.

Nested lists, icicle plots (Kruskal andLandwehr, 1983), and treemaps (Shneider-man, 1998) are commonly used for visualizinghierarchical data, but they have limitations and do

not easily support data-dense hierarchies, such ashierarchical topic models. Nested lists can be hardto navigate as they fail to maintain the same sizeand approximate structure during exploration. Anicicle plot, which is a vertical representation ofa partition chart, suffers from similar renderingconstraints and limits positioning, sizing, andreadability of text labeling. Treemaps use nestedrectangles to display hierarchical data, but havebeen criticized as not cognitively plausible (Fab-rikant and Skupin, 2005), making them difficultto interpret. Additionally, as is the case fornested lists and icicle plots, treemaps obscure thestructure of the underlying data to accommodatelayout and sizing constraints.

Hierarchie uses an interactive sunburstchart (Stasko et al., 2000), which is a partitionchart with radial orientation that supports visual-izing large or small hierarchies without requiringscrolling or other interaction. The sunburst chartimplementation used by Hierarchie is directlybased upon the Sequences Sunburst (Rodden,2013) and Zoomable Sunburst (Bostock, 2012b)examples that are implemented in the Data-DrivenDocuments library (Bostock, 2012a).

3 Hierarchical Topic Modeling

The HLDA algorithm takes a simple, top-downapproach for producing hierarchical topic modelsby recursively splitting and re-modeling a corpus.Standard LDA discovers the distribution of wordsin topics and topics in documents through an infer-ence process; our implementation uses Gibbs sam-pling (Griffiths and Steyvers, 2004) for inference.As a result of this process, each word in a docu-ment is assigned to a topic. At the end of sampling,HLDA uses these word-to-topic assignments toconstruct new synthetic documents for each topicfrom each of the initial documents. These syn-thetic documents contain only those words fromthe original document that are assigned to the topicand make up the synthetic corpus for the topic. So,if there are 10 topics in the topic model, up to 10new synthetic documents — one for each topic —will be created for each document, and these doc-uments will be merged into the topic’s syntheticcorpus.

For each topic, t, we then construct a new topicmodel, m

t

, using the synthetic corpus correspond-ing to t. The discovered topics in m

t

representthe subtopics of t. This process, illustrated in

72

Figure 1: Overview of the HLDA algorithm. The algorithm runs LDA over the original corpus whichresults in a topic model and word-topic assignments. These word-topic assignments are used to createsynthetic documents — one for each document/topic pair. The synthetic documents are grouped intosynthetic corpora by topic, and LDA is run for each of the synthetic corpora. This process continuesrecursively until the synthetic corpus and documents are too small to model. The result is a hierarchy oftopic distributions.

Figure 1, can be repeated recursively, until thesynthetic corpus and synthetic documents are toosmall to model.2 While the number of topics ateach level in the hierarchy must be specified, theoverall number of topics discovered by this ap-proach is a byproduct of the algorithm.

This modeling approach is a wrapper algorithmthat can be applied to any modeling approach thatassigns individual tokens in documents to specifictopics.

4 Hierarchie

To effectively visualize the topic hierarchy out-put from HLDA, it is important to properly con-vey the relevance and structure of the topics. In-tuitive interaction with the visualization is impor-tant so users can easily explore topics and identifypatterns. Without effective visualization, formingconclusions becomes as difficult as approachingthe raw documents without the benefit of algorith-mic analysis.

In practice, a diverse set of visualizations areused to display hierarchical data. An effective vi-sualization of a hierarchical topic model shouldsupport the following Use Cases:

1. Accuracy - display topics without hiding orskewing the hierarchical structure

2. Granularity - interact with the visualization

2This is parameterized and can be set based on tolerablequality degradation from short documents or small corpora.

to explore the topics at all levels of the hier-archy

3. Accessibility - view the underlying data as-sociated with the topics

Many of the visualizations we considered forviewing topic hierarchies obscure or misrepresentthe true structure of their underlying data, largelydue to the amount of space required for rendering.Others provide less skewing of the structure, yet,for large hierarchies, require a high degree of userinteraction (clicking and navigating) to expose theunderlying data. We found that a sunburst chart isbest suited to our purposes as it supports visual-izing large or small hierarchies without requiringscrolling or other interaction. Unlike other hierar-chical visualizations, the sunburst can accommo-date the size of a typical computer screen withouthiding or minimizing structure.

Figure 2 displays a top-level view of theHierarchie visualization for a dataset of Tweets,Reddit comments, and news articles regarding theMalaysia Airlines flight. Each level of the hierar-chical topic model is represented as a ring of theSunburst chart where the arcs comprising the ringsrepresent the individual topics. By not labelingeach arc, or “slice,” within the sunburst, the high-level overview of the hierarchical topic model ispresented to the user with minimal complexity.

The initial, high-level view of the sunburstchart follows the design principle of overviewfirst, zoom and filter, details on demand (Shnei-

73

Figure 2: The top-level view of the Hierarchie visualization. This visualization uses a sunburst chart,which is optimal for displaying the topic hierarchy created by the HLDA algorithm without hiding orskewing the hierarchical structure.

derman, 1996) and does not display details for ev-ery topic, requiring user interaction to expose ad-ditional data. In our sunburst visualization, userinteraction allows for exploration of the informa-tion at a finer granularity. When hovering over atopic of interest, the words of the topic are dis-played in the empty center of the sunburst. This isan efficient use of space and prevents disorienta-tion, since minimal eye movement is required be-tween the slice of interest (where the user’s mouseis located) and the center list of topics.

When a user selects a slice of interest, the sun-burst zooms in to display the selected topic andsub-topics. This allows the user to analyze a spe-cific section of the hierarchy. This interaction isshown in Figures 4 and 5. The sunburst has re-oriented to display the selected sub-topic, (plane,crash, crashed) as the visualization root.

To provide a clean and meaningful display oftopic information for each slice, only one slice’sinformation can be shown at a time. As the sun-burst zooms to display selected topics, it is use-ful to provide context for the location of the topicwithin the overall topic hierarchy. Therefore, twocontextual visualizations — a breadcrumb trailand a contextual anchor — are provided. Bread-crumb trails are often utilized to provide contextduring navigation, such as when navigating a filestructure or large retail website. The breadcrumb

trail displays the hierarchical path leading to thecurrent topic (Aery, 2007). A contextual anchor,or contextual snapshot (Mindek et al., 2013), isused to provide additional context to the user. Thecontextual anchor displays the entire hierarchicaltopic model to the user at all times. When the userselects a topic slice to view a section of the hier-archy in more detail, the contextual anchor high-lights the position of the selected topic within thehierarchical topic model. This offers context tothe user, regardless of their location within the hi-erarchy. An example of the breadcrumb trail andcontextual anchor is displayed in Figure 3.

5 Case Study

The search for Malaysia Flight MH-370 was on-going during the composition of this paper, withfew clues indicating what might have actually oc-curred. In an attempt to organize the various the-ories, we collected 1600 Tweets and 970 Red-dit comments containing the keyword “MH370”in addition to 27 Daily Beast articles returned bya URL filter for any of the key words “malay,”“370”, “flight,” “missing,” “hijack,” “radar,” “pi-lot,” “plane,” “airplane,” and “wreckage.” Thiscorpus offers a diverse sampling of discussionconcerning the missing airliner that is too largefor a human alone to quickly analyze. We pro-

74

Figure 3: Our simple breadcrumb trail and contex-tual anchor offer constant context as the user ex-plores the visualization. Highlighted slices withinthe contextual anchor are those currently dis-played in the sunburst visualization.

cessed the corpus with HLDA using 10 topics foreach level. This number of topics balances gran-ularity and accuracy. Using too many narrow top-ics results in information overload, whereas toofew broad topics could be difficult to understand3.We then visualized the resulting hierarchical topicmodel with Hierarchie as shown in Figure 2. Aswe were most interested in looking at the vari-ous theories surrounding the flight, we chose toexplore one of the high-level topics, (plane, peo-ple, pilot, think, know), in more detail, becausemany of this topic’s sub-topics suggest specifictheories related to the outcome of MH-370. Ta-ble 1 shows the 10 sub-topics for the “theory”topic represented by their 3 most probable terms.The bolded topics are those that suggest theories.Figure 4 shows the sunburst graph reoriented af-ter the selection of the main “theory” topic. Thesunburst graph is labeled with the sub-topics thatrepresent the selection of interesting theories.

These topics suggest four primary theories: thatthe plane landed, the plane crashed, the planewas hijacked by terrorists, or the pilot crashedthe plane in an act of suicide. Hovering over the(plane, crash, crashed) topic shows the sub topics,and clicking the topic reorients the sunburst chart,

3Deviating from this number slightly may also be effec-tive, and experimentation is required to determine the num-ber of topics that is the best fit for the current data set and endgoal.

plane, crash, crashedplane, landed, landplane, think, people

pilot, plane, hijackingterrorist, terrorism, passports

suicide, pilot, oceanShah, Anwar, political

plane, China, worldphone, phones, cell

evidence, think, make

Table 1: The 10 high-level topics of the modelgenerated from running HLDA on the MalaysiaFlight MH-370 corpus. The bolded topics suggestspecific theories regarding the status of the plane.

crash, water, crashedfailure, catastrophic, mayday

mechanical, failure, daysplane, ocean, didplane, error, lost

Table 2: A selection of the sub-topics of discus-sion surrounding a plane crash scenario. Thesesub-topics suggest more detailed discussion. Forexample, that the plane crash may have resultedfrom a catastrophic mechanical failure or other er-ror.

as shown in Figure 5. The sub-topics under (plane,crash, crashed) suggest more detailed discussionof a crash scenario, such as the plane crashing intothe water, and that there may have been a catas-trophic mechanical failure or other error. Table 2contains a selection of these sub-topics.

An alternate theory is suggested by the (terror-ist, terrorism, passports) topic, which is shown inFigure 6. The sub-topics here suggest more de-tailed discussion involving terrorism as the causefor the plane’s disappearance. Table 3 contains aselection of these sub-topics.

The hierarchical topic model produced byHLDA and visualized with Hiearchie provide au-tomated organization of the many theories regard-ing the missing Malaysian airliner. The high-leveloverview provides a quick summary of all of thediscussion surrounding the event, while the hi-erarchical organization and intuitive explorationallows the discussion, and specifically each the-ory, to be explored in depth, exposing potentially

75

passports, stolen, usingterrorists, crash, terrorist

Muslim, Muslims, Islamicattack, going, terror

responsibility, common, group

Table 3: A selection of the sub-topics of discus-sion surrounding a terrorism scenario. These sub-topics include more details, such as the discussionof stolen passports, relevant to the theory that theplane disappearance is the result of an act of ter-rorism.

Figure 4: Sub-categories of interest have beenpurposely numbered for clarity. 1:(plane, crash,crashed); 2: (plane, landed, land); 3: (terrorist,terrorism, passports); 4: (suicide, pilot, ocean).

relevant information. Organizing all of this databy hand would be difficult and time consuming.This intuitive visualization in combination withour method for organizing the underlying datatransforms a disparate corpus of documents intoa useful and manageable information source.

6 Future Work and Conclusion

The Hiearchie visualization and related hierarchi-cal topic modeling algorithm support the under-standing and exploration of text corpora that aretoo large to read. Although existing topic mod-eling algorithms effectively process large corpora,the resulting topic models are difficult to interpretin their raw format. Current visualization meth-ods only scale to a small number of topics, whichcannot accurately represent a diverse corpus. Ad-ditional structure is required to organize a repre-sentative topic model of a large dataset into an un-

Figure 5: Clicking the (plane, crash, crashed)topic slice in the top-level (plane, people, pilot)visualization reorients the sunburst to display theslice as its root, enabling more detailed explo-ration of sub-topics.

Figure 6: The(terrorist, terrorism, passports)topic slice in the top-level (plane, people, pilot)visualization.

76

derstandable and navigable analysis tool.Our approach visualizes the hierarchical topic

model produced by the HLDA algorithm to sup-port intuitive, directed browsing of topical struc-ture within a diverse collection of documents. Asdemonstrated in the Malaysia Airlines case study,this technique can be used to quickly gain insightabout the diverse speculation surrounding a signif-icant, inconclusive event. Hiearchie enables usersto examine and gain insight from large, diversedatasets more efficiently than if they had to inter-pret complicated algorithmic output or read rawdocuments.

The sunburst visualization provides a clearoverview of the structure of the model; however,individual topics are currently represented as listsof words ordered by their probability for the topic.This is non-optimal for topic understanding. Ad-ditionally, this topic information is displayed onhover, which does not easily support making com-parisons between topics. Future work includes im-plementing alternative techniques for displayingthe topic information and performing an evalua-tion to determine which technique is most appro-priate for the intended use cases.

Future work also includes adding additional in-formation to the visualization through color andtopic placement. In the current implementation,topic slices are currently colored by the mostprevalent topic word. Coloring slices by sentimentor other topic-level metrics will enrich the visual-ization and improve the user’s ability to quicklydiscern different topics and their meaning withinthe model as a whole. Similarly, topic position inthe sunburst does not currently provide any usefulinformation. One possible layout is based on topiccovariance, which is a metric of topic relatednessbased on the frequency of topic pair co-occurrencewithin the documents of the corpus. An improvedsunburst layout could take into account topic co-variance to optimize the layout such that relatedtopics were positioned together at each level of thehierarchy.

Acknowledgements

We would like to thank Andrew McCallum fordiscussions related to the ideas behind our topicmodeling approach. We would also like to thankMark Frymire, Peter David, Jen Sikos, and Nic-hole Hansen for their support in the writing pro-cess. Additionally, we would like to acknowl-

edge that this work was performed under AFRLcontract FA8750-12-C-0077. The opinions andconclusions do not reflect the position of the AirForce.

77

ReferencesSean C Aery. 2007. Breadcrumb navigation deploy-

ment patterns in retail web sites.

David M Blei, Thomas L Griffiths, Michael I Jordan,and Joshua B Tenenbaum. 2003a. Hierarchicaltopic models and the nested chinese restaurant pro-cess. In NIPS, volume 16.

David M Blei, Andrew Y Ng, and Michael I Jordan.2003b. Latent dirichlet allocation. the Journal ofmachine Learning research, 3:993–1022.

Mike Bostock. 2012a. Data Driven Documents (d3).http://d3js.org.

Mike Bostock. 2012b. Zoomable sunburst. http://bl.ocks.org/mbostock/4348373.

Allison June-Barlow Chaney and David M Blei. 2012.Visualizing topic models. In ICWSM.

Jason Chuang, Christopher D Manning, and JeffreyHeer. 2012. Termite: Visualization techniques forassessing textual topic models. In Proceedings ofthe International Working Conference on AdvancedVisual Interfaces, pages 74–77. ACM.

Jacob Eisenstein, Duen Horng Chau, Aniket Kittur, andEric Xing. 2012. Topicviz: interactive topic explo-ration in document collections. In CHI’12 ExtendedAbstracts, pages 2177–2182. ACM.

Sara Irina Fabrikant and Andre Skupin. 2005. Cogni-tively plausible information visualization.

Matthew J Gardner, Joshua Lutes, Jeff Lund, JoshHansen, Dan Walker, Eric Ringger, and Kevin Seppi.2010. The topic browser: An interactive tool forbrowsing topic models. In NIPS Workshop on Chal-lenges of Data Visualization.

Thomas L Griffiths and Mark Steyvers. 2004. Find-ing scientific topics. Proceedings of the Nationalacademy of Sciences of the United States of Amer-ica, 101(Suppl 1):5228–5235.

Joseph B Kruskal and James M Landwehr. 1983. Ici-cle plots: Better displays for hierarchical clustering.The American Statistician, 37(2):162–168.

Wei Li and Andrew McCallum. 2006. Pachinko allo-cation: Dag-structured mixture models of topic cor-relations.

David Mimno and Andrew McCallum. 2007. Orga-nizing the oca: learning faceted subjects from a li-brary of digital books. In Proceedings of the 7thACM/IEEE-CS joint conference on Digital libraries,pages 376–385. ACM.

Peter Mindek, Stefan Bruckner, and M Eduard Groller.2013. Contextual snapshots: Enriched visualizationwith interactive spatial annotations. In Proceedingsof the 29th Spring conference on Computer Graph-ics (SCCG 2013).

Kerry Rodden. 2013. Sequences sunburst. http://bl.ocks.org/kerryrodden/7090426.

Ben Shneiderman. 1996. The eyes have it: A taskby data type taxonomy for information visualiza-tions. In Visual Languages, 1996. Proceedings.,IEEE Symposium on, pages 336–343. IEEE.

Ben Shneiderman. 1998. Treemaps for space-constrained visualization of hierarchies.

Alison Smith, Sana Malik, and Ben Shneiderman.2014. Visual analysis of topical evolution in un-structured text: Design and evaluation of topicflow.

John Stasko, Richard Catrambone, Mark Guzdial,and Kevin McDonald. 2000. An evaluation ofspace-filling information visualizations for depict-ing hierarchical structures. International Journal ofHuman-Computer Studies, 53(5):663–694.

78