Chemical Visualization of Human Pathogens:the Retroviral Capsids

Juan R. Perilla∗†‡, Boon Chong Goh∗†‡, John E. Stone∗, and Klaus Schulten∗†∗Beckman Institute, University of Illinois at Urbana-Champaign

†Department of Physics, University of Illinois at Urbana-Champaign‡Co-first author

Abstract—

Retroviruses are pathogens characterized by their ability toincorporate viral DNA into a host cell’s genome. Retroviruseslike Rous Sarcoma Virus (RSV) infect cells during mitosis, whenthe chromatin is exposed to the cytoplasm. Conversely, the genusof lentiviruses, like the human immunodeficiency virus (HIV),have evolved to infect non-dividing cells [1]. Despite infectingcells at different stages of their life cycles, RSV and HIV sharea similar late stage replication cycle that is highly dependenton the group actin polyprotein precursor (Gag), which containsthe matrix (MA), capsid (CA) and nucleocapsid (NC) proteins.Both HIV’s CA and Gag are considered unexploited targets forpharmaceutical intervention. We describe the techniques thatwere used to build, simulate, analyze and visualize the structuresof both Gag and CA, and we discuss scientific visualization needsthat spurred development of an interactive GPU-accelerated raytracing engine and the use of remote visualization technologies.

I. I NTRODUCTION

Retroviruses are parasites that pose a major health threat tohumans (e.g., HIV and human T-cell leukemia virus), as well asother animals (e.g., chickens (RSV), monkeys (M-PMV), mice(MLV), etc). In the case of HIV, numerous treatments havebeen developed, but the virus adapts quickly to antiviral drugssuch that new compounds need to be refined continuously. HIVinfects the human cell by inserting its genes, packaged into acapsid, into the cell’s nucleus, and thus taking control over thecell’s machinery. Once the cell is infected, the virus replicatesand is released into the host’s bloodstream.

As a virus that infects non-dividing cells, which have theirgenome protected behind a nuclear membrane, HIV has totake advantage of natural cell responses to induce cooperationof the host to reach inside the nucleus. Some species ofmonkeys are immune to HIV by spoiling this cooperationwith the capsid, attacking the capsid instead. Likewise, phar-macological interventions seek to attack the HIV capsid toprevent cooperation with the cellular machinery, or simplyby breaking the capsid apart. Such interventions, however,require knowledge of the chemical and physical propertiesof the capsid. In an accompanying movie we show the HIVcapsid, a colossal structure, containing about 1,300 proteinswith altogether 4 million atoms. Although the capsid proteinsare all identical, they nevertheless arrange themselves intoa largely asymmetric structure. The large size and lack ofsymmetry posed a huge challenge to resolving the chemicalstructure of the HIV capsid [2]. Embedding the structure intoa computational model that also included physiological solventled to a 64-million-atom simulation, the largest such simulation

achieved to date [3]. While the solution of the HIV capsidstructure was a great achievement, it was just the starting pointfor the main research agenda, which is developing new an-tiviral drugs. Computational biologists presently simulate howthe capsid exploits cellular proteins to be guided to the cell’snucleus and how antiviral drugs interfere with that process. Thecurrently identified drug candidates, while effective against thecapsid, are unfortunately also highly toxic to the patient. Anincredibly precise picture of the interactions of the capsid withthe cell and drugs has emerged from the simulation, which canhelp guide future drug development. Such research, however,requires the most extreme computer power available today.

Newly released viruses are unable to infect healthy cellsuntil they reach a mature state. An alternative strategy ofpreventing virus spread is therefore, to lock the viral particlesin their immature, non-infectious state. However, to makethe immature virus an attractive target for structure-baseddrug development one needs to know its chemical structure.Unfortunately, the complexity and size of the viral particle– an incomplete hexagonal shell close to 100 nm in size –have prevented the experimental determination of the atomic-detail chemical structure of the virus. Nonetheless, we haverecently determined an atomistic structure of the immatureretroviral lattice for Rous sarcoma virus [4] as presented in theaccompanying movie. The multi-domain RSV model revealsnovel features of the packing and dynamics of the immaturecapsid protein with implications for the maturation processand confirms the stabilizing roles of the so-called upstreamand downstream domains of the immature RSV. Since RSV isa close cousin of HIV, the RSV structure may lead to a betterexperimentation and targeting of HIV.

II. COMPUTATIONAL CHALLENGES

Computer simulations enable researchers to study themechanics of virus processes at atomic resolution, probingspatial and temporal resolutions that are currently inaccessibleto experimental methods alone [3]. Petascale supercomputersprovide the computing power necessary to simulate very largebiomolecular systems such as the HIV virus in all-atomdetail [2]. These studies require tremendous computationalcapabilities for both the simulations as well as the analysisand visualization of the results.

The immature RSV hexamer model (Figure 1 right) wasderived through a combination of state-of-the-art modelingtechniques, including, cryo-EM-guided homology modeling,and large-scale molecular dynamics simulations. Chemicalstructures of RSV capsid proteins, including the p10 peptide,

Fig. 1. The immature retroviral capsid for Rous sarcoma virus [4] (left)are composed of hundreds of identical hexamer models (right). The immatureRSV hexamer model consists of p10 peptide (green), NTD (cream), CTD(violet-blue) and SP-NC (blue-gray).

N-terminal domain (NTD), and C-terminal domain (CTD) [5],[6], were docked in a cryo-EM density map of an imma-ture retroviral lattice [7]. To complete the construction ofan immature RSV model, a crucial domain downstream ofthe capsid protein, namely spacer peptide-nucleocapsid (SP-NC), must be incorporated into the model. However, structuralknowledge of this domain is scarce as experimentalists strug-gle to elucidate the structure of the SP-NC domain at highresolution. Therefore, the SP-NC domain has to be modeledusing purely computational techniques. The SP-NC domainwas previously proposed to be a six-helix bundle (6HB) anda 6HB model was proposed whose structure is based on anion channel [8]. The tertiary arrangement of the 6HB modelhad to be optimized such that a ring of salt bridges can beformed to stabilize the 6HB model [4]. To refine the structureof the immature RSV hexamer model, an enhanced samplingtechnique named replica exchange MD simulation, a featureavailable in NAMD [9], was employed to simultaneouslysimulate>100 copies of a 0.6-million-atom system for a totalsimulation time of 5µs. Such massive calculation require 5,000Cray XK7 GPU-accelerated compute nodes and can only becompleted using petascale supercomputers.

The scaffold of an entire immature RSV capsid model(Figure 1 left) was constructed using the positional informa-tion derived from the Thomson problem [10]. Four-hundrednegatively-charged particles were placed on the surface ofa sphere of radius 60 nm, and subsequently equilibrated byCoulombic repulsion. Once equilibrated, particles were deletedmanually to mimic the pattern of the defects as observedin the cryo-electron tomogram of authentic immature RSVcapsids [11]. The all-atom model of the immature RSV hex-amer model was subsequently mapped to the final positions ofthe remaining particles. The hexamers were rotated along theazimuthal direction to minimize steric clashes, the resultingmodel is showed in the accompanying movie and in Fig. 1.Visualizing the incomplete spherical shell of the immatureRSV capsid allows us to inspect the possible mechanisms forenzymatic reactions that initiate the maturation process.

The model of the mature HIV-1 capsid [2] required an in-genious combination of computational and experimental tech-niques. The hexamer-hexamer interactions were derived usingthe molecular dynamics flexible fitting (MDFF) method [12].This method incorporates the electronic density obtained by

the microscope, with a quantum-mechanically derived atomicforce-field [13], allowing determination of the location ofindividual atoms. Due to the lack of experimental informationregarding the interactions between pentamers and hexamers,the model was derived entirely by computational modelingthrough long-time scale simulations. Remarkably, the 1.5µsof simulation obtained for the pentamer-of-hexamers – a 1.5-million-atom system – required the continuous use of 400Cray XK7 nodes for four months [2].

With the building blocks of a capsid at hand, namelythe hexamer-of-hexamers and pentamer-of-hexamers, it wasessential to determine a realistic scaffold of an HIV-1 capsid.Thus, to sample the conformational space spanned by matureretroviral capsids, the spiral algorithm was employed [14]. Thealgorithm treats the capsid as a cubic planar graph, analogousto fullerenes, enabling the search of candidate conformationsby shuffling of pentagons [15]. As illustrated in the movie,determination of the 3-D geometry of each graph, was thenperformed by embedding of the cubic graph in two dimensionsemploying the Tutte algorithm [16], followed by mapping ofthe 2-D graph into a sphere of arbitrary radius [15]. Since theconnectivity of the edges is dictated by the graph, a globalgeometry optimization is performed by treating each edge asa Carbon atom and employing a molecular forcefield. Afterthe candidate structures are generated, the cross-correlationbetween each model and cryo-electron tomography of authen-tic capsids was calculated to measure the likelihood of eachmodel [2]. The scaffold was then used to place hexamers-of-hexamers, as well as pentamers-of-hexamers, yielding a com-plete model of a retroviral capsid. Furthermore, the model wasfurther optimized by MD simulations and validated in livinghuman cells [2]. The HIV capsid is the largest biomoleculardynamics simulation – a 64-million-atom system – performedto date [3]. The simulation required the sustained use ofNAMD on 8,000 Cray XK7 nodes for a period of two months,in order to obtain enough sampling.

Parallel MD simulations were performed using NAMD [9],running on the GPU-accelerated Cray XK7 compute nodesof Titan at ORNL and Blue Waters at NCSA. NAMD andVMD use the Cray XK7 GPUs for acceleration of large-size and long-timescale simulation, analysis, and visualizationtasks [17]–[21], and they provide built-in scripting and plug-in interfaces allowing them to be extended and customized forthe research tasks described above.

III. V ISUALIZATION TECHNIQUES

All of the renderings of the all-atom virus capsids, molecu-lar dynamics simulations, and experimental cryo-EM densitieswere produced using VMD [22]. The visualizations encom-pass a wide variety of molecular representations. In severaloverview renderings of the virus capsids, capsid subunits aredrawn using “QuickSurf”, a fast GPU-accelerated molecularsurface representation provided in VMD [20], [23], as shownin the viral overview renderings shown in Figs. 1 and 2. TheQuickSurf representation allows the structure to be depictedwith smoothly variable detail, enabling atomic-scale featuresto be clearly shown or abstracted via smoothing. Renderingsof the pentameric and hexameric lattice scaffolding used toconstruct initial all-atom virus capsid models were producedusing the VMD scripting interfaces [24].

Fig. 2. VMD overview rendering of the immature retroviral lattice for Roussarcoma virus [4] (left) and the mature HIV capsid (right) composited with aconceptual rendering of the viral envelope.

During fitting of the all-atom virus capsid to experimentalcryo-EM density, individual protein domains are shown in so-called “ribbon” or “cartoon” representations in combinationwith the target experimental cryo-EM density map drawn witha transparent material that allows the interior protein backbonestructure to be clearly seen. The combination of multiple su-perimposed molecular representations makes it possible to seethe improving fit between the experimental cryo-EM densitymap and the all-atom virus structure as fitting progresses.

Colors were selected to clearly delineate key componentsof the immature and mature virus capsid structures, highlight-ing for example, the distinction between the large numberof hexameric subunits and the 12 pentameric subunits inthe mature HIV capsid. Different colors were also selectedfor latch proteins, and N-terminal and C-terminal subunits.Shadows and broad-area lighting techniques such such asambient occlusion help provide the viewer with visual cues thatelucidate pores, crevices, pockets, and cavities in molecularsurfaces. The use of ambient occlusion lighting and carefullyselected surface parameters clarifies the boundaries of individ-ual hexamer and pentamer subunits, and provides shading thathelps differentiate the N-terminal and C-terminal subunits.

The VMD ViewChangeRender and VMDMovie plu-gins were used to develop, manage, and render all of theindividual movie segments. Movie segments were producedusing keyframes composed from user-specified view orienta-tions, with associated simulation trajectory frames, molecularrepresentations, and lighting, shading, and material properties.

IV. I NTERACTIVE GPU-ACCELERATEDRAY TRACING

Built-in parallel ray tracing (RT) engines in VMD eliminatethe need for intermediate disk I/O that would be harmful forperformance [25], and they provide advanced lighting andshading capabilities unavailable in most rasterization-basedvisualization tools, including ambient occlusion lighting andshadows, depth-of-field focal blur, reflection and high qualitytransparency, stereoscopic cameras, and panoramic projectionsAll of the renderings produced with VMD used the built-inTachyonL-OptiX GPU-accelerated RT engine [20], a light-weight adaptation of the Tachyon RT engine [19], [26] basedon OptiX [27] and CUDA [28].

A fully-interactive version of the TachyonL-OptiX GPU-accelerated RT engine allows the user to view the molecularscene in exactly the way it will appear in final rendering, but

achieving high interactivity through a progressive-refinementRT algorithm that updates the display continuously as stochas-tic samples accumulated. When the user changes the vieworientation or global lighting or shading parameters, VMDclears the RT accumulation buffer and stochastic samplingstarts anew. Large scenes with complex shading can be in-teractively rendered with faster display or sample convergencerates using multiple GPUs in parallel.

To facilitate accurate and rapid previewing of HIV capsidstructures, VMD interactive RT was used extensively on a highperformance remote visualization server with four NVIDIATesla K80 GPUs. Remote visualization allowed researchers toexploit the interactive RT feature of VMD while traveling,using conventional laptop computers that would otherwise beentirely incapable of such high-quality interactive rendering.The remote visualization server provided GPU-acceleratedH.264 video streaming via NICE DCV and the NVIDIA GRIDand NVENC video SDKs, encoding directly from the GPUframebuffer to achieve high frame rates and reduced latencyby minimizing PCIe transfers between the GPUs and the host.

V. FINAL RENDERING AND POST PRODUCTION EDITING

Final rendering of movie frames was completed on the re-mote visualization server described above and on the Blue Wa-ters Cray XK7 nodes [19], [20], [29]. Movie frames were ren-dered in 16:9 aspect ratio at HD1920× 1080 resolution, with12 antialiasing samples per pixel and 12 ambient occlusionshadow rays per hit, and two directional lights; transmissionrays and shadow filtering were performed for transparent ge-ometry. The final movie was composed from individual moviesegments, still images, artwork, and audio narration compo-nents that were assembled using the Adobe After Effects non-linear video editing software. Narration audio was acquiredand composed using Adobe Audition. All audio tracks werenormalized to +0.1 db and each voice track was individuallyequalized and processed with a noise reduction filter. Musicwas composed and performed by: http://www.bensound.com.

VI. CONCLUSIONS

Ongoing advances in experimental imaging and all-atommodeling of virus structures promise new insights into thebasis of viral infection. The processes of structure determi-nation, simulation, analysis, and visualization of large virusesconstantly pose new methodological, software, and hardwarechallenges due to their size and complexity. By harassing thepower of the state-of-the-art supercomputers and cryo-electronmicroscopy, we are now able to derive atomic models of themature and immature retroviral capsids. These capsid modelswill serve as a platform to investigate the delicate interactionsof the capsid with cellular host factors and drug molecules.

The HIV modeling and visualization tasks described hereinare only possible with the use of petascale supercomputers,and represent the current threshold for computational virol-ogy. As such, extensive and routine use of the capabilitiesof the Blue Waters and Titan supercomputers was required,and high performance remote visualization was employed toaccommodate a mix of highly parallel, and highly interactivemodeling and visualization tasks. In the future we expect thathigh-interactivity remote visualization will be a feature of all

high performance computing centers, eliminating the need fortransfer of large datasets and enabling interactive modeling andvisualization tools to be easily and effectively used.

ACKNOWLEDGMENTS

This research is part of the Blue Waters sustained-petascalecomputing project supported by NSF awards OCI-0725070and ACI-1238993, the state of Illinois, and “The Compu-tational Microscope” NSF PRAC awards OCI-0832673 andACI-1440026. This research used the Oak Ridge Leader-ship Computing Facility at Oak Ridge National Laboratory,supported by DOE Contract DE-AC05-00OR22725. The au-thors acknowledge support from NSF rapid response researchaward ACI-1524703, and NIH grants 9P41GM104601 and5R01GM098243-02. The authors acknowledge Dr. Jodi Had-den for narrating the introduction of the movie.

REFERENCES

[1] Laura Hilditch and Greg J Towers. A model for cofactor use duringHIV-1 reverse transcription and nuclear entry.Curr. Opin. Virol., 4:32–36, 2014.

[2] Gongpu Zhao, Juan R. Perilla, Ernest L. Yufenyuy, Xin Meng, Bo Chen,Jiying Ning, Jinwoo Ahn, Angela M. Gronenborn, Klaus Schulten,Christopher Aiken, and Peijun Zhang. Mature HIV-1 capsid structureby cryo-electron microscopy and all-atom molecular dynamics.Nature,497:643–646, 2013.

[3] Juan R. Perilla, Boon Chong Goh, C. Keith Cassidy, Bo Liu, Rafael C.Bernardi, Till Rudack, Hang Yu, Zhe Wu, and Klaus Schulten. Molec-ular dynamics simulations of large macromolecular complexes.Curr.Opin. Struct. Biol., 31:64–74, 2015.

[4] Boon Chong Goh, Juan R. Perilla, Matthew R. England, Katrina J.Heyrana, Rebecca C. Craven, and Klaus Schulten. Atomic modeling ofan immature retroviral lattice using molecular dynamics and mutagen-esis. Structure, 23:1414–1425, 2015.

[5] Narayanasamy Nandhagopal, Alan A. Simpson, Marc C. Johnson,Adam B. Francisco, Gisela W. Schatz, Michael G. Rossmann, andVolker M. Vogt. Dimeric rous sarcoma virus capsid protein structurerelevant to immature Gag assembly.J. Mol. Biol., 335:275–282, 2004.

[6] R Campos-Olivas, J L Newman, and M F Summers. Solution structureand dynamics of the Rous sarcoma virus capsid protein and comparisonwith capsid proteins of other retroviruses.J. Mol. Biol., 296(2):633–49,2000.

[7] Tanmay A M Bharat, Norman E Davey, Pavel Ulbrich, James D Riches,Alex de Marco, Michaela Rumlova, Carsten Sachse, Tomas Ruml, andJohn A G Briggs. Structure of the immature retroviral capsid at 8Aresolution by cryo-electron microscopy.Nature, 487(7407):385–9, July2012.

[8] Di L Bush, Eric B Monroe, Gregory J Bedwell, Peter E Prevelige,Judith M Phillips, and Volker M Vogt. Higher order structure of theRous sarcoma virus SP assembly domain.J. Virol., 88(March):5617–5629, 2014.

[9] James C. Phillips, Rosemary Braun, Wei Wang, James Gumbart, EmadTajkhorshid, Elizabeth Villa, Christophe Chipot, Robert D. Skeel,Laxmikant Kale, and Klaus Schulten. Scalable molecular dynamicswith NAMD. J. Comp. Chem., 26:1781–1802, 2005.

[10] J J Thomson. XXIV. on the structure of the atom: an investigation of thestability and periods of oscillation of a number of corpuscles arrangedat equal intervals around the circumference of a circle; with applicationof the results to the theory of atomic structure.Philosophical MagazineSeries 6, 7(39):237–265, 1904.

[11] Alex de Marco, Norman E Davey, Pavel Ulbrich, Judith M Phillips,Vanda Lux, James D Riches, Tibor Fuzik, Tomas Ruml, Hans-GeorgKrausslich, Volker M Vogt, and John A G Briggs. Conserved and vari-able features of Gag structure and arrangement in immature retrovirusparticles.J. Virol., 84(22):11729–36, 2010.

[12] Leonardo G. Trabuco, Elizabeth Villa, Kakoli Mitra, Joachim Frank,and Klaus Schulten. Flexible fitting of atomic structures into electronmicroscopy maps using molecular dynamics.Structure, 16:673–683,2008.

[13] Robert B. Best, Xiao Zhu, Jihyun Shim, Pedro E. M. Lopes, JeetainMittal, Michael Feig, and Alexander D. MacKerell. Optimization ofthe additive charmm all-atom protein force field targeting improvedsampling of the backboneφ, ψ and side-chainχ1 and χ2 dihedralangles.J. Chem. Theor. Comp., 8(9):3257–3273, 2012.

[14] David E Manolopoulos and Patrick W Fowler. Molecular graphs, pointgroups, and fullerenes.The Journal of chemical physics, 96(10):7603–7614, 1992.

[15] Peter Schwerdtfeger, Lukas Wirz, and James Avery. Program fullerene:a software package for constructing and analyzing structures of regularfullerenes. Journal of computational chemistry, 34(17):1508–1526,2013.

[16] William T Tutte. How to draw a graph.Proc. London Math. Soc,13(3):743–768, 1963.

[17] John E. Stone, James C. Phillips, Peter L. Freddolino, David J. Hardy,Leonardo G. Trabuco, and Klaus Schulten. Accelerating molecularmodeling applications with graphics processors.J. Comp. Chem.,28:2618–2640, 2007.

[18] James C. Phillips, John E. Stone, and Klaus Schulten. Adapting amessage-driven parallel application to GPU-accelerated clusters. InSC’08: Proceedings of the 2008 ACM/IEEE Conference on Supercomput-ing, Piscataway, NJ, USA, 2008. IEEE Press.

[19] John E. Stone, Barry Isralewitz, and Klaus Schulten. Early experiencesscaling VMD molecular visualization and analysis jobs on Blue Waters.In Extreme Scaling Workshop (XSW), 2013, pages 43–50, Aug. 2013.

[20] John E. Stone, Kirby L. Vandivort, and Klaus Schulten. GPU-accelerated molecular visualization on petascale supercomputing plat-forms. InProceedings of the 8th International Workshop on UltrascaleVisualization, UltraVis ’13, pages 6:1–6:8, New York, NY, USA, 2013.ACM.

[21] John E. Stone, Ryan McGreevy, Barry Isralewitz, and Klaus Schulten.GPU-accelerated analysis and visualization of large structures solvedby molecular dynamics flexible fitting.Faraday Discuss., 169:265–283,2014.

[22] William Humphrey, Andrew Dalke, and Klaus Schulten. VMD – VisualMolecular Dynamics.J. Mol. Graphics, 14:33–38, 1996.

[23] Michael Krone, John E. Stone, Thomas Ertl, and Klaus Schulten. Fastvisualization of Gaussian density surfaces for molecular dynamics andparticle system trajectories. InEuroVis - Short Papers 2012, pages67–71, 2012.

[24] James C. Phillips, John E. Stone, Kirby L. Vandivort, Timothy G.Armstrong, Justin M. Wozniak, Michael Wilde, and Klaus Schulten.Petascale Tcl with NAMD, VMD, and Swift/T. InSC’14 workshop onHigh Performance Technical Computing in Dynamic Languages, SC’14, pages 6–17. IEEE Press, 2014.

[25] John Stone and Mark Underwood. Rendering of numerical flowsimulations using MPI. InSecond MPI Developer’s Conference, pages138–141. IEEE Computer Society Technical Committee on DistributedProcessing, IEEE Computer Society Press, 1996.

[26] John E. Stone. An Efficient Library for Parallel Ray Tracing andAnimation. Master’s thesis, Computer Science Department, Universityof Missouri-Rolla, April 1998.

[27] Steven G. Parker, James Bigler, Andreas Dietrich, Heiko Friedrich,Jared Hoberock, David Luebke, David McAllister, Morgan McGuire,Keith Morley, Austin Robison, and Martin Stich. OptiX: a general pur-pose ray tracing engine. InACM SIGGRAPH 2010 papers, SIGGRAPH’10, pages 66:1–66:13, New York, NY, USA, 2010. ACM.

[28] John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron. Scalableparallel programming with CUDA.ACM Queue, 6(2):40–53, 2008.

[29] Melih Sener, John E. Stone, Angela Barragan, Abhishek Singharoy,Ivan Teo, Kirby L. Vandivort, Barry Isralewitz, Bo Liu, Boon ChongGoh, James C. Phillips, Lena F. Kourkoutis, C. Neil Hunter, and KlausSchulten. Visualization of energy conversion processes in a lightharvesting organelle at atomic detail. InProceedings of the InternationalConference on High Performance Computing, Networking, Storage andAnalysis, SC ’14. IEEE Press, 2014.