16
CI-IFUSTOPHER J. LANCZYCKI, CAWIN A. JOHNSON, BENES L. TRUS, JAMES F. CONWAY, ALASDAIR c. STEVEN, AND ROBERT L. MARTINO US National Instztutes of Health To calculate a full 30 structural model of a virus capsid, researchers analyze cryo- electron micrographs that contain many randomly oriented images of the virus. The authors use parallel computing techniques to improve the pe~ormance of the computational algorithms that determine each particle 's orientation and generate the 30 model. This enhanced computational peg5ormance allows analysis of many more particles and a more precise determination of their orientations, letting researchers study important details of virus capsids at higher resolutions. hile some viral infections, such as small- pox and polio, have been eradicated or almost eradicated, others continue to cause widespread illness. In addition, novel viral pathogens-for example, HIV and ebola- continue to emerge. In their quest to control such viruses, biomedical researchers benefit from an understanding of the viruses' detailed structure and biochemical proper- ties. They derive this structural information from two main sources-X-ray crystallography and electron mi- croscopy. (Nuclear magnetic resonance spectroscopypro- vides an additional approach to solving the structures of relatively small viral proteins or their individual domains.) X-ray crystallography has solved the structures of more than 20 icosahedralvirus capsids to h g h resolution-3 to 4 angstroms. 1,2 Cryoelectron microscopy3has yet to acheve this resolution for such large structures, although it has re- cently yielded resolutions better than 10 A.4 Nevertheless, thls technique has many compensating advantages. It is ap- plicable to the large class of noncrystallizingstructures and works for virus capsids of all sizes, even those available only in very small quantities. Using cryoelectron microscopy, researchers can visualize the interactions of viruses with antibodies, receptors, or membranes; they can also visual- ize conformational changes that might affect the capsid structure during the replication cycle. New computational methods and high-performance computer hardware can aid considerably in attaining res- olution improvements for cryoelectron microscopy. Some of the factors that limit resolution are imprecision in particle-orientation angles, insufficient numbers of particles included in the analysis, and the presence of aberrant capsids whose inclusion would degrade the cal- culated structure. In this article, we demonstrate that parallel computing provides a reduction in computation time that, for a given reconstruction, significantly miti- 76 1070-9924/98/$10.00 0 1998 IEEE IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

  • Upload
    rl

  • View
    219

  • Download
    4

Embed Size (px)

Citation preview

Page 1: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

CI-IFUSTOPHER J. LANCZYCKI, CAWIN A. JOHNSON, BENES L. TRUS, JAMES F. CONWAY, ALASDAIR c . STEVEN, AND ROBERT L. MARTINO

US National Instztutes of Health

To calculate a full 3 0 structural model of a virus capsid, researchers analyze cryo- electron micrographs that contain many randomly oriented images of the virus. The

authors use parallel computing techniques to improve the pe~ormance of the computational algorithms that determine each particle 's orientation and generate the 3 0 model. This enhanced computational peg5ormance allows analysis of many

more particles and a more precise determination of their orientations, letting researchers study important details of virus capsids at higher resolutions.

hile some viral infections, such as small- pox and polio, have been eradicated or almost eradicated, others continue to cause widespread illness. In addition,

novel viral pathogens-for example, HIV and ebola- continue to emerge. In their quest to control such viruses, biomedical researchers benefit from an understanding of the viruses' detailed structure and biochemical proper- ties. They derive this structural information from two main sources-X-ray crystallography and electron mi- croscopy. (Nuclear magnetic resonance spectroscopy pro- vides an additional approach to solving the structures of relatively small viral proteins or their individual domains.)

X-ray crystallography has solved the structures of more than 20 icosahedral virus capsids to hgh resolution-3 to 4 angstroms. 1,2 Cryoelectron microscopy3 has yet to acheve this resolution for such large structures, although it has re- cently yielded resolutions better than 10 A.4 Nevertheless,

thls technique has many compensating advantages. It is ap- plicable to the large class of noncrystallizing structures and works for virus capsids of all sizes, even those available only in very small quantities. Using cryoelectron microscopy, researchers can visualize the interactions of viruses with antibodies, receptors, or membranes; they can also visual- ize conformational changes that might affect the capsid structure during the replication cycle.

New computational methods and high-performance computer hardware can aid considerably in attaining res- olution improvements for cryoelectron microscopy. Some of the factors that limit resolution are imprecision in particle-orientation angles, insufficient numbers of particles included in the analysis, and the presence of aberrant capsids whose inclusion would degrade the cal- culated structure. In this article, we demonstrate that parallel computing provides a reduction in computation time that, for a given reconstruction, significantly miti-

76 1070-9924/98/$10.00 0 1998 IEEE IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Page 2: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

gates the first and second factors; we cannot, however, entirely eliminate the third.

Initially, we had hoped to be able to use cnjo- electron microscopy to process a few Ihundred particles. Now, reconstructions with several thousand images are quite feasible at a level where resolutions better than 10 A are attain- able.+’ Parallel computing is a powerful tool to help continue the push to yet higher resolutions.

How cryoelectron microscopy works The primary data in cryoelectron microscopy are 2D projections of randomly oriented 3D vi- ral s t r~c tu res .~ By combining many 21D views, we calculate a 3D model of the original stnic- ture. Often, virus capsids (the protein shells sur- rounding and protecting the viral DNA or RNA core) possess a high degree of symmetry in spite of the complex conformations that the individ- ual protein molecules assume.’4 This symme- try reduces the computational effort required to generate the 3D model.

For example, the image in the lower left of Figure 1 is a surface rendering of a 3D model of the capsid of the herpes simplex virus at a reso- lution of 25 The HSV capsid is made up of 162 capsomers (visible in Figure 1 as five- and six-membered rings), whose members are pro- tein subunits. The 3D model was calculated from a collection of 110 2D images, such as those in the background of Figure I ; the calcu- lation exploited the HSV capsid’s icosahe- dral symmetry, common among the spherical v i ru~es .~ Figure 1 also contains several other examples of icosahedral virus capsids. These ex- amples illustrate some basic structural differ- ences in virus capsid structure, which are fre- quently linked to the biochemical mechanisms whereby the virus infects host cells. (Although this article focuses on current methods for (de- termining icosahedrally symmetric structures, symmetry is not a prerequisite for electron mi- croscopy. Indeed, this method has the potential to reveal asymmetric viral components, includ- ing the nucleic-acid core comprising the vinusS genetic material.)

An important feature of cryoelectron Ini- croscopy is its ability to preserve the capsid’s na- tive structure by quickly freezing specimens in a thin layer of vitreous ice, thus maintaining an aqueous environment.’ (For more details, see the sidebar, “Virus Structure and Cryoelecti-on Microscopy.”) All the particles imaged are pre- sumed to be identical in structure and to differ

Figure 1.3D reconstructions of viral capsids studied at NIH.3 Clock- wise from lower l e k herpes simplex virus (HSV) type I mature cap- sid, HSV type I procapsid, T=3 hepatitis B virus, T=4 hepatitis B virus, HK97 procapsid, HK97 mature capsid, and bovine papillomavirus. The background is a cryoelectron micrograph con- taining images of various HSV capsids.

only in the random orientations in which they are observed. The goal is to determine accu- rately the individual particle orientations and combine the information within the set of 2D projections to reconstruct the 3D structure.

Highly resolved 3D capsid reconstructions are useful for answering important biological ques- tions and permit a detailed study of the capsid’s constituent proteins.* This knowledge helps re- searchers understand how the proteins assem- ble to form a capsid. Reconstructions of bio- chemically modified viruses (for example, those with one or more proteins removed) help to lo- calize capsid protein^.^ In addition, the binding of antibodies for specific virus proteins can help researchers deduce their placement and can re- veal mechanisms of viral neutralization.‘ Struc- tural information can also assist in the design of pharmaceuticals that prevent infection or inhibit virus replication.‘ We thus exploit the differ- ences between native and bound or modified capsids to infer important structure and func- tion relationships at the viral capsid.

How parallel computing can help The computational resources required to gen- erate 3D reconstructions of viruses at good res- olution (better than 20 A) are formidable; this feat requires hundreds or even thousands of 2D projections. When we began this work in 1990,

APRIL-JUNE 1998 77

Page 3: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

we performed our serial reconstructions in a V W S environment, which severely limited the number of projections we could process in a given reconstruction. Upgrading to Alpha workstations has improved speed considerably, but removing the sofhvare limitations on parti- cle numbers has returned us to the situation where large, high-resolution reconstructions are still taxing. Therefore, we developed parallel- computing techniques to execute the recon- struction algorithms on a 128-processor Intel iPSC/860 MIMD parallel ~ o m p u t e r ; ~ we re- cently ported &IS software to both the IBM SP2 and SGI Origin parallel computers.

We have network access to these systems via new, menu-driven, front-end software running on each user’s workstation. From this easy-to- use environment, scientists can achieve a con- siderable speedup for a given reconstruction, and they can select and incorporate an arbi- trarily large number of particles in the compu- tation. The practical limitations of the iPSC/ 860 include slow I/O rates for moving data onto the system, IOW processor availability in the shared environment, and a relatively small per-processor memory size. Although we can currently overcome this final problem by care- ful software design, our current-generation

tities that contain the n, although they are not

hich includes infection eering of the host’s biochemi-

roduction. Their medical

fection cycle is the delivery ost cell, which could be me-

several other mecha- host cell functions to

cell. The new viri-

of shapes and forms, major classes. The di-

reater than 1,500

78 IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Page 4: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

parallel machines (the IBM SP2 and SGI Ori- gin) effectively eliminate the issue. Despite the remaining limitations, parallel computing re- lieves many computational limitations in gen- erating 3D reconstructions.

In addition, parallel computing makes it more tcnable to use ncw algorithms, such as thc Polar Fourier Transform method,” and to generate rcconsmctions in cases of arbitrary (or even no) symmetry. Improvements in electron micro- scopy techniques will give scientists the poten- tial to probe virus structures at higher resolu- tion; parallel computing can help them realize this potential more rapidly.

The computational process Figure 2 diagrams the reconstruction process. The first task is t o acquire good electron micro- graphs, from which we select undamaged, repre- scntative particlcs. After digitizing the micro- graphs, we extract individual capsid images and preprocess them (that is, normalizing them uniformly, etc.). Thcn, we perform an oricntation determination on each particle using Fourier transform methods and exploiting symmetry properties?J-12 By subsequently making cross- comparisons between independent particlcs, wc further optimize these preliminary orientations. Fourier-based methods also undcrlic the algo-

tion of the protein shell in the same orientation as the origi- nal images. When subtracted, the difference image has clearly shown 2D images of DNA.’

Obtaining a 3D viral reconstruction involves three basic phases: virus growth and purification, cryoelectron micro- scopy, and image pro~essing.~-”

For cryoelectron microscopy, we freeze the sample rapidly in a layer of vitreous ice, thus preserving the virions in their native state. Ideally, each micrograph contains many distinct virus particles, each randomly oriented. Next, we digitize the micrographs with a microdensitometer (for example, the Perkin-Elmer 101 OMG) or a high-resolution CCD camera, generating intensity values on a 2D array. Ultimately, many experimental factors influence the resolution of the final im- age: the purity, homogeneity, and intactness of the prepara- tion; the quality of the cryoelectron micrograph image; and the fidelity of digitization.

We evaluate the individual particles manually, excluding those that are visibly damaged or otherwise atypical. In- cluding the damaged particles would lower the signal-to- noise ratio in the averaged reconstruction, whereas intact but noisy particle data improve the signal-to-noise ratio by increasing the overall information content in the data set.

Finally, we petform preprocessing operations such as background intensity subtraction, masking out of intruding neighboring particles, removal of stray dust particles, inten- sity normalization, and so on. We then input the data files to our parallelized reconstruction software suite.

References 1. D.L.D. Caspar and A. Klug, “Physical Principles in the Con-

struction of Regular Viruses,” Proc. Cold Spring Harbor Symp. Quantitative Biology, Vol. 27, Cold Spring Harbor Laboratory Press, Cold Spring Harbor, N.Y., 1962, pp.

2. C. Branden and J. Tooze, introduction to Protein Structure, 1-24.

Garland Publishing, New York, 1991. 3. W.W. Newcomb et al., ”Structure of the Herpes Simplex

Virus Capsid: Molecular Composition of the Pentons and the Triplexes,” I. Molecular Biology, Vol. 232, No. 2, Jul. 20,

4. F.P. Booy et al., ”Finding a Needle in a Haystack: Detection 1993, pp. 499-51 1.

of a Small Protein (the 12-kDa VP26) in a Large Complex (the 200-MDa Capsid of Herpes Simplex Virus),” Proc. Nat’l Academy of Science, USA, Vol. 91, No. 12, Jun. 7, 1994, pp. 5652-5656.

5. F.P. Booy et al., ”Liquid-Crystalline, Phage-Like Packing of Encapsidated DNA in Herpes Simplex Virus,” Cell, Vol. 64, No. 5, Mar. 1991, pp. 1007-1 01 5.

6. R.H. Cheng et. al., ”Fungal Virus Capsids, Cytoplasmic Compartments for the Replication of Double-Stranded RNA, Formed as Icosahedral Shells of Asymmetric Gag Dimers,” /. Molecular Biology, Vol. 244, No. 3, Dec. 2, 1994, pp. 255-258.

7. R.H. Cheng et al., “Functional Implications of Quasi-Equiv- alence in a T=3 Icosahedral Animal Virus Established by Cryo-Electron Microscopy and X-Ray Crystallography,” Structure, Vol. 2, No. 4, Apr. 15, 1994, pp. 271 -282.

8. B.L. Trus et al., “Distinct Monoclonal Antibodies Separately Label the Hexons or the Pentons of Herpes Simplex Virus Capsid,” Proc. Nat‘l Academy of Science, USA, Vol. 89, No. 23, Dec. 1,1992, pp. 1 1508-1 151 2.

Icosahedral Particles-The Uncommon Line,” /. Structural Biology, Vol. 11 6, No. 1, Jan. 1996, pp. 48-55.

10. S.D. Fuller, “The T=4 Envelope of Sinbis Virus is Organized by Interactions with a Complementary T=3 Capsid,” Cell, Vol. 48, No. 6, Mar. 27, 1987, pp. 923-934.

Three-Dimensional Structure of Simian Virus 40 and Visual- ization of Chromatin Core,” Proc. Not? Academy of Science, USA, Vol. 85, No. 2, Jan. 1988, pp. 422-426.

9. S.D. Fuller et al., ”Three-Dimensional Reconstruction of

11. T.S. Baker, J. Drak, and M. Bina, ”Reconstructions of the

APRIL-JUNE 1998 79

Page 5: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

Electron

digitization microscopy and - Preprocessing -

L

Surface render Refinement Multiple Findview process: - particle - and display Refine reconstruction

reprojections I I

Figure 2. A block diagram of a 3D reconstruction of icosahedral virus images.

rithms for reconstructing the 2D images into a single 3D model; we use methods pioneered by R. Anthony Crowther and colleagues.” Repro- jections of the 3D model are fed back to iteratively refine the orientation parameters as required.’”

In broad terms, the parallelized portion of the computation encompasses two tasks:

* orientation determination and refinement

computation of the 3D electron density for each 2D particle image and

map from the 2D images.

Improved resolution of the final reconstruc- tion depends on advances in both these areas. Our parallel algorithms are built upon software developed by Stephen Fuller and Tim Baker and coworkers, 12-14 which we adapted for the Intel iPSC/860. In addition, we created front-end software to interface with the parallel computer either directly or over the network.

This particular application is a good exam- ple of a complex, multifaceted computational task. The computational process is not a black box whereby we plug in data and automati- cally receive an answer. Rather, we must eval- uate intermediate results to determine how the computation should proceed. Most no- tably, the orientation determination and re- finement involves considerable user interac- tion that cannot readily be automated.

Furthermore, the major computational tasks have diverse properties, so employlng a single, global, parallel programming paradigm is not appropriate. The reconstruction suite contains subprograms that are perfectly parallelizable, others that require data distribution across the available processors, and still another that is best implemented using a worker-manager scheme.

Table 1 lists all relevant stages in the recon- struction suite, each with its corresponding par- allelization strategy. The table also gives a mea- sure of the computational complexity in each case, emphasizing the dominant variables. Be-

cause of the problem’s heterogeneous nature, throughput depends to varylng degrees on sev- eral parameters. In particular, while the number of 2D images, Np, dictates the parallelization strategy at first, the size of the images in recip- rocal and direct space (R,,, and r,,,, respec- tively) eventually become relevant parameters.

Orientation determination Given a set of Np 2D images, the first task is to determine each virus particle’s As described in the sidebar, “Icosahedral Sym- metry,” we carry this out in Fourier space by ap- plying the Fourier Projection Theorem. Hence, each 2D virus image p becomes a planar slice P (in reciprocal space) through the parent 3D Fourier transform F. The orientation of P is given by a pair of Euler angles (0, 4). Once we assume icosahedral symmetry,

these angles can be restricted to the a s p - metric search unit shown in Figure A (see the “Icosahedral Symmetry” sidebar), and there are 59 other symmetry-related pro- jections p,. equivalent to P.

Consider the case where one of the symmetry- related planes P,. intersects P in a line I,. The symmetry operation taking P, into P can also map the line 1, onto a second distinct line l6 of P: the lines I, and lb are termed a pair of common lines for P.9,11-12 The values of P along the com- Inon-line pair I, and Zb must be equal, as they are identical lines by symmetry (apart from noise). By comparing the values of P for all indepen- dent pairs of common lines, we infer which ori- entations (e, $) are most consistent with the data

timal particle orientations will minimize the phase midual, namely the sum of the squared dif- ferences between the Fourier phases along all common-line pairs. In practice, image noise can make a unique determination difficult, so we

and the underlylngvirus ~y”etr ies .~J-’* O P-

80 IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Page 6: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

accumulate a best list of the no most promising alterna- tives, typically with 10 to 40 members.

Findview The program Findview' im-

plements the common-lines technique to determine a ranked set of best orientations of a 2D virus particle image by scan- ning the asymmetric search unit. A third Euler angle, the rotation angle R E [O", lSO"] of the plane P about its normal, locates the common-line pairs within the plane.

By incrementing the angles

Table 1. Computational complexity and parallelization strategy summary. The first three programs determine the particle orientations; the next four compute the 3D virus map. N,, is the number of particles in the reconstruction; r,,, and R,, are the radii (in pixels) of the particles in direct and reciprocal space, respectively. In practice, r,, and R,, are not significantly modified for a given set of Np images.

Computational Parallelization Program

Findview Local Refine Global Refine Merge Invert Bessel MAP

tioning the asymmetric unit, as shown in Figure A of the "Icosahedral Sym- metry" sidebar. A global collection operation then assembles the full I- and Q arrays and, for each parti- cle, Findview compiles the best list of no orientations with the lowest residual. To further optimize the phase residuals, the pro- gram performs an origin correction for each candi- date solution by imposing a set of small shifts in the image origin. We have also successfully parallelized the

complexity strategy

Distribute computation in asymmetric unit Worker-manager Distribute particle plan Distribute particle data Distribute computation by rings (R, Z) Distribute computation by planes Z 1) Distribute computation by planes 2) Distribute map data

8 and @, with typical increments ranging from 2" down to 0.25", Findview conducts a direct search within the asymmetric unit. Using the indepen- dent common-line pairs, the program calculates phase residuals I- for all rotation angles Q = o", lo, ..., 180". For each particle, the n minimum phase residuals r(6, @) and angles Q(6, 4) spec- ify the best list of the most likely particle orien- tations. Because the number of common-line pairs can change between views, a 2 weighting scheme12 compares the residuals for the differ- ent orientations.

Findview expends most of its effort searching the asymmetric unit. The dependence of I-@, @) on a single orientation allows the asymmetric unit to be divided among the available proces- sors. Therefore, the individual images are se- quentially processed, but we parallelize the residual calculation for each particle by parti-

origin correction by distribution of the residual calculation.' AI1 remaining serial elements of Findview contribute minimally to the computa- tion except on a large number of processors (that is, if N 2 32; see Table 2).

Refine Findview generates a best list of the most likely

orientations by comparing each particle with its own symmetry-related projections. But all parti- cles in the study set represent the same 3D capsid structure, implying that the Orientations of the Np particles constrain each other and should be in- ternally self-consistent. Therefore, we now refine the best orientations from Findview by compar- ing several different particles simultaneously.

Those lines in the Fourier transform where the planes Pi and , derived from two dzffeerent parti- cles, intersect are called c~oss common lines. These

Table 2. Single particle benchmark of Findview parallelization; timing of global Refine with a Np= 55 particle study set; timing of local Refine. N,n, = 60 individual runs on N processors. From the scaling behavior in Table 1, we can derive timing estimates for arbitrary Np'

Findview Global Refine Local Refine Search Total Search Total Time Time

N time (sec) time (sec) speedup speedup (min.) Speedup (min.) Speedup

1 423.67 490.25 1.00 1.00 165.02* 1.00 31.60 1.02 2 211.94 247.39 2.00 1.98 85.06 1.94 34.56 0.92 4 106.17 126.91 3.99 3.86 43.35 3.80 11.65 2.71 8 53.32 66.93 7.95 7.32 22.32 7.08 5.24 6.03

2.92 10.8 16 27.06 37.19 15.66 13.18 13.17 12.53 32 13.75 22.63 30.82 21.66 7.26 22.8 1.96 16.2

64 7.23 15.23 58.6 32.1 8 4.21 37.2 1.80 17.6 - - - - 128 3.71 11.80 114 41.52

* Projection

APRIL-JUNE 1998 81

Page 7: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

intersections imply geometrical relations between the two image planes that the angles e,, 9, &, 4 should obey. The program Refine (called EmicoGrad in previous work? performs this analysis, comparing the cross common lines gen- erated by the intersection of two arbitrary planes Pi and pi. Because the data in both planes should be identical along the line of intersection, optimally consistent views of the ith andfi particles mini- mize the cross-phase residual Rii. For each particle combination, Rq contains a term hom all pairs of cross common lines between particles i andj.

To start, the user must be able to identify a ba- sis set of particles of known orientation-for ex- ample, by appraisal of the best list. Using this basis set as the reference, one collectively refines all Np orientations. Two variants of this opera- tion are computationally usefu1:'J2

0 a local refinement, which sequentially re- fines each of the N, nonbasis particles

Many spherical virus capsids exhibit icosahedral symmetry. An icosahedron has a twofold axis through the center of all 30 edges, a threefold rotational axis through the center of the 20 triangular faces, and fivefold axes at the 12 vertices. Figure A shows an icosahedron in an orientation in which all three coor- dinate axes coincide with twofold rotational symme- try axes. When the icosahedron is rotated so that a fivefold axis is along the z-axis, what is known as the fivefold orientation results. (This is the orientation assumed for most of the computation.)

More generally, the orientation of the capsid seen experimentally is given by the angles 8 and 4. The icosahedral symmetry implies that there are 60 equiv- alent views of the object. Alternatively, I /60th of the full capsid is the basic repeat unit that defines the en- tire structure. As such, the range of unique orienta- tions is restricted to the asymmetric search unit defined by 8 €[69.09", 90°] and 8 E [-31.72", 31.72"] (see Figure A).

are 2 0 projections p(x, y) of an icosahedrally symmetric 3 0 structure of orientation (8, 4) pro- jected onto a plane. The Fourier transform of the projection, P(X, v), can be simply related to the 3D density yx, E I) of the virus particle by the Fourier Projection Theorem. Specifically, if F is the full 3 0 Fourier transform of f, then P(X, v) is exactly the planar slice through Fcontaining the origin and

The images obtained by cryoelectron microscopy

(4

69.09" -

8

90" ~

against the fixed basis set; and 0 a global refinement, which refines all parti-

cles (including those in the basis set) together.

The programs search for lower values of the cross residual Ri. by using the Nelder-Mead sim- plex neth hod.^?'^-'^ The complete refinement process comprises many separate runs of Refine with different subsets of particles. We have de- veloped a sophisticated fi-ont-end program to ex- pedite and simplify the process of assembling ba- sis and refinement s e ~ . ~

In the local refinement, the program refines N, individual images against the basis set by trying the no orientations from the Findview best l i s t in each case. However, the computation in the N& cases proves to be very uneven, and an acceptable distri- bution of the computational work among proces- sors cannot be predetermined. Therefore, for the local refinement, we implemented a worker-man- ager parallelization, in which processor 0 assigns

(b) -31.72" 31.72"

Fzpre A. (a) Icosahedron in the twofold view showzng the Euler angles 8 and 4 dejining its orientation. All unique pairs of angles lie in the unshaded asymmetrrc unit, expanded in (b). (b) The parallelization of the asymmet- ric unit, where numbers represent the processor asaped to that onentation in the four-processor case.

82 IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Page 8: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

tasks to processors as they fall idle. The lowest value of the residual found during the local refine- ment typically is chosen as best, and processor 0 reports this information back to the user as views are tested. Interaction via the workstation allows the user to select or reject the resulting orientation &om local refinements for each image. The man- ager processor is very busy as it sends each worker vs. slave processor all the data required to analyze a view, of which there are typically thousands.

In global refinement, all images P, are uncon- strained-that is, the basis set is empty. Refine computes residuals Ry against each particle 4 for a set of simplex moves in 0, @, and Q. Because memory constraints prohibit the replication of all images on each processor, we switch to a domain decomposition scheme in which the particles are distributed in memory among the processors (see Figure 3).

A given processor computes only those residu- als R, corresponding to its assigned particles 6,

but it must analyze all simplex moves for every particle Pi. This requires that image Pi be broad- cast for use by the other nodes when computing RY, temporarily replicating the image Pi on all processors as shown in Figure 3. Once Pi has been processed, Refine subsequently broadcasts the (i+ 1)th image to all processing elements from its home processor. On the iPSC/860, the maxi- mum number of images in a single Refine run ranges from 500 to 1,000 depending on the memory overhead required and the number of processors employed. This increases significantly on NIH’s IBM SP2 and SGI Origin.

Performance results As shown in Table 1, Findview’s runtime scales

as Np because the particles are independently an- alyzed. By distributing the asymmetric unit among the various processors, Findview displays an almost linear speedup in the search, with just under 90% efficiency on N = 128 processors (see

Projection of 20 virus particle for orientation 0

Values of F sampled on the line with orientation 0

where the orientation (0, 4) is pre- served in transform space.’ In Figure B we illustrate, in two dimensions for simplicity, how the projection p relates to Fthrough the slices P. The heavy line in the upper right represents the line along which the micrograph p provides a sampling of F. As each p represents 60 symmetry-related planar slices P through F, the Np projections sample Fon (maximally) 60Np planes (see Figure 5).

Figure B A simplified 2D schematic description of the Fourier Pro- jection Theorem (FPT). The electron density ficnction f (whose 2 0 Founer tran$orm is F) is imaged by an electron beam yielding a prqection p(.) defined by the beam orientation angle 0, where s IS a

P(S), which the FPT states are precisely the values of F along a line (shown in bold) with angle 0. A set o f electron mzcrographs with dts- tinct onentations samples F along many different lines.

coordinate normal t o the beam. The Fourier wan$orm ofp(s) zs

References 1 . R.A. Crowther, “Procedures for

Three-Dimensional Reconstruction of Spherical Viruses by Fourier Syn- thesis from Electron Micrographs,“

Philosophical Trans. Royal Soc. of London, Part 6, Vol. 261, No. 837, May 27,1971, pp. 221-230.

APRIL-JUNE I998 83

Page 9: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

Figure 3. Global refine- ment as carried out in Refine by distributing particle data for a four- processing-element (PE), six-image exam- ple. PE 0 broadcasts im- age P I . A receiving PE temporarily stores PI (light ball) to compute R l j for all images j in i t s domain (dark balls). a

PE 2

itEl> I P E 3

Table 2). The overall speedup falls off because the origin correction and other components of the code do not exhibit the perfect paralleliz- ability of the search. Both variants of Refine, however, scale strongly as Np2 due to the need to compute phase residuals between pairs of par- ticles. Despite considerable internode commu- nication, the global refinement phase of Refine gives a reasonable 58% efficiency on 64 nodes. In general, improved performance occurs when the number of processors N is a factor of Np.

The local refinement displays more complex behavior as Nincreases. For small N, I/O laten- cies are significant, the manager experiences re- duced computational capability, and we measure low efficiencies. We found a peak efficiency of

Reconstructed 3D virus

Figure 4. Steps of the 3D reconstruction once the particle orienta- tions (e, 4, Q) are determined. The virus particles and reconstruc- tion are of the herpes simplex virus.

75% for eight processors with a relatively small number of refinement runs. In practice, the number of runs is much larger, and the speedup is considerably improved at higher values of N. Given the poorer scaling of a naive paralleliza- tion strategy that distributes the N4, refinement cases, this represents a notable improvement in performance.

Constructing the 3D image Using the set of optimal refined particle orien- tations, the reconstruction algorithm merges all Np electron micrograph images into a single re- constructed 3D virus density map. Figure 4 de- composes the algorithm into its four main steps. As illustrated in Figure 5, the Np projections P provide an irregularly sampled representation of the Fourier transformed density map, F(R, Y , 2). Our current parallel implementation adapts the serial code developed by Fuller and Baker and coworker^,^^-^^ which follows from earlier work of Crowther and coworkers.” Crowther’s equations detail a straightforward prescription to obtain the real-space mapf; and such recon- structions present several challenges that paral- lel computing can significantly mitigate.

In the following section, we describe in detail our parallel reconstruction algorithm. While we discuss the algorithm in the context of icosahe- dral viruses, the strategies we developed and the underlying formalism are broadly applicable. The sidebar “Mathematical Formalism” presents the relevant definitions and mathematical details.

Merge: Combine all micrographs The subprogram Merge reduces the digitized

representations of all Np images into a single ma- trix BHB and vector BHF on each ring (R, 2). The results of Merge thus represent a conden- sation of all virus particles, and no further refer- ence to Np occurs in the reconstruction process.

Each image is described by the function p(x, y), which represents a 2D projection of the 3D densityf(r, y, z), together with its orientation angles, (6, $, Q), and origin coordinates, (xz,yz). A 2D FFT ofp(x, y) generates the function P(X, Y). The Fourier Projection Theorem says that P(X, Y) is a planar slice through the full 3D rec- iprocal space function F(R, Y , Z ) (see the related sidebar and Figure B). Two such slices appear in Figure 5. Given the set of 60 orientations (0, 4, Q) for the ith particle and its 59 symmetry- related mates, it is straightforward to determine those angles Yi at which these 60 planes inter-

84 IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Page 10: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

sect an arbitrary ring (R, 2). The indexj = 1,2 , ..., jvzux runs over allj" intersections of the planes from all N, particles with the ring in ques- tion. Interpolation of P(X, Yj generates the values of the 3D transform F at the points

The properties of the matrix B and the vector F allow for simplifications in solving for Fourier- Bessel amplitudes G. Because of icosahedral sym- metry, an intersection at angle Yl implies another at -Yl, and one can show that Im{F(R,-Y, 2)) = -Im{F(R, Y, Z)}. Therefore the imaginary parts of the products BHF and B'IB, wluch contain im- plicit sums over the angles TI, must vanish.

In Equation 6 (see the "Mathematical For- malism" sidebar), the quantities G, BHF, and BHB have an R-dependent dimension (2N, + l), where (2N, + 1) is the number of nonzero FE amplitudes required on the ring (R, Z'). How- ever, the symmetry properties of F and B allow the computation on each ring to be recast into two smaller problems with dimensionalities N, + 1 and N, by considering the linear combinations G', = G, + G, and G-, = G, - G-,.ll Significant savings in computational and memory resources result from such a decomposition because the matrices are considerably smaller.

Our strategy for parallelizing Merge, sketched in Figure 6, is based on the fact that Merge is an O(Np) code. We employ a data distribution scheme in which a subset of particles is assigned to each processor, which sequentially processes the members of its subset. The partial sums (BHB)I and (BHF)I maintained on processor 1 for its own subset of particles is incremented as each particle is analyzed. FinalIy, the partial sums are globally added across the processors to yield B"B and BHF for the entire data set. Because these products are manifestly linear with respect to the summed angular indexj,

(R, y,, 9.

Merge is almost perfectly parallelizable. Table 3 illustrates that deviations from 100% efficiency arise only when the number of processors N is not a factor of Np (that is, for N 2 32 proces- sors). An unequal division of particles is thus the primary cause of load imbalances, as inter- processor communication occurs only in the fi- nal global addition.

Memory utilization in Merge is independent of Np and depends on the resolution limit-that is, Rmx, the maximum radius in reciprocal space. In terms of both computation and memory us- age, Merge is 0 (R4,3, but only at very high resolutions where N,(R) is approximately 100 does this become a serious issue. Therefore, an

Table 3. Reconstruction timing results for N processors. The numbers in parentheses are efficiencies based on the one-node timings. Times are for Np = 48 electron micrograph images. Because Merge scales linearly with Np and the other codes scale independent of Np, times for arbitrary N,, are simply estimated.

N Total time (sec) Merge Invert Bessel Map

1 4,116* 3,094 82.9 92.8 1,126" 2 2,226 (92%)* 1,553 (100%) 44.1 (94%) 58.1 (80%) 545.2 (103%)* 4 1,248 (83%) 774.5 (100%) 23.9 (87%) 30.2 (77%) 280.2 (100%) 8 726.2 (71%) 388.6 (100%) 13.9 (75%) 20.6 (56%) 147.9 (95%)

16 438.7 (59Vo) 196.0 (990/,) 8.5 (66%) 14.8 (39%) 98.7 (71%) 32 330.3 (39%) 132.5 (73%) 7.0 (38%) 13.1 (23%) 59.7 (59%) 64 251.8 (26%) 68.4 (71%) 4.9 (26%) 10.8 (13%) 43.8 (40%)

Figure 5. A pair of elec- tron micro- graph images produce planes of data PI and Pz in reciprocal space. The in- tersection points (circles) sample the function F(R, Y, Z) on the cylindrical grid (R,Z), where the angular coordinate Y is considered continuous.

" Extrapolation based on a quadratic fit to the data from N = 4 to 16.

APRIL-JUNE 1998 85

Page 11: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

4

Figure 6. Parallelization strategy for Merge. A subset of particles i s assigned to each processor I = 0,1, 2,3, which computes (BHB), and (BHF), for that subset. These partial answers are globally combined to generate BHB and BHF for the complete set of Np particles.

Set of 2D virJs imaqe data

added benefit of our parallelization is that only one particle image (typically 256 X 256 pixels, or approximately 130 Kbytes) is in memory at a time, and redundant disk I70 and interprocessor communication are removed.

Invert: Compute the FB amplitudes G,(R, Z) In Figure 7, we plot N,, the number of posi-

tive-n Bessel terms kept in the expansion off(or equivalently, the dimensionality of the vector G,). Strictly, N, = N,(R), and the 2-dependence seen in Figure 7 is an artifact of resolution lim- itations. On those rings with nonzeroj,,,, we seek the (2N, + 1) FB amplitudes G,'. For small annuli close to the Z-axis, N, = 0, and solving for G,' is trivial. But when R = R,,, we obtain (B"B)-l by performing matrix inversions of di- mensionality (N, + 1) and N,, where maximally N, = 23 in our example. Once the inverse of (BHB) is computed from its eigenspectrum, sim- ple matrix operations give Gi(R, 2). For such small matrices, this method is reasonably effi- cient and scales as N:-that is, -(Riu,).

Because Invert inverts a pair of matrices on each ring, a basic parallelization strategy is obvi- ous: Assign each processor a set of rings. Each of the N processors is assigned -R2&N rings (see Figure 8). Invert rejects rings beyond the resolu- tion limit as it encounters them. To minimize communication overhead, Invert replicates the FB amplitudes after all computation is complete.

e mathematics of the 3D reconstruction of virus density

form by F= F(R, Y, Z). We use cylindrical coordinates to ex- ploit the fivefold symmetry to simplify the computation and efficiently use the raw data,' but these formulas also apply to the reconstruction of asymmetric bodies as well.

The images generate many planes of data in reciprocal space via the Fourier Projection Theorem and icosahedral

symmetry (See the "Icosa A and B). This leads to irregu ure 5 demonstrates that the

ploy a Fourier-Bessel expansion, yr, y, z). The problem then becomes a set of much smaller 1 D problems on the rings (R, Z), where R = 1, 2, ..., R,, and Z= 1, 2, ..., Z,, as follows.

We write the density map

g,,(r,Z) = jam Gn(R,Z)e~'"~'2~,,(2zR

and b ( x ) is the nth-order Besse

plitudes. In the presen latory angular depend

ing the Z-axis as the fi less n = 0, +5, +_I 0, +_I

Here, nmox depends on R an

86 IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Page 12: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

In test reconstructions with = 70 there are 4,970 rings, of which 3,849 are within the reso- lution limit. And while it is small, the fraction of rings with large N, is nonnegligible. Thus, a complication can arise in ensuring an acceptable load balance, because the matrix dimensionality depends on the coordinate R (see Figure 7).

Imbalances arise when a processor handles an unusually high number of rejected rings (and ends much earlier than the other processors) or large N, rings (and works much longer than the other processors). Despite this, we observe ex- cellently balanced loads (with a spread of only 0.3 to 0.5 seconds) and computational efficiencies of 97%. (See Table 3 ; we expect load imbalances to impact performance most in the N+ limit.) Arranging for roughly equivalent distributions of R values on all processors did not result in im- proved performance. Invert loses efficiency due to initialization overhead and global communi- cation in summing Gi, and such effects are am- plified by Invert's short execution times.

Bessel: Compute the Fourier amplitudes gn(r, Z) Bessel integrates over R according to Equa-

tion 2 (see the "Mathematical Formalism" side- bar), and is the simplest step in the reconstruc-

2'

R

Figure 7. Region of support for Invert. N, is proportional to the dimensionality of BHB, so that the matrix size varies over the re- gion of support.

tion. Of course, the infinite upper limit of the integration in Equation 2 now becomes be- cause of resolution limitations. The real-space maximum, r-, defaults to the radius (in pixels)

index n, and Nn4nmoX/5l. At each point (R, Y, Z), the FB am- plitudes G, are related to the unevenly sampled transform F:

F(R, yf, Z) = G, (R, Z)einV n

Given the orientation (e, @, Q) of each plane, finding all angles Y at which these planes intersect an arbitrary ring (R, Z) is a simple geometrical problem, illustrated graphically in Figure 5. There exist jmox (R, Z) nonuniformly distributed an- gles y/ on each ring, where the index j runs over all jmox in- tersections generated by all particles on a given ring (R, Z).

By defining the matrix elements Bjn = exp(inYj) we more compactly rewrite Equation 3 as

or, in matrix notation

F = BG. (5 )

Here, F = F(R, Z) is a complex jmOx element vector containing data at each y o n the (R, Z) ring, while C is the corresponding 2Nn + 1 element vector of FB amplitudes. In practice, having many particles and a broad range of parti-

cle orientations ensures many intersections on a ring. Typi- cally then, jmox > 2N, + 1, and the data overdetermines the least-squares solution for C in Equation 5, given by

where BH is the Hermitian conjugate of B. We must solve Equation 6 separately for C on each of the rings (R, Z). The normal matrix BHB is real symmetric, and both BHB and BHF implicitly contain a summation over all intersections Yjfor a given ring. Given G, Equations 1 and 2 are used to solve for the map f.

References 1. R.A. Crowther, "Procedures for Three-Dimensional

Reconstruction of Spherical Viruses by Fourier Synthesis from Electron Micrographs," Philosophical Trans. Royal Soc. of London, Part B, Vol. 261, No. 837, May 27,1971, pp. 221-230; references therein.

2. S.D. Fuller et al., "Three-Dimensional Reconstruction of Icosahedral Particles-The Uncommon Line," 1. Structural BiO/OgY, Vol. 11 6, No. 1, Jan. 1996, pp. 48-55.

APRIL-JUNE 1998 87

Page 13: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

Figure 8. Paral- lelization of In- vert: rings are assigned to processors ac- cording to the color coding in this four-node, five-plane ex- ample. The re- sults provide the FB ampli- tudes G,(R, Z) following a global summa- tion.

B ~ B &

B ~ F

0 1 ‘ 2 3

of the electron micrographs. In our example, Y,, = 88. The complex coefficients gn(r, Z) can easily be replicated on the individual processors because we have only (2Nn + 1) nontrival imag- inary coefficients g, for each (Y, 2) pair.

We must calculate the R-integral for each (n, z) pair for every value of Y. Bessel’s paral- lelization strategy simply assigns a set of planes of constant 2 to each processor. The computa- tional load balance achieved proves surprisingly good, with the processor times varylng by about 1.6 seconds, independent of cube size. Because Bessel only consumes about 3 % of the entire re- construction time, we have not pursued further sophistication in the parallelization. Table 3 contains parallelization statistics for Bessel.

One of Bessel’s novel feature is a compression of the g, coefficients. It turns out that numerous (n, 2) pairs have g&, Z ) = 0 for all Y. In our ex- ample, nonzero data exist for only 2,476 of the possible 4,048 (72, Z ) pairs. Compressing the data to reduce the memory requirements proves useful in the final program, Map, which can approach the per-processor memory limits. Hence, memory savings in storing the g,, in Bessel help avoid memory overflows in Map.

Map: Assemble the map To generate the final 3D density mapffr, y, z),

we integrate over 2 and sum over the discrete index n in Equation I (see the “Mathematical Formalism” sidebar). Note that Equation 1 is not a Fourier transform, because the number of

ns changes while the number of angles I,V re- mains fixed. Furthermore, the particle radii in real and reciprocal spaces are unequal in prac- tice: In our example, Y,,, = z,,, = 88 (deter- mined by image size), while &,, = Z,,, = 75 (by specification of the input resolution limit).

O n a subset of planes of constant z , each processor calculatesflr, y,z) according to Equa- tion l. For these planes, Map expands the five- fold symmetry about the z-axis and makes a bi- linear interpolation off from a cylindrical to Cartesian representation, desirable for image rendering and display.

We distribute the computation by z planes, and not more finely according to (Y, z ) rings, be- cause the resulting map segments are conve- niently aligned in memory for an efficient sub- sequent rotation of the particle (see Figure 9). The load balance achieved suffers slightly due to the coarse-grained parallelization, with a vari- ability of S % in execution time for eight proces- sors, climbing to 15% for 64 processors. But a fine-grained redistribution by rings is costlier because of increased communication during the rotation. The average computational efficiency scales quite well, at 98% for eight processors and 79% for 64 processors (see Table 3 ) .

After solving for& Map rotates the distributed five-fold map into a twofold orientation, ex- panding it to enforce the symmetry. In this ro- tation, the original y-axis becomes the new z- axis (see Figure 9), and data redistribution assigns each processor planes of constant y rather than constant z. We can now rotate the planes in parallel independently. The computa- tional efficiency was excellent at 94% for 64 processors; it also resulted in a good load bal- ance, with less than l % spread in rotation times.

Finally, the full 3 D reconstructed virus map f lx , y, z ) is written to disk, or used to generate reprojections for further image refinement.

Timing and computational issues Table 3 collects the timing results for the al-

gorithms discussed in this section. Clearly, the major computational effort resides in Merge and Map. While the load balance remains quite good in both Bessel and Invert, their small ab- solute run times (a combined 4% to 6% of the total) make their efficiency highly sensitive to overhead operations such as initializations, dy- namic array allocation, VO, and so on.

Merge tends to be the most time-intensive procedure, dictated by the average run time per particle, but it is nearly perfectly parallelizable.

88 IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Page 14: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

Efficiency is maximized when the number of particles Np is evenly divisible by the number of processors N. In theory, a perfectly paralleliz- able application on identical particles should possess an efficiency of

For Np = 48, q = 0.75 for both N = 32 and N = 64. We find is about 0.71 to 0.73, close to the theoretical limit for these cases. Interprocessor communication enters only via an optimized global sum and does not affect speedup.

Map also constitutes a significant portion of the computing time, and (ignoring the U 0 com- ponent) its efficiency remains acceptably high out to N = 64. Of all the steps, the integration and summation in Equation 1 by far dominate the load. Both the integration and rotation par- allelize quite well; for N = 64,q = 79% for inte- gration and 94% for rotation.

A final comment is in order regarding the computational complexity of the reconstruction (see Table 1). The most obvious point is that the particle number Np only enters into Merge, which scales linearly with Np. The computation time also scales with the radii in reciprocal and real space, parameters controlled by the image resolution. For a given set of particles, though, the parameters R,, and r,, seldom undergo significant variation, and increasing Np is the usual means of improving the final map. So while Merge scales strongly with the radius as R4,,, in practice its Np scaling is the crucial de- pendence. Hence, Merge can accommodate large increases in Np. But in the large Np limit, the orientation determination steps are severely limiting, because Refine scales as Ni. To miti- gate this limitation, in the future we plan to re- place Refine with the Polar Fourier Transform method,” which scales as Np.

In Invert, each of the RZmm normal equations are solved in a roughly O(R2,,,) time, making it an order-R4,,, program overall. In Bessel, for each of the rma, radii, a sum on R is performed for all (n , Z ) pairs, leading to the scaling O(r,J?,,). Finally, we found that Map runs in O(r2,,,,R ’,,,) time. For a fixed resolution, the computation scales as a constant with respect to Np after Merge; with a fixed particle set, on the other hand, changing the resolution (by alter- ing either &, or Y,,,) can induce drastic rescal- ing of the execution time. With a parallel im- plementation such as the one we described here,

=t =t

Data swap ==--=-+

Figure 9. Parallelization of MAP. Data is distributed to the nodes ac- cording to the color scheme. After solving in parallel for fix, y, z) on i t s assigned z-planes for the fivefold map (left), a data swap redis- tributes the data in planes of constant y (right). Rotation about the y-axis then proceeds in parallel across the y-planes to give a map in the twofold orientation.

improvements in resolution are more accessible from a computational point of view than for se- rially executed code. But practically, experimen- tal concerns most strongly dictate improve- ments in the resolution limit.

ne of our objectives in this article was to illustrate with a concrete ap- plication the complexities involved in parallelizing a large, diverse suite

of scientific programs. This application is in- dicative of many computationally intensive sci- entific problems-multiple sequentially per- formed stages and an appreciable degree of user interaction and direction both before and dur- ing execution. For such problems, a range of parallel programming paradigms must be inte- grated. Furthermore, the parallelization strat- egy must be mindful of the user interaction with the program and work with, rather than circumscribe, the user’s direction: we constructed our menu-driven, front-end programs with this goal in mind.

We have merged a wide variety of approaches in a natural way and obtained very good perfor- mance, while reducing the tedious “housekeep- ing” required of users in the pax. Consequently, we have succeeded in generating 3D maps of the virus capsid faster (for a given reconstruction job) and at higher resolution (for a given time of computation) than previously attainable.

APRIL-JUNE 1998 89

Page 15: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

These enhanced capabilities for analyzing virus particles demonstrate the value of parallel computing by providing important insights into the virus structure and function through im- provements in resolution and signal-to-noise ra- tios for our reconstructions. For example, our parallelized computations allowed the visualiza- tion of a small (12-kilodalton) minor capsid pro- tein on the tips of the hexamers of the HSV cap- sid (see Booy et al.'). Other studies that have utilized parallel technology include work on the L-A virus'* and bovine papill~mavirus.~

Continued improvement in the computing re- sources available to structural biologists can be expected to extend cryoelectron microscopy resolution to below 9 A.* Compared to the iPSC/860, our current parallel computer, the IBM SP2, provides an order of magnitude im- provement in both speed and available memory toward such goals. Continuing algorithmic re- search focuses on achieving greater accuracy in the orientation determination-for example with the Polar Fourier Transform technique.'

Implementing such a wide-ranging scheme has proven to be labor-intensive and is highly nontrivial, and adapting to hardware and soft- ware advances and upgrades are real challenges. But with the advent of standardized parallel li- braries and message-passing interfaces, in par- ticular MPI, a much smoother transition to par- allel computing is now available. Research to make the parallelism of the programs and archi- tecture transparent to the end user will also be important in malung parallel computation a more widely used technique in biomedical and many other applications.

References 1. W. Chiu, "What Does Electron Cryomicroscopy

Provide That X-ray Crystallography and NMR Spectroscopy Cannot?" Ann. Rev. Biophysical and Biomoleculav Structure, Vol. 22, 1993, pp.

2. M. Stewart and G. Vigers, "Electron Microscopy of Frozen Hydrated Biological Material," Nature, Vol. 31 9, No. 6055, Feb. 20, 1986, pp. 631-636.

3. A.C. Steven et al., "The Making and Breaking of Symmetry in Virus Capsid Assembly: Glimpses of Capsid Biology from Cryoelectron Microscopy," FASEB j., Vol. 11, No. I O , Aug. 1997, pp. 733-742.

4. B.L. Trus et al., "Novel Structural Features of Bo- vine Papillomavirus Capsid Revealed by Three-Di-

2 3 3-255.

mensional Reconstructions to 9 A Resolution," Nature: Structural Biology, Vol. 4, No. 5, May 1997, pp, 41 3420.

5. R. Henderson, "The Potential and Limitations of Neutrons, Electrons and X-Rays for Atomic Reso- lution Microscopy of Unstained Biological Mole- cules,'' Quarterly Rev. Biophysics, Vol. 28, No. 2, May 1995, pp. 171-1 93.

6. B.L. Trus et al., "Distinct Monoclonal Antibodies Separately Label the Hexons or the Pentons of Herpes Simplex Virus Capsid," Proc. Nat'l Acad- emy of Science, USA, Vol. 89, No. 23, Dec. 1, 1992,

7. F.P. Booy e t al., "Finding a Needle in a Haystack: Detection of a Small Protein (the 12-kDa VP26) in a Large Complex (the 200-MDa Capsid of Herpes Simplex Virus)," Proc. Nat'l Academy of Science, USA, Vol. 91, No. 12, Jun. 7, 1994, pp. 5 652-5 65 6.

8. J.F. Conway et al., "Visualization of Three-Dimen- sional Density Maps Reconstructed from Cryo- electron Micrographs of Viral Capsids," j . Struc- tural Biology, Vol. 11 6, No. 1, Jan. 1996, pp. 200-208.

9. C.A. Johnson et al., "Orientation Determination in the 3D Reconstruction of Icosahedral Viruses Using a Parallel Computer," Proc. Supercomputing, IEEE Computer Society Press, Los Alamitos, Calif.,

10. T.S. Baker and R.H. Cheng, "A Model-Based Ap- proach for Determining Orientations of Biological Macromolecules Imaged by Cryoelectron Mi- croscopy," ]. Structural Biology, Vol. l 16, No. l ., Jan. 1996, pp. 120-1 30.

11. R.A. Crowther, "Procedures for Three-Dimensional Reconstruction of Spherical Viruses by Fourier Syn- thesis from Electron Micrographs," Philosophical Trans. Royal Soc. of London, Part B, Vol. 261, No. 837, May 27, 1971, pp. 221-230.

12. S.D. Fuller et al., "Three-Dimensional Reconstruc- tion of Icosahedral Particles-The Uncommon Line," 1. Structural Biology, Vol. 11 6, No. 1, Jan. 1996, pp. 48-55.

13. S.D. Fuller, "The T=4 Envelope of Sinbis Virus is Organized by Interactions with a Complementary T=3 Capsid," Cell, Vol. 48, No. 6, Mar. 27, 1987, pp. 923-934.

14. T.S. Baker, J. Drak, and M. Bina, "Reconstructions of the Three-Dimensional Structure of Simian Virus 40 and Visualization of Chromatin Core," Proc. Nat'l Academy of Science, USA, Vol. 85, No. 2, Jan. 1988, pp. 422-426.

15. R.H. Cheng et. al., "Fungal Virus Capsids, Cyto- plasmic Compartments for the Replication of Double-Stranded RNA, Formed as Icosahedral

pp. 11 508-1 151 2.

1994, pp. 550-559.

90 IEEE COMPUTATIONAL SCIENCE & ENGINEERING

Page 16: Parallel computing strategies for determining viral capsid structure by cryoelectron microscopy

Shells of Asymmetric Gag Dimers," /. Molecular Biology, Vol. 244, No. 3, Dec. 2, 1994, pp. 255-258.

Christopher J. Lanczycki is a computational scientist in the Computational Bioscience and Engineering Lab- oratory, Center for Information Technology, National Institutes of Health (CBEL/CIT/NIH). His research in- terests encompass the physics of self-organizing sys- tems and parallel computing for scientific applica- tions-particularly computational structural biology and protein structure determination. Following a BS in physics from the University of Connecticut a t Storrs, he obtained his PhD from the University of Maryland in computational/theoretical condensed matter physics. He is a member of the American Phys- ical Society and the American Crystallographic Asso- ciation. Contact him at the Computational Bioscience and Engineering Laboratory, the National Institutes of Health, Bldg. 12A, Rm. 2033, Bethesda, MD, 20892- 5624; la ncz@alw. ni h . gov.

Calvin A. Johnson is a computer scientist and chief of the High Performance Computing Section of CBEL/CIT/NIH. His research interests include nonlin- ear optimization, parallel computing, scientific com- puting, and image processing. He received a BS in electrical engineering from Syracuse University, an MS in electrical engineering from the Johns Hopkins Uni- versity, and a PhD in information technology (opera- tions research) from George Mason University. John- son is a member of the IEEE, SIAM, and Informs. Contact him a t the Computational Bioscience and En- gineering Laboratory, the National Institutes of Health, Bethesda, MD, 20892-5624; [email protected].

Benes L. Trus is the chief of the Image Processing Re- search Section, CBEL/CIT/NIH. Image processing, structural biology, virus structure, and 3D reconstruc- tions are among his research interests. He obtained a BS in chemistry from Tulane University and a PhD in physical chemistry from the California Institute of Technology. Trus is a member of the NY Academy of Sciences, the Microscopy Society of America, the Chesapeake Society for Microscopy, Phi Beta Kappa, and Sigma Xi. Contact him a t the Computational Bio- science and Engineering Laboratory, the National In- stitutes of Health, Bethesda, MD, 20892-5624; trus@ helix.nih.gov.

James F. Conway is a staff scientist in the Laboratory of Structural Biology Research, National Institute of Arthritis, Musculoskeletal and Skin Diseases, at the NIH (LSBR/NIAMS/NIH). His research focuses on the 3D structure determination of icosahedral viruses and other, nonicosahedral macromolecular assemblies, from cryoelectron microscopy data. This includes work in developing new software procedures for attaining higher resolution reconstructions and the use of par- allel processing in analyzing large data sets to high resolution. He earned both his BS and PhD from the Department of Physics and Biophysics at Massey Uni- versity, New Zealand. Contact him a t the Laboratory of Structural Biology Research, the National Institutes of Health, Bethesda, MD, 20892; conway@calvin. n ia ms. n i h .g ov.

Alasdair C. Steven is the chief of LSBR/NIAMS/NIH. His interests lie in the fields of structural biology, elec- tron microscopy, image processing, virology, macro- molecular assembly, protein and virus engineering, and computational biology. Steven is editor-in-chief of the journal of Structural Biology and author of Ani- mal Virus Structure (Elsevier, 1988). He took his MA in mathematics and natural philosophy a t Edinburgh University, Scotland. A PhD in theoretical elementary particle physics from Cambridge University and an SKMB certificate in molecular biology followed. He is a member of the Microscopy Society of America, Amer- ican Society for Microbiology, Biophysical Society, Pro- tein Society, and Chesapeake Society for Microscopy. Contact him a t the Laboratory of Structural Biology Research, the National Institutes of Health, Bethesda, MD, 20892; [email protected].

Robert L. Martino is an associate director of CIT as well as chief of CBEL. He is also the NIH representative to the national, multiagency Computing, Information, and Communications Research and Development Pro- gram, and an adjunct faculty member in the Whiting School of Engineering a t Johns Hopkins University. His research interests include the application of computers in medicine and biology, parallel computing, meth- ods for processing physiological signals and medical images, computer architecture, and digital-system the- ory. He received a BS from Northeastern University and his MS and PhD from the University of Maryland, all in electrical engineering. Martino is a member of the IEEE, Computer Society, and IEEE Engineering in Med- icine and Biology Society. Contact him at the Compu- tational Bioscience and Engineering Laboratory, the National Institutes of Health, Bethesda, MD, 20892- 5624; [email protected].

APRIL-JUNE 1998 91