Towards automatic electron tomography

Ultramicroscopy 40 (1992) 71-87 ! ~ North-Holland

Towards automatic electron tomography

K. Die rksen , D. T y p k e *, R. Heger l , A.J. K o s t e r 1 and W. B a u m e i s t e r Max-Planck-lnstitut fiir Biochemie, W-8033 Martinsried, Germany

Received 10 October 1991

Electron microscope control, t ha t allows the automatic recording of tilt series for the 3D reconstruction of individual objects, has been realized. The experimental set-up includes a 200 kV TEM equipped with a 1K× 1K CCD camera, both controlled externally by a fast dedicated image-processing computer. For the goniometer control an accurate electronic readout of the tilt angle and a board driving the goniometer motor have been installed. For low-dose imaging, three to five different specimen areas are used: one (or two) for the determination of object displacements during tilting, one (or two) for autofocusing, and another one for recording the tilt series to be used for the 3D reconstruction. Tilt series can be recorded with a rather low total dose, the lower limit being set by the requirement that subsequent projection images have to be aligned by means of cross-correlation functions. The method has been tested with graphitized carbon particles on carbon film and with negatively stained proteasomes from the archaebacter ium Thermoplasma acidophilum. Some future develop- ments towards fully automatic electron tomography are discussed.

I. Introduction

The first papers outlining the principles of three-dimensional (3D) reconstruction from a set of projections were published in 1968 [1-3]. Since then three-dimensional electron microscopy has become a thriving discipline and an important asset to the arsenal of methods which structural biology has at its disposal. A variety of methods to collect and to exploit 3D data sets have been developed over the past two decades. The terms "electron crystallography" - emphasizing the close methodological relationship to X-ray crystallography - and "electron tomography" have come into use. Both methods are, in fact, closely related and many intermediate forms exist (for more recent reviews see refs. [4-8]). While electron crystallography has meanwhile succeeded in reaching near-atomic resolution with some pro-

* To whom correspondence should be addressed. i Present address: Depar tment of Biochemistry and Bio-

physics, University of California, San Francisco, CA 94143, USA.

teins [9,10], electron tomography has hitherto re- mained a low-resolution technique. This is mainly due to the fact that the whole data set is obtained from an individual object thus exposing it to a rather high dose. Collecting a full data set for tomography means that images of the specimen have to be taken at a set of different tilt angles distributed over a large angular range. Recording such a "tilt series" normally requires several intermediate operations between two successive projections such as setting a new tilt angle, recen- tering the object and focusing; with beam-sensi- tive objects all these operations should be carried out under low-dose conditions. Therefore manual recording of tilt series is cumbersome, prone to errors and, in practice, expensive in terms of dose. Automatic recording can be expected to improve the situation dramatically.

There have been early attempts, beginning in the late seventies, to set up automatic recording of tilt series [11]. Part of this endeavour was, e.g., the development of a device using an "electronic correlation valve" to correlate image pairs in order to determine object displacements during

0304-3991/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved

72 K. Dierksen et a L / Automatic electron tomography

tilting [12]. On the whole, the project was over- ambitious and beyond technical feasibility at that time.

The recent development of microprocessor- controlled transmission electron microscopes, and the availability of large-area slow-scan CCD cam- eras and fast image-processing computers, have provided us with the essential tools to realize automatic electron tomography. Once the electron microscope is equipped with the necessary auxiliaries, the main task is to develop appropriate procedures. Techniques such as automatic focusing, which have been developed recently [13-15], can be incorporated into these procedures. In this communication we describe our experimental set-up and the procedures we have so far developed for automatic tomography. Moreover, we report on first applications, which not only demonstrate that the methods work, but also indicate that the total electron exposure of the specimen can indeed be kept within reason- able limits.

molecular images are aligned, classified and as- signed a position in a virtual tilting experiment (angular reconstitution [22], random conical tilting [23]). It is therefore an inherent feature of these methods, distinguishing them from tomog-

SPECIMEN PREPARATION

2. Scope for automatization in electron tomography

The methodology of electron tomography, i.e., the three-dimensional reconstruction of individual objects from projection images obtained by tilting the specimen, has been worked out some time ago [16,17]. In "brute force" approaches, accepting that the specimen will be exposed to very high and probably "lethal" doses, the method has been applied to various biological specimens, ranging in size from molecules [18,19] to cells [20,211.

More recently, clever reconstruction schemes, more or less closely akin to tomography, have been developed which allow one to reduce the dose to a level required for recording a single image with a signal-to-noise ratio sufficient for alignment and classification. Advantage is taken of the existence of many different orientations and hence different projection images of individual molecules in a large population of molecules on a single micrograph; basically the individual

'! ' " , " ' '"

A~IGNME~ OF ~ E ~ C ~ i O N S

3:0 ~ECON~CTi~N

FURTHER PROCESSING OF ~ D DATA

REFINEMENT OF THE ALIGNMENT OF PROJECTIONS

FILLING-IN OF MISSING DATA BY THE USE OF CONSTRAINTS (POCS, MAXIMUM ENTROPY ...)

CLASSIFICATION, ALIGNMENT AND MERGING OF 3-D DATA

Fig. 1. Flow-diagram for electron tomography. Steps on dark- grey background are part of our present automatic procedures, steps on light-grey background are targeted for future

automatization.

K. Dierksen et a L / Automatic electron tomography 73

raphy sensu strictu, that the 3D reconstruction is tied up with averaging. This limits the use of these methods to situations where multiple, strictly isomorphous, structures exist.

The flow-diagram shown in fig. 1 summarizes the basic steps of a tomographic structure determination, and it indicates which of them are executed automatically by our current installation, and which steps will be targeted for automatization in the near future. While we believe that "automatic electron tomography" should eventu- ally include quasi-on-line 3D reconstructions, the final processing steps, which involve sophisticated image analysis and restoration techniques as well as merging of several independent 3D reconstructions, will for quite some time be done off-line.

3. Some dose considerations

The main concern in electron tomography of biological material is, in fact, whether or not the dose necessary for the acquisition of a full 3D data set can be kept within tolerable limits. The key step in the 3D reconstruction is the correct alignment of the images of a tilt series. This is done by cross-correlation of successive images after stretching them in a direction perpendicular to the tilt axis [24,25]. The lower limit for the dose is given by the requirement that the correlation peaks for subsequent projections must be statistically significant. It is a distinct advantage of the tomographic approach that, unlike other methods which rely on the alignment and classification of single molecular images, it allows the use of the whole field of view, i.e., the whole image frame provided by the CCD camera, for cross-correlation. Therefore, the cross-correlation peak may become significant in spite of a low signal-to-noise ratio of the projections.

In the following we explain, based on simple considerations, that a full 3D series can be recorded with a rather low total dose. We assume that electron shot noise is the major source of noise in the images and that the specimen can be

considered as a statistical object. For the estimate we use the expression [26,27]

v~pZ P = /~-p2+ 1 (1)

which relates the signal-to-noise ratio P of the peak of the cross-correlation function between two independent images of the same object to the number M of independent picture elements in each image and the signal-to-noise ratio p of a single independent picture element (note that one independent picture element will normally contain several pixels). To account for the degradation of the correlation peak occurring if projections, taken at different tilt angles, are cross-correlated, even when they have been stretched be- forehand, we may assume a somewhat larger value of the signal-to-noise ratio P of the peak.

As a realistic example we assume that the projection images to be reconstructed consist of M = 104 independent picture elements and that the signal-to-noise ratio of the correlation peak (without the aforementioned degradation) is P =

1 10. From eq. (1) we obtain p = 3 as signal-to-noise ratio of the independent picture elements in one 2D projection; the corresponding dose is denoted as D 2. On the other hand a statistically significant 2D image requires a signal-to-noise ratio Ps = 5 of the image elements [28]. As the noise contrast is inversely proportional, and therefore p is proportional to the square root of the dose, the dose Dzs , required to obtain a statistically significant image, will be greater than D 2 by a factor (p s /p ) 2 = 225. Thus the alignment of a tilt series, comprising as many as 45 projections, would work

1 even with an accumulated dose as low as ~ of the dose required for obtaining a statistically significant projection. This means that the 3D reconstruction could be performed with rather noisy projection data; the result, however, would be a very noisy structure that has to be improved by averaging.

The above considerations indicate a lower the- oretical limit. With a nearly perfect image pick-up system, this limit can be approached experimen- tally. On the other hand only ratios between doses have been considered, not absolute dose

74 K. Dierksen et al. / Automatic electron tomography

values which depend of course on the real object contrast. Due to their low contrast, images of a frozen-hydrated specimen will require a signifi- cantly higher dose than images of negatively stained specimens.

In some cases the assumption that the object can be considered to be "statistical" is not adequate. If one, for instance, enhances the contrast of a frozen-hydrated sample by adding heavy atom clusters, this will cause strong deviations from a statistical contrast distribution, and the high-contrast additives will considerably lower the dose required for cross-correlation.

4. The experimental set-up

A scheme of the experimental set-up is shown in fig. 2. It includes a CM 20 transmission electron microscope (Philips) equipped with a 1K x 1K charge-coupled device (CCD) camera (Photo- metrics) and a TEMDIP S computer (TVIPS GmbH). The goniometer of the CM 20 with TWIN objective lens normally permits a tilt range of _+ 60 °. However, with an appropriate specimen holder the range can be extended; specimen holders allowing unlimited tilt have become available [29]. As the goniometer is not incorporated into the microprocessor control of the microscope, an electronic readout of the tilt angle and

a board driving the goniometer motor had to be installed.

The control of imaging procedures is carried out by the TEMDIPS computer which has been optimized for fast image processing (a scheme is shown in fig. 3). It is connected to the microscope, the tilt stage and the camera. Essential parts of the control procedures are image- processing steps, e.g., the cross-correlation of images. In order to execute these steps in a reason- able time, the processor has to be quite fast. So far these operations have been executed using the board " F F T 2000" designed for fast Fourier transformation (FFT). With this board, a 5122 FFT takes about 3 s, a full cross-correlation of two 5122 images approximately 10 s. The "VAP 80" array processor, which will be incorporated in the near future, will not only reduce the operation time by a factor of 10, but also improve the accuracy since operations can be executed with floating point numbers.

4.1. T E M c o n t r o l

The remote control of the CM 20 uses the C M O N I T O R program package. It allows the exe- cution of nearly all functions of the microscope that can be performed by the operator via knobs, buttons and soft keys, and in addition a number of direct operations. C M O N I T O R commands can

GONIOMETER IMAGE CONTROL PROCESSING

TE. ] CONTROL

DONTROL J - D A T A

Fig. 2. Experimental set-up including the CM 20 TEM, the externally controlled goniometer G, the computer, and the CCD camera.

K. Dierksen et aL ,/Automatic electron tomography 75

Fig. 3. Configuration of the TEMDIPS computer. Black frames indicate modules of the present installation. Modules with grey frames will be installed in the near future.

be combined to procedures which again may contain subprocedures up to a depth of 10. Com- mands may also be integrated into higher-language programs, using for example the languages PASCAL or C.

4.2. Goniometer control

The goniometer stage of the CM series microscopes is designed only for manual operation. For our purpose it is necessary to read the tilt angle and to control the goniometer motor by the exter- nal computer; therefore some changes had to be made (see fig. 4). For the angle read-out an incremental rotary decoder ( R O D 900, Heiden- hain) was coupled backlash-free to the goniometer (gear ratio 1 : 2). A bidirectional counter (VRZ

"1 ~ VRZ 143

,CD out

Ooml~ut@~" 1 i/o-ports q 16 bit parallelJ motor driver }

Fig. 4. Elements of the goniometer control: incremental rotary encoder ROD 900, bidirectional counter VRZ 143, computer,

and goniometer motor GM.

143, Heidenhain) interprets the electronic signal and displays the tilt angle. The resolution is 0.01 °, which is certainly finer than is normally required. The displayed tilt angle is also provided to a parallel data output in BCD code and transferred to general purpose I / O ports on an APAL board in the TEMDIPS computer. A relay allows one to switch between manual and computer-controlled operation of the goniometer. In the latter case a motor driver, located on a separate board in the computer and controlled by T T L signals from the APAL board, operates the motor. A C-program handles the communication between the computer and the goniometer and enables the user to set any desired tilt angle within the allowed range.

4.3. The CCD camera

The camera is of the slow-scan type, equipped with a 1060 x 1024 chip T H X 31156 from Thom- son CSF. The chip is cooled to - 2 5 ° C by a two-stage Peltier cooler. For use with the EM the chip is tightly coupled to a fibre-optic element (50 mm thickness, 28 mm diameter) at the entrance of the camera head. The camera head is mounted on a home-made rotary flange [30], coupled to the microscope. It allows one to orient one side of the CCD chip parallel to the direction of the tilt axis in the image. An additional fibre-optic plate carrying a multicrystalline P20 fluorescent screen (thickness ca. 20 p.m) is optically coupled to the fibre-optic input of the camera.


For camera control and readout, T E M D I P S contains the program package CAMC which per- forms the grabbing and manipulation of images. The electric charge ( "CCD electrons") accumulated on the chip during an exposure is digitized by the camera control unit at a pixel rate of 500 kHz; grabbing of a 10242 pixel image thus takes 2 s. It is possible to take smaller images or to bin several pixels, thus reducing the acquisition time. In the digitizer 11 (or 44) CCD-electrons are

converted into 1 analog-digital unit (ADU) at the higher (or lower) setting of the amplifier in the control unit.

The camera delivers a raw image on which bias and dark current contributions are superimposed. In addition, the raw image is affected by different sensitivities of single CCD picture elements, by inhomogeneities of the fluorescent screen, and, more seriously, by transparency variations of the fibre-optic input. All these influences can largely

Fig. 5. Example of (a) a raw image taken with the CCD camera and (b) the corrected image with corresponding diffractograms (a' and b').

K. Dierksen et al. / Automatic electron tomography 77

be eliminated by correcting the raw image according to the expression

( I~aw -/dark) I . . . . ( / f i a t - - / da rk ) ' (2)

where Ina , and Idark are images taken with homo- geneous or without illumination, respectively. Also the bias is eliminated in eq. (2) as it is contained in /dark and Ijla, as well as in /raw" Fig. 5 shows an example of an uncorrected and a corrected image together with the corresponding power spectra. The hexagonal lattice ("chicken wire") showing up in the first power spectrum is caused by the hexagonal packing of fibre bundles in the fibre-optic input of the camera. The dark current of our CCD chip amounts to a charge of ca. 98 " C C D " electrons per pixel per second at -25°C, which gives an output of 9 A D U / ( p i x e l - s) at the the higher amplifier setting.

Important features of the camera are its linearity, sensitivity, and resolution. The linearity is excellent over the whole dynamic range of 4096 ADU. Sensitivity and resolution strongly depend on the fluorescent screen. A light-optical knife- edge test of the camera alone with nearly parallel light resulted in a modulation transfer function (MTF) approaching the ideal MTF (for which only the finite pixel size is taken into account). The electron-optical resolution of the camera with the presently used fluorescent screen and fibre- optic element is considerably worse: At 80 kV accelerating voltage the half maximum radius of the MTF is smaller by a factor of about 2.5. At higher voltages it becomes even worse. For this reason we usually record 5122 images with 2 × 2 binning. This is quite adequate for our present purposes; however, in order to fully exploit the possibilities of the camera, in the long run either the resolution of the fluorescent screen has to be improved, or a convergent fibre-optic element has to be inserted. The mean pixel sensitivity, as measured with the microscope's screen current amplifier, is approximately 50 A D U for every 80 kV electron impinging on the fluorescent screen when measured at the higher setting of the camera amplifier. This measurement might imply sys- tematic deviations due to backscattering, but cer-

tainly not due to secondary electrons since the screen is held at +22.5 V potential with respect to ground.

5. Procedures for automatic tilt series recording

It depends on the object under scrutiny, in fact basically on the magnification being used, how sophisticated a procedure for automatic tilt series recording has to be. At low magnifications the influences of mechanical inaccuracies of the goniometer or of small deviations from the eucentric z-position of the specimen will be tolerable; therefore, procedures can be very simple. How- ever, this applies to a rather limited magnification range when images are recorded with the CCD camera. In our camera the image area is 19.5 mm × 19.5 mm; with the fluorescent screen of the camera being situated ca. 300 mm below the microscope; this corresponds to 10.1 mm × 10.1 mm in the EM film plane. On the other hand, the mechanical accuracy of the tilt stage is about _+ 1 /~m over the whole tilt range, which sets a lower limit to tilt displacements even if the specimen has been adjusted to the eucentric height. As objects are usually not exactly flat, the eucentric height depends on the xy position of the specimen and may vary by several microme- ters. The z position should therefore be corrected before a tilt series is started, even at low magnification. If we accept a displacement within a tilt series corresponding to + 10% of one side of the image area, no displacement correction is necessary up to an EM magnification of about 1000 × , provided the specimen has been adjusted to the eucentric height.

At higher magnifications one has to determine the tilt displacements and correct for them. As the displacement has components in x, y, and z direction, the correction implies that the image has to be shifted back laterally and the focus has to be readjusted. The lateral displacement is measured by cross-correlation of an image with a reference image recorded previously, and corrected by using the electromagnetic image shift facility of the microscope. For the focus adjustment we use the autofocus procedures developed


by Koster et al. [13-15]. These procedures have been translated to the programming language C and adapted to our equipment. For beam-sensi- tive specimens all these operations have to be carried out under low-dose conditions. This requires that images needed for the determination of lateral displacement or for autofocusing have to be recorded on object areas different from the area to be used for the 3D reconstruction.

The Philips software version 4.1 which we have used up to now to establish automatic procedures offers several options to produce a defined image shift. Of these only the shift facility in the FO- CUS state of the TEM L O W - D O S E mode is appropriate, since it provides a fine enough step size for the image shift, and the primary beam is moved together with the image but can also be shifted independently for fine adjustment. The step size is compensated for magnification and almost constant in the SA (selected area) magnification range. In the EM film plane the steps are g r = 115 /xm per tick in radial and gq~ =4 .09 mrad per tick in azimuthal direction. On the camera screen this corresponds to 5.8 pixels per tick (radial) for recording with 2 × 2 binning.

5.1. Procedures correcting for tilt displacements only

Several procedures with increasing complexity have been developed for automatic control of tilt series. All programs have been written in the C programming language. Initially, a rather simple procedure without a low-dose option and autofocusing was used in order to study the behaviour of the correction for lateral displacements, which is certainly the most critical step. In this procedure all images were recorded at the same magnification. The tilt angle increment was 2°; the reference image was changed at 4 ° increments. It was found that a single displacement correction step was not reliable enough. However, with two correction steps for every tilt angle the procedure worked even at 50000 x EM magnification (96000 x on the CCD camera 's fluorescent screen), provided that the first image was not taken at an extreme tilt angle thus avoiding the

strong displacement which occurs when the tilt direction is reversed. The tilt series shown in fig. 8a was taken using this procedure.

It is worthwhile to consider the displacement correction in more detail. There are several sources of error which may lead to an incorrect shift of the image:

(1) If projections taken at different tilt angles are cross-correlated, the correlation peak will be stretched perpendicular to the tilt axis. Its length depends on the object thickness, the tilt angle, the angular increment and the area of overlap. At high tilt angles this is by no means a minor effect: With 5122 images taken at tilt angles 60 ° and 56 ° the peak may become stretched over a distance of approximately 50 pixels. The height of the peak is strongly diminished and the peak maximum may be anywhere in this elongated peak. It could happen that a background maximum becomes even higher, thus causing a strong displacement error. Normally the object thickness is much smaller than the lateral extension of an image. Then the images should be stretched according to 1 /cos ce before the cross-correlation function is calculated. (Correlating image i with i - 1 , one may also stretch image i according to cos a i_ ~/ cos ai.) This reduces the stretching of the correlation peak to an extent given by the object thickness and, of course, improves the peak shape and height. The peak can be further improved by appropriate filtering.

(2) A nonlinearity of the electro-magnetic image deflection could cause considerable errors. In particular it would be very difficult to determine the linear and nonlinear shift parameters with sufficient accuracy to allow for a measurement of displacements on areas different from the exposure area. In our case it turned out that the image shifts are sufficiently linear within a range of several /~m.

(3) The finite step size of the image shift introduces errors which will accumulate during a tilt series if they occur in the reference images. However, as the directions and magnitudes of single shift errors are statistically distributed, the total error will be acceptable even for tilt series with 60 or more projections. When images are stretched before the correlation, one may use a


smaller number of projections as references in order to avoid the accumulation of errors.

(4) If, in low-dose procedures, the tilt displacement is determined on a tilt area which is different from the exposure area, a specimen inclination in any direction may lead to a rather large displacement error: For an inclination angle of several degrees the chosen object area may even disappear from the field of view of the camera. This can be overcome by using tilt areas on both sides of the exposure area.

(5) Specimen drift is another source of error. After any mechanical movement , shift or tilt, a drift will occur, and it takes a few seconds before a stable situation is reached. In our automatic tilt procedures only the tilt angle is changed mechan- ically. Therefore, at least with room-tempera ture specimen holders, the drift is rather small. Cry- oholders normally have a larger long-term drift of up to several n m / s . So far we have not tried to run automatic procedures with a cryoholder. However, it should be possible to cope with the drift problem as long as the image shift remains in the accessible range.

5.2. Procedures for low-dose imaging and including auto focusing

When procedures for automatic recording of tilt series are applied to biomolecular specimens, low-dose techniques and autofocusing at every tilt angle become essential. The CM low-dose facility is not suitable for automatic low-dose procedures, since the E X P O S U R E state does not allow for an image shift. Therefore, we have implemented a similar concept using the FOCUS state in the TEM L O W D O S E mode. Several areas have been defined: exposure, focus 1, focus 2, tilt 1, tilt 2 (see fig. 6). For every area different conditions (position, magnif icat ion, e i ther imaging or diffraction) can be chosen. These data together with the data of the EM mode are stored in a data pool. A simple command (e.g., C M N D > SET focus l) sets the microscope to the state and object position defined by the corresponding data. To minimize irradiation of the exposure area, autofocusing and corrections for tilt displacements are carried out on the focus or tilt

SPECIMEN Fig. 6. Specimen areas used in low-dose procedures for automatic recording o f t i l t series: t i l t areas ~ , focus areas F i and e×posure area E, i = 1, 2. The ti l t areas ~ are shown with a larger size to indicate that t i l t displacements are normal ly

measured and corrected at lower magnif icat ion.

areas, respectively. The scheme of an automatic low-dose procedure, in which one area is used for autofocusing and two areas for displacement correction, is shown in fig. 7.

Autofocusing is done by taking two images formed with opposite beam tilts. Those two pic- tures will be displaced by an amount proportional to defocus (relative to Gaussian focus). The displacement is measured by cross-correlating both images. Finally, after the actual amount of defocus has been found, the microscope is set to the desired defocus. The reproducibility of the autotuning (autofocus) method has been discussed by Koster and de Ruijter [15]. These authors applied autotuning to untilted specimens using a video camera as input image device. We have tested the reproducibility of autofocusing using the CCD camera as image pickup device. In the experiment a series of images of an untilted specimen, shadowed with approximately 1.5 nm P t / C , was recorded after setting the focus automatically. During the series the focus was reset each time to Az = 2 /xm before autofocusing. The focus of the images was determined from the calculated diffractograms using a least-squares routine available in the EM image processing system [31]. In setting the focus automatically, a standard deviation of gAz = 4.1 nm at a magnification of 38000 × was found, and at 50000 × a standard deviation of gAz = 2.3 nm. Consequently, autofocus worked with a sufficiently high accuracy and reli- ability on untilted specimens. Although the dose for autofocusing is not of importance in our low-


dose procedures, it was found to be low for typical imaging conditions, e.g., at 38000 × approximately 100 e - / n m 2. With a tilted specimen the accuracy of autotuning was less satisfactory; hence some improvements of the autofocusing procedure for tilted specimens have to be made.

Autotuning as part of automatic tomography procedures offers several options to ensure opti- mum imaging conditions. These incude the alignment of the primary beam to a coma-free axis of

the microscope, focusing, or focusing combined with correction of astigmatism. The beam alignment has to be carried out before a tilt series is started. In most cases it is also sufficient to correct astigmatism once, prior to the tilt series. However, if one aims at higher resolution, it will be necessary to incorporate autofocusing with astigmatism correction into the tilt procedures, because the astigmatism may change during a tilt series, at least at higher tilt angles.

J . . . . . . . . . . . . . . . .

Fig. 7. Flow diagram of a low-dose procedure for automatic recording of tilt series which uses two tilt areas and one focus area.

K. Dierksen et al. /Automatic electron tomography 81

6. Procedures for automatic tilt series recording on trial

In order to test the procedures, we have performed 3D reconstructions from automatically recorded tilt series of two specimens. The first one was a carbon film with an arbitrarily chosen graphitized-carbon particle on top. In the second experiment negatively stained proteasomes iso- lated from the archaebacter ium Thermoplasma acidophilum were chosen as a specimen. Protea- somes are barrel-shaped protein complexes with a molecular weight of approximately 700 kD and proteolytic activity [32]. In the preparat ion we used, most particles appear in the side-on orientation. Gold clusters, approximately 20 nm in diameter, were added to the specimen in order to provide markers facilitating the on-line alignment. The total exposure of the area used for the reconstruction was about 5000 e - / n m 2.

3D reconstructions from tilt series involve two basic steps: At first all projections must be aligned with respect to a common coordinate system; once this is accomplished, a 3D density distribution is calculated from the aligned projections. Although the on-line correction for specimen displacements during tilting ensures that the specimen area chosen is largely contained in all projections, the residual displacements are too large to be tolerable for a 3D reconstruction. There- fore the alignment has to be refined. For the second step of the reconstruction we have used a series expansion technique [33] which is available in the EM program system [31].

For the alignment it was assumed that the orientation of the projections as well as the direction of the tilt axis does not change throughout the whole tilt series, the latter being parallel to one side of the image frames. The displacement from one projection to another was determined by cross-correlation functions using the whole image frames of 512 × 512 pixels. Starting with the 0 ° projection, each projection was aligned with respect to the neighbouring one. In order to combat the elongation of the cross-correlation peak perpendicular to the tilt axis all projections were stretched by resampling according to a factor 1 /cos a, a denoting the tilt angle.

A specimen area was cut out from the 0 ° projection using a circular mask with a diameter of 400 pixels and smooth edges; the mean value inside and all pixels outside were set to zero. This masking eliminates distortions caused by the image boundary and reduces the influence of a not quite perfect correction of the fixed pat tern of the CCD camera. The corresponding specimen area was identified in the neighbouring projection by cross-correlation and the displacement was corrected for. After the mask had been im- posed on this projection, it was used as the reference for the alignment of the subsequent projection.

The two experiments are documented in figs. 8-10. The full tilt series of the carbon particle is presented in fig. 8a. The gallery in fig. 8b shows horizontal sections through the reconstructed particle. The reconstruction was performed using a coarser sampling grid with a pixel size of 1.6 nm which is also the distance between neighbouring sections. Apar t from some imperfections which are probably due to the restricted range of tilt angles (or the absence of data in the "missing wedge" in reciprocal space), affecting noticeably the boundaries near the top and the bot tom of the particle, the structure appears to be well defined in the 3D reconstruction.

In fig. 9a several of the pre-aligned projections of the proteasome preparat ion are shown. A gold cluster is located near to the centre of each projection and surrounded by a number of proteasomes. The relatively low contrast of the protein makes it difficult to identify the particles unambiguously. The 3D reconstruction is shown in the form of horizontal sections in fig. 9b. The pixel size as well as the distance between adja- cent projections is 2.1 nm. As in the projections, the gold cluster at the centre shows up with high contrast in the 3D reconstruction, but now also proteasomes become clearly visible. Particles appear and disappear over a sequence of 5 to 6 sections which corresponds to a thickness of 10- 12 nm. Obviously, the vertical positions of the particles depend on their xy positions indicating a slight inclination of the specimen. In fig. 10 sections through the 3D reconstruction of a subframe are shown. Here the original sampling dis-



tance o f 0.52 nm was used; this is also the distance be tween the sections. The reconstruct ion is based on the al ignment, which has been performed using the whole image frames. In extract- ing the subframes one has to take into account that the distance of the subframe centre f rom the tilt axis is shor tened with increasing tilt angles. In close ag reement with a previous 3D reconstruction using r andom conical tilting [34], the proteasomes show clearly the barrel shape with the triparti te inner compar tment .

The most striking finding is probably that the visibility of the pro teasomes is much enhanced in the 3D reconstruct ion. This is less surprising when considering that the exposure, which largely de- termines the signal-to-noise ratio, is much lower in the case of a single project ion than for the accumula ted exposure of the 3D data set. It proves that informat ion f rom noisy projections can be combined properly, thus revealing structural details not recognizable in individual projections of the tilt series. Moreover , it is wor th ment ioning that the 3D reconstruct ion of sub- f rames can be pe r fo rmed on the basis o f the al ignment of the whole image frames, even if the correlat ion of the subframes fails due to the smaller number of resolution elements.

7. P o s s i b l e s o u r c e s o f e r r o r

The 3D reconstruct ions described in the previous section were carr ied out assuming that changes in magnificat ion within the tilt series and image aberrat ions such as distort ion can be neglected. It is fur ther supposed that the tilt axis is perpendicular to the direct ion of project ion and stable in orientat ion. Al though the quality of the reconstruct ions indicates that these possible

sources of imperfect ion do not p roduce serious errors, it seems worthwhile to discuss and evalu- ate them in more detail.

Apar t f rom linearity and sensitivity, the CCD camera has the advantage of a high geometr ic accuracy. Its position and orientat ion is, within a series of images, fixed with respect to the microscope. Therefore it can be used as a tool for a precise character izat ion of the microscope; using cross-correlations, a very high precision can be at tained since correlat ion peak positions can be de te rmined with subpixel accuracy.

7.1. Orientation and stability of the tilt axis

The precision rotary bearing of the goniometer defines the or ientat ion of the tilt axis very accu- rately, certainly within 0.1 mrad. However, there may be a deviation f rom the normal between the tilt axis and the pr imary beam. Two factors are involved: the mechanical al ignment of the go- n iometer stage and the al ignment of the objective pole pieces, which in turn defines the direction of the pr imary beam under the condit ion of opti- mum beam al ignment on to a coma-free axis. There are no specifications given by the manufac- turer; it can be expected, however, that the mis- al ignment due to both factors is in the range of several mrad. An error of this magni tude will certainly affect the quality of a reconstruct ion f rom 5122 or 10242 pixel images; it will cause a resolution fall-off towards the per iphery of the reconstruct ion volume.

7.2. Changes of image rotation and magnification

Changes of the image rotat ion and the magnification can be caused ei ther by z-displacements due to the limited mechanical accuracy of the tilt

Fig. 8. Automatically recorded tilt series and 3D reconstruction of a graphitized-carbon particle on carbon film. (a) Tilt series. Tilt range from - 59 ° to + 59 ° with 2 ° angle increment; accelerating voltage 80 kV; EM magnification 50000 x, magnification on CCD camera screen 96000x. Images were recorded as 512×512 pixel frames with 2x2 binning and a pixel size of 38 p.m. Referred to the specimen the pixel size is 0.4 nm, the image area (200 nm) 2. Three images of the same specimen area were recorded at each tilt angle, two of which were used for displacement correction. No focus correction was made; however, the eucentric height had been carefully adjusted; the focus was about 2 /xm underfocus. The electron dose of one image was approx. 100 e - /nm 2, the total dose approximately 18000 e - /nm 2. (b) Series of sections through the reconstructed particle at reduced resolution. Pixel size and

distance between successive sections are 1.6 nm.

84 K. Dierksen et a L / Automatic electron tomography

axis, by changes of the lens currents or by magnetic hysteresis of the lenses. With a Philips CM12, we have measured the dependence of these magnitudes on z-displacements: the image

rota t ion/z-displacement was 0.18 mrad / /xm, and the relative magnification change/z-displacement was 2.4 × 10-4/ /xm. This means that a 4 /xm z-displacement will cause a shift of 1 pixel over a

Fig. 9. (a) Selected projections ( - 6 4 °, 0 °, + 20 °, + 64 °) of an automatically recorded tilt series of proteasomes. Tilt range from - 6 4 ° to + 64 ° with 4 ° angle increment; accelerating voltage 80 kV; EM magnification 38000 x (corresponding to 73 000 x on the CCD camera); 5122 pixels with pixel size 0.52 nm. For recording the tilt series a procedure without autofocusing and not applying strict low-dose conditions was used: Displacements were first measured and corrected using two tilt areas; then a fine correction was done on the exposure area. The total exposure of the exposure area was thus approximately 5000 e - / n m z. The proteasomes are hardly recognizable in the projections; only a gold particle (ca. 20 nm in diameter), used to facilitate the displacement determination, is displayed with strong contrast. (b) Horizontal sections through the central part of the 3D reconstruction, calculated at reduced resolution. Pixel size and distance between successive sections are 2.1 nm. The supporting film is obviously

slightly inclined (ca. 20-3 ° ) with respect to the object plane of the objective lens.


distance of 1024 pixels. Both effects can therefore be neglected provided that the specimen has been adjusted to the eucentric height.

The lens currents of a modern microscope certainly have a sufficient long-term stability in not contributing to these errors. However, hysteresis effects may occur when different magnifications are used within a procedure. Such effects can, of course, be avoided by normalizing the imaging lenses after every change of the magnification.

Within the relatively small image field of the CCD camera, third-order distortion can be neglected. Elliptic distortion, however, even exists on the optical axis. Though this is only a misalign- ment error, it will transform a circle into an ellipse, the axes of which may differ by several per mille. It may therefore affect the quality of 3D reconstructions.

8. Future prospects

7.3. Third-order and elliptic distortions 8.1. On-line image restoration

The influence of both kinds of distortion becomes obvious in displacement maps as calculated in the course of examining large image fields of near-perfect crystalline specimens [9].

At least for untilted specimens, autofocusing has been demonstrated to work and to provide a preselected focus with high accuracy (see section 5 and ref. [15]). This, in conjunction with on-line

Fig. 10. Horizontal sections through the central part of a 3D reconstruction of a subframe of the tilt series shown in fig. 9, calculated with the full resolution. The subframe size in the projections was 1282 pixels, and the pixel size and distance between

the projections 0.52 nm.


I

o

-I

--2-

o 1 2 3 4 o 1 2 ,3 4

1 / n r n 1 / n r n

Fig. 11. Two sets of phase contrast transfer functions suitable for on-line image restoration of biological specimens, (a) for a 100 kV TEM with LaB6-cathode , and (b) for a 200 kV TEM with thermal field emission gun. The parameters of the two cases are (the values in parentheses refer to case (b)): Energy spread (FWHM) 1.5 eV (1.0 eV), illumination aperture 0.2 mrad (0.1 mrad), defocus values (underfocus) 103, 264, 1200, 6000 nm (85, 218, 1000, 5000 nm). In reduced units the defocus values are 1.20, 3.07, 14,

70 Scherzer.

image acquisition, allows one to implement on- line image restoration using a small number of images taken at well defined focus settings. Apart from corrections for the transfer function, the aim is mainly to extend the range of "good" transfer to low frequencies which are important for the contrast of biological material. For this purpose a series of underfocus values, spaced approximately in a geometric sequence, should be suitable. The focus values can be assumed to be known in advance with sufficient accuracy; however, there may be displacements between the images, due, e.g., to specimen drift, which have to be determined by cross-correlation. Two sets of phase contrast transfer functions which should be suitable for on-line restoration are shown in fig. 11.

8.2. Quasi-on-line 3D reconstruction

It would surely be desirable to obtain a first 3D reconstruction soon after automatic recording of a tilt series has been completed, particularly in low-dose work where the structure cannot be recognized in the projections. The first step, the alignment of the projections, can certainly be incorporated into automatic procedures. If an array processor is available, it can easily perform the necessary operations during times it is not used for EM control. A 3D reconstruction, performed immediately after completion of the tilt series, would enable the operator to decide

whether the series is worth storing for further evaluation or not.

Acknowledgements

We would like to thank Drs. J. Frank, J. Chal- croft and Bing Jap for critical reading of the manuscript. This work has been supported by the Deutsche Forschungsgemeinschaft (SFB 266, project A2).

References

[1] D.J. De Rosier and A. Klug, Nature 217 (1968) 130. [2] W. Hoppe, R. Langer, G. Knesch and Ch. Poppe, Natur-

wissenschaften 5 (1968) 33. [3] R. Hart, Science 159 (1968) 1464. [4] L.A. Amos, R. Henderson and P.N.T. Unwin, Prog.

Biophys. Mol. Biol. 39 (1982) 183. [5] R.M. Glaeser, Annu. Rev. Phys. Chem. 36 (1985) 243. [6] W. Hoppe, Angew. Chem. Int. Ed. Engl. 22 (1983) 456. [7] M. Radermacher, J. El. Microsc. Techn. 9 (1988) 359. [8] J. Frank, Electron Microsc. Rev. 2 (1989) 53. [9] R. Henderson, J.M. Baldwin, T. Ceska, F. Zemlin, E.

Beckmann and K.H. Downing, J. Mol. Biol. 213 (1990) 889.

[10] B.K. Jap, P.J. Walian and K. Gehring, Nature 350 (1991) 167.

[11] W. Hoppe, Chem. Scr. 14 (1978/79) 227. [12] R. Guckenberger, Dissertation (Techn. Univ. Miinchen,

1979).

K. Dierksen et a L / Automatic electron tomography 87

[13] A.J. Koster, A. van den Bos and K.D. van der Mast, Ultramicroscopy 21 (1987) 209.

[14] A.J. Koster, W.J. de Ruijter, A. van den Bos and K.D. van der Mast, Ultramicroscopy 27 (1989) 251.

[15] A.J. Koster and W.J. de Ruijter, Ultramicroscopy 40 (1992) 89.

[16] W. Hoppe, H.J. Schramm, M. Sturm, N. Hunsmann and J. Gassmann, Z. Naturforsch. 31a (1976) 645.

[17] W. Hoppe and R. Hegerl, in: Computer Processing of Electron Microscope Images, Topics in Current Physics, Vol. 13, Ed. P.W. Hawkes (Springer, Berlin, 1980) p. 127.

[18] W. Hoppe, J. Gassmann, N. Hunsmann, H.J. Schramm and M. Sturm, Hoppe-Seyler's Z. Physiol. Chem. 355 (1974) 1483.

[19] V. Knauer, R. Hegerl and W. Hoppe, J. Mol. Biol. 163 (1983) 409.

[20] B.F. McEwen, M. Radermacher, C.L. Rieder and J. Frank, Proc. Nat. Aead. Sci. USA 83 (1986) 9040.

[21] A.L. Olins, D.E. Olins, H.A. Levy, S,M. Margle, E.P. Tinnel and R.C. Durfee, J. Microscopy 154 (1989) 257.

[22] M. van Heel, Ultramicroscopy 21 (1987) I l l . [23] J. Frank, W. Goldfarb, D. Eisenberg and T.S. Baker,

Ultrarnicroscopy 3 (1978) 283.

[24] W. Hoppe, Naturwissenschaflen 61 (1974) 534. [25] R. Guckenberger, Ultramicroscopy 9 (1982) 167. [26] W. Hoppe, R. Hegerl and R, Guckenberger, Z. Natur-

forsch. 33a (1978) 857. [27] W.O. Saxton, Computer Techniques for Image Process-

ing in Electron Microscopy (Academic Press, New York, 1978).

[28] A. Rose, Vision: Human and Electronic (Plenum, New York, 1973).

[29] J. Chalcroft and C.L. Davey, J. Microscopy 134 (1984) 41, [30] I. Daberkow and K.H. Herrmann, private communica-

tion. [31] R. Hegerl and A. Altbauer, Ultramicroscopy 9 (1982)

109. [32] B. Dahlmann, F. Kopp, L. Kuehn, B. Niedel, G. Pfeifer,

R. Hegerl and W. Baumeister, FEBS Lett. 251 (1989) 125.

[33] P.R. Smith, T.M. Peters and R.H.T. Bates, J. Phys. 16 (1973) 361.

[34] R. Hegerl, G. Pfeifer, G. PiJhler, B. Dahlmann and W. Baumeister, FEBS Lett. 283 (1991) 117.

Documents

Towards automatic electron tomography