A Bayesian reconstruction method for micro-rotation imaging in light microscopy

A Bayesian Reconstruction Method for Micro-RotationImaging in Light MicroscopyDANAI LAKSAMEETHANASAN,1* SAMI S. BRANDT,1 PETER ENGELHARDT,1,2

OLIVIER RENAUD,3 AND SPENCER L. SHORTE3

1Laboratory of Computational Engineering, Helsinki University of Technology, FI-02015 TKK, Finland2Department of Pathology and Virology, Haartman Institute, University of Helsinki,FI-00014 University of Helsinki; and HUSLAB, Helsinki, Finland3Institut Pasteur, Plate-forme d’Imagerie Dynamique, Imagopole, 25-28 ru du Dr. Roux, 75015 Paris, France

KEY WORDS micro-rotation imaging; fluorescence imaging; cell biology; expectation-maximization; total variation

ABSTRACT The authors present a three-dimensional (3D) reconstruction algorithm and recon-struction-based deblurring method for light microscopy using a micro-rotation device. In contrast toconventional 3D optical imaging where the focal plane is shifted along the optical axis, micro-rota-tion imaging employs dielectric fields to rotate the object inside a fixed optical set-up. To address thisentirely new 3D-imaging modality, the authors present a reconstruction algorithm based on Bayes-ian inversion theory and use the total variation function as a structure prior. The spectral propertiesof the reconstruction by simulations that illustrate the strengths and the weaknesses of the micro-rotation approach, compared with conventional 3D optical imaging, were studied. The reconstructionfrom real data sets shows that this method is promising for 3D reconstruction and offers itself as adeblurring method using a reconstruction-based procedure for removing out-of-focus light from themicro-rotation image series.Microsc. Res. Tech. 71:158–167, 2008. VVC 2007Wiley-Liss, Inc.

INTRODUCTION

Three-dimensional (3D) fluorescence imaging of indi-vidual live cells is an essential tool for cellular biology.Previously, 3D fluorescence imaging has employed so-called optical sectioning (z-stack) techniques (Agardet al., 1989; Fay et al., 1989). These techniques requirethat samples are stabilized by attachment to a trans-parent surface, and the focal plane of the microscope isshifted in small steps along the optical axis (z-axis) tocollect a through-stack of 2D images (see Fig. 1a). Onlypart of the sample located near the focal plane is infocus while the other parts appear blurred. This imagestack series can be computationally restored using 3Ddeconvolution techniques to remove this out-of-focusblur (Carrington et al., 1995; Holmes et al., 1995; Shaw,1995; van Kempen et al., 1997; Verveer et al., 1999; Vici-domini et al., 2006). Perhaps, some of the most confound-ing limitations to conventional 3D imaging arise fromthe fact that in optical microscopy the z-axis resolution isalways two fold to three fold lower than that of the xy-axis. Through-stacking protocols compound this problemsuch that any resulting 3D reconstruction is distorted bythis x, y, z resolution asymmetry, and the objects surfaceand volume are convolved in a complex and mostly irre-cuperable manner. To overcome these problems, axial to-mography (tilted-view) has been proposed by Shaw et al.(1989), Cogswell et al. (1996), Bradl et al. (1996), Heintz-mann and Cremer (2002) and Renaud et al. (2007). Inaxial tomography, several z-stack data sets are collectedfrom different orientations by rotating the specimen, byknown angles and then all the views are fused into a sin-gle reconstruction volume (see Fig. 1b). This techniquesignificantly enhances the axial resolution while the lat-eral resolution slightly degrades, comparing with the

reconstruction using the conventional single stack. How-ever, this approach requires complex data acquisition,nonsimultaneous recording of the oriented volumes.Also, these methods again required stabilization of thespecimen on a glass capillary.

The principle of cell rotation involves organizing theintensity and phase difference between eight electrodesto create a 3D dielectric field cage (Fuhr et al., 1994;Muller et al., 1999; Schnelle et al., 1993, 2000) and pro-duce rotational force (Shorte et al., 2003). Thisapproach provides nontactile manipulation and fastdata acquisition, which allows facile quasi-3D imagingof non-adherent living cells by virtue of the fact thatcells can be positioned to access almost any point ofview, making 3D hidden features accessible (Lizundiaet al., 2005). In micro-rotation imaging (Korlach et al.,2005; Lizundia et al., 2005), instead of moving the focalplane of the objective, dielectric fields trap nonadherentcells in 3D electrode cages and rotate them fullythrough 360 degrees approximately around a singleaxis parallel to the focal plane, see the schematic viewin Figure 2. In contrast to the z-stacking and the axialimaging approaches, only a single 2D image is takenfrom each view as shown in Figure 1c. Today, the majorchallenge in micro-rotation imaging is to provide ro-bust, automated 3D reconstruction solutions. Here, we

*Correspondence to: Danai Laksameethanasan, Laboratory of ComputationalEngineering, Helsinki University of Technology, FI-02015 TKK, Finland. E-mail:[email protected]

Received 14 May 2007; accepted in revised form 17 October 2007

Contract grant sponsor: European Union (NEST-FP6-AUTOMATION)-project;Contract grant number: 4803 (NEST).

DOI 10.1002/jemt.20550

Published online 28 November 2007 in Wiley InterScience (www.interscience.wiley.com).

VVC 2007 WILEY-LISS, INC.

MICROSCOPY RESEARCH AND TECHNIQUE 71:158–167 (2008)

propose a 3D reconstruction approach based on theBayesian inversion theory, aimed precisely to facilitatequalitative micro-rotation imaging. In addition, ourmethod can be seen as a reconstruction-based deblur-ring method where the out-of-focus light in observedimages is removed using the Bayesian reconstruction.

Bayesian inversion is a statistical method for solvingill-conditioned inverse problems associated with mea-surement noise as well as incomplete data (see Vardiet al., 1985). The Bayesian approach provides a flexibleand consistent way to incorporate prior informationwith observed data. The Bayesian method has beensuccessfully used in many applications including medi-cal imaging (Kolehmainen et al., 2003), electron tomog-raphy (Engelhardt, n.d., Scheres et al., 2007; Skoglundet al., 1996), and conventional 3D optical imaging(Holmes et al., 1995; Verveer et al., 1999; Vicidominiet al., 2006). In our Bayesian model, we assume Pois-son noise in the measurements and use the total varia-tion (TV) prior for the object densities. The advantageof the TV is that it has an edge preserving propertywhile it smooths out the homogeneous regions of theobject. This structural prior has been applied in imagedenoising (Rudin et al., 1992), limited angle tomogra-phy (Kolehmainen et al., 2003), as well as 3D opticalsectioning in confocal microscopy (Dey et al., 2006).Also, other edge-preserving priors exist that may havedifferent performance from TV (Green, 1990; Vicido-mini et al., 2006).

To solve a 3D reconstruction problem posed by micro-rotation imaging, the projection directions must beknown. The standard approaches for solving the corre-sponding motion estimation problem, including fiducialand nonfiducial techniques (Brandt, 2006; Brandt andZiese, 2006; Brandt et al., 2001a,b), seem difficult forour test-data because of the lack of detail in the targetobject. We have therefore used the method of motionrecovery without correspondence (Brandt and Koleh-mainen, 2007; Brandt and Mevorah, 2006) that bettersuits estimating the motion for these kinds of objects.

The study begins by presenting the image formationmodel, the statistical setting, and the minimization ofthe statistical functions in Materials and Methods sec-tion. The results of reconstructing the simulated andthe real micro-rotation series are reported in Resultssection. Finally, we discuss and conclude the study inDiscussion section.

MATERIALS ANDMETHODSAcquisition of Micro-Rotation Time-Lapse

Image Series

The method for handling and trapping living cell indielectric-field-cage (DFC3) chips has been describedpreviously, for example, by Lizundia et al. (2005) andRenaud et al. (2007). In this study, SW13/20 cell line, akind gift from Vaughan et al. (2001), was chosenbecause they display stable, high-expression levels of anuclear lamin A GFP fusion protein, resulting instrong fluorescence patterns tightly localized to the nu-clear envelope volume, and easily detected in mostcells. Before observation, adherent cells in culturewere resuspended in CytoconTM buffer II (Evotec Tech-nologies, Germany) at a final concentration of 105 to106 cells per mL. Briefly, suspended cells were flowedinto a microfluidic chamber (DFC3 chip, Evotec Tech-nologies). Electrodes of the octode cage are separatedby 40 lm from tip to tip and are 40 lm height; andinside the cage isolated cells in suspension weretrapped in suspension (see Fig. 2) by activating elec-trode current typically with frequency 700 kHz andvoltage between 3 and 10 V. Control of the DFC3-chipelectrodes used either a Cytocon 400TM or a CytoSmartTM

generator via SwitchTM software (Evotec Technologies).Trapped cells were rotated around the x or y-axis bymodulating the phase of currents at each electrode ofthe cage; and trapped rotating cells were imaged using

Fig. 1. Ideal geometries of the data acquisition of (a) the z-stacking, (b) the axial z-stacking (threeorientations) and (c) the micro-rotation techniques. Each plane represents the focal plane in the objectcoordinate frame and the spherical object locates in the middle.

Fig. 2. (a) Schematic view of the dielectric field channel with eightelectrode strips:four attached on the top glass (300 lm thickness) andfour on the bottom glass (150 lm thickness). (b) Schematic view of the3D electrode cage with the eight electrodes and the spherical object atthe middle.

Microscopy Research and Technique DOI 10.1002/jemt

159MICRO-ROTATION IMAGING IN LIGHT MICROSCOPY

an inverted optical microscope (excitation 450–490 nm;emission 515–565 nm), sensitive camera (DV885 fromAndor Technology, UK), a 633 water immersion objec-tive (numerical aperture 1.2).

Image Formation Model

In this section, we will present how the measure-ment plane can be constructed from the 3D object origi-nating from our previous work (Laksameethanasanet al., 2006). First, we consider a single projection andthereafter multiple ones taking all the projections to-gether.

Single Projection. The forward model can be math-ematically described by the linear operator Ai: C3 ? C2

such that

miðx; yÞ ¼ Aif ðx; y; zÞ; ð1Þwhere f 2 C3 is the piecewise continuous density volumeevaluated at (x,y,z) 2 R3, mi 2 C2 presents the piecewisecontinuous measurement image for the ith projection,evaluated at (x,y) 2 R2, and C2 and C3 are the spaces ofpiecewise continuous functions in R2 and R3, respec-tively. The forward model is constructed as follows.

Assuming a linear system, a wide-field microscopecan be characterized by a 3D point spread function(PSF), which is the response volume for a point source.We assume that the optical axis is in the direction of z-axis. If the microscope system is shift invariant, themeasurement image mi, obtained as the plane corre-sponding to the focal plane z 5 d, is represented by

miðx; yÞ ¼Z Z Z

hðx� x0;y� y0;d� z0Þfiðx0;y0; z0Þ dx0dy0dz0;

¼ hðx;y; zÞ � fiðx;y; zÞ��z¼d

ð2Þ

where fi 5 Rif is the rotated object density for the ithprojection, Ri is the rotation operator, h is the PSF, and* is the 3D convolution operator. Taking the 2D Fouriertransform with respect to x, y of the both sides of (2),we obtain

Miðkx;kyÞ ¼Z

Hðkx;ky;d� z0ÞFiðkx;ky; z0Þ dz0; ð3Þ

where Mi, Fi, and H denote the 2D Fourier transformof mi and fi and h.

In the minimization problem, we additionally needthe adjoint operator Ai*: C2 ? C3, which makes a 3Dback-projection from a measurement image. Let gi 5Ai*mi, we get

giðx; y; zÞ ¼ R�i

nhð�x;�y;�zÞ � �dðz� dÞmiðx; yÞ

�o; ð4Þ

where Ri* is the adjoint of the rotation operator and d(z)is the Dirac-delta function. In other words, the adjointoperator consists of zero-padding, correlation with thePSF, and rotation of the volumes. The derivation is pre-sented in Appendix. By taking the 2D Fourier trans-form with respect to x, y in (4), we obtain

Gðkx; ky; zÞ ¼ R�i

nH�ðkx; ky;d� zÞMiðkx; kyÞ

o; ð5Þ

where G is the 2D Fourier spectrum of g with respectto x, y in the function of z.

In practice, we must work with a discretized objectand the PSF. When the discretized object is rotated, weadditionally need to interpolate the rotated volume tothe grid of PSF to compute the convolution in (2). Fromthe many interpolation methods (see Lewitt, 1990,1992; Mueller, 1998), we pick the trilinear interpola-tion method as it is computationally cheap and pro-vides reasonable quality. The discretized model of (1)and (2) can be represented as a linear system

mi ¼ Aif ¼ PHRif ; ð6Þwhere mi 2 RM is a vector of measurement values (Mpixels) for the projection i, f 2 RN is a vector of densityvalues (N voxels), Ai 2 RM3N is a matrix of projection i,Ri is the rotation-translation matrix for the projectioni, H is the Toeplitz matrix representing the 2D convolu-tion operations, and P is a projection matrix that picksthe object plane coinciding with the focal plane. For theadjoint operator, the corresponding discretized modelis

gi ¼ ATi mi ¼ RT

i HTPTmi; ð7Þ

where gi has the same size as f. In practice, the resultof matrix-vector product between H and the rotatedobject Rif can efficiently computed using the FFT algo-rithm in (3) and (5). The rotation-interpolation matrixRi is a function of six motion parameters (three anglesand three displacements), which are estimated by thethe method of motion recovery without correspondence(Brandt and Mevorah, 2006). After the rotation andthe translation, Ri performs the trilinear interpolationfor the object density. Although Heintzmann andCremer (2002) suggested to use trilinear interpolationin both the forward and backward projections, weemphasize that the trilinear interpolation and itsadjoint are not identical since the basis kernel of trilin-ear interpolation is not spherically symmetric (Lewitt,1992; Mueller, 1998). Hence, Ri

T requires the adjoint ofthe trilinear interpolation instead, which can be com-puted by constructing the transpose of trilinear inter-polation matrix for each elementary cube, defined byeight grid points.

Multiple Projections. In the estimation problem,we solve the equation (6) simultaneously for all the pro-jections. The total linear model is composed from (6),which is given by

m ¼ Af ; ð8Þwhere m 5 [m1

T, . . ., mKT]T 2 RKM is a joint vector of the

K measurement vectors and A 5 [A1T, . . ., AK

T]T 2RKM3N is a joint projection matrix including the K pro-jection matrices. For the adjoint operator, the corre-sponding discretized model is

g ¼ ATm ¼XKi¼1

ATi mi: ð9Þ

We note that this multiple-projection model resemblesthe model of Verveer and Jovin (1999), which combines


160 D. LAKSAMEETHANASAN ET AL.

a confocal-microscopic image with a wide-field imagefrom the same object for improving the quality ofrestored confocal images with high signal to noise ra-tio, and also relates to the model of Heintzmann andCremer (2002) where several oriented z-stacks aresuperimposed by rotating the sample, aiming toimprove the axial resolution. Furthermore, Anconelliet al. (2006) proposed the similar concept for deconvo-lution of multiple images in the field of astronomy.

So far, we have neglected measurement noise in theimage formation model (2) where in reality this modeldoes not hold. Noise modeling is discussed in the fol-lowing section.

Bayesian Inversion and Estimation

This section discusses how to estimate the objectdensity f for the given measurement images m, assum-ing known motion parameters for the object.

In Bayesian inversion theory, the complete solutionfor an inverse problem is represented by the posteriordistribution, given by the Bayes’ formula

pðf jmÞ ¼ pðfÞpðmjfÞpðmÞ / pðfÞpðmjfÞ; ð10Þ

where p(m|f) is the likelihood density, p(f) is the priordensity for f, and p(m) is the total probability densitiesfor m. The advantage of the Bayesian approach is thatall the available prior knowledge relating to theunknown structure can be used in a systematic way tosolve the reconstruction problem. Rather than solvingreconstruction, we can use the posterior distribution toassess their reliability. For the complex posterior den-sity, the characterization of the posterior distributionmay require Monte Carlo simulations that can be com-putationally heavy. In simple cases, one often usesmaximum a posteriori (MAP) estimates, which usuallycoincide with traditional estimators with regulariza-tion. The general theory of statistical inversion is pre-sented, for instance, by Kaipio and Somersalo (2004).

In this study, we compute the MAP estimate

f ¼ arg maxf

pðf jmÞ: ð11Þ

It means that, for the given prior density p(f) and themeasurement data m, we determine the unknown val-ues f that approximate the data m the best. The likeli-hood density p(m|f) is constructed from the measure-ment noise model as follows.

In fluorescence microscopy, the amount of photonscounted by the detector is well described by the Poissonprocess, i.e., the pixel measurement mj is a sample of aPoisson distributed random variable with the expectedvalue (Af)j. Assuming that the measurements are inde-pendent, the likelihood density is

pðmjfÞ ¼YKMj¼1

1

mj!

� �exp mT logðAfÞ � 1TAf

� �; ð12Þ

where 1 5 [1, 1, . . ., 1]T 2 RKM.Prior Selection. Since the reconstruction problem

is inherently ill-posed, we have to incorporate some

prior knowledge of the structure to be reconstructed tomake the problem well-posed. The widely used priormodels are the Gaussian and Entropy priors that pro-vide smooth solutions and prevent noise amplification.However, one must compromise between the noisereduction and the object edge preservation (vanKempen et al., 1997, Verveer et al., 1999). To overcomethis problem, the TV prior has been proposed by Rudinet al. (1992) and Green (1990), and it has been used invarious applications including 3D light microscopicimaging (Dey et al., 2006; Vicidomini et al., 2006).We make some modifications to the TV definition asfollows.

Let us define the piecewise continuous object densityas f (r) 5 Sknkg(r 2 rk), where g(r) is represented bythe Gaussian kernel and f 5 [n1, n2, . . ., nN]

T. The TVregularized function is defined as

VðfÞ ¼Z

jDf ðrÞj dr �Xl

��Xk

nlDgðrl � rkÞ��

� 1TjGf j; ð13Þ

where D is Laplacian operator and G is the Toeplitzmatrix representing the 3D convolution with the Lap-lacian of Gaussian kernel Dg. The TV regularized prioris then

pðfÞ / exp �kVðfÞð Þ; ð14Þ

where the regularization parameter k controls weight-ing between the likelihood and prior terms.

Minimization

In this section, we discuss how to minimize the sta-tistical objective function. There are several ways toproceed. One possibility is the direct optimization bythe conventional conjugate gradient (CG) method, asinvestigated by Verveer et al. (1999), with various priorfunctions for 3D deconvolution problems with z-stack-ing. However, it is relatively difficult to deal with thepositivity constraint. An alternative approach is to usethe expectation maximization (EM) algorithm wherethe positivity constraint is built-in and the method iseasy to implement.

With the objective function of multiple projections,Verveer and Jovin (1999) used the CG method in theminimization assuming Gaussian noise with Gaussianprior whereas Heintzmann and Cremer (2002)employed the EM method with no prior. Here, we usethe extension of the EM algorithm for the TV prior inthe multiple projections (8).

Expectation Maximization. The EM algorithm isa statistical method for finding the maximum likeli-hood (ML) estimate of unknown parameters, givenobservations with missing information (incompletedata). First, the algorithm estimates the expected com-plete data likelihood from incomplete data (expectationstep) and then searches for the estimate maximizingthe expected complete data likelihood (maximizationstep). It is well known that the likelihood increasesmonotonically during the EM iterations until a station-ary point of the likelihood function is found (Dempsteret al., 1977). There are many advantages of using the



EM algorithm, including adaptability and low cost periteration.

The EM algorithm can be used to compute the recon-struction, even with no explicit prior where the maxi-mization problem is equivalent to minimizing

f ¼ argminf

1TAf �mT logðAfÞ� : ð15Þ

The EM iterations (Holmes, 1989; Vardi et al., 1985)for the reconstruction are then computed from

fkþ1 ¼ fk

AT1AT m

Afk

� �; ð16Þ

where the divisions and vector multiplications are per-formed element by element and k is the iteration index.An important property of this algorithm is that itensures non-negativity of the solution. However, sincethis inverse problem is ill-posed, neglecting the priorknowledge may lead to a useless solution if a largenumber of iterations is taken. So, to obtain meaningfulresults without an explicit prior, the iterations must bestopped early enough to avoid underregularization andnoise amplification.

Expectation Maximization With Total Varia-tion. The basic EM algorithm can be extended to com-pute the MAP estimate to take the prior informationinto account. Here, we use this extension for the TV

prior. Inserting (12) and (14) into (10), the computationof the MAP estimate (11) is equivalent to minimizing

f ¼ argminf

1TðAfÞ �mT logðAfÞ þ kVðfÞ� : ð17Þ

Since the absolute function is not differentiable at zero,we use the smooth approximation |t| � b21 cosh (bt),as used by Green (1990) and Kolehmainen et al. (2003),where b is a parameter controlling the approximationaccuracy. The larger the parameter b, the more accu-rate approximation we will get. Minimizing (17) yieldsthe iterative scheme

fkþ1 ¼ fk

AT1þ krVðfkÞAT m

Afk

� �; ð18Þ

derived by Green (1990), that we call it EMTV algo-rithm; we approximate the gradient by !V(fk) � GT

tanh(bGfk). The regularization parameter k must alsobe selected properly; overregularization (too large k)leads the algorithm to diverge whereas underregulari-zation amplifies noise. Our EMTV algorithm imple-mentation is similar to the works of Green (1990) andDey et al. (2006) except we define the piecewise contin-uous object density f through Gaussian interpolation sothat (13) simply contains the convolution with the Lap-lacian of Gaussian kernel G as explained in BayesianInversion and Estimation Section.

Fig. 3. Simulated 3D point spread function. The upper row shows intensity profile along x and z-axis(optical axis) at the centre of the PSF; the lower row displays two 2D-slices of the xy-plane (corre-sponding to the focal plane) and the yz-plane.



In this work, we used the Cisiszar I-divergence as astopping criterion (Csiszar, 1991). In the simulation,we measure the I-divergence between the true object fand the reconstructed object fk whereas, for real data,we compute the criterion between the measurementimages m and the projected images Afk. We stop theiterations when the change in I-divergence betweentwo consecutive iterations is smaller than 1024, or thealgorithm reaches the maximum number of iterations.

RESULTSSimulation Results

To evaluate the properties of the proposed methodand to compare the micro-rotation imaging with thestandard z-stack imaging, we first report our simula-tions with the wide-field fluorescence microscopedescribed in Acquisition of Micro-Rotation Time-LapseImage Series Section. In our simulations, we used SVIHuygensTM software to compute the 3D PSF that cor-responds to the wide-field microscope (see Fig. 3). Anobject was created with 100 3 100 3 100 samples inthe volume. Figure 4a shows three 2D-slices corre-sponding to the xy, xz, and yz-planes located at the vol-ume centre and its iso-surface, respectively, from top to

bottom. The object consisted of a sphere at the middleand three sets of three plates and nine bars, arrangedaround the sphere in different directions. The volumehad only two intensity values, one for the backgroundand the other one for the object. The object wascorrupted by Poisson noise (SNR 5 20) and projectedusing the micro-rotation imaging model (2) and theconventional z-stack model (Verveer et al., 1999)with the wide-field PSF. Figure 4b illustrates micro-rotation images (127 3 127 nm lateral, 3.6 degreesbetween two consecutive images) where the object wasrotated around x-axis, and Figure 4c shows simulatedz-stack images (1273 127 nm lateral, 254 nmaxial).

The z-stack reconstruction using the EMTV algo-rithm is shown in Figure 4d; the result contains highresolution details in the lateral view (xy), but some blurremains in the axial views (xz, yz), particularly alongthe optical axis (z). Figure 4e illustrates the micro-rota-tion reconstruction using EMTV; the reconstruction issharp in the views parallel to the rotation axis (xy andxz-slices), but it is blurred in the view orthogonal to therotation axis (yz-plane), especially in the tangentialdirection of the rotation. In other words, the plates andbars oriented in the radial direction could not be recon-structed. We note that the rotation geometry (see

Fig. 4. Reconstruction comparison of z-stacking and micro-rotation models using the EMTV algo-rithm. (a) three orthogonal slices and iso-surface of the original 3D synthetic object; (b) raw micro-rota-tion images (rotated around the x-axis); (c) three orthogonal slices of the simulated z-stack object; (d)EMTV reconstruction from the z-stack images; (e) EMTV reconstruction from the micro-rotation images.



Fig. 1) and the elongation of the PSF function make theresolution weaker in the tangential direction.

We additionally compared the EMTV algorithm tothe standard EM algorithm as well as the prior func-tion suggested by Green (1990) (we call it EMGR). Theresults are shown in Figure 5. Obviously, the noisyartefacts inside the homogeneous region (the sphere)appear in the EM reconstructions whereas the EMTVand EMGR significantly improve the result: the noisyartefacts are removed while the edges of the sphereremain sharp. We also noticed that the differencebetween the EMTVand the EMGR is fairly small.

We also experimented with a 3D simulated cell. Fig-ure 6a shows three rotating views of the original celland Figure 6b correspondingly displays micro-rotation

images obtained by the simulated image formationmodel with Poisson noise (SNR5 20). The EMTVrecon-struction with the size 1003 1003 100 is illustrated inFigure 6c, which shows the rotated slices of the recon-struction corresponding to the true images. Figure 6dshows three orthogonal slices (xy, xz, yz) of the true cellwhereas Figures 6e and 6f illustrate the correspondingslices of the reconstructions using EMGR and EMTV.As can be seen, our reconstruction method was success-ful even though there are some smearing artefacts,caused by the rotation geometry, visible on the yz-plane.

Results With Real Data

The proposed method was tested with a micro-rota-tion set representing the nuclear envelope of the living

Fig. 5. Comparison of micro-rotation reconstruction using (a) EM, (b) EMGR, and (c) EMTV algo-rithms; (d) intensity profiles along y-axis for the three reconstructions. [Color figure can be viewed in theonline issue, which is available at www.interscience.wiley.com.]



cell, see Acquisition of Micro-Rotation Time-LapseImage Series section. This set consisted of 378 images,with 295 3 298 pixels (pixel size 127 3 127 nm)acquired using the pure micro-rotation imaging modal-ity. After the translation correction and cropping theregion of interest (see Brandt and Mevorah, 2006), thearea used was 150 3 150 pixels and the fundamentalperiod was 143 images per a rotation. Figure 7a shows

some example images from the set. The EMTV recon-struction (150 3 150 3 150 voxels) is shown in Figures7b and 7c; Figure 7b displays rotated slices of thereconstruction corresponding to Figure 7a whereasFigure 7c shows xy, xz, and yz-slices of the reconstruc-tion; Figure 8 displays its 3D-stereo view. As can beseen, our reconstruction method was successful eventhough the rotation geometry leads to some artefacts

Fig. 6. Micro-rotation reconstruction of synthetic cell using EMTValgorithm. (a) three rotating viewsof a original 3D cell; (b) raw micro-rotation images corresponding to slices in (a); (c) EMTV reconstruc-tion corresponding to slices in (a); (d) three orthogonal slices of the original cell; (e) EMGR and (f) EMTVreconstructions corresponding to slices in (d).

Fig. 7. Micro-rotation reconstruction of nuclear envelope of a real cell using EMTV algorithm. (a) sixraw images from micro-rotation set; (b) reconstructed slices corresponding to (a); (c) three reconstructedslices from the xy, xz, yz views, respectively.



visible on the planes parallel to the yz-plane, which issimilar to the reconstruction of the simulated cell inFigure 6f.

DISCUSSION

In this study, we have proposed the 3D reconstruc-tion and deblurring method for a rotating object frommicroscope images assuming that the motion estimatesand the PSF are available. Our method is based uponBayesian inversion theory where we have assumedPoisson noise in the observations and total variationstructure prior for the object density. The proposedmethod is an attractive way for approaching the recon-struction problem since the structure prior knowledgecan be taken into account in a statistical way. For solv-ing the minimization problem, we used the standardEM iterative algorithm and its extension to the totalvariation prior.

We evaluated the proposed method with both syn-thetic phantom and real wide-field data. The experi-ments show that the proposed method is promisingsince it is able to reconstruct the rotating object as wellas removes the blur in the original images. We alsoinvestigated the properties of the reconstruction, com-pared with the z-stack approach. The simulation showsthat the micro-rotation reconstruction on the xz and yz-views is better than the z-stack reconstruction while itgives worse resolution on the yz-view (orthogonal tothe rotation axis x). This implies that the resolution ofreconstruction depends on the geometry of the data ac-quisition. More precisely, the rotation geometry andthe elongation of the PSF make the resolution weakerin the view orthogonal to the measurement imageplanes. Even though the micro-rotation setting wasfound to have bounded resolution in the tangentialdirection of the rotation, the micro-rotation images canbe deblurred by our method which can be understoodas a model-based deconvolution approach.

In this study, the method was implemented in Mat-lab. The estimation of the reconstruction took about20 h for the real data. However, we expect that thecomputation times can be greatly reduced using paral-lel computing since each projection could be computedsimultaneously. Moreover, the developed algorithmsare generic and hence directly usable in other geome-tries in addition to the micro-rotation setting.

ACKNOWLEDGMENTS

We thank Scientific Volume Imaging for prov-iding the 3D cell phantom for testing and the PSFused in this study, both in the context of EU projectAUTOMATION.

APPENDIXAdjoint of Projection Model

Here, we derive the adjoint of the forward operatorAi in (4).

An operator A*i is said to be an adjoint of Ai, if andonly if, for all f 2 C3 and for allm 2 C2, it satisfies

Aif ;mh im¼ f ;A�i m

�f; ðA1Þ

where h., .im and h., .if are the inner products in themeasurement and object spaces, respectively. Theinner product between two measurement images m1

and m2 is defined as

m1;m2h im ¼Z Z

A

m1ðx; yÞm2ðx; yÞ dxdy; ðA2Þ

where the integral is taken over the image domain A.The inner product between two object volumes f1 and f2is defined as

f1; f2h if ¼Z Z Z

V

f1ðx; y; zÞf2ðx; y; zÞ dxdydz; ðA3Þ

where the integral is taken over the volume domain V.Thus we have

Aif ;mih im ¼Z Z

A

�Rif ðx;y;zÞ�hðx;y;zÞ

��z¼d

miðx;yÞdxdy

¼Z Z Z

V

�Rif ðx;y;zÞ �hðx;y;zÞ

�miðx;yÞdðz�dÞ

dxdydz

¼Z Z Z

V

Rif ðx;y;zÞ�hð�x;�y;�zÞ � �miðx;yÞdðz�dÞ� dxdydz

¼Z Z Z

V

f ðx;y;zÞ

3R�i

nhð�x;�y;�zÞ� �miðx;yÞdðz�dÞ�odxdydz

¼ f ;A�i mi

�f

ðA4Þ

where the adjoint operator of Ai* : C2 ? C3 is

ðA�i miÞðx;y;zÞ¼R�

i

nhð�x;�y;�zÞ� �miðx;yÞdðz�dÞ�o:

ðA5Þ

Fig. 8. 3D wall-eye stereo-view of the reconstruction. The horizontal artefact around the centre ofthe object is due to slightly inaccurate motion estimates. [Color figure can be viewed in the online issue,which is available at www.interscience.wiley.com.]



REFERENCES

Agard DA, Hiraoka Y, Shaw PJ, Sedat JW. 1989. Fluorescence micros-copy in three dimensions. Methods Cell Biol 30:353–377.

Anconelli B, Bertero M, Boccacci P, Desider G, Carbillet M, Lanteri H.2006. Deconvolution of multiple images with high dynamic rangeand an application to LBT LINC-NIRVANA. Astronomy Astrophys460:349–355.

Bradl J, Rinke B, Schneider B, Edelmann P, Krieger H, Hausmann H,Cremer C. 1996. Resolution improvement in 3-D microscopy byobject tilting. Microsc Anal 44:99–111.

Brandt S, Heikkonen J, Engelhardt P. 2001a. Automatic alignmentof transmission electron microscope images using markers. In:Proceedings of the 12th Scandinavian Conference on Image Analy-sis. Bergen, Norway. pp. 33–40.

Brandt S, Heikkonen J, Engelhardt P. 2001b. ‘Multiphase method forautomatic alignment of transmission electron microscope imagesusing markers’. J Struct Biol 133:10–22.

Brandt SS. 2006. Markerless alignment in electron tomography. In:Frank J, editor. Electron tomography, 2nd ed. New York: Springer.

Brandt SS, Kolehmainen V. 2007. Structure-from-motion without cor-respondence from tomographic projections by bayesian inversiontheory. IEEE Trans Med Imaging 26:238–248.

Brandt SS, Mevorah M. 2006. Camera motion recovery without corre-pondence from micro-rotation sets in wide-field light microscopy.In: Proceedings of Statistical Methods in Multi-Image and VideoProcessing Workshop, in conjuction with ECCV 2006, Graz, Austria.

Brandt SS, Ziese U. 2006. Automatic TEM image alignment by trifo-cal geometry. J Microsc 222:1–14.

Carrington WA, Lynch RM, Moore ED, Isenberg G, Fogarty KE, FayFS. 1995. Superresolution three-dimensional images of fluorescencein cells with minimal light exposure. Science 268:1483–1487.

Cogswell CJ, Larkin KG, Klemm HU. 1996. Fluorescence microto-mography: Multi-angle image acquisition and 3D digital recon-struction. Proc SPIE 2655:109–115.

Csiszar I. 1991. Why least squares and maximum entropy? An axio-matic approach to inference for linear inverse problems. Ann Stat19:2032–2066.

Dempster A, Laird N, Rubin D. 1977. Maximum likelihood fromincomplete data via the EM algorithm. J R Stat Soc Ser B 39:1–38.

Dey N, Blanc-Fraud L, Zimmer C, Kam Z, Roux P, Olivo-Marin J, Zer-ubia J. 2006. Richardson-Lucy algorithm with total variation regu-larization for 3D confocal microscope deconvolution. Microsc ResTechn 69:260–266.

Engelhardt P. 2000. Electron tomography of chromosome structure.In Encyclopedia of analytical chemistry, Vol. 6. New York: Chiches-ter Wiley.

Fay FS, Carrington W, Fogarty KE. 1989. Three-dimensional molecu-lar distribution in single cells analysed using the digital imagingmicroscope. J Microsc 153:133–149.

Fuhr G, Muller T, Schnelle T, Hagedorn R, Voigt A, Fiedler S, ArnoldWM, Zimmermann U, Wagner B, Heuberger A. 1994. Radio-fre-quency microtools for particle and live cell manipulation. Naturwis-sencschaften 81:528–535.

Green PJ. 1990. Bayesian reconstructions from emission tomographydata using a modified EM algorithm. IEEE Trans Med Imaging9:84–93.

Heintzmann R, Cremer C. 2002. Axial tomographic confocal fluores-cence microscopy. J Microsc 206:7–23.

Holmes TJ. 1989. Expectation-maximization restoration of band-lim-ited, truncated point-process intensities with application in micros-copy. J Optical Soc Am A 6:1006–1014.

Holmes TJ, Bhattacharyya S, Cooper JA, Hanzel D, KrishnamurthiV, Lin W, Roysam B, Szarowski DH, Turner JN. 1995. Light-micro-scopic images reconstructed by maximum likelihood deconvolution,In: Pawley JB, editor. Handbook of biological confocal microscopy,2nd ed. New York: Plenum Press.

Kaipio J, Somersalo E. 2004. Statistical and computational inverseproblems. Berlin: Springer Verlag.

Kolehmainen V, Siltanen S, Jarvenpaa S, Kaipio J, Koistinen P, Las-sas M, Pirttila J, Somersalo E. 2003. Statistical inversion for medi-cal x-ray tomography with few radiographs. II. Application to den-tal radiology. Phys Med Biol 48:1465–1490.

Korlach J, Reichle C, Muller T, Schnelle T, Web WW. 2005. Trapping,deformation, and rotation of giant unilamellar vesicles in octodedielectrophoretic field cage. Biophy J 89:554–562.

Laksameethanasan D, Brandt SS, Engelhardt P. 2006. A three-dimensional Bayesian reconstruction method with the point spreadfunction for micro-rotation sequences in wide-field microscopy. In:Proceedings of IEEE International Symposium on BiomedicalImaging, Arlington, VA, USA, pp. 1276–1279.

Lewitt RM. 1990. Multidimensional digital image representationsusing generalized kaiserbessel window functions. J Opt Soc Am A7:1834–1846.

Lewitt RM. 1992. Alternatives to voxels for image representation initerative reconstruction algorithms. Phys Med Biol 37:705–716.

Lizundia R, Sengmanivong L, Guergnon J, Muller T, Schnelle T,Langsley G, Shorte SL. 2005. Using micro-rotation imaging tostudy jnk-mediated cell survival in theileria parva-infected b-lym-phocytes. Parasitology 130:1–7.

Mueller K. 1998. Fast and accurate three-dimensional reconstructionfrom cone-beam projection data using algebraic methods. PhD the-sis, The Ohio State University.

Muller T, Gradl G, Howitz S, Shirley S, Schnelle T, Fuhr G. 1999. A3D-microelectrode system for handling and caging single cells andparticles. Biosens Bioelectron 14:247–256.

Renaud O, Heintzmann R, Saez-Cirion A, Schnelle T, Muller T, ShorteSL. 2007. A system and methodology for high-content visual screen-ing of individual intact living cells in suspension. Proc SPIE6441:64410.

Rudin L, Osher S, Fatemi E. 1992. Nonlinear total variation basednoise removal algorithms. Physica D 60:259–268.

Scheres SHW, Gao H, Valle M, Herman GT, Eggermont PPB, FrankJ, Carazo J-M. 2007. Disentangling conformational states ofmacromolecules in 3D-EM through likelihood optimization. NatMethods 4:27–29.

Schnelle T, Hagedorn R, Fuhr G, Fiedler S, Muller T. 1993. Three-dimensional electric field traps for manipulation of cells-calculationand experimental verification. Biochim Biophys Acta 1157:127–140.

Schnelle T, Muller T, Gradl G, Shirley SG, Fuhr G. 2000. Dielectro-phoretic manipulation of suspended submicron particles. Electro-phoresis 21:66–73.

Shaw PJ. 1995. Comparison of wide-field/deconvolution and confocalmicroscopy for three-dimensional imaging. In: Pawley JB, editor.Handbook of biological confocal microscopy, 2nd ed. New York: Ple-num Press.

Shaw PJ, Agard DA, Hiraoka Y, Sedat JW. 1989. Tilted view re-construction in optical microscopy. Three-dimensional reconstruc-tion of drosophila melanogaster embryo nuclei. Biophys J 55:101–110.

Shorte SL, Muller T, Schnelle T. DI200243, Invention: method anddevice for 3 dimensional imaging of suspended micro-objects provid-ing high-resolution microscopy. Patent Number; EP 02 292 658.8;file date: 25 October 2002.

Skoglund U, Ofverstedt LG, Burnett R, Bricogne G. 1996. Maximum-entropy three-dimensional reconstruction with deconvolution of thecontrast transfer function: A test application with adenovirus.J Struct Biol 117:173–188.

van Kempen GMP, van Vliet LJ, Verveer PJ, van der Voort HTM.1997. A quantitative comparison of image restoration methods forconfocal microscopy. J Microsc 185:354–365.

Vardi Y, Shepp LA, Kaufman L. 1985. A statistical model for positronemission tomography. J Am Stat Assoc 80:8–37.

Vaughan OA, Alvarez-Reyes M, Bridger J, Broers J, Ramaekers F,Morris G, Whitfield W, Hutchison CJ, Wehnert M. 2001. Bothemerin and lamin C depend on lamin A for localization at the nu-clear envelope. J Cell Sci 114:2577–2590.

Verveer PJ, Jovin TM. 1999. Improved restoration from multipleimages of a single object: Application to fluorescence microscopy.Appl Optics 37:6240–6246.

Verveer PJ, Gemkow MJ, Jovin TM. 1999. A comparison of image res-toration approaches applied to three-dimensional confocal andwide-field fluorescence microscopy. J Microsc 193:50–61.

Vicidomini G, Mondal PP, Diaspro A. 2006. Fuzzy logic and maximuma-posteriori based image restoration for confocal microscopy. OpticsLett 31:3582–3584.



Documents

A Bayesian reconstruction method for micro-rotation imaging in light microscopy