Neuroinform (2011) 9:247–261DOI 10.1007/s12021-011-9120-3

ORIGINAL ARTICLE

Automated Reconstruction of Neuronal MorphologyBased on Local Geometrical and Global Structural Models

Ting Zhao · Jun Xie · Fernando Amat · Nathan Clack ·Parvez Ahammad · Hanchuan Peng · Fuhui Long · Eugene Myers

Published online: 6 May 2011© The Author(s) 2011. This article is published with open access at Springerlink.com

Abstract Digital reconstruction of neurons frommicroscope images is an important and challengingproblem in neuroscience. In this paper, we proposea model-based method to tackle this problem. Wefirst formulate a model structure, then develop analgorithm for computing it by carefully taking intoaccount morphological characteristics of neurons, aswell as the image properties under typical imagingprotocols. The method has been tested on the datasets used in the DIADEM competition and producedpromising results for four out of the five data sets.

Keywords DIADEM · Neuron tracing ·Tube models · Tree structure reconstruction ·3D microscopy

Introduction

Progress in microscopy has generated a need forautomated computational analysis and research intomethods for such analyses has been termed bioimage

T. Zhao (B)Qiushi Academy for Advanced Studies, Zhejiang University,38 ZheDa Road, Hangzhou, 310027, Chinae-mail: tingzhao@gmail.com

J. Xie · F. Amat · N. Clack · P. Ahammad · H. Peng ·F. Long · E. MyersHHMI Janelia Farm Research Campus, 19700 Helix Dr.,Ashburn, VA 20147, USA

F. Longe-mail: longf@janelia.hhmi.org

informatics, e.g. Peng (2008). In this paper we addressa specific problem in this field, that of reconstructingneuronal morphologies from 3D microscopy images. Itcan be stated as follows:

Given a 3D image that is acquired via a com-mon 3D microscopy protocol, automatically re-construct the morphology of the neurons presentin the image.

This specific problem is important because a goodsolution will have a significant impact on neuro-science. The ultimate goal of neuroscience is to un-derstand how nervous systems work. This cannot beachieved without obtaining the structure of a realneuronal network, which reveals how neurons areconnected. This in turn requires the extraction of mor-phologies of individual neurons in the system. How-ever, biologists have to rely on time-consuming manualor semi-manual methods to turn the images into mor-phological models. Given the number of neurons in anervous system, we can easily foresee the emergenceof a bottleneck. Therefore, an accurate fully-automatedreconstruction method is critical to the advancement ofneuroscience.

The importance of the problem has led to quite a fewattempts at developing automated methods for neuronreconstruction (Meijering 2010). To understand thesemethods, we need to remember two important commonproperties of the morphology of a neuron:

1. The fibers of a neuron form a tree.2. At the typical resolution of a light microscope,

a fiber of a neuron often has a smooth tubularshape.

248 Neuroinform (2011) 9:247–261

The most successful automated methods take advan-tages of one or both of these two properties. They canbe further classified into two main categories:

Local Tracing A method in this category usually startsfrom a seed, which can be located automatically ormanually, and traces where a neuronal fiber goes inthe given image. The direction of tracing is determinedbased on the signal around the current location. Arepresentative of this class is the method that wasproposed in Al-Kofahi et al. (2002). The authors usedtemplate fitting to determine the direction of tracing.The template they used consists of four parallel edgedetectors, each of which locates a part of the branchboundary with a greedy search. A local tracing methodproduces one branch at a time and does not directlytrace through branch points where a fiber forks into twoor more continuing fibers. So a separate branch pointdetection method is required to complete the recon-struction (Al-Kofahi et al. 2007). Model-free strategiesof local tracing, such as Rayburst Sampling (Rodriguezet al. 2006) and Voxel Scooping (Rodriguez et al. 2009),can trace multiple branches with one seed. However,the quality of the tracing relies heavily on the existenceof a preprocessing step that accurately separates fore-ground and background.

Global Skeletonization In contrast to local tracingmethods, the other category of methods extract skele-tons of neurons based on the global signal distributionin an image. One straightforward way of doing thisis to use a selected segmentation algorithm to turnthe image into a binary foreground/background form,and then apply a 3D binary skeletonization algorithm(Dima et al. 2002; Weaver et al. 2004; Evers et al.2005; Narro et al. 2007). But this has not always beena good choice because of the difficulty of the first step,binarization. A Hessian-based vesselness measurementhas been used to enhance line structures in imagesand improve skeletonization (Abdul-Karim et al. 2005;Yuan et al. 2009; Vasilkoski and Stepanyants 2009).But it requires predefined scales and is computationallyexpensive when the number of scales is large. In analternate strategy, Dijkstra’s shortest path algorithmhas been adopted to extract skeletons directly fromgray-scale images (Zhang et al. 2007; Peng et al. 2010b).This algorithm was applied to find the path that givesthe shortest geodesic distance between two points. Aslong as the geodesic distance is reasonably defined,the path found is indeed the one desired. Since Dijk-stra’s algorithm provides the global optimal solution,the method is robust when the seed points are welllocated (Meijering et al. 2004; Xie et al. 2010; Peng et al.

2010a). But such a method is not reliable when thereare multiple neurons in an image and it requires manualselection of the termini to trace between.

Both types of methods have been shown to be usefulfor neuron tracing. However, these automated resultsstill require significant human curation and correctionin order to be perfect. In this paper our goal is todevelop a better method for reconstructing neuronswhose morphology adheres to the two common prop-erties mentioned above. For tracing neurons that dono show smooth neurites in the images (e.g. neuronsimaged by electron microscopy), we would expect verydifferent methods to work and it is beyond the scopeof this paper. So we employ the idea of local tracing,and then refine the result with the use of shortest paths.Especially, we formulate the procedure of model basedtracing, in the spirit of Al-Kofahi et al. (2002) but byrealizing it with a formal tube model, improve per-formance significantly. A general framework of tree-like morphology reconstruction from the resulting fibersegments is also proposed. Based on the framework,we have developed an automated pipeline to produce afinal reconstruction from a raw or preprocessed image.The method was tested on the DIADEM data sets(Brown et al. 2011). The main purpose of this paperis to describe the method that brought us to the DIA-DEM final, so we do not formally evaluate it againstthe many algorithmic ideas that have been exploredpreviously.

Methods

Model-based Neurite Fiber Tracing in 3D

The shape of a neurite fiber can be approximated as aseries of circular cross sections along a continuous curve(Fig. 1) that are deformed to an ellipse along the axialdirection by the anisotropy of the point spread function(PSF) of the microscope. More formally, its surfaceN(t, u) for parameters t, u ∈ [0, 1] can be defined as:

N(t, u) = C(t) + O(u, t)R(t)A (1)

where C(t) = ( x(t), y(t), z(t) ) is a continuous anddifferentiable curve, O(u, t) = (r(t) cos 2πu, r(t) sin2πu, 0) is a circle of radius r(t) in the z = 0 plane,

R(t) =

⎛⎜⎜⎝

ry × rz

ry = k × rz

rz = �C(t)‖�C(t)‖

⎞⎟⎟⎠

Neuroinform (2011) 9:247–261 249

Fig. 1 Neurite model andexample. a is a cartoon toshow that the model iscomposed of a set of crosssections along a principle axis.b–e are neurites from the realimages of the 4 DIADEMdatasets. b, c are examples ofdarkfield microscopy andd, e are examples ofbrightfield microscopy. Referto the “Results” section formore details about theimaging conditions

5

0

5

10

05

1015

5

0

5

10

y

x

z

ba

c d e

is a fixed matrix at each value of t that maps the circlefrom the z = 0 plane into a plane perpendicular to thetangent of the curve C,1 and

A =⎛⎝

1 0 00 1 00 0 a(t)

⎞⎠

stretches or squeezes the z-dimension by the parametera(t) > 0 reflecting the asymmetry of the PSF and thesampling rate.

When a neurite is smooth enough, which is oftentrue in real data, we can approximate a neurite segmentas a discrete series of elliptical cylinders. So one wayto reliably detect if a point (x, y, z) is on the neurite’sprinciple axis C(t), is to see if there is a good fit between

1The product for ry can give a singular 0-vector when rz = k. Inthis case, its value is the limit of the expression as the singularityis reached.

the signal and a cylindrical model centered at (x, y, z)

of fixed height h reflecting the “mesh size” of theproposed series. Such an elliptic cylinder has four freeparameters: its radius r, two Euler angles φ and ψ thatorient it along a vector in space, and the anisotropycalibration a in the axial direction. This also defines adeformable model of a canonical cylinder, which hasunit radius and orientation parallel to the axial or z-axis. Therefore designing a more reasonable filter-likestructure for the canonical cylinder will allow us to esti-mate the parameters more efficiently and accurately.

A 3D Cylinder Filter

To evaluate the fit of a cylinder model to the data, wederived a 3D filter for a cylinder, which we also calla template. The fitting score between the model andthe data is defined as the the correlation between theintensity data and the template. Due to its averagingeffect, this simple defintion can effectively suppressthe disturbance of imaging noise or artifcacts, even

250 Neuroinform (2011) 9:247–261

if our modeling procedure does not consider imagingnoise or artifcacts explcitly. Our template is designedfor darkfield images, where signal is brighter thanbackground. One can simply invert the template forbrightfield images.

In cross-section, a neurite fiber looks like a Gaussian-diffused spot possibly scaled along the axial direction,so we use an elliptical Mexican Hat filter (Laplacianof a Gaussian) to convolve with the signal along thesymmetric axis of the cylinder. The choice of usinga Mexican Hat filter is similar to the one in Schmittet al. (2004), except that the shape of our filter can beelliptical. So our canonical unit vector is given by:

U(x, y, z) = (1 − (

x2 + y2))

e−(x2+y2) (2)

where z ∈ [−h/2, h/2]. To obtain the space of all pos-sible cylinder templates, one can scale U in x and y byradius r, then rotate it in any manner desired, and finalscale it along z (making it elliptical) by factor a.

Of course, the support of the filter must be finitenot only for computational purposes, but also because,while the model demands that background surround(most) of the neurite fiber, making this zone too largemeans that fibers passing very near by will disruptthe fit of the filter. So for a given filter instance, re-gardless of a and r, we limit the support of the filterto τ pixels beyond the zero-crossing of the MexicanHat. The second consideration, is that for very thickneurons, their intensity profile often has a flattenedtop due to saturation of the signal in the center (e.g.Fig. 2). For such a profile a slight deformation of thetemplate induces little or no change in score, so weempirically found that regularizing the template familyby multiplying the positive part of the Mexican Hat by(ar2)1/4 solves the problem.

As long as the neurite fiber section under considera-tion is surrounded by background, the filter essentially

gives a large score for any cross section surrounding thebright region. To rectify this the cross correlation scoreof the filter convolved with the signal is normalizedby dividing by the integral of the absolute value ofthe filter over its support. In the optimization step tofollow, the r and a parameters are varied along withthe orientation angles at a specified center point inspace to those for which the template yields the largestnormalized score when convolved with the signal.

Note that h and τ are the two parameters that needto be chosen prior to running our method for any partic-ular image acquisition conditions and they are primarilya function of pixel size. For example, for most imagesat 40X h = 10 and τ = 2 suffice. In our experience themethod is robust to these parameter choices and theyremain fixed for any set of images acquired with thesame microscope and imaging protocol.

Tracing a Neurite Fiber

One could use a greedy search to optimize the fourfree parameters (two orientation parameters and twoscale parameters) of the template when it is placed ata given point in space. But this is less appropriate forsuch a large number of parameters because search timeincreases exponentially in the number of parameters.If the score function is smooth and has a single peak,gradient descent is certainly preferred. Fortunately, ourscoring function does have this property in a fairly largezone of the parameter space around the true optimum.Moreover, the symmetry of the function makes it wellfit by a quadratic function within this zone, so as longas the template is placed in a reasonable initial po-sition (which could for example be determined by acoarse discrete sampling of the parameter space), onecan use the conjugate gradient descent method, whichhas been shown to be the best for optimizing general

Fig. 2 Example of a thickneuron cross-section wherethe intensity profile isflattened in the center. a isthe actual image crosssection. b is a plot of theintensity profile of the crosssection in a

010

2030

4050

010

2030

400

50

100

150

200

250

300

xy

inte

nsity

a b

Neuroinform (2011) 9:247–261 251

quadratic functions. In our implementation, we usedthe Polak–Ribière conjugate gradient descent method(Wright and Nocedal 2006). Results showed that it issignificantly better than steepest gradient descent (e.g.Fig. 3).

There is no closed form expression for calculatingthe gradients because of the lack of a closed formdescribing the image. So numerical estimation is nec-essary, and as per standard practice we use the approx-imation (S(u + �u) − S(u − �u))/(2�u) for the partialderivative ∂S/∂u of a variable u.

After fitting a cylinder at a given position, we haveonly modeled a small section of the target neurite fiber.To cover the whole neurite, we use a strategy similar tothe one introduced in Al-Kofahi et al. (2002) to traversethe entire fiber in mesh steps of size h/2. Once wehave fitted a cylinder ci, we then place a cylinder ci+1

with its center at one end of ci’s symmetric axis in thesame orientation and of the same cross-section shape,and then optimize the orientation and shape of ci+1

from this starting position. In this way, given an initialcylinder c0, one can walk in both direction to traversea fiber. The pseudo-code for tracing in one direction issimply as follows.i ← 0repeat

ci+1 ← Axial translation of ci by h/2 pixelsOptimize ci+1

i ← i + 1until ci has a score below the threshold or ci hits a

previously traced region

In the next subsection, we discuss how to arrive at aset of seed cylinders c0 for an entire image.

0 5 10 15 20 25 30 35 40 45 50600

700

800

900

1000

1100

1200

1300

1400

Number of Iteration

Sco

re

ConjugateSteepest

Fig. 3 Conjugate gradient descent reaches the optimal scorefaster than steepest gradient descent

Tracing All Neurite Fibers

In the previous section we showed how to trace aneurite fiber given an initial filter fit c0. To find aset of initial filters, one for each fiber, we developeda seed detection method to locate all locations andapproximate orientations of points on the curve of aneurite based on two image features. The first feature ispixel intensity: when a pixel is brighter it is more likelyon a neurite. The second feature is local geometry. Weprefer a seed to be on a line structure. This can bemeasured by examining the eigenvalues of a Hessianmatrix (Sato et al. 1998) computed at a point, typicallyat a number of scales. But the calculation of a Hessianmatrix is expensive in both time and space, so we onlydo the calculation once at the smallest scale (3 × 3 × 3),and assume that thick fibers are bright enough that theywill be picked up by the intensity feature. For each ofthese two features a threshold is computed with thetriangle method over the histogram of the local maximaof feature scores at all pixels. Any pixel that has eitherof its two features above threshold is considered to bean on pixel in a binarization of the image that roughlycovers all the neurites or at least a portion of everyneurite fiber.

Since we want to place our initial filter fits c0 at thecenter of a neurite, we consider as possible seed loca-tions those points that are local maxima of the distancemap of the binary image. Obviously, many possibleseeds can be reported for a given neurite fiber whereasonly one is needed. So we optimize the fit of a cylindertemplate at every possible seed. The optimization ateach point is initialized with the best-scoring τ -pixelcircular cylinder over a fixed set of orientation samplingthe space of all possible orientations. We then sort theseeds in the order of their optimized model scores forc0. We start with the best seed and trace its neuritefiber, removing from consideration any seeds that arecovered by the tracing. We then pick the next bestremaining seed and trace it, and continue greedily inthis fashion until all the seeds are exhausted. In this waywe build a model of (almost) every neurite fiber in theimage. The caveats are that if the signal is sufficientlydim in parts then the fiber may not be spanned leadingto the occasional broken fiber, and in some cases thetracing moves directly through a branch point, but thisis benign in that it is technically the fusion of two fibersthat should ultimately be fused anyway.

Reconstruction of Neuronal Morphology

Once the set of individual neurite fibers or fragmentarytrace have been obtained for an image, they can then

252 Neuroinform (2011) 9:247–261

be assembled to form a neuronal tree structure. Thereare typically many ways to join them together, but byprohibiting the formation of cycles during the assemblyprocess we guarantee that the final structure is a tree.This can be formulated as a graph problem.

Suppose we have a set of neurite fibers, {Nk}Nk=1. The

goal is to determine how they are connected. We firstcreate a neurite graph, in wich each node is a neuriteand each edge indicates a possible connection betweentwo neurites. In contrast to a usual graph, a node in aneurite graph has three parts, two ends and a body. Soan edge between two nodes in this undirected graphmust also specify at each node whether it connectsto an end or the body. The connection pattern of anedge can be either end-to-end or end-to-body (Fig. 4),but never body-to-body. Moreover, only one edge isallowed between two neurites. Our problem, in theframework of this graph, is to find a minimum weightspanning tree in the case one neuron is under view and

all the Nk are true positives. The whole procedure isillustrated in Fig. 5. In our method, we do not considerthe connection between a pair of neurites that are toofar away from each other. The distance threshold is setto 20 pixels, which is twice as long as the height ofthe cylinder templates. Any two neurites that have adistance larger than the threshold are supposed to befrom different neurons.

So the design issue is how to assign a cost to eachpossible edge, which will join fibers if selected to be inthe result. The cost must be such that the smaller thevalue the more desirable the edge for inclusion. One ofthe simplest schemes for the cost of connecting Ni toN j is the distance between the center of the end of Ni

and the center of the end of N j in the case of an end-to-end edge, and the distance of the center of the endof Ni to the nearest point on the surface of the body ofN j in the case of an end-to-body edge. This definitionis quite natural because two fibers tend to be close

Fig. 4 Graph of neurites. Aneurite is modeled as a 3-partnode a, which can formend-to-end connection b orend-to-body connection cwith another node. Realexamples of end-to-endconnection and end-to-bodyconnection are shown in dand e respectively. Colorsindicate different neurites

BodyEnd EndEnd

End

Body

a

b

c

d e

Neuroinform (2011) 9:247–261 253

Initialize G as an empty neurite graph

Has every neurite Ni been

checked?

Has everyneurite Nj (i=j)been checked?

Calculate the connection cost

between Ni and Nj

Calculate the shortest distance between the ends of Ni and the

surface of Nj

Is the distance no greater than 20 pixels?

Add the edge between Ni and Nj

to the graph G

Yes

No

No

Yes

Build MST on G

Yes

No

/

Fig. 5 Flowchart of the tree reconstruction procedure

to each other when they join to form a branch pointin the image. However, when such gaps start being aslarge as the nearest fiber to one of the fibers underconsideration, a wrong choice can be made. This doeshappen with some low frequency. For example, therewas one such occurrence for a traced neuron which hasabout 40 branches. So other cues must be considered.

One such strong cue to tell if two tubes are truly con-nected is the strength of the image signal between them.If we can find a path from one neurite fiber end or bodyto the other endpoint along a path of bright pixels, thenit is highly likely that the two tubes connect to each

other. Therefore we added another measurement, thegeodesic distance, as an option. The geodesic distancebetween two points x0 and x1 is defined as

geog(x0, x1) = minc∈{all possible paths}

∫c

g[I(c(t))] ||dc(t)|| (3)

where c(0) = x0, c(1) = x1, ||dc(t)|| =√dc2

x(t) + dc2y(t) + dc2

z(t) is the differential of the

arc length of c(t), and I(x) is the image intensity at x.The function g defines how image intensity contributesto the geodesic distance. In our study, we expect theshortest path c, to be on the foreground. This can beexpressed by a sigmoid function

g(x) = 1

1 + ex−αβ

(4)

where α and β are selected to reflect the separabilitybetween the foreground and background and we do nothave to set them arbitrarily. For example, we can takethe signal inside a tube as foreground and those aroundthe tube as background and calculate their averagevalues, which are denoted as c f and cb , respectively.If we want g(c f ) = 0.99 and g(cb ) = 0.01, it is easy tosolve for α and β yielding α = c f +cb

2 and β = c f −cb

2ln(99).

A neurite graph becomes a typical weighted undi-rected graph if we ignore the internal structure ofits nodes and the connection pattern of its edges. Insome sense, the minimal spanning tree of this graphmaximizes the likelihood of being the appropriate treereconstruction if the cost function on edges properly

X

Y

Z

Fig. 6 An example of crossover from real data. The top imageand the bottom image are showing the same two neurites pro-jected onto X-Y plane and X-Z plane respectively. The color linesindicate how the two neurites cross

254 Neuroinform (2011) 9:247–261

Fig. 7 Crossover a in aneurite graph b. The solutionc, d gives the minimal sum ofthe angles between matchedneurites. The red and greenlines in c and d show how theneurites are merged aftermatching

1

2

3

4

1 2

34

1 2

34

1

2

3

4

a

c

b

d

reflects the likelihood that two neurites involved have adirect connection. Unfortunately there is the case thatwhen two neuron fibers cross each other very closely,even the quite reasonable geodesic cost function failsto meet this criterion.

When one fiber passes another very closely, espe-cially in the axial z-dimension where resolution is lowerwith most microscopes, their signal may overlap to theextent that the geodesic cost function (falsely) indicatesthat they should be joined (Fig. 6). In other words,the overlap pattern cannot be identified by the MSTalgorithm. We call such a pattern a crossover, and itmust be distinguished from the real fusions that needto occur at branch points in a neuron. In the neuritegraph, a crossover has one of two special signaturesdepending on whether the fiber tracing passes throughthe crossover region or not. If the tracing does not pass,the pattern will be pairwise end-to-end connectionsamong four or more nodes (Fig. 7). Otherwise, thepattern will be the connections from the ends of two ormore neurites to the body of another neurite (imaginefibers 1 and 3 are already joined into a single fiber inFig. 7). For the latter case, the problem is compoundedas the local tracing that went through the crossoverregion may have “jumped tracks” to another fiber. Soconservatively, we break any fibers that pass throughthe connection region. This has the further advantageof reducing the problem to the first case where allpossible connections are end-to-end.

Crossover cannot be solved directly with minimumspanning tree as it will tend to just join all the fiberstogether (in the case that multiple neurons are present),or join fibers depending on the signal strength betweenthem which is not a good indicator of which fiber shouldbe joined with which. A better indicator is the anglebetween two neurite end vectors: if there is no or littlechange in direction as one goes from one fiber to thenext, then it is likely that the two fibers should bejoined. So for the fibers that end at the crossover regionwe find the best set of pairings based on minimizing thetotal of the join angles using the Hungarian algorithm.These neurites are then fused. The cross-over processtakes place prior to the greedy minimum spanning treealgorithm (MST).

Results

We tested the algorithm on 4 of the 5 data sets2 fromthe Digital Reconstruction of Axonal and Dentritic

2The excluded dataset, Neuromuscular Projectio Fibers, containsneurons that have irregular surfaces and uneven internal stain-ings. Therefore we used a model-free region growing method toprocess the dataset instead.

Neuroinform (2011) 9:247–261 255

Table 1 Summary of the datasets

Dataset CF HC NL OP

Species Rat Rat Mouse DrosophilaNervous system Cerebellar cortex Hippocampus CA3 Peripheral Olfactory bulbFiber type Axon terminals Dendrites and axons Axons AxonsMicroscopy Transmitted light brightfield Transmitted light brightfield 2-photon microscopy Confocal microscopyVoxel size 1 × 1 × 8.8 μm 0.217 × 0.217 × 0.33 μm 0.294 × 0.294 × 1 μm 0.33 × 0.33 × 1 μm

Fig. 8 The tracing result(green) of an CF image stackoverlaps with a slice of theoriginal stack. The result istranslated a little from itsoriginal position for betterview

Fig. 9 The tracing result(red) of an HC image stackoverlaps with a slice of theoriginal stack. The result istranslated a little from itsoriginal position for betterview

256 Neuroinform (2011) 9:247–261

Fig. 10 The tracing result(colored) of an NL imagestack overlaps with thevolume rendering of theoriginal stack. The result istranslated a little from itsoriginal position for betterview. Note that there is plentyof signal that has nocorresponding neuronalstructure. This is because thestructures are excluded fromthe specified neuron set,despite that the tracing didnot miss them

Morphology (DIADEM) competition (http://www.diademchallenge.org). Named after the stained neu-ron types, the 4 datasets are CF (Cerebellar ClimbingFibers) (Sugihara et al. 1999), HC (Hippocampal CA3Interneuron) (Calixto et al. 2008), NL (Neocortical

Layer6 Axons) (De Paola et al. 2006) and OP (Olfac-tory Projection Fibers) (Jefferis et al. 2007). The imag-ing protocol for each dataset is listed briefly in Table 1.The images of the datasets CF and HC are from bright-field microscopy and some special preprocessing steps,

Fig. 11 The tracing result(red) of an OP image stackoverlaps with the volumerendering of the originalstack. The result is translateda little from its originalposition for better view

Neuroinform (2011) 9:247–261 257

ba c d e

Fig. 12 An OP image stack was contaminated by different levels of Gaussian noise: a σ = 20, b σ = 40, c σ = 60, d σ = 80, e σ = 100

to be described, are necessary before applying modelfitting on them. For the NL dataset, an image containsmultiple neurons. Given the coordinates of a point oneach neuron, we built a single tree first and then cutthe edges with largest connection cost to get multipletrees. Also the crossover prior described in the previoussection was applied to the NL data only. The finalresults on all 4 datasets are illustrated in Fig. 8 (CF),Fig. 9 (HC), Fig. 10 (NL), Fig. 11 (OP).

Testing for Robustness to Noise We also tested ourmethod on different noise levels to show its strength.The testing images were created by adding Gaussiannoise to the OP image stack (Fig. 11). We used fivenoise levels, measured by the standard deviation ofthe noise distribution: σ = 20, σ = 40, σ = 60, σ = 80,σ = 100 where values in the image range from 0 to

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Noise Level

DIA

DE

M S

core

Our methodNeuroStudio

Fig. 13 The tracing results on different noise levels generatedby our method (solid line) are compared to the results obtainedfrom NeuroStudio (dash-dot line). It shows that our result is muchmore robust to the noise

255. The neuron becomes less visible when σ is larger(Fig. 12), making tracing progressively more difficult.To quantitatively evaluate how robust our method isto noise, we calculated the scores of tracing quality us-ing the DIADEM metric (http://www.diademchallenge.org/metric.html). The results were also compared withthose obtained from the free software NeuroStudio(Wearne et al. 2005), which was developed based onthe Rayburst Sampling tracing algorithm. As shown inFig. 13, our method produced reasonable result (score= 0.866) even when the noise level is at 100. In contrast,NeuroStudio’s performance degrades rapidly reachinga score of 0 when the noise is at 80. Moreover, ourmethod outperformed NeuroStudio significantly for allthe noise levels, including the case where there is nonoise.

Preprocessing for the CF Dataset In the CF imagesneurons are brown and nuclei are blue. In some ar-eas they overlap due to the nature of the bright-fieldpoint spread function which does not optically sectionin z. This overlap can not be resolved by extractinga certain color component as in some regions lightfrom both objects is present. We solved the problembased on a color mixing model. In this model, there isan average (bright) background B and there are twodifferent materials, m1 and m2, in the imaging fieldand each material absorbs light of a certain color, c1

and c2, respectively. At any voxel x, the total observedintensity I(x) is:

I(x) = B − [c1, c2] ·[

p1(x)

p2(x)

](5)

where pi(x) denotes the absorption contribution frommaterial mi. We estimate B as the average intensityof each frame which is a good estimate as the back-ground occupies most of each frame. c1 and c2 are cal-culated reliably by taking the average values of the two

258 Neuroinform (2011) 9:247–261

Fig. 14 The color componentcorresponding to neurons b isextracted from an image ofthe CF dataset a

ba

separable peaks in the color histogram of B − I. Oncewe have these three values, we compute the amount ofeach material at each voxel by solving Eq. 5 by linearregression. The result is shown in Fig. 14.

Preprocessing for the HC Dataset Much of the pre-processing required for this dataset was due to eitherpoor microscopy or poor stitching of the multiple 3Dstacks that made up each data set. In particular, thez-planes of a given tile were offset with respect toeach other and the luminence of each plane variedsignificantly. We do not know how these artifacts wereintroduced, but we perforce had to correct them.

First the images were converted to gray-scale for theneuron intensity using the color model scheme usedfor CF, save that here matters were particular easy asthere was only one material. Next the motion artifactswere corrected by splitting the full volume into a se-ries of subvolumes corresponding to the lateral tilingused to originally acquire the data. Each subvolume

was then processed independently. For each pair ofsuccessive z-planes, a lateral translation was estimatedas that maximizing the correlation between each edge-enhanced representations of the respective planes.Edge enhanced images were computed by filtering witha difference-of-boxes filter (box sizes 5 and 15 pixels)followed by masking with the original data binarizedaccording to the Ostu threshold. Linear interpolationwas used to align images according to the estimatedtranslation. Border regions were filled by clamping tothe edges. After motion correction, the volume was me-dian filtered along z (window size 1 × 1 × 13) in orderto correct for intensity fluctuations between planes. Thewindow size was chosen to approximate the observedextent of a single neurite image along the z direction.Finally, the vertical blur from strongly labeled objects,such as the cell soma, added an approximate constantsignal to all planes above and below the object. Toremove this signal, we estimated it as the median planealong z and subtracted that from each plane clamping

Fig. 15 Preprocessing resultfor the HC dataset: a Originalimage; b Processed image

ba

Neuroinform (2011) 9:247–261 259

negative values to zero. An example of processing re-sult is shown in Fig. 15.

Discussion

There have been numerous articles on methods forthe automated reconstruction of neuronal morphology.However, the quest for better, more powerful recon-struction algorithms remains. Existing methods fall intothe two main classes of local tracing and global skele-tonization. Both paradigms have proved to be useful,but we feel their potential has not been fully explored.In this paper, we refined a local tracing approachby designing a more sophisticated template for whichwe carefully optimize its orientation and positioningas the tracing progresses. Combining it with a novelneurite fiber graph model, we have developed a ro-bust neuron reconstruction method. The method workson various types of images with few modifications.Even though the method was originally designed fordarkfield images, it generates reliable results for pre-processed brightfield images as well.

We proposed a geometrical model to formulate theshape of a neurite fiber. From the model, we deriveda tracing algorithm based on deformable templates.We also found that the template parameters can beestimated efficiently by conjugate gradient descent. Asa result the method can deal with neurites with varioussizes and produce accurate measurement of the surfaceof a neuron.

One main advantage of our model-based approachis its robustness to noise. This is due to the intrinsicaveraging effect of the score calculation. So it doesnot need an additional noise reduction method to getacceptable results. This is especially well-suited forinteractive tracing, which allows a user to extract aneurite with one click on raw data.

We have also defined a special graph model to derivealgorithms for finding the most likely tree structuregiven a set of neurite fibers. The graph model is flexibleand allowed us to add useful priors. For example, wehave shown that adding a prior for crossover patternscan improve the reconstruction results of the NL dataset dramatically.

The positive feature of our template model is thatgiven sufficient support h it clearly distinguishes neuritefrom non-neurite artifacts. That is it has a low falsepositive rate of identification (an example is shown inFig. 16). The problem is that it does not detect short fea-tures that are at fewer than h pixels in size such as shortside branches, or sometimes it will erroneously fit themodel over a small break that actually separates two

Fig. 16 Our method can avoid tracing blobs (marked by greenarrows) that are as bright as some neurites

fibers. To further improve the method, its limitation inthese circumstances must be addressed (Fig. 17) and wesuggest approaches as follows:

Dense Branches When many branches are present ina small region, such as the end of an axonal projection,the fitting template may jump from one branch tip toanother or fail to fit on a short segment between twobranch points. Both mistakes will result in topologicalerrors. The first type of mistake is hard to fix because ofthe loop structures formed in the image due to the lim-ited resolution. We may have to correct them manuallyor add topological constraints for reconstruction. Thesecond one may be less a problem because it is easierto detect the missing signal than to correct an over-fitting. Once we find a bright region not being includedin the reconstruction, we can extract it and estimate theorientation and size by principle component analysis ifthe cylinder model does not fit.

Broken Branch Signal Although a neurite is contin-uous, we may only see a sequence of isolated brightdots when it is imaged. This can be caused by unevenstaining or uneven distribution of GFP particles and isgenerally indicative of poor sample preparation. Our

260 Neuroinform (2011) 9:247–261

Fig. 17 Scenarios causingproblems for our tracingmethod: a The two branchtips are so close that thetracing jumps from one to theother; b The cylinder modelmay fail to fit on a shortbranch between two closebranch points; c Twoexamples of broken neuritesignal on which the tracingwill stop too early at anyseeding point

ba

c

tube model will generally fail on data of this qualitybecause the assumption of continuity is no longer valid.One could treat this as a special case and use a differenttracing method, such as the one proposed in Peng et al.(2010a) to attempt to span such dark zones.

Information Sharing Statement

All codes for our neuron tracing pipeline are availablein open source and can be found on the DIADEMwebsite (http://diademchallenge.org).

Acknowledgements The authors thank Shiv Vitaladevuni,Mark Longair, and Saket Navlakha for helpful discussions. Thiswork was supported by Howard Hughes Medical Institute.

Open Access This article is distributed under the terms of theCreative Commons Attribution Noncommercial License whichpermits any noncommercial use, distribution, and reproductionin any medium, provided the original author(s) and source arecredited.

References

Abdul-Karim, M., Roysam, B., Dowell-Mesfin, N., Jeromin, A.,Yuksel, M., & Kalyanaraman, S. (2005). Automatic selectionof parameters for vessel/neurite segmentation algorithms.IEEE Transactions on Image Processing, 14(9), 1338–1350.

Al-Kofahi, K., Lasek, S., Szarowski, D., Pace, C., Nagy, G.,Turner, J., et al. (2002). Rapid automated three-dimensionaltracing of neurons from confocal image stacks. IEEE Trans-actions on Information Technology in Biomedicine, 6(2),171–187.

Al-Kofahi, Y., Dowell-Mesfin, N., Pace, C., Shain, W., Turner, J.,Roysam, B. (2007). Improved detection of branching pointsin algorithms for automated neuron tracing from 3D confo-cal images. Cytometry Part A, 73(1), 36–43.

Brown, K., Barrionuevo, G., Canty, A., De Paola, V., Hirsch, J.,Jefferis, G., et al. (2011). The DIADEM data sets: Repre-sentative light microscopy images of neuronal morphologyto advance automation of digital reconstructions. Neuroin-formatics. doi:10.1007/s12021-010-9095-5.

Calixto, E., Galván, E., Card, J., & Barrionuevo, G. (2008). Co-incidence detection of convergent perforant path and mossyfibre inputs by CA3 interneurons. The Journal of Physiol-ogy, 586(11), 2695–2712.

De Paola V., Holtmaat A., Knott G., Song S., Wilbrecht L.,Caroni P., et al. (2006). Cell type-specific structural plastic-ity of axonal branches and boutons in the adult neocortex.Neuron, 49(6), 861–875.

Dima, A., Scholz, M., & Obermayer, K. (2002). Automatic seg-mentation and skeletonization of neurons from confocal mi-

Neuroinform (2011) 9:247–261 261

croscopy images based on the 3-D wavelet transform. IEEETransactions on Image Processing, 11(7), 790–801.

Evers, J., Schmitt, S., Sibila, M., & Duch, C. (2005). Progress infunctional neuroanatomy: Precise automatic geometric re-construction of neuronal morphology from confocal imagestacks. Journal of Neurophysiology, 93(4), 2331.

Jefferis, G., Potter, C., Chan, A., Marin, E., Rohlfing, T., Maurer,C., et al. (2007). Comprehensive maps of Drosophila higherolfactory centers: Spatially segregated fruit and pheromonerepresentation. Cell, 128(6), 1187–1203.

Meijering, E. (2010). Neuron tracing in perspective. CytometryPart A, 77(7), 693–704.

Meijering, E., Jacob, M., Sarria, J., Steiner, P., Hirling, H., &Unser, M. (2004). Design and validation of a tool for neu-rite tracing and analysis in fluorescence microscopy images.Cytometry Part A, 58(2), 167–176.

Narro, M., Yang, F., Kraft, R., Wenk, C., Efrat, A., &Restifo, L. (2007). NeuronMetrics: Software for semi-automated processing of cultured neuron images. Brain Re-search, 1138, 57–75.

Peng, H. (2008). Bioimage informatics: A new area of engineer-ing biology. Bioinformatics, 24(17), 1827–1836.

Peng, H., Ruan, Z., Atasoy, D., & Sternson, S. (2010a). Auto-matic reconstruction of 3D neuron structures using a graph-augmented deformable model. Bioinformatics, 26(12):i38–i46.

Peng, H., Ruan, Z., Long, F., Simpson, J., & Myers, E. (2010b).V3D enables real-time 3D visualization and quantitativeanalysis of large-scale biological image data sets. NatureBiotechnology, 28(4), 348–353.

Rodriguez, A., Ehlenberger, D., Hof, P., & Wearne S. (2006).Rayburst sampling, an algorithm for automated three-dimensional shape analysis from laser scanning microscopyimages. Nature Protocols, 1(4), 2152–2161.

Rodriguez, A., Ehlenberger, D., Hof, P., & Wearne, S. (2009).Three-dimensional neuron tracing by voxel scooping. Jour-nal of Neuroscience Methods, 184(1), 169–175.

Sato, Y., Nakajima, S., Shiraga, N., Atsumi, H., Yoshida, S.,Koller, T., et al. (1998). Three-dimensional multi-scale line

filter for segmentation and visualization of curvilinear struc-tures in medical images. Medical Image Analysis, 2(2), 143–168.

Schmitt, S., Evers, J., Duch, C., Scholz, M., & Obermayer, K.(2004). New methods for the computer-assisted 3-D recon-struction of neurons from confocal image stacks. Neuroim-age, 23(4), 1283–1298.

Sugihara, I., Wu, H., & Shinoda, Y. (1999). Morphology of sin-gle olivocerebellar axons labeled with biotinylated dextranamine in the rat. The Journal of Comparative Neurology,414(2), 131–148.

Vasilkoski, Z., & Stepanyants, A. (2009). Detection of the opti-mal neuron traces in confocal microscopy images. Journal ofNeuroscience Methods, 178(1), 197–204.

Wearne, S., Rodriguez, A., Ehlenberger, D., Rocher, A.,Henderson, S., & Hof, P. (2005). New techniques for imag-ing, digitization and analysis of three-dimensional neuralmorphology on multiple scales. Neuroscience, 136(3), 661–680.

Weaver, C., Hof, P., Wearne, S., & Lindquist, W. (2004). Auto-mated algorithms for multiscale morphometry of neuronaldendrites. Neural Computation, 16(7), 1353–1383.

Wright, S., & Nocedal, J. (2006). Numerical optimization. NewYork: Springer.

Xie, J., Zhao, T., Lee, T., Myers, E., & Peng, H. (2010). Auto-matic neuron tracing in volumetric microscopy images withanisotropic path searching. In: Medical image computingand computer-assisted intervention–MICCAI 2010 (pp. 427–429). Springer.

Yuan, X., Trachtenberg, J., Potter, S., & Roysam, B. (2009).MDL constrained 3-D grayscale skeletonization algorithmfor automated extraction of dendrites and spines fromfluorescence confocal images. Neuroinformatics, 7(4), 213–232.

Zhang, Y., Zhou, X., Degterev, A., Lipinski, M., Adjeroh,D., Yuan, J., et al. (2007) Automated neurite extractionusing dynamic programming for high-throughput screen-ing of neuron-based assays. NeuroImage, 35(4), 1502–1515.