1

3-D Active Meshes for Cell Tracking

Prashant Pal (16EC65R23)M.Tech, VIPESIIT Kharagpur

2

Topics to be discussed• Introduction• Chan-Vese-Mumford-Shah model• Proposed 3-D Active Meshes model• Mesh merging strategy• Collision detection• Implementation of the Energy Functional• Mesh resampling• Mesh splitting• Applications• Results

3

Cell segmentation & tracking past challenges• Some Variational deformable models have proven over the past decades a high

efficiency for segmentation and tracking in 2-D sequences. Yet, their application to 3-D time-lapse images has been hampered by discretization issues, heavy computational loads.

• Cells that evolve freely in their environment are difficult to track.• Typical experiments for cell tracking can produce several tens of gigabytes of

images in multiple colors and dimensions in a just a few hours.• Present softwares only provide basic image analysis algorithms and are not

suited for efficient 3-D cell tracking.

4

3D Active meshes • This method is proposed to address these limitations by reformulating

the problem entirely in the discrete domain.• We use 3-D active meshes, which express a surface as a discrete

triangular mesh, and minimize the energy functional accordingly.• By performing computations in the discrete domain, computational

costs are drastically reduced, whilst the mesh formalism allows to benefit from real-time 3-D rendering.

5

Chan-Vese-Mumford-Shah model

Fig. 1[4]. The final image has piecewise-constant intensities

6

Chan-Vese-Mumford-Shah model

C – Any variable curveu0 (x,y) – Image Pixel value at (x,y)C1 – Average of image inside CC2 – Average of image outside C

(1)

Fig. 2[2]. Consider all possible cases in the position of the curve. The fitting term is minimized only in the case when the curve is on the boundary of the object.

7

Proposed method

S – Any variable surfaceI – Image Voxel intensity at (x,y,z)CI – Average of image inside SCo – Average of image outside S

(2)

8

Implementation of the Energy FunctionalEvolution equation per mesh is given by

Mi – Triangular mesh for i’th objectFint – Internal Force Fext – External ForceFcpl – Coupling ForceFvol – Volume Conservation Force

(3)

9

1. Internal Force : Minimizes the internal energy term required to minimize the geodesic contour

length

– Empirical constantKv – Curvature unit vectorNv – Outer unit normal gI – Result of edge detector function

2. External Force : The external force driving each vertex towards the object boundary is given

by

Co – Average intensity of backgroundCi – Average intensity of object0 , – Empirical weightsI(v) – Image intensity value of nearest image voxels to vertex vNv – Normal vector to the mesh at vertex v pointing outwards

(4)

(5)

10

3. Coupling Force: The coupling force is meant to repel the surface at locations where it overlaps with

another surface

Colldist(v.Mj) – Penetration depth of v within the mesh Mj

4. Volume Conservation Force : An additional feedback force preventing vertices from moving in

such a way that the mesh volume changes drastically

Ai – Reference volume

Vol(Mj) – Current volume of mesh Mj

(6)

(7)

11

Local mesh resampling operations

If an edge gets too long (top left), it is either split to create two new faces or inverted if the resulting edge length is within the distance interval. If an edge gets too short (top right), the two corresponding faces are deleted and the vertices are merged.

Fig. 3[1]

12

Description of the mesh splitting scheme

As vertices A and B come closer and call for a merge operation, there exists a third common neighbor C such that ABC is not a mesh face . The virtual face is then used to cut the mesh and fill the holes in each new mesh. (a) Before splitting(b) After splitting

Fig. 4[1]

13

Mesh merging strategy

a) Collision is detected when the vertex of the black mesh penetrates a face of the red mesh.

b) We then delete the colliding vertex, the faces it belongs to as well as the penetrated face.

c) Finally, both meshes are merged by triangulating the gap between each mesh hole.

Fig. 5[1]

14

Collision detectionConsider two meshes M1, M2 and their bounding spheres S1, S2. We describe below the main steps of the algorithm to determine if M1 is colliding with M2 (the other case is done by analogy).a) If S1 and S2 do not intersect, stop.b) Extract all vertices of M1 located within S2. If the list is empty, stop.c) Intersection test: For each vertex of the previous list, construct a

vector from Vi to the center of M1, and test the intersection between this ray and all faces of M2 with same orientation. This technique greatly reduces the number of tests and, hence, computations.

15

Collision Detection Process

Close-up views of the sequence to illustrating the collision detection system implemented in the framework to handle cell contacts. (a) Before contact .(b) After contact

Fig. 6[1]

c) Before separationd) After separation

16

Segmentation result on a low contrast cell

Image size is 80x80x34 voxels, and voxel resolution is 0.327x0.327x3 m. (a) Maximum intensity projection (MIP) of the image. (b) Pseudo-colored MIP. (c) Final mesh after convergence (resolution: 0.327 m; size: 16300 faces; time: 11 s).

Fig. 7[1]

17

Tracking results on two amoeba crawling in a 3-D collagen matrix

Last image of a 4-D sequence with 150 images of size 400x379x20 voxels and resolution 0.327x0.327x3 m. (a) MIP of the image at t = 150. (b) Cell tracks superimposed on the final meshes at t = 150(resolution: 1 m; size:5000 to 7000 faces each; time: 3–7 s per image).

Fig. 8[1]

18

Conclusion

Table 1. - COMPARISON OF SEGMENTATION RESULTS ON SIMULATED DATASETS

19

Comparison of tracking errors on simulated contact events

Errors represent the overlap between cells, the total call background and cell-cell confusion. Bold values indicate best results.

Table. 2[1]

20

References[1] A. Dufour, R. Thibeaux, E. Labruyere, N. Guillen and J. C. Olivo-Marin, "3-D Active Meshes: Fast Discrete Deformable Models for Cell Tracking in 3-D Time-Lapse Microscopy," in IEEE Transactions on Image Processing, vol. 20, no. 7, pp. 1925-1937, July 2011.[2] T. F. Chan and L. A. Vese, "Active contours without edges," in IEEE Transactions on Image Processing, vol. 10, no. 2, pp. 266-277, Feb 2001.[3] A. Dufour, V. Shinin, S. Tajbakhsh, N. Guillen-Aghion, J. C. Olivo-Marin and C. Zimmer, "Segmenting and tracking fluorescent cells in dynamic 3-D microscopy with coupled active surfaces," in IEEE Transactions on Image Processing, vol. 14, no. 9, pp. 1396-1410, Sept. 2005.[4] D. Mumford and J. Shah, “Optimal approximations of piecewise smooth functions and associated variational problems,” Commun. Pure Appl. Math., vol. 42, no. 4, pp. 577–685, 1989.

21

Thank You !