31
Craniofacial Superimposition in Forensic Identification using Genetic Algorithms Lucia Ballerini 1 , Oscar Cord´ on 1 , Sergio Damas 1 , Jose Santamar´ ıa 2 , Inmaculada Alem´ an 3 , and Miguel Botella 3 1 European Centre for Soft Computing, Edf. Cient´ ıfico Tecnol´ ogico, Mieres, Spain {lucia.ballerini,oscar.cordon,sergio.damas}@softcomputing.es 2 Department of Computer Science, University of Ja´ en, Ja´ en, Spain [email protected] 3 Physical Anthropology Lab, University of Granada, Granada, Spain {ialeman,mbotella}@ugr.es Summary. Photographic superimposition is a forensic process that aims to iden- tify a missing person from a photograph and a skull found. One of the crucial tasks throughout all this process is the craniofacial superimposition which is usually car- ried out manually by forensic anthropologists; thus being very time consuming and presenting several difficulties when trying to find a good fit between the 3D model of the skull and the 2D photo of the face. In this chapter we introduce a detailed description of the problem, an automatic procedure based on the use of a real-coded genetic algorithm to tackle it and the analysis of its performance when solving dif- ferent synthetic and real superimposition problems from our Physical Anthropology lab at the University of Granada. 1 Introduction Forensic anthropology is best conceptualized more broadly as a field of foren- sic assessment of human skeletonized remains and their environments [1]. This assessment includes both the identification of the victims’ physical character- istics and cause and manner of death from the skeleton. This way, the most important application of forensic anthropology is the identification of human beings from their skeletal remains. Photographic supra-projection [2] is a forensic process where photographs or video shots of a missing person are compared with the skull that is found. By projecting both photographs on top of each other (or, even better, matching a scanned three-dimensional skull model against the face photo/video shot), the forensic anthropologist can try to establish whether that is the same person. This work is supported by the Spanish Ministerio de Educaci´ on y Ciencia (ref. TIN2006-00829), and by the Andalusian Dpt. of Innovaci´ on, Ciencia y Empresa (ref. TIC-1619), both including EDRF fundings.

Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition in ForensicIdentification using Genetic Algorithms?

Lucia Ballerini1, Oscar Cordon1, Sergio Damas1, Jose Santamarıa2,Inmaculada Aleman3, and Miguel Botella3

1 European Centre for Soft Computing, Edf. Cientıfico Tecnologico, Mieres, Spain{lucia.ballerini,oscar.cordon,sergio.damas}@softcomputing.es

2 Department of Computer Science, University of Jaen, Jaen, [email protected]

3 Physical Anthropology Lab, University of Granada, Granada, Spain{ialeman,mbotella}@ugr.es

Summary. Photographic superimposition is a forensic process that aims to iden-tify a missing person from a photograph and a skull found. One of the crucial tasksthroughout all this process is the craniofacial superimposition which is usually car-ried out manually by forensic anthropologists; thus being very time consuming andpresenting several difficulties when trying to find a good fit between the 3D modelof the skull and the 2D photo of the face. In this chapter we introduce a detaileddescription of the problem, an automatic procedure based on the use of a real-codedgenetic algorithm to tackle it and the analysis of its performance when solving dif-ferent synthetic and real superimposition problems from our Physical Anthropologylab at the University of Granada.

1 Introduction

Forensic anthropology is best conceptualized more broadly as a field of foren-sic assessment of human skeletonized remains and their environments [1]. Thisassessment includes both the identification of the victims’ physical character-istics and cause and manner of death from the skeleton. This way, the mostimportant application of forensic anthropology is the identification of humanbeings from their skeletal remains.

Photographic supra-projection [2] is a forensic process where photographsor video shots of a missing person are compared with the skull that is found. Byprojecting both photographs on top of each other (or, even better, matching ascanned three-dimensional skull model against the face photo/video shot), theforensic anthropologist can try to establish whether that is the same person.? This work is supported by the Spanish Ministerio de Educacion y Ciencia (ref.

TIN2006-00829), and by the Andalusian Dpt. of Innovacion, Ciencia y Empresa(ref. TIC-1619), both including EDRF fundings.

Page 2: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

2 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

To do so, an accurate 3D model of the skull is first demanded. Next, thematching of two sets of radiometric points (facial anthropometric landmarksin the subject photograph, and cranial anthropometric landmarks in the ob-tained skull model) is considered [2]. Then, a decision making stage starts byanalyzing the different kinds of achieved matchings between landmarks. Someof them will perfectly match, some will partially do so, and finally some otherswill not. After the whole process, the forensic expert must declare whetherthe analyzed skull corresponds to the missing person or not.

As can be seen, the latter procedure is very time consuming and there is nota systematic methodology but every expert usually apply his/her particularprocess. Hence, there is a strong interest in designing automatic methods tosupport the forensic anthropologist to put it into effect.

On the other hand, image registration (IR) is the image processing taskthat aims to find the optimal point/surface correspondence/overlapping be-tween two (or more) images, captured in a local coordinate system by a spe-cific acquisition procedure, i.e. from different points of view (multiple views),at different times, and by different sensors [3]. Thus, the key idea of the IRprocess is achieving the transformation (rotation, translation, etc.), noted asf, that places different images in a common coordinate system bringing thepoints as close together as possible by minimizing the error of a given metricof resemblance.

We aim to propose an automatic methodology to develop photographicsupra-projection that will be divided in three stages, somehow reproducingthe steps of the procedure employed by the forensic experts, and which willbe based on IR techniques.

The first stage involves achieving the “virtual model” of the physical ob-ject itself, so that it constitutes an accurate model of the real object we aretrying to represent, i.e., a skull model. Different views are required to supplythe information needed to construct the 3D model. Range IR (RIR) aims toregister 3D images acquired by a range scanner (a device able to capture 3Dimages). Hence, in RIR f consists of a transformation between different rangeimages. Therefore, the more accurate the alignment of the views the betterthe reconstruction of the object. In [4, 5] we acquired different views of theskull to be modeled by using our lab range scanner and succeeded tacklingthe RIR problem using evolutionary algorithms [6].

Likewise, the second task known as craniofacial superimposition aims toproperly superimpose the skull 3D model and the photograph. In our ap-proach, forensic experts extract different landmarks from the skull 3D modelobtained in the first stage and the face photograph that will guide the auto-matic search for the f parameters that accurately overlay them. In this case,f corresponds to a more complicated 3D/2D transformation. The goal of thiscontribution is to formulate the corresponding IR problem and to solve it bymeans of genetic algorithms with a suitable design.

Finally, the last stage of the craniofacial identification process correspondsto the decision making. Typically, once the best superimposition is achieved,

Page 3: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 3

the forensic experts analyze the final correspondence between the significantlandmarks of the skull and the face and make a decision on five possiblechoices: positive, negative, highly probable positive, lowly probable positiveor unknown identification.

In this book chapter, we will focus our attention on the second stage of theidentification task, known as craniofacial superimposition. Shortly, it involvesautomatically finding the best overlaying between the virtual model of theskull and the 2D photo of the missing person.

As said, the superimposition process is one of the most time consumingtasks for the forensic experts. Therefore, software tools for the automationof their work are a real need. It is fundamental to adopt a proper techniqueto align the 3D model and the 2D image in a common coordinate frame bymeans of IR techniques [3]. Unfortunately, we can find only few proposalsto automate the process and, even worse, the results are not suitable forthe forensic experts. From our previous experience on medical IR [7, 8] andRIR [4], we will address the problem by taking an existing one as a base,Nickerson et al.’s [9], which in its original form shows several flaws making itinaccurate for the problem solving.

The structure of this chapter is as follows. In Section 2 we review thebasics of evolutionary algorithms, IR, and we summarize the first stage ofour methodology, i.e. the reconstruction of 3D skull models. We make a shortreview of the state of the art on craniofacial superimposition in Section 3.Section 4 is devoted to describe Nickerson et al.’s approach. In Section 5 wedetail our proposal. Experiments on synthetic and real data are presented inSection 6. Finally, Section 7 collects some conclusions and future works.

2 Preliminaries

The craniofacial superimposition task is so complicated that it worths ad-dressing some important concepts before tackling the problem itself. In orderto ease the reader’s understanding, we first present some basics of evolu-tionary computation in Section 2.1. Then, Section 2.2 describes the IR prob-lem, Section 2.3 presents different existing 3D registration transformation andSection 2.4 introduces the similarity metric. Finally, Section 2.5 summarizesdifferent approaches to the 3D reconstruction of forensic objects, since it con-stitutes the preliminary stage previous to the superimposition.

2.1 Evolutionary Computation

Evolutionary Computation (EC) uses computational models of evolutionaryprocesses as key elements in the design and implementation of computer-based problem solving systems [6]. Genetic algorithms (GAs) [10], based onthe mechanisms of natural genetics and selection, are maybe the most knownevolutionary algorithms.

Page 4: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

4 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

A GA works on a population of solutions. A fitness value, derived from theproblem’s objective function is assigned to each member of the population.Individuals that represent better solutions are awarded higher fitness values,thus giving them more chances to survive to the next generation. Starting witha random initial population, successive generations are created by the geneticoperators: reproduction, crossover and mutation. Finally, the best solution inthe last population is given as output.

To get more knowledge on EC and GAs, the interested reader can refer torecent books like [11, 12].

2.2 The image registration problem

The key idea of the IR process is to achieve the transformation that placesdifferent 2D/3D images in a common coordinate system. There is not a uni-versal design for a hypothetical IR method that could be applicable to allregistration tasks, because various considerations of the particular applica-tion must be taken into account. Nevertheless, IR methods usually requirethe four following components: two input 2D or 3D Images named as SceneIs = {p1,p2, . . . ,pn} and Model Im = {p ′

1 ,p ′2 , . . . ,p ′

m}, with pi and p ′j be-

ing image points; a Registration transformation f (see Section 2.3), beinga parametric function relating both images; a Similarity metric functionF (see Section 2.4), in order to measure a qualitative value of closeness ordegree of fitting between the scene and the model images; and an Optimizerwhich looks for the optimal transformation f inside the defined solution searchspace.

Hence, IR is the process of finding the optimal spatial transformation fachieving the best fitting (measured using F ) between the model and thetransformed scene image points, f(Is). Such transformation estimation is in-terpreted into an iterative optimization process in order to properly explorethe search space. Figure 1 graphically represents the IR optimization process.

Fig. 1. The IR optimization process

2.3 3D registration transformations

We can sort the different kinds of transformations f according to the main-tenance (or not) of an established relationship between geometric primitives(points, lines, etc.) in both the scene and the model images (see Figure 2).

Page 5: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 5

Fig. 2. From left to right and top to bottom: scene and model 3D images of atent overlays. Next, the scene image suffers a translation and, in the last two cells,a rotation transformation and a uniform scaling, respectively. In the next row, anon uniform scaling is applied. On the right, top view of the tents when an affinetransformation has been applied (two of the vertices of the tent floor keep unchangedwhile the other two are moved). In the next cell, projective transformation of thetent (while the floor keeps unchanged, both tent entrances fall down). Finally, acurved transformation is depicted

• Rigid transformation: if the distance between every couple of points ispreserved, we find a rigid (or isometric) transformation. 3D ones can bedescribed using the equation:

x′

y′

z′

1

=

r11 r12 r13 txr21 r22 r23 tyr31 r32 r33 tz0 0 0 1

xyz1

(1)

where t = (tx, ty, tz)T is an arbitrary translation vector and rij is an entryof a rotation matrix R (given in the X, Y , and Z coordinate axes order)defined by:

R =

CβCγ CβSγ −Sβ

SαSβCγ − CαSγ SαSβSγ + CαCγ SαCβ

CαSβCγ + SαSγ CαSβSγ − SαCγ CαCβ

3×3

,

where Ci = cos(i) and Si = sin(i), for i = α, β, γ.So, in the 3D rigid transformation space, there are six parameters thatmust be determined to solve the IR problem: three rotation angles (α, β, γ)(corresponding to each of the X, Y , and Z principal coordinate axes),called as Euler angles, and a three dimensional translation vector t.

Page 6: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

6 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

• Similarity transformation: if every angle in the scene is preserved afterapplying the transformation f then we work on an similarity transforma-tion. Furthermore, depending on what kind of changes affect the segmentsdimension, we have uniform or non uniform scalings. In the former case,the relative change in segments dimension is the same in all directions(adding a new parameter to be estimated), whereas in non uniform trans-formations such change is different in different directions (adding as manyparameters as different size changes).

• Affine transformation: if neither angles nor segment sizes are main-tained, we are dealing with an affine transformation. After the transfor-mation, only parallelism is kept. Every couple of parallel lines keeps onbeing parallel after the transformation.

• Perspective transformation: if we consider a Pin Hole cameramodel [13] the perspective transformation provides a representation of a3D scene projected into a 2D image plane. Every ray connecting every 3Dpoint and its counterpart in the 2D image plane passes through a spe-cific point known as center of projection. The perspective transformationis characterized because planarity and linearity are preserved. However,neither distances nor angles are preserved.

• Curved transformation: these transformations can not in general berepresented using constant matrices. Many applications represent them interms of the scene coordinates as polynomial transformations. For the caseof trilinear transformations we have:

p = (x, y, z)T , p′ = (x′, y′, z′)T = T (p)

p′ =1∑

i=0

1∑j=0

1∑k=0

(ai,j,k, bi,j,k, ci,j,k)T xiyjzk (2)

where ai,j,k, bi,j,k and ci,j,k are the transformation coefficients. Pointingout that i, j and k are either 0 or 1, the dimension of the search space isthe 3× 23 = 24.Note that even junctions are not preserved when a curved transformationis applied.

2.4 Similarity metric

One of the most important components of any IR method is the similaritymetric. This is considered as a function F that measures the goodness of agiven registration solution, that is, the registration transformation f , and thefinal performance of any IR method will depend on its accurate estimation.

Each solution is evaluated by F applying such transformation f to one ofthe two images, usually to the scene image (f(Is)). Once this has been done,the second step will be to determine the degree of closeness or fitting, Ψ(), ofthe transformed scene and the model images:

Page 7: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 7

F (Is, Im, f) = Ψ(f(Is), Im). (3)

There are many approaches trying to estimate such function Ψ() dependingon the dimensionality (2D or 3D) and the nature of the considered images.The most usual when following a feature-based IR approach, i.e. when thesame relevant features are selected prior to tackling the IR problem, is theMean Square Error [14, 15, 16, 4]:

F (Is, Im, f) =1r

r∑i=1

‖ f(pi)− p ′j ‖2 (4)

where r is the number of points in the scene image, f(pi) is the resultingpoint after the transformation is applied to the scene point pi, and p ′

j is theclosest model point to f(pi) of the scene.

2.5 Summary of the first stage: skull 3D model building

As said in the Introduction, the first stage of the photographic supra-projection process aims to achieve an accurate 3D model of the physical skullthat was found. The more accurate the 3D model, the more reliable the finalidentification decision.

On the other hand, the 3D object reconstruction problem can be formu-lated as an IR problem where f is defined as the transformation linking thedifferent views acquired by a given 3D scanner. In particular, RIR handles im-ages that come from a range scanner [17]. RIR must tackle several difficultiesthat arise due to: i) the object shape acquisition regarding the illuminanceconditions (recovering noisy shapes); ii) the presence of occlusion (region ofthe scanned object shape only present in one of the two consecutive scannedimages); iii) the fact that the object to be scanned has either a globally sym-metric shape or the symmetry appears along the shape of two consecutiveviews; and iv) the absence of an a priori known transformation or, at least,an approximated motion between consecutive scanned images. In many caseswhen these drawbacks appear, the reconstruction procedure is forced to beperformed by a manual and time consuming RIR process without ensuringthe best possible outcomes.

Due to the latter, several approaches for the automatic registration of thescanned views have been proposed in the last few years. In pair-wise RIR [18],the object modeling process starts by firstly registering two adjacent views,goes on with the next pair, and so on, building a partially reconstructed shapeeach time. We succeeded tackling the pair-wise RIR problem to build skull3D models using evolutionary algorithms in [4, 5].

Page 8: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

8 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

3 Related work and discussion on craniofacialsuperimposition

Successful comparison of human skeleton remains with artistic or photo-graphic replicas has been achieved many times, beginning with the studiesof the skeletal remains of Dante in the nineteenth century [19], till the iden-tification of victims of the recent Indian Ocean tsunami [20].

We have only found two really interesting proposals performing cranio-facial superimposition in a fully automatic way, based on the use of neuralnetworks [21] and GAs [9], respectively.

The method proposed by Ghosh and Sinha [21] is an adaptation of theirprevious proposal for face recognition problems [22]. The Extended SymmetryPerceiving Adaptive Neuronet (ESPAN) consists of two neural networks to beapplied to two different parts of the overlaying and allows the user to selectfuzzy facial features to account for ambiguities due to soft tissue thickness.More in details the system can implement an objective assessment of thesymmetry between two nearly front images: the cranial image and the facialimage that are the inputs as the source and the target images, respectively.The output is the mapped cranial image suitable for superimposition. Twonetworks need to be trained separately because each of them can correctly maponly a part of the cranial image. Two limitations arise, already pointed out bythe authors: a part of the cranial image that will not be properly mapped andthe need for a front view image. Moreover, this method is not fully applicablebecause of its long computation time, and the need of separately applying twodifferent networks to the upper skull contour and to front view cranial features,where the overlaying found by the first one is disrupted by the second.

On the other hand, Nickerson et al.’s method [9] uses a GA to find the opti-mal parameters of the similarity and perspective transformation that overlaysthe 3D skull model on the photo (see Section 4 for a detailed description ofthis method).

Nevertheless, to date, no automatic method is used in practical applica-tions despite the high number of cases examined [23]. Indeed, studies on thevalidity of “classic” superimposition methods are still in progress [24]. At thesame time, it is affirmed that visual assessment is more effective than metricalstudies [25].

Recent papers confirm that some authors think the most advanced methodis based on digital superimposition through the use of Photoshop r© and theimaging tools provided by these software packages [20, 26, 27]. We agree withthese authors that working with digital images is definitively simpler andcheaper than photo/video superimposition, but we should note that the meth-ods they use are not automatic as they manually resize, shift and rotate theimage by trial and error, thus dealing with a very time consuming and erroraffected process.

Regardless the automatic or non automatic approach to tackle photo-graphic supra-projection, some authors [25, 28, 29] agree that this technique

Page 9: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 9

should be used only for excluding identity, rather than for positive identifica-tion. Seta and Yoshino [30] state the following general rule that superimpo-sition is of greater value in ruling out a match, because it can be definitelystated that the skull and photograph are not those of the same person. How-ever, if they do align, it can only be stated that the skull might possibly bethat of the person in the photograph. This statement can justify the thirdstage of our proposal, where we plan to express the identification decisionaccording to several confidence levels using a fuzzy decision making system.

On the other hand, recent literature has considerably developed the po-tential of a somehow related task, the craniofacial reconstruction, preferablyreferred to as facial approximation [31, 32]. The 3D facial image may be recon-structed by either building muscle and soft tissue using clay, or by means ofcomputer graphics. Data concerning the reliability of these methods for foren-sic anthropology and lack of relationship between facial approximation andresemblance rating have been reported [33]. An interesting review of currentsystems for computerized forensic facial reconstruction can be found in [34]

Finally, we should mention that several automatic approaches regardingthe 3D face model – 2D face photograph superimposition have been also pre-sented and extensive reviews can be found like the one in [35]. However, theyare out of the scope of this discussion, as they deal with face recognition thatis a completely different problem.

4 GA-based Nickerson et al.’s proposal

The method proposed by Nickerson et al. [9] included the following tasks:

• 2D digitalization of an antemortem facial photograph.• 3D digitalization of the surface mesh of the skull.• Application of digital filtering techniques to the 2D photo image and the

3D model to reduce or eliminate systematic error.• Selection of four landmark points on the digital facial image and four

equivalent noncoplanar landmarks on the skull surface mesh.• Calculation of the near-optimal affine and perspective transformations re-

quired to map the skull surface mesh into two dimensions and onto theface image.

• Joint solid rendering of the digital facial photograph and transformed skullsurface mesh for visual analysis.

A digital camera and a range scanner were used for 2D and 3D digitaliza-tions, respectively. Well known image processing algorithms were consideredfor image enhancement (median filtering, histogram equalization, Wiener fil-tering [36]). Rendering was done through computer graphics techniques, afterpolygonal texture mapping of the 2D image.

The most important novelty of this technique was the automatic calcula-tion of the mapping of the skull surface mesh on the digital facial photograph.

Page 10: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

10 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

This mapping was achieved from the matching of the four landmarks previ-ously identified both in the face and the skull. The landmarks used in theirwork were: either the glabella or the nasion, the two ectocanthion points, andan upper mandibular dentition point, if present, or the subnasal point (seeFigure 3).

Fig. 3. Cranial and facial landmarks (arrows indicate the selected ones)

The mappings were developed from sets of similarity transformations and aperspective projection. These transformations consisted of the following stepsperformed on the skull mesh:

1. general rotation about an arbitrary 3D point and rotation axis;2. uniform scaling;3. arbitrary translation in 3D space;4. mirroring about the z-axis;5. perspective projection from 3D to 2D as a function of angle of view.

This mapping problem was specified as a set of eight equations in twelveunknowns.

The unknowns were:

(rx, ry, rz) = origin of the general 3D rotation(dx, dy, dz) = direction cosines of the axis of rotation

θ = general angle of rotation of the cranial meshs = constant 3D mesh scaling factor

(tx, ty, tz) = general 3D mesh translation vectorφ = perspective angle of view

The known quantities were the set of landmark points:

(xfi, yfi, 1) = facial landmarks(xci, yci, zci) = cranial landmarks

Page 11: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 11

The entire systems of equations (similarity transformations and perspec-tive mapping) can be written using a set of 4 by 4 matrices as follows:

F =

xf1 yf1 1 1xf2 yf2 1 1xf3 yf3 1 1xf4 yf4 1 1

C =

xc1 yc1 zc1 1xc2 yc2 zc2 1xc3 yc3 zc3 1xc4 yc4 zc4 1

A =

1 0 0 00 1 0 00 0 1 0

−rx −ry −rz 1

D1 =

1 0 0 00 dz/ν dy/ν 00 −dy/ν dz/ν 00 0 0 1

D2 =

ν 0 dx 00 1 0 0

−dx 0 ν 00 0 0 1

R =

cos θ − sin θ 0 0sin θ cos θ 0 0

0 0 1 00 0 0 1

S =

s 0 0 00 s 0 00 0 s 00 0 0 1

T =

1 0 0 00 1 0 00 0 1 0tx ty tz 1

M =

1 0 0 00 1 0 00 0 −1 00 0 0 1

P =

1 0 0 00 1 0 00 0 tan(φ/2) tan(φ/2)0 0 0 1

where:

F = four facial landmark points mapped on the polygonC = four cranial landmarks points in 3D spaceA = translation of rotation origin to 3D space originD1 = initial setup rotation of the cranial meshD2 = secondary setup rotation of the cranial meshR = rotation about the axis or rotationS = general constant scaling of the meshT = general 3D translation of the meshM = mirroring about the z-axisP = perspective projectionν =

√d2

y + d2z

Then the resulting system can be computed as:

F = C · (A ·D1 ·D2 ·R ·D−12 ·D−1

1 ·A−1)·S · T ·M · P (5)

where the matrix operations in brackets perform the general rotation.4

4 Notice that some information is missing in Eq. 5 to know how to match the 2Dand 3D coordinates of the different landmarks. Moreover, we could deduce that itis only a matter of selecting the first two cranial coordinates (xci, yci) once theyare transformed by Eq. 5 (see Section 5).

Page 12: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

12 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

Three approaches were considered to solve the problem: a heuristic, aclassic numerical optimization and a binary-coded GA methods. Results basedon the GA outperformed the remainder.

The objective function to be minimized was the sum of the squared 2DEuclidean distances between the facial landmarks and the transformed craniallandmarks.

In order to work with an accuracy of at least 0.25 mm, the transformationparameters bounds related to the binary-coded GA were defined as:

−210 ≤ [{tx, ty, tz} · 4] ≤ 210

−210 ≤ [{rx, ry, rz} · 4] ≤ 210

0 ≤[{dx,dy,dz}

0.072

]≤ 213

0 ≤[

θ0.072

]≤ 213

0 ≤[

φ0.072

]≤ 213

0 ≤[

1024·(s−1)−5

]≤ 210

(6)

The total sum of bits required to encode the twelve discretized unknowns(chromosome size) was then 141. Unfortunately the authors did not specifyany other component of the GA. Their system required from 4 to 6 hours toconverge. Finally, they showed only qualitative results on a few of the twelvecases studied.

We implemented this approach, using the GAucsd package [37]. We alsomade experiments by empirically changing GA parameters and using the graycoding provided by the package. None of these experiments gave acceptableresults on our data (either synthetic or real ones), as will be showed in Sec-tion 6.

It is worth to note that beside our critics, this was a very pioneering workin the field, especially considering that it was published in the early nineties.

According to Ghosh et al. [21], Nickerson et al.’s method did not gainmuch popularity due to the fact that it essentially required a digitalized 3Dcranial image reconstruction, which could not be implemented either by aneasy/simple procedure or by an economic hardware configuration. We thinkthe reason for not finding practical applications of this method is probably alsodue to the difficulties for the forensic experts to understand the mathemat-ical formulation of the registration problem and the evolutionary framework(specially considering there are important issues missing, as said).

5 Upgraded GA-based craniofacial superimposition

We take Nickerson et al.’s approach as a base and try to solve its flaws. There-fore we made several adaptations of this proposal as follows: (1) improvementof the registration transformation; (2) use of a real coding scheme; (3) betterdesign of GA components.

Page 13: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 13

In order to properly match the 3D and 2D landmarks there are two pro-cesses to be considered regarding each of them, once Eq. 5 is applied. On theone hand, the transformed cranial landmarks Ct = (xt

ci, ytci, z

tci, w

tci) must be

mapped on the 3D Euclidean plane by dividing all the four terms of everylandmark by its fourth one: C ′ = Ct/wt

ci. On the other hand, the coordinatesof the 2D facial landmarks are usually provided with respect to the originwhich is located in the top-left corner of the digital image. Hence, they mustbe translated so that the origin is in the centre of the photo, mirrored in Xso that we deal with the perspective assumptions, and normalized in [−1, 1].The matrix M is not used in our implementation.

The range of the transformation parameters are calculated as follows:

ri ∈ [Ci − radius, Ci + radius], i ∈ {x, y, z}di ∈ [−1, 1], i ∈ {x, y, z}θ ∈ [0◦, 360◦]s ∈ [0.25, 2]φ ∈ [10◦, 150◦]tx ∈ [−lengthFB − (Cx + radius), lengthFB − (Cx − radius)]ty ∈ [−lengthFB − (Cy + radius), lengthFB − (Cy − radius)]tz ∈ [NCP − (Cz + radius), FCP − (Cz − radius)]

where radius = max(d(C,Lj)), with C being the centroid of the 3D land-marks, Lj being the jth 3D landmark, and d() being the Euclidean distancebetween two points,

minFD =1

tan(φmax

2 )

(considering a 2 × 2 projection plane centered in the Z axis),

lengthFB =(minFD + FCP ) ∗ sin(φmax

2 )

sin(90o − (φmax

2 ))

with FCP and NCP being the Far and Near Clipping Planes, respectively, FBthe Frustrum Base and FD the Focal Distance.

In preliminary experiments, we considered a higher scaling upper boundbut we did not achieve better results. On the other hand, we should note thatthe field of view of a typical camera is φ = 45◦. However, in professional oneswhere even the lens can be changed, φ ∈ [5◦, 180◦].

IR is an inherent real coding problem, therefore a real-coded GA is moresuitable for craniofacial superimposition. It is reported by several authors thatwhen dealing with real parameters, a real-coded GA performs better than abinary-coded one nine out of ten times [6, 38].

In our implementation, the IR parameters are represented by their naturalform. This overcomes several difficulties of the binary coding and let us avoidall the mapping needed by Nickerson between real-valued variables and fixed-length strings (see Eq. 6).

Page 14: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

14 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

Our fitness function is the original one proposed by Nickerson et al., i.e.:

fitness =Nlm∑i=1

√(x′ci − x′fi)2 + (y′ci − y′fi)2 (7)

where: Nlm is the number of landmarks used, (x′ci, y′ci) are the positions in the

projection plane of the transformed cranial landmarks calculated by the GA,and (x′fi, y

′fi) are the facial landmarks (xfi, yfi), after applying the aforemen-

tioned transformations.In the first implementation with a binary-coded GA we used the roulette-

wheel selection, two-point crossover and simple mutation that were the typicalGA components by the time that Nickerson et al.’s approach was proposed.Nowadays we can use more efficient genetic operators. In particular, we usedthe tournament selection method [39], the blend (BLX-α) crossover [40] andthe random mutation.

In tournament selection, t individuals are selected at random from thepopulation and the best of them is inserted into the new population for fur-ther genetic processing. This procedure is repeated until the mating pool isfilled. Though this scheme selects good individuals for the next generation, itcannot guarantee that the best solution obtained survives through the opti-mization process. For this reason, the process called elitism is also used. Oncean individual with highest fitness among the current generation is found, itwill be kept unchanged in the next generation.

The BLX-α crossover is applied with a probability Pc to the floating-point numbers representing the real variables. BLX-α uniformly picks newindividuals with values that lie in [x′, y′], where x and y represent the twoparents’ gene values for a particular variable. The offsprings are sampled asfollows: x′ + r× (y′−x′) where: x′ = x−α× (y−x); y′ = y +α× (y−x); r isan uniform random number ∈ [0, 1], checking that x′ and y′ will lie betweenthe variable’s lower and upper bounds.

A random mutation operator is applied with a probability Pm. It randomlyselects one of the genes of a parent and sets it equal to an uniform randomnumber between the gene’s lower and upper bounds.

6 Experiments

Once the specific design of our GA proposal is presented, we will study itsperformance. The experimental setup will consider both synthetic and realidentification cases:

• Synthetic problems: the synthetic problem consists of the 3D-2D superim-position of the 3D models of real-world objects acquired by the Konica-Minolta c© 3D Lasserscanner VI-910, used in the Physical Anthropologylab at the University of Granada, and their corresponding 2D photographs

Page 15: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 15

acquired by a digital camera. The tuning of the most suitable parametersrelated to the proposal will be tackled by a broad number of experiments.

• Real-world identification case: once the most suitable parameter tuning isachieved in the previous study, we will test it on a real-world case solvedby the lab staff in the past following a manual approach for photographicsupra-projection. We will consider the 2D photograph of the missing per-son and its corresponding 3D model of the skull.

6.1 Synthetic problem: calculator

The first considered object is shown in Figure 4. The 3D model comprises356,143 points, stored as x, y, z coordinates. The 2D image is a 3,488 × 2,616RGB (red, green and blue) image. It has been resampled to 1,280 × 960 pixelsfor practical reasons. For this object a set of 12 landmarks has been manuallyselected on the 3D model and on the corresponding 2D image.

The proposed algorithm has been applied to find a suitable set of pa-rameters that map the 3D landmarks on the 2D ones through a similaritytransformation and a perspective projection, i.e. to solve the superimpositionproblem.

Note that we should theoretically have a perfect fit between the 3D and 2Dlandmarks. We do not have to tackle with the uncertainty involved in the realcases of craniofacial superimposition, where the thickness of the soft tissue hasto be taken in account. However, we have to remind that the landmarks aremanually selected by users and therefore the uncertainty in localizing themshould be taken into account. Indeed, it is almost impossible to select a pointin the 3D model to identify the unique pixel corresponding to it in the 2Dimage.

Fig. 4. Example of an object used for a synthetic problem

Page 16: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

16 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

In the following paragraphs we will describe the experiments we performedin order to tune the GA parameters and to study the influence of the choiceof the landmarks.

GA parameters

In the first set of experiments the fitness is calculated on 4 landmarks, whilethe performance is evaluated on the remaining 8 landmarks, using the MeanSquare Error (MSE), given by:

MSE =∑N

i=1 ||f(Ci)− Fi||N

(8)

where N is the number of landmarks, f(Ci) are the positions of the trans-formed 3D landmarks in the projection plane, and Fi are the 2D landmarks. Inother words, MSE is the mean distance between the corresponding landmarks.

The ranges of transformation parameters, calculated as described in sec-tion 5, are:

rx ∈ [−67.29, 91.65]ry ∈ [−80.89, 78.05]rz ∈ [−80.90, 78.04]tx ∈ [−18752.9, 18728.5]ty ∈ [−18739.3, 18742.1]tz ∈ [−78.0346, 5080.9](dx, dy, dz) ∈ [−1, 1]θ ∈ [0◦, 360◦]φ ∈ [10◦, 150◦]s ∈ [0.25, 2]

We performed experiments using the following GA parameters:

generations = 300population size = {50, 100, 300}tournament size = 2BLX-α parameter = {0.1, 0.3, 0.5, 0.7, 0.9}crossover probability = 0.9mutation probability = 0.2

Tournament size and crossover and mutation probability values are fixedto find a trade-off between the diversification and intensification capabilitiesof the GA.

In order to avoid execution dependence, thirty different runs for each pa-rameter setting have been performed. Table 1 shows the MSE calculated onthe landmarks used during the training (MSEtraining) and on the not usedones (MSEtest) while varying the population size and the BLX-α parame-ter value. Namely, for any parameter setting the best/minimum value (m),

Page 17: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 17

Table 1. Minimum (m), maximum (M), mean (µ), and standard deviation (σ) MSEvalues over 30 runs on the training and test data sets, when varying the populationsize and the BLX-α parameter value. Best µ values are highlighted using bold font

population α MSEtraining MSEtest

size m M µ σ m M µ σ

0.1 0.017 0.341 0.271 0.102 0.034 0.438 0.361 0.1130.3 0.008 0.346 0.247 0.125 0.016 0.443 0.331 0.145

50 0.5 0.005 0.346 0.230 0.124 0.019 0.443 0.310 0.1440.7 0.007 0.345 0.159 0.141 0.016 0.442 0.229 0.1680.9 0.011 0.346 0.181 0.134 0.031 0.443 0.260 0.160

0.1 0.070 0.345 0.283 0.073 0.145 0.441 0.376 0.0790.3 0.045 0.344 0.254 0.106 0.077 0.440 0.341 0.118

100 0.5 0.012 0.347 0.139 0.126 0.032 0.444 0.202 0.1490.7 0.008 0.340 0.078 0.092 0.024 0.435 0.140 0.1140.9 0.004 0.339 0.109 0.117 0.017 0.434 0.162 0.145

0.1 0.033 0.334 0.266 0.097 0.083 0.428 0.351 0.1100.3 0.010 0.332 0.218 0.115 0.029 0.426 0.303 0.129

300 0.5 0.007 0.340 0.101 0.110 0.025 0.434 0.156 0.1300.7 0.006 0.093 0.038 0.023 0.021 0.186 0.076 0.0420.9 0.007 0.342 0.066 0.074 0.022 0.438 0.111 0.101

the worst/maximum value (M), the mean value (µ) and the related standarddeviation (σ) of the MSE are given.

As expected, the performance improves when the population size increases.Nevertheless, we should observe that we obtain quite good results with relativesmall populations and in some cases the minimum MSE is even better withsmaller populations.

Based on the average MSE on the test dataset, which relates to the ro-bustness of the procedure, the best results are obtained for α = 0.7 for everypopulation size, confirming that this value is a good trade off between explo-ration and exploitation of the search space.

The standard deviation (σ) is not too high, i.e. it is always less than 0.17.Hence, regardless of the different initial populations, a good set of transfor-mation parameters can always been found.

The number of generations considered is related to the generalizationpower, as in any learning process. In our case, we considered 300 generationsafter observing that the maximum performance is achieved on the trainingset, while the ability to obtain a similar performance on the test set decreaseswhen overpassing this value. Figure 5 depicts the MSE on the training andtest data sets. It can be observed that the best results on the test data occurnearby generation 300 and increasing the number of generation only improvesMSEtraining but not MSEtest, thus causing overfitting.

It should be remarked that MSEtest is the error on the test data setobtained with the individual having the best fitness. In general, this error does

Page 18: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

18 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

0

0.05

0.1

0.15

0.2

0.25

0.3

0 100 200 300 400 500 600

MSE

Generation

traintest

Fig. 5. MSE on the training and test set of landmarks during a typical run

not coincide with the lowest error value of the generation, instead it is usuallygreater than this latter. Indeed, our fitness does not take into account the erroron the test set. Hence, evolution is blind with respect to that error, as it shouldbe. In fact, it does not know which individual has the best performance on thetest set. The same considerations are valid for the MSEtest values reportedin all the tables.

The comparison between the results obtained with our real-coded GA(RCGA) and the original binary-coded GA (BCGA), implemented as de-scribed in Section 5, is reported in Table 2.

Table 2. Minimum (m), maximum (M), mean (µ), and standard deviation (σ)MSE values obtained with the RCGA and the BCGA

MSEtraining MSEtest

m M µ σ m M µ σ

RCGA 0.006 0.093 0.038 0.023 0.021 0.186 0.076 0.042BCGA 0.361 0.406 0.384 0.014 0.406 0.486 0.457 0.026

As can be seen, the RCGA outperforms the BCGA in every run. Valuesreported in this table are obtained with a population of 300 individuals bothfor the RCGA and the BCGA. Comparing Table 2 with Table 1, we canalso observe that even the RCGA with a smaller population (50 individuals)outperforms the BCGA.

The best overlay obtained by each method are shown in Figure 6. Noticethat the BCGA gets stuck in a local minimum and projects the 3D model ina very small zone located at the middle-right region (see circle). It is evidentthat it is not able to find the proper transformation parameters to superimposethe 3D model on the 2D photo.

Page 19: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 19

Fig. 6. Synthetic image superimposition using RCGA and BCGA

Choice of landmarks

In this section we aim to study the influence of the number and locationof landmarks used to calculate the transformation parameters that give theoptimal superimposition.

For all the experiments described in this section, the ranges of the trans-formation parameters are the same as in the previous experiments. The bestGA parameter values determined in our previous set of experiments are used,i.e. population size = 300 and α = 0.7 for the BLX-α crossover operator. Weagain performed 30 runs on each dataset.

First, three different data sets, containing 4, 6, and 8 landmarks respec-tively, have been used during the training. To evaluate the results, the MSEhas been computed on both the used and not used landmarks.

Table 3 reports statistics on MSE values obtained with the three datasets.We report also MSE values obtained using all the 12 landmarks as a referencebaseline.

Table 3. Minimum (m), maximum (M), mean (µ), and standard deviation (σ)MSE values on the training and test dataset at varying the number of landmarks.Best µ values are highlighted using bold font

dataset size MSEtraining MSEtest

training test m M µ σ m M µ σ

4 8 0.006 0.093 0.038 0.023 0.021 0.186 0.076 0.0426 6 0.012 0.337 0.076 0.081 0.014 0.482 0.115 0.1148 4 0.004 0.115 0.027 0.023 0.011 0.237 0.049 0.048

12 0.010 0.066 0.024 0.013

We observe that the best performance is obtained using 8 landmarks. How-ever, not noticeable differences can be observed between using 4 and 8 land-marks. For instance, looking at the maximum MSE, we can see that it is

Page 20: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

20 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

smaller using 4 landmarks, meaning that, in some runs, experiments usingonly 4 landmarks outperform experiments using 8 of them. Therefore, as con-sidering the use of a lower number of landmarks implies a reduced computationtime, we can conclude that 4 is the optimal number of landmarks to be usedin the superimposition process.

The run times for these three problems range from 2 to 3 seconds, accord-ing to the number of used landmarks.

Once the proper number of landmarks has been studied, we tackle anotherset of experiments dedicated to study how the location of the landmarksinfluences the results.

Five sets containing 4 landmarks each have been used in the training phase.The landmarks of every set are chosen in different locations of the object (seeFigure 7).

dataset A dataset B dataset C dataset D dataset E

Fig. 7. Landmarks selected for each training dataset are highlighted with brightcircles

The test datasets have been composed of the remaining 8 landmarks. MSEon both training and test sets are reported in Table 4.

Table 4. Minimum (m), maximum (M), mean (µ), and standard deviation (σ) MSEvalues obtained with different datasets of landmarks. Best µ values are highlightedusing bold font

training MSEtraining MSEtest

dataset m M µ σ m M µ σ

A 0.006 0.093 0.038 0.023 0.021 0.186 0.076 0.042B 0.006 0.048 0.021 0.012 0.012 0.186 0.085 0.046C 0.009 0.205 0.114 0.061 0.026 0.481 0.290 0.141D 0.007 0.063 0.024 0.012 0.018 0.111 0.041 0.020E 0.006 0.090 0.020 0.016 0.011 0.191 0.043 0.034

The best results are achieved using datasets D and E, where the landmarksare chosen close to the external borders of the object; the worst ones are in-stead achieved with C, where the landmarks used in the training phase are

Page 21: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 21

coplanar and concentrated on the bottom part of the object. Relative goodresults are obtained with datasets A and B, where the landmarks are selectedin positions that somehow remind to the anthropological landmarks on skullsand faces. The overall results confirm that as the separation among the land-marks grows, better fits between the 3D and 2D images will be achieved. Asexpected, the closer the landmarks, the more likely to be coplanar, and theworse the superimposition results.

The best final superimposition result, obtained using 12 landmarks, isshown in Figure 8.

Fig. 8. Final synthetic image superimposition: on the right image the 2D landmarksare marked with “o” and the 3D transformed and projected landmarks are markedwith “x”

6.2 3D-2D skull superimposition

A human skull belonging to the physical Anthropology Lab at the Universityof Granada has been used for the 3D-2D superimposition problem.

The 3D model (427,100 points stored as x, y, z coordinates) has been ac-quired by the aforementioned 3D range scanner, and the 2D photograph (1280× 960 RGB image) has been acquired by a SONY DSC-P71 digital camera.

A set of 11 anthropological landmarks were initially selected by the forensicexperts on the 3D model and on the corresponding 2D image (Figure 9).

Our algorithm was applied to find a suitable set of parameters that mapthe 3D landmarks on the 2D ones through a similarity transformation and aperspective projection. As in the previous synthetic experiments, we shouldtheoretically have a perfect fit between the 3D and 2D landmarks, regard-less the uncertainty in localizing them. Indeed, this is not a real craniofacialsuperimposition problem, but only the superimposition of a skull with itscorresponding photograph.

First, the anthropological landmarks are used for the training phase, thenwe did some experiments to determine the best landmarks able to solve ourproblem.

Page 22: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

22 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

Fig. 9. A skull used as for the 3D-2D superimposition problem with the landmarkslocated by the forensic experts: 3D model on the left, 2D image on the right

In all the experiments, the fitness function has been calculated as in Eq. 7on all the landmarks used in that experiment.

The initial position of the 3D model was a frontal view of the skull.The ranges of transformation parameters are:

rx ∈ [−108.571, 110.15]ry ∈ [−91.2757, 127.445]rz ∈ [−45.7997, 172.921]tx ∈ [−18771.4, 18769.8]ty ∈ [−18788.7, 18752.5]tz ∈ [−172.917, 5045.8](dx, dy, dz) ∈ [−1, 1]θ ∈ [0◦, 360◦]φ ∈ [10◦, 150◦]s ∈ [0.25, 2]

We performed experiments using the following GA parameters:

runs = 30generations = 300population size = 300tournament size = 2BLX-α parameter = 0.7crossover probability = 0.9mutation probability = 0.2,

the best parameters determined in the previous set of synthetic experiments.In our first experiment, using the 11 anthropological landmarks (see Fig-

ure 10) we observed the difficulties of this problem, mainly due to coplanarityof the selected landmarks. Indeed the GA found many non optimal solutions,i.e. local minima.

Page 23: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 23

As can be seen in the figure, all the landmarks were almost properlymapped. Therefore, the frontal part of the model matched quite well withthe corresponding part of the photo. However, the projection of the posteriorpart of the skull did not fit with the photo due to an overestimation of thescaling.

Fig. 10. 3D-2D skull superimposition using 11 anthropological landmarks (the 2Dlandmarks are marked with “o” and the 3D transformed and projected landmarksare marked with “x”)

Second, taking into account the low level of confidence of the forensicexperts in localizing the vertex landmark, we did not consider it in our follow-ing experiments. As can be observed in figure 11, the fit improved, but thecoplanarity problem was still affecting our results.

Then, the forensic experts selected other landmarks non coplanar with theanthropological ones. Figure 12 shows the obtained results with 13 landmarkswhich are much better than the previous ones. Once more, the reason of thesegood results are due to the less coplanarity of the latter landmarks.

We can observe than using landmarks on the lateral part of the skull guidesthe GA towards a better solution. Nevertheless, the matching is good but notexcellent and there is a lot of room for improvement. Besides, the run timefor our RCGA is approximately 2.7 seconds.

Indeed, this is a difficult problem for several reasons. As we already men-tioned, most of the landmarks are nearly coplanar, making the system ofequations indetermined. Moreover, we should notice the large range of theparameters we used, in order to consider one of the worse possible setting ofthe camera. This makes the search space of the GA much bigger than anypossible real identification problem.

The comparison between the results obtained with our RCGA and theoriginal BCGA considering the final set of 13 landmarks is reported in Table 5.

Page 24: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

24 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

Fig. 11. 3D-2D skull superimposition using 10 anthropological landmarks by re-moving the vertex (the 2D landmarks are marked with “o” and the 3D transformedand projected landmarks are marked with “x”)

Fig. 12. 3D-2D skull superimposition using 13 landmarks (the 2D landmarks aremarked with “o” and the 3D transformed and projected landmarks are marked with“x”)

As expected, the RCGA outperforms the BCGA in every statistical valuereported in the table. Actually, in the majority of the runs, the BCGA wasnot able to find an acceptable solution. In Figure 13 we can clearly see thatall the 3D landmarks are projected in a very small zone at the centre of theimage.

6.3 Craniofacial superimposition of a real world case

We focused our study on a real case that has been solved by the staff ofthe physical Anthropology Lab at the University of Granada in collaboration

Page 25: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 25

Table 5. Minimum (m), maximum (M), mean (µ), and standard deviation (σ)MSE values obtained with the RCGA and the BCGA

MSEtraining

m M µ σ

RCGA 0.030 0.122 0.045 0.018BCGA 0.226 0.330 0.312 0.030

Fig. 13. 3D-2D skull superimposition using BCGA

with the Spanish scientific police. Facial photographs of this person were pro-vided by the family, and the final identification done by photographic supra-projection has been confirmed. We studied this real case with the consent ofthe relatives.

The forensic anthropologists manually selected a set of 3D landmarks onthe skull and the corresponding 2D landmarks on the face (see Figure 14).

This is a real craniofacial superimposition case, where the thickness of thesoft tissue, as well as the uncertainty of localizing landmarks, will influencethe final results of our algorithm.

On these images we performed experiments with the same GA parametersused for the synthetic cases. The transformation parameters are calculatedas described in Section 5. Since our aim is to actually solve the craniofacialsuperimposition problem, the fitness is calculated on all the landmarks andthe performance is evaluated by computing the MSE on them.

The best superimposition results obtained by our RCGA are shown inFigures 15 and 16. In the latter one (obtained using only 6 landmarks), theMSE is higher, but the obtained superimposition is better. The reason forexcluding one of the landmarks comes from the observation of the manualsuperimposition obtained by the forensic experts (shown in Figure 17), wherethe landmark close to the ear is not properly matched.

Page 26: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

26 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

Fig. 14. Real world case: 3D skull model on the left, 2D face photo on the right

Fig. 15. Superimposition results using 7 landmarks

It is worth to note that our method can reach these results in 2.25 seconds,while the forensic expert employed approximately 24 hours to accomplish thesame task.

A good solution was find in each run. Error values over 30 runs are reportedin Table 6. For comparison we also report the results obtained with BCGA.As in the synthetic experiments, the RCGA outperforms the BCGA in everystatistical value.

The best superimposition obtained with BCGA is shown in Figure 18.Again it does not constitute a valid solution for the problem.

Page 27: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 27

Fig. 16. Superimposition results using 6 landmarks

Fig. 17. Manual superimposition obtained by the forensic experts

7 Concluding remarks and future works

The aim of our project is to design a complete, automatic, soft computing-based procedure to aid the forensic anthropologist in the identification task byphotographic supra-projection. In this paper we have proposed and validatedthe use of RCGAs for the craniofacial superimposition stage of the process thatinvolves the alignment of the 3D skull model with the 2D face photograph.The method is fast and accurate, and therefore very useful for solving one ofthe most tedious works performed by the forensic anthropologists.

Page 28: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

28 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

Table 6. Minimum (m), maximum (M), mean (µ), and standard deviation (σ)MSE values obtained with the RCGA and the BCGA on a real case

MSEtraining

m M µ σ

RCGA 0.024 0.309 0.190 0.097BCGA 0.487 0.609 0.565 0.035

Fig. 18. Superimposition results using BCGA

We have presented results obtained on a large number of experiments tovalidate our proposal and to tune the most suitable parameters. Our goodresults show a high improvement of our method with respect to the originalBCGA proposed by Nickerson et al. [9], that did not successfully work on ourdata. However, there is still a lot of room for improvement, that will be thegoal of our future research.

We plan to extend this study by using other evolutionary algorithms, as forexample the scatter search algorithm [41], that we already applied to medicalIR and RIR real-world problems with very successful results [4, 7, 8, 5]. Weare also planning to consider a higher number of real cases of identificationprovided and solved by the Physical Anthropology Lab at the University ofGranada.

Finally, we aim to tackle the third identification stage, i.e. the decisionmaking by using fuzzy logic, in order to assist the forensic expert to take thefinal identification decision.

Page 29: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 29

References

1. Krogman, W.M., Iscan, M.Y.: The human skeleton in forensic medicine. CharlesC. Thomas, Springfield, IL (1986) 2nd edition.

2. Iscan, M.Y.: Introduction to techniques for photographic comparison. In Iscan,M.Y., Helmer, R., eds.: Forensic Analysis of the Skull. Wiley (1993) 57–90

3. Zitova, B., Flusser, J.: Image registration methods: a survey. Image and VisionComputing 21 (2003) 977–1000

4. Santamarıa, J., Cordon, O., Damas, S., Aleman, I., Botella, M.: A scattersearch-based technique for pair-wise 3D range image registration in forensicanthropology. Soft Computing 11 (2007) 819–828

5. Santamarıa, J., Cordon, O., Damas, S.: Evolutionary approaches for automatic3D modeling of skulls in forensic identification. In: Applications of EvolutionaryComputing. Number 4448 in LNCS (2007) 415–422

6. Back, T., Fogel, D.B., Michalewicz, Z., eds.: Handbook of evolutionary compu-tation. IOP Publishing Ltd and Oxford University Press (1997)

7. Cordon, O., Damas, S., Santamarıa, J.: A Fast and Accurate Approach for 3DImage Registration using the Scatter Search Evolutionary Algorithm. PatternRecognition Letters 27 (2006) 1191–1200

8. Cordon, O., Damas, S., Santamarıa, J.: Feature-based image registration bymeans of the CHC evolutionary algorithm. Image and Vision Computing 24(2006) 525–533

9. Nickerson, B.A., Fitzhorn, P.A., Koch, S.K., Charney, M.: A methodology fornear-optimal computational superimposition of two-dimensional digital facialphotographs and three-dimensional cranial surface meshes. Journal of ForensicSciences 36 (1991) 480–500

10. Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and MachineLearning. Addison-Wesley, Reading, MA (1989)

11. Eiben, A.E., Smith, J.E.: Introduction to evolutionary computation.SpringerVerlag (2003)

12. Goldberg, D.E.: The design of innovation: lessons from and for competent ge-netic algorithms. Kluwer Academic, Boston (2002)

13. Faugeras, O.D.: Three-Dimensional Computer Vision. MIT Press, Cambridge,MA (1993)

14. Besl, P.J., McKay, N.D.: Iterative point matching for registration of free-formcurves and surfaces. IEEE Transactions on Pattern Analysis and Machine In-telligence 14 (1992) 239–256

15. Zhang, Z.: Iterative point matching for registration of free-form curves andsurfaces. International Journal of Computer Vision 13 (1994) 119–152

16. Yamany, S.M., Ahmed, M.N., Farag, A.A.: A new genetic-based technique formatching 3D curves and surfaces. Pattern Recognition 32 (1999) 1817–1820

17. Ikeuchi, K., Sato, Y.: Modeling from Reality. Kluwer Academic Publishers(2001)

18. Silva, L., Bellon, O., Boyer, K.: Robust range image registration using geneticalgorithms and the surface interpenetration measure. World Scientific (2005)

19. Welcker, H.: Der schadel Dantes (in German). In Witte, K., Boehmer, G., eds.:Jahrbuch der deutschen Dantegesellschaft. Volume 1. Brockhaus, Liepzig (1867)35–56

Page 30: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

30 Ballerini, Cordon, Damas, Santamarıa, Aleman, Botella

20. Al-Amad, S., McCullough, M., Graham, J., Clement, J., Hill, A.: Craniofacialidentification by computer-mediated superimposition. J. Forensic Odontostomal24 (2006) 47–52

21. Ghosh, A., Sinha, P.: An economised craniofacial identification system. ForensicScience International 117 (2001) 109–119

22. Sinha, P.: A symmetry perceiving adaptive neural network and facial imagerecognition. Forensic Science International 98 (1998) 67–89

23. Ubelaker, D.H.: A history of Smithsonian-FBI collaboration in forensic anthro-pology, especially in regard to facial imagery. Forensic Science Communications2 (2000) [online]

24. Scully, B., Nambiar, P.: Determining the validity of Furue’s method of craniofa-cial superimposition for identification. Malaysian Journal of Computer Science9 (2002) 17–22

25. Jayaprakash, P.T., Srinivasan, G.J., Amravaneswaran, M.G.: Craniofacial mor-phoanalysis: a new method for enhancing reliability while identifying skulls byphotosuperimposition. Forensic Science International 117 (2001) 121–143

26. Bilge, Y., Kedici, P., Alakoc, Y., K., U., Ilkyaz, Y.: The identification of adismembered human body: a multidisciplinary approach. Forensic Science In-ternational 137 (2003) 141–146

27. Ross, A.H.: Use of digital imaging in the identification of fragmentary humanskeletal remains: A case from the Republic of Panama. Forensic Science Com-munications 6 (2004) [online]

28. Yoshino, M., Matsuda, H., Kubota, S., Imaizumi, K., Miyasaka, S., Seta,S.: Computer-assisted skull identification system using video superimposition.Forensic Science International 90 (1997) 231–244

29. Cattaneo, C.: Forensic anthropology: development of a classical discipline in thenew millennium. Forensic Science International 165 (2007) 185–193

30. Seta, S., Yoshino, M.: A combined apparatus for photographic and video su-perimposition. In Iscan, M.Y., Helmer, R., eds.: Forensic Analysis of the Skull.Wiley (1993) 161–169

31. Vanezis, P., Vanezis, M., McCombe, G., Niblett, T.: Facial reconstruction using3-D computer graphics. Forensic Science International 108 (2000) 81–95

32. Claes, P., Vandermeulen, D., De Greef, S., Willems, G., Suetens, P.: Craniofacialreconstruction using a combined statistical model of face shape and soft tissuedepths: methodology and validation. Forensic Science International 159S (2006)S147–S158

33. Stephan, C.N., Arthur, R.S.: Assessing facial approximation accuracy: How doresemblance ratings of disparate faces compare to recognition tests? ForensicScience International 159S (2006) S159–S163

34. Wilkinson, C.: Computerized forensic facial reconstruction: A review of currentsystems. Forensic Science, Medicine, and Pathology 1 (2005) 173–177

35. Bowyer, K.W., Chang, K., Flynn, P.: A survey of approaches and challengesin 3D and multi-modal 3D + 2D face recognition. Computer Vision and ImageUnderstanding 101 (2006) 1–15

36. Gonzalez, R., Woods, R.: Digital image processing. 2nd edn. Prentice Hall,Upper Saddle River, New Jersey (2002)

37. Schraudolph, N.N., Grefenstette, J.J.: A user’s guide to GAucsd 1.4. TechnicalReport CS92-249, Computer Science and Engineering Department, Universityof California, San Diego, La Jolla, CA (1992)

Page 31: Craniofacial Superimposition in Forensic Identification …docshare01.docshare.tips/files/16217/162172505.pdfCraniofacial Superimposition using GAs 3 the forensic experts analyze

Craniofacial Superimposition using GAs 31

38. Herrera, F., Lozano, M., Verdegay, J.L.: Tackling real-coded genetic algorithms:operators and tools for the behavioural analysis. Artificial Intelligence Reviews12 (1998) 265–319

39. Blickle, T.: C.2.3: Tournament Selection. In: Handbook of evolutionary com-putation. IOP Publishing Ltd and Oxford University Press (1997)

40. Eshelman, L.J.: Real-coded genetic algorithms and interval schemata. In Whit-ley, L.D., ed.: Foundations of Genetic Algorithms 2. Morgan Kaufmann, SanMateo (1993) 187–202

41. Laguna, M., Martı, R.: Scatter search: methodology and implementations in C.Kluwer Academic Publishers (2003)