Determination of the sequence of line crossings

by means of 3D laser profilometry

Giuseppe Schirripa Spagnolo , Carla Simonetti, Lorenzo Cozzella

Dipartimento di Ingegneria Elettronica, Università degli Studi “Roma Tre”

ABSTRACT

The determination of the sequence of line crossings is still a current problem in the field of forensic documents

examination. This paper presents the potentiality of the 3D micro-topography to resolve the writing order of signatures

(if partially superimposed), to detect the tampering of manuscripts (if the added inscription superimposes itself on parts

already written), to analyze pressure variation, and to identify strokes in handwritten. The system used, in this paper for

creating 3D micro-topography, is based on conoscopic holography. It is a non-contact three-dimensional measuring

system that allows producing holograms, even with incoherent light, with fringe periods that can be measured precisely

to determine the exact distance to the point measured. This technique is suitable to obtain 3D micro-topography with

high resolution also on surfaces with unevenness reflectivity (which is usual on the surface of the handwritten

document). The proposed technique is able to obtained 3D profile in non-invading way. Therefore, the original draft do

not be physically or chemically modified, allowing the possibility of multi-analysis in different time, also in case of

forensic analysis with the necessity to preserving the original sample. The experiments performed with line crossings

data base show that the proposed method is able of “positive identification” of writing sequence in the majority of the

tests. When we have not had a positive identification, the result has been “inconclusive”. The proposed technique, if

correctly used, does not supply “false positive” or “probable” identifications. The possible results are only: “positive

identification” and “inconclusive”.

Keywords: forensics, questioned documents, 3D laser profilometry, determination of writing sequence.

1. INTRODUCTION

Handwriting-based writer identification is one of the most important research areas in the field of validation of signature

as well as of handwritten documents [1-3]. The identification of handwriting is based on the proposition that “people are

all alike; people are all different”. The characteristics of the process of handwriting (i.e. velocity, variations of the pen

pressure during the writing, stroke, etc.) are unique to every individual. To be able to make a positive identification of a

person’s handwriting, a document examiner must be able to observe the distinguishing individual features that separate

one person from all others. When people attempt to forge handwriting, they focus on the general appearance, letter

formation, size, style and slant of the writing. Handwriting consists, however, of individualized strokes and pressure

that are not obvious in 2D images of the written.

When someone tries to copy, another style of writing two difficult things must be done simultaneously. First, the forger

must alter her/his own way of writing. Second, the forger must adopt all the writing habits of the writer being imitated.

Some forgers copy a signature over an indirect light source, but most try to simulate the writing after practice indicates

that the signature “looks close”. Normal writing is done automatically. We do not think of how we write letters but the

schirrip@uniroma3.it; phone +39 0655177046; fax 0039 065579078; Università degli Studi “Roma Tre”, Dipartimento di

Ingegneria Elettronica, Via Della Vasca Navale 84, I-00146, Roma, Italy.

Optical Security Systems, edited by Zbigniew Jaroszewicz, Sergei Y. Popov, Frank Wyrowski,Proc. of SPIE Vol. 5954, 59540V, (2005) · 0277-786X/05/$15 · doi: 10.1117/12.623117

Proc. of SPIE Vol. 5954 59540V-1

forger does. Consequently, the natural flow of the pen in a simulated (i.e. forgery) signature becomes halting and

pressures vary greatly. In addition, the stroke structure and sequence often do not exhibit the same class as those found

in the original writing [4]. In conclusion, the reconstruction of handwriting movement is considered, in forensic

handwriting examination, the key to conclusions about the authorship of questioned handwriting or the authenticity of

disputed signatures.

When compared with online handwriting analysis, which uses special input device such as tablets, offline handwriting

analysis is much more difficult because the only available information is a static two-dimension (2D) image of the

handwritten on the paper; the dynamic information of the pen-tip (stylus) movement such as pen-tip pressure, velocity,

and pen-up and pen-down can not be captured by means of 2D images. Difficulty also lies in the fact that it is hard to

segment signature strokes due to highly stylish and unconventional writing styles. The non-repetitive nature of variation

of the signatures, because of age, illness, geographic location and perhaps to some extent the emotional state of the

person, accentuates the problem.

However, it is possible to observe that, when we effect a handwriting (for example is affixed a signature) on a common

paper sheet, the pen-tip, besides releasing the ink, deforms the paper. In other words, the writing pressure leaves some

cuts; several or less deep ones according to the type of pen and the pressure used in handwrite. Therefore a three-

dimensional analysis of handwriting would be able to extract information on stroke sequence and on the pressure

applied to the paper during writing, so as pen-up and pen-down [5,6]. In this paper, we propose the use of the 3D laser

profilometry, realized by means of the conoscopic holography, to transform seemingly flat handwritten letters into

landscapes of hills and valleys that reveal the pressure and stroke sequence used to create each word documents.

Conoscopic holography is a non-contact three-dimensional measuring technique that makes possible to produce

holograms, even with incoherent light, with fringe periods that can be measured precisely to determine the exact

distance to the point measured. It is suitable to obtain 3D micro-topography with high resolution also on surface with

unevenness reflectivity (this situation is usual on the surface of the handwritten document). The technique is able to

obtained 3D profile in non-invading way. Therefore the system leaves the investigate surface unaltered so that the

questioned document can be investigated by means of other destructive or non-destructive technique in different time,

also in case of forensic analysis with the necessity to preserve the original sample. For the application proposed in this

paper, we have used, as laser profilometry, a commercially available conoscopic rangefinder made by Optimet®.

The methodology proposed in this work, so as the classical 2D analysis, is a comparative technique. Therefore, to reach

an opinion, the document examiner must have adequate known samples of writing to compare to the questioned writing.

The quality and quantity of the known samples (referred to as exemplars or standards of comparison) that are available

will determine whether a positive opinion can be reached.

The basic requirements for handwriting identification are:

Adequate standards of comparison. This must be the greater possible number. More standard am available,

better is. It is best to have example written before and after the date of the questioned handwritten documents.

This helps to assess the variability of the writer.

The questioned and known writing should be of the same style. Script is compared to script. Printing is

compared to printing. The same wording is also preferable.

Examination of original handwriting is necessary. Fax and copy machines do not reproduce the micro-

topography of the handwritten. Has not the availability of the original thing blocks the use of the technique

here introduced.

The determination of the 3D micro-topography, in the field of forensic document examination, can be obtaining using

of the scanning electron microscope (SEM) [7] or Atomic Force Microscopy [8]. SEM and AFM are promising 3D-

technique, but have a limited range in the vertical direction (~ 5 µm) and in the scanning area (< 2 square centimeters).

By means of laser profilometry overcoming the limits imposed by the use of the techniques SEM and AFM is possible.

In fact, with the laser profilometry technique non-destructive examination of "wide" zones of documents can be made.

The analysis, of questioned documents, by means of laser profilometry has, potentially, several applications. In

particular, to resolve the writing order of signatures (if partially superimposed), to detect the tampering of manuscripts

(if the added inscription superimposes itself on parts already written), to analyze pressure variation and to identify

strokes in handwritten is possible.

This paper is organized as follows: In Section 2, the conoscopic holography and used conoscopic range finder are

briefly described. In Section 3, the proposed method is described and the experimental results are presented. The

conclusions are presented in Section 4.

Proc. of SPIE Vol. 5954 59540V-2

Circular polarizer Circular polarizer

Handwritingsample

2. CONOSCOPIC HOLOGRAPHY AND CONOSCOPIC RANGE-FINDER

A conoscopic range finder (based on conoscopic holography) is well suited to provide an accurate 3D profile of

handwritten documents. At present, cheap conoscopic systems, well adapted to work in the field, are available.

Conoscopic holography has already been described in depth many times [9,10], here we present only what is necessary

to understand the following discussion.

Conoscopic holography is a simple implementation of a particular type of polarized light interference process, which

uses a birefringent crystal.

In ordinary holography an interference pattern is formed between an object beam and a reference beam using a coherent

light source. The object and reference beams propagate with the same velocity, but follow different geometrical paths.

In conoscopic holography, the object and reference beams of coherent holography are replaced by the ordinary and the

extraordinary components of a single beam propagating in birefringent media. Therefore, the signal and reference

beams have the same geometrical paths but different optical path-lengths; the two beams are naturally coherent one with

the other and therefore the technique allows producing holograms, even with non coherent light.

Conoscopic holography has some advantages over classical holography if spatially limited illuminated area is

concerned (one singular point):

much greater stability than classical holography because the geometrical paths of both wave fronts are almost

the same;

an interfringe distance adjustable to common CCD camera resolution; thus interfacing with a computer system

is facilitated;

the possibility of using not spatially coherent quasi-monochromatic light because of the small phase difference,

which is introduced.

The basic principle resides in considering a crystal sandwiched between two circular polarizers in order to provide an

interference pattern (see Figure 1).

Figure 1: Conoscopic principle.

Each object point P(x, y, z) either emits, diffuses or reflects quasi-monochromatic, non-polarized and spatial incoherent

light intensity I(P). A ray, with wavelength , making an angle with the system optical axis (see Figure 2) passes

through the first circular polarizer, which generates two orthogonal polarized, 90° phase shifted, rays. Within the

uniaxial (birefringent) crystal, the two rays propagate according to two modes, namely the ordinary and the

extraordinary mode, with different velocities.

Proc. of SPIE Vol. 5954 59540V-3

Figure 2: Propagation of light into a uniaxial crystal.

The wave velocity in a medium depends on the refractive index. In uniaxial crystals, the ordinary refractive index is

constant, while the extraordinary index depends on the angle between the ray and the crystal optical axis, and therefore

on the object’s point location. If the ordinary and the extraordinary indices of refraction are 0n and En , respectively,

and their difference is 0 En n n , then two orthogonal polarized waves will propagate, one with an index of

refraction 0n (ordinary ray), and the second, the extraordinary ray, with an index of refraction nE( ) given

approximately by [11]

2

0 sinEn n n (1)

The retardation angle, or the difference of optical path between extraordinary and ordinary waves, is given by

2 2if is small2 2

sincos

Ln L n . (2)

The intensity at ', ', 0R x y on the recording plane 0z due to point source , ,P x y z , see Figure 2, will be given

by

2 2

2

2( , ) ( ) , 0.5 0.5cos

L nI R P I P T R P I P x x y y

z (3)

Equation (3) represents a Fresnel zone plate (FZP) centered at ' , 'x x y y , plus a constant bias. We note that the

distance z (the depth coordinate) acts as a parameter, which changes the fringe spacing.

The number of fringes (NF) in the Fresnel zone plate is:

2 2

02 2

0

2 FF F

NL n L n AN N z

z L n A (4)

where 2A is the size of the FZP and z0 is the mean distance from the illuminated surface of the object to the recording

plane.

By Equation (4), depth estimation is possible by illuminating a very small area (by a focalized laser beam) and counting

the fringe number NF . Obviously, for continuous depth estimation, a calibration curve z0 (NF) is needed. This curve

yields the distance z0 from the object surface to the hologram center as a function of fringe number NF. At fractional NF,

depth estimation is achieved by using polynomial interpolation.

A particular of the used range finder is shown in Figure 3.

Proc. of SPIE Vol. 5954 59540V-4

'; '-

Figure 3: Particular of the used range finder.

The vertical resolution and the dynamical range of the system are 0.2 and 2000 m, respectively. The system used in

this work is in the fix-probe configuration. The fix-probe configuration consists of keeping the conoscopic probe fixed

and placing the questioned document to be analyzed on a translation-table. The maximum dislocation of the translation-

table is 12 cm with a position accuracy of 0.5 m. The documents are resting on a support made of porous copper. By

evacuating the air through the microscopic holes in this material, the position of the document remains unaltered

throughout the measurement.

3. THREE-DIMENSIONAL ANALYSIS OF HANDWRITTEN DOCUMENTS

The conoscopic range finder determines the micro-topography of the examined surface. The resulting 3D profile shows

the pen-tip strokes as an impression in the paper [6].

A typical handwritten document surface profile consists of roughness, pen-tip strokes (waviness) and form (in many

cases the paper sheet may also contain significant form such as deformations) [12]. It is important to notice that: the

paper roughness range is, usually, 2-5 m (dependent on type of paper used); the pen strokes depth range is 2 – 15 m

(dependent on writing pressure and material); the deformations of paper sheets can be also 100 times greater than the

pen-tip stroke impression.

Figure 4(a) shows the obtained raw data from the analysis of handwritten symbol “a ”.

This raw data contain information on pen-tip strokes, paper roughness and paper deformation. So, after the acquisition

of the data, 3D image processing and filtering are necessary to getting a surface model that can be used to identify the

lines that describe the handwriting [13].

The first elaboration that comes effected is the individualization, and elimination, of the paper deformation by means of

suitable 3D filter. The handwriting of a word or symbol cause the incision of the sheet, besides, it cause a global

deformation on the sheet. Figure 4(b) shows the global deformation drawn out by means of “3D form removal”

techniques. The surface profile of the symbol “a ”, after the operation of “form removal”, is shown in figure 4(c). The

filtered data is then passed through a new elaboration to separate the roughness and waviness components of the

surface. Figure 4(d) shows the roughness component present on the paper sheet. The pen-tip strokes (waviness)

impression is shown in Figure 4(e). Finally, 3D view of the pen-tip strokes, with a mirror along the z-axis, is shown in

Figure 4(f).

Proc. of SPIE Vol. 5954 59540V-5

0 2 4 6 8mm mm o 2 4 6 8mm pm

0.5

I 0lII: I I i!!i E O.O6 6040

V 20

0,•mm mm (D)

0 2 4 6 8mm tim o 2 4 6 8mm tim

P'r pj—100.5-I I-

L1.5 L

-20 2SJ L

2 L

Is-I I

mm mm (d)

[

o_______ 10

mm (e) (f)

.

Figure 4: 3D image processing for separation of roughness, waviness and form in a handwritten document.

(a) Original data; (b) Form–deformation of sheet; (c) Profile of the euro symbol, after the operation “form removal”;

(d) Paper roughness; (e) Pen-tip strokes–waviness; (f) 3D view of the pen-tip strokes with a mirror along the z-axis.

Proc. of SPIE Vol. 5954 59540V-6

Observing the reconstructed 3D image, strokes appear like furrows, but for recognizing the crossing dynamics the

observer needs to pay attention to some particular characteristics. A first activity to perform is to verify the presence of

some ‘bumps’, irregularities in the grooves produced in the paper by the pen-tip used for the writing. These bumps are

located at the sides of the second stroke that crosses the first one. The presence of these irregularities is localized in the

strokes’ crossing zone, along the first handwritten stroke line, and they have to stand out against the paper depression

created by the double pen passage. The strokes’ and bumps’ visibility is due to the kind of paper on which we have

written and to the kind of pen that we have used.

Figure 5 shows a typical example of analysis. This example was realized using a common black ballpoint pen (Bic®

E25) on 80 g/m2 white paper commonly used for printing and photocopying. Figures 5(a) and (b) show the crossing

zones from two different angles for highlighting the presence of bumps along the first (F) stroke direction, as indicated

in Figure 5(b). Figure 5(c) is the reverse along the z-axis image; it clearly shows that the second (S) stroke was made

after stroke (F), because its shape is continuous, whereas stroke (F) is interrupted. As can be noted, a characteristic of

the second stroke (S) is that it ‘cuts’ the first one (F). Using this assertion it is easier to reconstruct the crossing

dynamic.

Figure 5: Profilometric acquisition by means of conoscopic holography. The investigation area is about 5mm × 5mm.

(a) and (b) 3D view of the strokes’ profile. It is possible to note the regularity in the (S) line. The presence

of bumps is evident. (c) 3D view with a mirror along the z-axis.

In Figure 5(c), we note the presence of an “oval”, where the two strokes are really crossed. This oval is produced by the

“fall” of the tip of the pen into the furrow traced by the first stroke. It is possible, by the analysis of this hole, to note

that it is lengthened in the direction of flowing of the pen (i.e. along the writing direction) [5]. Therefore, from the

analysis of this type of “oval”, when present, it is possible to go back to the direction used for tracing the stroke (see

Figure 6).

Figure 6: 3D view of figure 5(c) that shows a different geometrical point of view.

In this image it is possible to note the lengthening in the direction of writing.

Proc. of SPIE Vol. 5954 59540V-7

Line (a)

NLine (b)

Bumps

1Writing direction Line (b) - Profile

End point Start point

C,,Start point (a)

C,,End point (b)

The analysis of the direction of writing can be effected, with more effectiveness, by means of profilometry along the

handwritten lines. Figure 7 shows this type of analysis made on the “ ” script.

Figure 7: Profilometry along the lines of writing.

Analyzing the Figure 7 it is possible to note that along the firth line are present two “evident” bumps. The presence of

these bumps, in the profile along the line (a), assures that this line is what we have traced for first. The presence of these

bumps, in our example, assures that the line (a) it is what traced for first. The profile along the line (b) shows an

interesting characteristic.

When the pen-tip arrives at the zone of crossing, where is present the first line, it downs within the groove already clear.

Successively it goes up again smoothing the first groove. This process produces the oval and its lengthening in the

direction of writing. Therefore, analyzing the presence of the bumps and of the oval we are able to determine the

drawing order of a handwritten script.

In Figure 8 are illustrated two symbols “& ” handwritten with different drawing orders.

Figure 8: Different drawing orders.

Proc. of SPIE Vol. 5954 59540V-8

End point

The 3D view, relating to the symbols “& ” of Figure 8, are shown in Figure 9.

Figure 9: Different drawing orders - 3D view of the pen-tip strokes.

Analyzing the 3D reconstruction of the grooves left by the pen-tip (in Figure 9 they are shown with a mirror along the

axis-z) it is possible to recognize the different dynamics used in the writing.

Our method is able, also, to resolve complex stroke sequence. Figure 10 shows a handwritten script with more than two

lines intersecting. The 3D view, relating to this script is shown in Figure 11. It is possible to see that, even in this case,

the correct sequence of drawing order is recovered.

Figure 10: Scrip with more two lines intersecting.

Proc. of SPIE Vol. 5954 59540V-9

(b) (c)

Figure 11: 3D view of script with more two lines intersecting. (a) shows show the micro-topography of the symbol.

(b) and (c) show the 3D reconstruction of the intersection from two different geometrical points of view.

.

Finally, the method is able to reconstruct the stroke sequence of a handwritten script. Figure 12 shows the handwritten

“campo” with labeled stroke sequence.

Figure 12: Handwritten with labeled stroke sequence.

Proc. of SPIE Vol. 5954 59540V-10

The 3D view of the handwriting allows the correct recovery of drawing order. The 3D view of the “campo” handwriting

is shown in Figure 13. Analyzing the features of the 3D view is possible to recover the stroke sequence.

Figure 13: 3D view of the “campo” handwriting. The features, of the 3D view, allow reconstructing the stroke sequence.

To confirm the ability of the proposed technique, to detect the dynamics of the strokes’ superposition, we have

performed numerous tests. In the tests, we have used different type of pens on different type of papers. The following

table report, in a synthetic way, the results of the tests.

Synthesis of results obtained for each kind of pen

Category of pen Positive identification Inconclusive

BIC pen 98 2

Ballpoint pen with fine point 49 1

Fountain pen 42 18

Liquid ink pen 32 8

Felt pen 48 12

Felt pen with fine point 31 9

The results explain in the table confirm that ballpoint pens are recommended in signing legal documents.

Proc. of SPIE Vol. 5954 59540V-11

4. CONCLUSION

In this paper, we have presented a new method that recovers the stroke sequence of a static handwritten sample from a

3D analysis of the script. This method is based on the detection of the strokes’ dynamics and, in particular, on the

superposition order. The system uses conoscopic holography and a 3D analysis of the acquired data. To the best of our

knowledge, this is the first time holograms have been used to analyze handwriting. It is very difficult to compare the

proposed method with other methods that use 2D analysis; therefore, other searches are necessary to comparing the

holographic method with current handwriting-analysis techniques to see how the new approach stacks up. In particular,

it is essential to develop a protocol of the measure and of the interpretation of the results, and that the forensic scientist

approve this protocol. However, the proposed method works well and it allows, in the most of the cases, to recover

correct stroke sequenced of a handwritten script. Besides, the method is able of analyzing the pressure variations used

during the writing.

Future work will be oriented on the "automatic" determination of 3D features of the handwriting. The principal limit of

the proposed technique is the time consumed for the 3D scanning of the handwritten. For this reason, it is essential to

carry out studies for the improvement of the used hardware. The instrumentation commercially available is inadequate

for extensive utilization of the technique.

REFERENCES

1. G. Poulin, “Establishing the sequence of strokes: the state of the art”, Int. J. Forensic Doc. Examiners, 2, pp.16-32,

1996.

2. R. Sabourin, G. Genest and F. Prêteux, "Off-line Signature Verification by Local Granulometric Size

Distributions", IEEE Trans. on PAMI, 19( 9), pp. 976-988, 1997.

3. Y. Kato, and M. Yasuhara, “Recovery of Drawing Order from Single-Stroke Handwriting Images”, IEEE Trans. on

PAMI, 22(8), 938-949, 2000.

4. R. J. Jalbert, “Detecting Forgeries Requires Analysis of Strokes and Pressures”, New York State Bar Journal,

November/December, pp.24-25, 2000.

5. V. Berx and J. De Kinder, “A 3D view on the ‘crossing lines’ problem in document investigation”, Proc. SPIE vol.

4709, pp.102-110, 2002.

6. Schirripa Spagnolo G., Simonetti C., Cozzella L., “Superposed strokes analysis by conoscopic holography as an aid

for a handwriting expert”, J. Opt. A: Pure Appl. Opt., 6, 869-874, 2004.

7. S. Tollkamp, H.G. Fackler, “Use of low voltage SEM in the detection of forgeries”, Int. J. Forensic Doc.

Examiners, 2, pp. 333–341, 1996.

8. S. Kasas, A. Khanmy-Vital, G. Dietler, “Examination of line crossings by atomic force microscopy”, Forensic Science International, 119, pp. 290–298, 2001.

9. Y. Malet, and G.Y. Sirat, “Conoscopic Holography application: multipurpose rangefinders”, J. Opt., 29, pp. 183-

187, 1998.

10. G.Y. Sirat, “Conoscopic holography: I. Basic principles and physical basis; II. Rigorous derivation”, J. Opt. Soc. A,

9, pp. 70-90, 1992.

11. M. Born and E. Wolf, Principles of Optics (New York: Pergamon), 1975.

12. ANSI B46.1, “Surface Texture (Surface Roughness Waviness and Lay)”, published by the American Society of

Mechanical Engineer, 2002.

13. J. Raja, B. Muralikrishnan, Shengyu Fu. Recent advances in separation of roughness, waviness and form, Precision

Engineering Journal of the International Societies for Precision Engineering and Nanotechnology, 26, pp. 222–

235, 2002.

Proc. of SPIE Vol. 5954 59540V-12