*Corresponding author.E-mail address: lhomer@cmla.ens-cachan.fr (E. L'Homer).

Pattern Recognition 33 (2000) 1147}1160

Extraction of strokes in handwritten characters

Eric L'Homer*

CMLA-DIAM, ENS de Cachan, 94235 Cachan Cedex, France

Received 26 August 1998; received in revised form 6 April 1999; accepted 6 April 1999

Abstract

Among the many handwritten character recognition algorithms that have been proposed in the past few years, few ofthem use models which are able to simulate handwriting. This can be explained by the fact that simulation models requirethe estimation of strokes starting from statistic images of letters, while crossing and overlapping strokes make thisestimation di$cult. The approach we suggest is to e$ciently deal with crossing areas and overlaps using parametricrepresentations of lines and thickness of stroke: a probabilistic model of strokes is described to extract non-overlappingstrokes of the image. A bayesian approach using a statistical study and a model of stroke crossing is described thatoptimizes the reconstruction of crossings and permits to characterize image of letters by robust graphs ofcurves. ( 2000 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.

Keywords: Handwritten character recognition; Thinning algorithms; Graphs of strokes; Stroke crossing; Stroke path detection

1. Introduction

An algorithm for reconstructing lines in cursive hand-writing using a parametric model is presented. Thinningalgorithms are used by handwritten characters recogni-tions techniques to represent letters as strokes, loops orcrossings. Many thinning algorithms have been proposedin the past few years, but these algorithms are not yetreally suited to images of handwritten letters.

Two techniques are usually used by thinning algo-rithms. Parallel thinning algorithms [1}3] are fast andeasy to code, but skeletons of closed loops and singlestrokes give the same results*a single line*and overlap-ping strokes often generate short segments which distortthe representation of a letter. Fig. 1 shows the skeleton ofan image of the word ` xssurea, which summarizes someof these di$culties. Some algorithms have been proposedto reduce the number of short segments which appear onthe crossing [4}6], or to estimate closed loops [7], butthese algorithms do not estimate overlapping strokepaths.

Vector thinning algorithms link boundary points ofletter in order to "nd the underlying strokes [8,9]. Thesealgorithms give plane lines as sequences of points, butcannot deal with crossings. Bodies of letters are then splitinto two parts: one part for regular strokes and one partfor components whose boundaries are not linked. Thisdual description of letters into lines and `crossing areasais better than the result given by parallel thinning algo-rithms, but is not satisfying enough to represent andrecognize letters.

Doermann and Rosenfeld [10,11] address this prob-lem of the interpretation of inferring strokes to recon-struct crossing areas. Their process "rst detects stroke-like and non-stroke-like regions of letter. Local con"g-uration of strokes (local measure of the con"dence thata given pair of segments are portions of the same stroke),and local compatibility of the reconstruction with theimage is used to interpret non-stroke region as high-curvature point, corner, crossing, etc. A cubic-spline ap-proximation is used to reconstruct strokes. Although thiswork seems to be the most useful, the approach proposeddoes not use a global model of crossing areas, and thenneeds to be extended to all simple cases.

The approach we suggest is to e$ciently deal withcrossing areas and overlaps using parametric representa-tions of lines and thickness of strokes, global model of

0031-3203/00/$20.00 ( 2000 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.PII: S 0 0 3 1 - 3 2 0 3 ( 9 9 ) 0 0 1 0 3 - X

Fig. 1. Squeleton obtained from a classical thinning algorithm:the closed loop is described as one single stroke (a), and paths ofoverlapping strokes are not well described (b, c, d).

Fig. 2. Relations between stroke path and tangents of strokeboundaries.

crossing of strokes and statistical studies to estimate themodel parameters.

Several remarks have convinced us to search suchresults, in the perspective of using strokes to recognizehandwritten words. First, crossings are less stable thanstrokes in letters and particularly in words, where over-laps between strokes of di!erent letters create crossingswhich do not appear in single letters. Moreover, strokesthat are cut by crossing are less stable than whole strokes.Finally, the process we have tested to decide if the markof a stroke is, in fact, the path of a closed loop seems toshow that the only way to decide this is to try to link thestroke with two other single strokes. This forces us totreat crossing areas, if we want to extract stable structuralinformation on handwritten letters and words.

We "rst de"ne a model of strokes, and present analgorithm to extract non-overlapping strokes. Thisallows us to split up letters into two parts. A secondalgorithm that optimizes the reconstruction of crossingsis then proposed. Finally, results of our algorithms ona NIST database is presented.

2. Extraction of simple strokes

We present here a simple model for strokes which usesrelations between boundaries and strokes to estimatelines. We consider in this section images of strokes with-out noise.

2.1. Dexnitions and properties

Let a stroke be a family of disc MDt, t3[0, 1]N, which is

de"ned by the stroke path l: [0, 1]PR2 and the radius ofthe discs r: [0, 1]PRH`. We assume that the functionsl and r verify the two following properties: l and r areC= and r5 2(t)(v2(t) for t3[0, 1], with v"dl/dt. The traceof a stroke on R2 is the set G"MM3R2, &t3[0, 1],M3D

tN, and the stroke boundaries is B"G1 CGs .

With these conditions on l and r, the two envelopes(E

1, E

2) of a stroke are always de"ned. The envelopes of

a family of discs D are points of each circumference

MCt,r, t3[0, 1], u3[!p, p]N of the family of discs which

verify: LCt,r/Lt is parallel to LC

t,r/Lu, that is, withlt"(x

tyt)*:

E1t"l(t)!r

t Ax5t

y5tB A

r5t

!J1!r5 2tv~2t

J1!r5 2tv~2t

r5t

B,

E2t"l(t)!r

t Ax5t

y5tB A

r5t

J1!r5 2tv~2t

!J1!r5 2tv~2t

r5t

B. (1)

Fig. 2 shows an example of a stroke with its envelopes.Let E be the set of points of the envelopes, then Bis strictly included in EXC

0XC

1, where C

0and C

1are

the circumferences of the "rst and the last discs of thestroke. So, to study relations between B and the strokepath, we "rst describe relations between E and the strokepath, and then we study cases where E is di!erent from B.

Given two points E1,t

and E2,t

on each stroke envel-ope, two relations may be used to compute the point l

tof

the path which "t E1,t

and E2,t

(Fig. 3):

Relation 1. (a) If the tangents t1

and t2

are dexned onE1,t

and E2,t

, then the two straight lines which are perpen-dicular to t

i, and contain E

i,t, for i"1, 2, cross on F

t, and

DE1,t

FtD"DE

2,tFtD. (b) l

t"F

t.

Relation 2. Let Htbe the center of [E

1,tE2,t

], then [Htlt] is

perpendicular to [E1,t

E2,t

], and Htlt"v

trtr5t.

Estimating the path link given the envelopes is thesame as matching the "rst envelope with the second, andone point E

1,tis "t with the point of E

2at the same

parameter value t using Relation 1(a).However B and E do not always "t, especially, when

the curvature of the stroke path is high: in this case, oneof the envelopes may be strictly included in G. In suchcases we cannot "t the two boundaries of the stroke toestimate the path link. We introduce the following de"ni-tion to describe this case:

1148 E. L'Homer / Pattern Recognition 33 (2000) 1147}1160

Fig. 3. Envelopes of the stroke is hidden between B1 and B4,and l.s.i between B2 and B3.

Fig. 4. Estimation of the stroke path for a `visiblea stroke.De5nition 1. The envelopes of a stroke are locally strictlyincluded in G (l.s.i.) at the parameter value t if:

&j'0 such that ∀e, DeD(j, di(t, e)(0, i"1 or 2

with

di(t, e)"[E

i,t!l

t`e]2!r2t`e.

Even if the envelopes are nowhere l.s.i, this is not su$-cient to force their inclusion in the boundaries of G, sinceenvelopes may be hidden by another part of the strokemark, or by another stroke. This de"nition, however,allows to obtain a simple and local way to decidewhether the two envelopes of a segment of stroke are`visiblea, i.e. is B included, and then to decide if it ispossible to "t both boundaries. Let o be the curvature ofthe stroke path. It is easy to show, with a Taylor'sexpansion, that the envelopes are l.s.i if

v2A1!orS1!Ar5 2v2BB#2

v5 rr5v!rrK!r5 2'0. (2)

In practice, we use a pathlink, so v(t)"1, and we assumerK"0. The relation becomes

1!orJ1!r2!r5 2(0. (3)

2.2. First algorithm

Our main hypothesis is to assume that handwrittencharacters images are a union of noisy stroke traces. Thegoal of this "rst algorithm is to divide the trace L ofa letter into two parts A and B. The "rst set A may bedescribed by segments of stroke in which both envelopesare included in theirs boundaries, and B"LCA.

Given a black and white letter image, our "rst step is tosmoothen the local boundaries strokes using classicalmasks, and to link the edge points (B) of the image, which

corresponds to discrete and noised sequences of points ofthe stroke boundaries.

Let A1 and B1 be two points of the same boundary, forwhich the distance between A1 and B1 is equal to the halfaverage h of the thickness of the letter's strokes. Thealgorithm tests the following hypothesis, H

v: (A1, B1)

is the end of a segment with visible envelopes. If the test ispositive, the algorithm estimates the corresponding edgepoints A2 and B2 of the second stroke boundary.

First, we estimate the two tangents tA1

and tB1

on bothpoints A1 and B1, using their neighboring edge points.Let FA (res. FB) be the "rst edge point that crosses thehalf-straight line which contains A1 (res. B1), and isperpendicular to t

A1(res. t

B1) (see Fig. 4(I)). These two

points are used to obtain, from a neighborhood S of FAand FB, a set of positions for A2 and B2.

For each couple of points (A@2, B@2) of S, which de"nesa segment of stroke ¹"[A1, B1; A2, B2], the algorithmtests if ¹ is l.s.i, which is made by testing if relation (3) isveri"ed. We estimate o, r and r5 using the centers(C@1, C@2) of both segments of edge points. (Fig. 4(II)).

We arbitrate among the possible couple of pointswhich verify condition (3) by choosing the ones whichbest verify relation 1(a). A cost function is de"ned, whichdepends on the angles between the tangents on the fouredge points. A Gaussian noise is introduced to deal withthe error on the estimation of these angles. Then the costfunction is de"ned as the quadratic distance between theexpectation of the four angles and their estimations.

This process is used to link, step by step, the twoboundaries of each stroke with visible envelopes: giventhe two ends (B1, B2) of the little segment of stroke, let D1be an edge point of the right neighborhood of B2, ata distance h from B2, and SD be the set of left neighboring

E. L'Homer / Pattern Recognition 33 (2000) 1147}1160 1149

Fig. 5. Examples of results with classical thinning algorithms (the AFP3 algorithm described by Guo and HallGuo [2].

edge points of B1, at a distance smaller than h from B2

(Fig. 4(IV)). The algorithm follows the process describedbelow to "t in SD the best corresponding end D2.

2.2.1. End of strokeStrokes end in three cases:

1. When r5 is too high on a little segment and thus is notcompatible with a de"nition of a stroke. This happenswhen the stroke ends at a crossing, like a T-junction forexample.

2. When the test is negative for all the points of S. Thiscorresponds to a segment of stroke whose curvature ishigh.

3. When the boundaries meet each other, the strokestops naturally at its end.

Finally we obtain, for each letter, one part described bysequences of linked edge points, which form simplestrokes, that is strokes with visible envelopes, and an-other part which corresponds to traces of stroke cross-ings and small segments of stroke with high curvature.Fig. 6(I) shows results of this "rst algorithms for imagesof the training set of the NIST form-based handprintrecognition system.

3. Models for stroke crossings

3.1. Motivation and Hypothesis

Results obtained by this "rst algorithm are similar tothose of vectorial thinning algorithms: letters are de-scribed as graphs of strokes, and part of letters wherestrokes cross are not described, apart from connectingstrokes.

The main disadvantages of such results are the descrip-tion of closed loops as single strokes, and the variation ofthe graph of curves for letters of the same class; thereforethe study and the parameter estimation of the deforma-tion of such graphs to simulate handwritten letters aredi$cult and ine$cient.

Let us take the following examples to clarify bothproblems: Fig. 5 shows four examples of letters a andthe corresponding thinning results obtained with a classi-cal parallel thinning algorithm (the AFP3 algorithmdescribed by Guo and Hall [2]. As usual, linesobtained are often bad estimations of overlapping strokepaths.

Fig. 6(I) shows the corresponding results obtained byour "rst algorithm, that can be described as graphs ofstrokes: the graph nodes are strokes and `crossing areasa,and a graph structure connects nodes that join each other(Fig. 6(II)).

For these four examples of the same class of letters, weobtain four di!erent graph structures, whereas a charac-terization of these examples using stable graphs of curvesis possible, if we are able to estimate stroke paths thatcross each other, and if we segment paths to obtainsmooth lines (Fig. 6(III)). In such descriptions, calledgraphs of curves, that are close to those of on-line hand-writing characters, crossings are `implicita and do notincrease the variation of graph structures by segmentingsmooth strokes: graph structures may be reduced to thenumber of curves.

Our goal is then to deal with crossings and overlap-ping strokes, to decrease the variation of graph struc-tures. Our approach is "rst to use a parametric model ofstrokes in order to merge strokes that are segmented bycrossing, and then to determine between the di!erentcon"gurations of crossing the one which best matches theobservations.

1150 E. L'Homer / Pattern Recognition 33 (2000) 1147}1160

Fig. 6. Graphs of strokes without crossings estimations (I and II), and graph of curves with crossing identi"cation (III).

Three main hypotheses are made to make an e$cientalgorithm of crossing reconstruction:

h1. Curvature of strokes has a low value when crossing.h2. A branch of a stroke may merge with at least two

other branchesh3. The algorithm deals with crossings with less than"ve branches.

A branch of a crossing Cr is a stroke which ends in Cr.The "rst hypothesis is used to merge two branches

A, B of a crossing into a stroke SA,B

only if SA,B

can bedescribed as one smooth stroke. S

A,Bis the link between

A and B. Fig. 7 shows a crossing with four branches ona letter `ka, and three di!erent ways to reconstruct strokepaths into these crossing, which correspond to threedi!erent descriptions of (A, B): as two strokes (Fig. 7(a)),as one stroke whose curvature is high in its center (Fig.7(b)), and as a loop (Fig. 7(c)). Observations on this imageare not su$cient to choose between these three solutionsbut, if we describe these three solutions using graphs ofcurves we obtain three similar graph with three curves(Fig. 7(II)). Thus, generally, links that form smooth strokesare the only ones that are useful and easy to estimate.

The second hypothesis is made to deal with closedloops, and the last one is done because in the NISTdatabase we use, the number of crossing with more thanfour branches is very low and, conversely, the number ofcon"guration of such crossing is too high to test each ofthem.

3.2. Parametric model of strokes

A crossing C is a set of overlapping strokes. The traceof a crossing is described by the branches of the crossing,parts of strokes that do not overlap, that are connectedwith the `crossing areaa, the center of the crossing that isnot described as a stroke.

The "rst algorithm does not use a parametric model tode"ne strokes, that are de"ned as sequences of edgepoints. To reconstruct smooth strokes whose edges arehidden by a crossing, we introduce the following para-metric models on paths and thickness of strokes D(l, r):

l(t)"Gx(t)"f (t)"+4

i/0aiti,

y(t)"g(t)"+4i/0

biti

(4)

and, if ¸ is the length of the line of the stroke, and p theaverage thickness on the letter:

r(t)"c0#c

1t#c

2expA!t

¸

pB#c3

expA(t!1)¸

pB" +

q/0.3

cq.h

q(t) , (5)

with 0)t)1. We note S(h) such a stroke, withh"(a, b, c).

Observations on a trace of a stroke is de"ned by twosequences of edge points. Let (X

l)1..n

and (Xr)1..n

be thesequences of these points. Noise on the edge points

E. L'Homer / Pattern Recognition 33 (2000) 1147}1160 1151

Fig. 7. Three di!erent ways to reconstruct lines into a crossing, which correspond to three similar graphs of curves.

depends on di!erent factors like pen or pencil type, paperand ink quality, scanning accuracy. We use a Gaussianadditive and independent noise on each edge point tomodel such variations. Let Mk

1,2,k

nN be the abscissa of

the sequences of edge points. We have supposed thatstroke paths are smooth, so we can assume that theenvelope is included in the edge stroke. Then, let t andn be the unit tangent and normal vectors of the strokepath, and (l

r, l

l, m

r, m

l) four n-dimensional Gaussian vec-

tors, with k"k1,2, k

n:

Xr(k)"E

r(k)#l

r,kn(k)#m

r,kt(k),

Xl(k)"E

l(k)#l

l,kn(k)#m

l,kt(k) (6)

with (Er, E

l) the two sequences of points of the envelopes

(Fig. 8):

Er(k)"l(k)!

r5 (k) r(k)

v(k)t(k)!r(k) n(k)S1!

r5 2(k)

v2(k),

El(k)"l(k)!

r5 (k) r(k)

v(k)t(k)#r(k) n(k)S1!

r5 2(k)

v2(k). (7)

3.3. Topology of crossings

Let Tm

be the set of m]m binary (0 or 1) symmetricmatrix c that verify: ∀j, +

ic(i, j))2 and c(i, i)"0 ∀

i. If

we assume that c(i, j)"1 corresponds to a link betweenbranches i and j of a crossing with m branches, andc(i, j)"0 corresponds to an absence of such link betweenboth branches, then T

mis the set of crossing con-

xgurations: according to the main hypothesis, Tm

describes the di!erent reconstructions of crossings withm branches. For example, the matrix c

3.2(8) corres-

ponds to a crossing with three branches A, B, C, para-meterized by h

B}C, the parameters of the link S(h

B,C), and

by hA, the parameters of the stroke S(h

A) whose trace is A,

and the matrix c3.3

corresponds to a three-branch cross-ing parametrized by (h

A}B, h

A}C), the parameters of the

two links S(hA}B

) and S(hA}C

) between the "rst branch andthe two others: this con"guration corresponds to a closedloop.

c3.2

"A0 0 0

0 0 1

0 1 0B, c3.3

"A0 1 1

1 0 0

1 0 0B. (8)

Fig. 9 shows the set T3.

Let Cm

be the group of cyclic permutations on sets withm elements. T

m/C

mis the set of types of crossings, i.e. the

set of con"gurations modulo cyclic permutations onbranches. We assume that con"gurations with the sametype have the same probability of appearance, as we

1152 E. L'Homer / Pattern Recognition 33 (2000) 1147}1160

Fig. 8. (I):sequences of points, observations on a 4-branch crossing. (II):a$ne interpolation of the abscissa of the edge points for themerging of two branches.

Fig. 9. The 8 con"gurations for a crossing with 3 branches.

Fig. 10. The 12 types of con"gurations for a crossing with4 branches.choose not to take account of the orientation of strokes

in our a priori knowledge of con"guration distribution.Fig. 10 shows the set of types of T

4.

Finally, a crossing Cr with m branches, whose con"g-uration is c3T

m, is parameterized with h

c"Mh

k,lD

c(k, l)O0NXMhkD+

lc(k, l)"0N, the parameters of links

S(hk, l

) and branches S(hk) with no links on Cr.

Then the likelihood of the observations (X)12r

is de-"ned as follows: de"ne for each point X

ithe point E

iin

the trajectories of S(h) which minimizes the di!erencebetween the thickness r

ion this point and DE

i!X

iD. Let

Ri"DE

i!X

iD!r

i. We assume that (R) is a su$cient

statistic for (X), and that R&N(0, p2.Id), where p2, thevariance of the noise on the edges of strokes, is estimatedon the image of letter.

4. Estimation of crossing con5gurations

In this section, we detail the principles of our secondalgorithm, an algorithm of crossing estimation. Let Cr bea stroke crossing with m branches. We assume, under thehypothesis of the existence of a link S(h

A,B) between two

branches A and B, that hA, B

is independent of the otherlinks and of the crossing branches. Thus in a "rst step, thesecond algorithm makes a least square estimation of allthe possible links between two branches on Cr. In a sec-ond step, the algorithm computes for each con"guration

E. L'Homer / Pattern Recognition 33 (2000) 1147}1160 1153

Fig. 11. Examples of the merging of a simple stroke with a closed loop (a}c), and the "nal estimation of the crossing (d). (a) and (b)correspond to false hypotheses, and (c) is the true one.

an estimation of the maximum a posteriori loglikelihoodof the observation, and "nally chooses the ones whichmaximize it.

4.1. Two branch merging

Let (A, B) be two branches of a crossing, and HA}B

thehypothesis that (A, B) merge into one stroke S(h

A}B)

whose path curvature remains at a low value. Given thesequences of edge points (Ar, Al)

1,2, nAand (Br, Bl)

1,2, nBwhich describe the two branches, let k"Mk

1,2, k

nA,

knA`1

,2, knA`nB

N be the abscissa of these sequences underH

A}B. (Ar, Al) and (Br, Bl) are sequences of noisy observa-

tions of points of the two envelopes of SA}B

. Let (Cr, C

l) be

the sequences (Ar, A

l)X(B

l, B

r): C

r(k

i)"A

r(k

i)1

ixnA#

Bl(k

i)1

i;nA. The estimator hK

A}Bis computed with the para-

metric model of strokes, using estimation of the abscissaand tangents, and simpli"cation of the formula 7.

First, we use the main hypothesis h1 to make an a$neinterpolation (kK ) of the abscissa (Fig. 8(II)). Unit tangentsand normal vectors are then locally estimated using thisa$ne interpolation of the path.

We note these estimators, tiand n

iat the abscissa i.

Furthermore, we assume that r5 /v;1, so as to neglectthe term (r5 (k

i)r(k

i)/v(k

i))t(k

i), and we simplify

r(ki) n(k

i)J1!r5 2(k

i)/v2(k

i) into r(k

i) n(k

i). Finally, we

obtain

Cr(k

i)"l(k

i)!r(k

i)n

i#l

r,ini#m

r,iti,

Cl(k

i)"l(k

i)#r(k

i)n

i#l

l,ini#m

l,iti. (9)

Then a least-squares linear estimation hKA}B

of the para-meters of S

A~Bis obtained from the linear system (9). Fig. 12

shows some examples of stroke estimated using this process.

Remark. The e$ciency of the a$ne estimation of theabscissa depends on the main hypothesis h1: without it,such linear estimations would not be acceptable.

If one uses this algorithm to merge a branch A withanother B whose thickness is higher, for instance whenB is the trace of a closed loop, then the result is oftenwrong because one of the edges of S

A~Bis hidden in the

trace B (Fig. 11). So in such a case, we introduce two sub-hypothesis H

A}Brand H

A}Bl:

1. HA}Br

Only the right edge of SA}B

is visible in the traceof the branch B.

2. HA}Bl

Only the left edge of SA}B

is visible in the trace ofthe branch B.

1154 E. L'Homer / Pattern Recognition 33 (2000) 1147}1160

Fig. 12. The six di!erent simple links between branches for a crossing with 4 branches.

Under these two hypotheses, we assume that the thick-ness of S

A}Bis constant, to estimate the hidden edge,

and to obtain a least-square estimation of hKA~Br

andhKA~Bl

(Fig. 11).We obtain for each hypothesis of branch merging

a quick and linear estimation of the correspondingstrokes. Using the same process to estimate the para-meters h

k,kof each branch of the crossing, an estimation

hKc

of the parameters of each con"guration c is done(Fig. 12).

The last step of the algorithm is to identify, within theset T

m, which con"guration best "ts the observations.

4.2. Crossing identixcation

Let (X)i/12r

be the observations on a crossing Cr withm branches, and Let n"2r be the dimension of theobservation on the crossing. Let # be the sample space ofhK (X)"MhK

k,l, k, l3M1,2, qNN, where q is the cardinal of

Tm. We use a bayesian approach to identify crossing

con"guration. We de"ne a decision function d :s]#PT, which selects estimate c("d(X, hK (X)) of thecon"guration of Cr.

To construct d, we introduce a loss functionC:T]TP[0, 1] which measures the cost introduced bythe identi"cation of a con"guration c by c( , and we take

the function which minimizes the expected risk R:

R(d, X, hK (X))"q+c/1

P(cDX, hK (X)) . C(c,c( ) (10)

"

q+c/1

p(XDhK (X) Dc).Pm(c)

p(X, hK (X)). C(c, c( ), (11)

R(d, X, hK )"q+c/1

p(XDhKc(X), c).p(hK (X)Dc).P

m(c)

p(X, hK (X)). C(c, c( ). (12)

The function C we used is such that C(c, c( )"0 if c"c(and C(c, c( )"1 if cOc, so the choice of c( becomes themaximum a posteriori likelihood (MAP) estimator:

c("argmaxc

¸(X, hK , c) (13)

"argmaxc

log p(XDhKc, c)#(log p(hK Dc)#log P

m(c)) , (14)

that can be split into two terms, one for the loglikelihoodof the observations, and one for an a priori logdensity ofthe parameter estimators.

4.2.1. A priori distribution of the parametersIn the context described in the beginning of this article,

no information about the class of the image of the letterwhich is treated is available. So we introduce a su$cientstatistic of the parameters h which does not depend on

E. L'Homer / Pattern Recognition 33 (2000) 1147}1160 1155

Fig. 13. A false link between two branches on a crossing with 4 branches.

Fig. 14. Examples of estimations of stroke paths for images of letters of a NIST database.

1156 E. L'Homer / Pattern Recognition 33 (2000) 1147}1160

Crossing by letter 0 1 2 3#Number of observed

letters162 291 62 5

% 31% 56% 12% 1%

Number of branches 3 4 5Number of observed

crossings354 75 2

% 82.1% 17.4% 0.5%

Types T3.1 T3.2 T3.3 othersNumber of strokes 3 2 2 *

Number ofcon"gurations

1 3 3 *

Number of observedcrossings

6 97 245 6

% 1.7% 27.4% 69.2% 1.7%

(Continued on next page)

the orientation of the stroke but depends only on theintrinsic shape of the stroke, that is a function of thecurvature. For each stroke S

k,l, which merges one branch

k whith another l, there are two options: this stroke isa real one, or this stroke does not exist, and we takea statistic /

k,l, function of the curvature, whose densities

are di!erent enough to discriminate the two hypothesis.For example, functions like the mean of the curvaturewill not be suitable because even if the link does not exist,the mean of the curvature of the line may have a lowvalue (Fig. 13).

Let ¸ be the length of a stroke. The statistics we use is:

*o(h)"PL

0

Do.o@(s)D ds, (15)

Thus *o(h) allows one to take into account the curvaturevariations.

As we assume that the density of *o depends only onthe fact that the stroke exists or does not exist, an empiricestimation of the density of *o for both hypotheses hasbeen done on 200 crossings of the database images.A parametric estimation of the law of these two empiricdensities was computed using a gamma law c(a

1, b

1) with

a1"0.68, b

1"2.15 when the connection really exists,

and a law c(a2, b

2) with a

2"1.51, b

2"0.01 when the

link is missing. Let g(. D 0) be such density when the linkdoes not exist, and g(. D 1) when the link exists. Eventual-ly, the a priori density of *o(hK ) is:

p(*o(hK )Dc)" <k,l/12q,kEl

g(Do(hKk,l

)Dc(k, l)). (16)

The a priori distribution of the con"guration Pm(c) is

estimated on a test set of letters of the database, under thehypothesis that the a priori probability of a con"gurationdepends only on the type of the con"guration.

4.2.2. Loglikelihood estimationTo compute the loglikelihood log p(XDc, hK ), we have to

estimate the series (R) of errors on the thickness on theedge points for each con"guration c. When X

istays on

a branch of the crossing that is described with only onestroke S, the least-square estimation of the parameters ofS computes straight the error R

iof the thickness on the

edge point Xi. When two strokes overlap on a branch, or

when Xiis an edge point of the area of the crossing, the

algorithm chooses for Xithe minimum of errors for all

the strokes of c. But in practice the series R is notcomputed for each con"guration and sub-con"gurations.A minimal di!erence R

i,Ml,kNis computed for each edge

points Xiand for each stroke ShK l,k, and for each con"g-

uration we take: Ri"min(R

i,Ml,kND c(k, l)"1).

A problem appears when one follows this process: thedimension of the parameters which describe a crossingdepends on the type of con"guration, i.e. on the numberof strokes of the con"guration. So the law of R2"

+i/12r

R2i, which is a su$cient satistics for the loglikeli-

hood, depends on the con"guration: a bias appears thatgives an advantage to the con"gurations whose numberof parameters is higher. So we use a loglikelihood that ispenalized by a penalty term pen(dim(X), dim(c)) to takeinto account this bias: it is described in the appendix.

Finally, the algorithm computes for each con"gurationand possible subcon"guration c, an estimation of themaximum of the penalized a posteriori loglikelihood:

¸C(X, hKc, c)"log(p(XDhK , c)!pen(dim(X), dim(c))

#log p(o(hK )Dc)#log Pm(c)

"log(p(RDhK , c)!pen(dim(X), dim(c)) (17)

#log Pm(c)# <

k,l/12m, kEl

g(*o(hKk,l

)Dc(k, l))

(18)

The estimator of the con"guration becomes:

c("argmaxc|T

¸C(X, hKc, c) (19)

and the crossing is described by strokes correspondingto c( .

5. Results

The estimation of the a priori distribution of the cross-ing types has been done on 520 images of letters (20letters for each class) of the NIST database. The follow-ing arrays summarize the distribution of the number ofcrossings by letters, the number of branches by crossing,and the distribution of crossing types for crossings with3 and 4 branches.

E. L'Homer / Pattern Recognition 33 (2000) 1147}1160 1157

Types T4.1 T4.2 T4.3 T4.4 T4.5 T4.6 T4.10 othersNumber of strokes 4 3 3 2 2 3 3 *

Number of con"gurations 1 4 2 1 2 8 4#4 *

Number of observed crossings 0 7 1 56 2 5 2 2% 0% 9.3% 1.3% 74.6% 2.7% 6.7% 2.7% 2.7%

The last column (`othersa) corresponds to crossingswhere one branch is the mark of three or more strokes,which is not described by our model.

Fig. 14 shows 24 examples of letters for which thecrossing identi"cation is true. The last four letters of this"gure show examples of wrong results, which happen inabout 10% of the letters. The image (a) shows a letterwith a 4-branch crossing and a 5-branch crossing that arenot reconstructed. The letters (b) and (c) show the mainfactor of errors: when the curvature of one branch ofa closed loop varies strongly, the corresponding linkbetween the branch and the closed loop may not bereconstructed.

In order to illustrate the importance of a prioriknowledge to estimate strokes, the last example (d)shows a crossing identi"cation that does not correspondto the true line of the letter, but will be acceptablewithout any knowledge of the class of the letter. Suchresult indicates that our method will not be useful forimages of word without any a priori knowledge of thegraph of curves to be estimated conditionally to the classof letters.

6. Conclusion

We have described an algorithm for reconstructingstroke paths of o!-line handwritten characters, whichdeals with the case of stroke crossings. A simple paramet-ric model of stroke has been introduced. The method isbased on the properties of the edges of this model, and ona statistical study of the stroke crossings on handwrittencharacters.

With this method, handwritten characters are de-scribed by simple graphs of curves that are similar to theon-line description of characters. Therefore, unlike thedescription obtained by classical thinning algorithms, anadaptation of on-line handwriting recognition methodsbecomes possible, and an e$cient model for the law ofgraphs of curves can be used.

Appendix A

In this section, we "rst analyze the bias of the loglikeli-hood estimator in a simple case, then we generalize theresults to the case of stroke crossings.

Let X be a random vector on R2d, which veri"es underthe hypothesis H

0:

X(i)"d~1+k/0

akA

i

2nBk#m

i, i"122n (20)

and, under the hypothesis H1:

X(i)"d~1+k/0

bkA

i

nBk1MixnN

#bk`dA

i

nBk1Mi;nN

#mi,

i"122n, (21)

m&N(0, I2n

). With an observation x of this randomvector, we would like to identify the true hypothesis,and estimate the corresponding unknown parametersA"(a

0,2, a

d~1)H or B"(b

0,2, b

2d~1)H.

We can recognize, with simpli"cations, the two hy-pothesis between two branches of a crossing: the twobranches are linked and form one stroke (H

0), or they are

the mark of two di!erent strokes (H1).

With the following notations: ¹2n"((i/2n)j), ¹

n"

((i/n)j) and ¹n0"(Tn

00Tn

), the two models can be rewrittenlike this:

H0: X"A¹

2n#m, (22)

H1: X"B¹

n0#m. (23)

For each hypothesis, the estimator of the maximumlikelihood for A and B corresponds to the least-squareestimator:

AK "(¹H2n¹

2n)~1¹H

2nX, (24)

BK "(¹Hn0¹

n0)~1¹H

n0X. (25)

Thus the estimator of the maximum of the loglikelihoodfor H

0and for H

1is:

H0: ¸

0(X)"!1

2(X!AK ¹

2n)H(X!AK ¹

2n)#cst(n), (26)

H1: ¸

1(X)"!1

2(X!BK ¹

n0)H(X!BK ¹

n0)#cst(n). (27)

Under H0

we have:

¸0(X)

@H0"cst(n)!1

2(A¹

2n#m!AK ¹

2n)H

](A¹2n#m!AK ¹

2n) (28)

"cst(n)!12mH*

2nm (29)

1158 E. L'Homer / Pattern Recognition 33 (2000) 1147}1160

with *2n"(I

2n!¹

2n(¹H

2n¹

2n)~1¹H

2n). Setting *

n0"

I2n!¹

n0(¹H

n0¹

n0)~1¹H

n0, we have, for the statistics of the

loglikelihood estimator of the hypothesis H1

under H0:

¸1(X)

@H0"cst(n)!1

2(¹

2nA#m)H*

n0(¹

2nA#m). (30)

Under H1, both statistics become:

¸0(X)

@H1"cst(n)!1

2(¹

n0B#m)H*

2n(¹

n0B#m) (31)

¸1(X)

@H1"cst(n)!1

2mH*

n0m (32)

Thus the di!erence ""¸1(X)!¸

0(X) which is used to

identify the hypothesis, is, under H0:

"@H0

"¸1(X)

@H0!¸

0(X)

@H0

"!12

(mH(*2n!*

n0)m!2(AH¹H

2n*n0¹

2nA (33)

#AV10H¹H2n

*n0

m)) (34)

and under H1:

"@H1

"¸1(X)

@H1!¸

0(X)

@H1

"!12

(mH(*n0!*

2n)m!2(BH¹H

n0*2n¹

n0B (35)

#BH¹Hn0

*2n

m)) (36)

In both cases, the di!erence is the sum of two terms; thelaw of the "rst term is !1

2s2(d), and the law of the second

is a Gaussian N(!2D(n), 4D(n)). Under H0, D(n)"

AH¹H2n

*n0¹

2nA, and under H

1, D(n)"BH¹H

n0*2n¹

n0B,

but in both cases, D(n) is equal to the quadratic distancebetween the series of points of the true curve and the onesestimated on the false hypothesis: under H

0, ¹

2nA is the

vector of the true points of the curve (without any noise),and ¹

n0B is the vector of the estimation of these points

under the false hypothesis. Therfore,

D(n)@H0

"(¹2n

A!¹n0

BK )H(¹2n

A!¹n0

BK )

"AH¹ H2n

*n0¹

2nA.

The calculus is the same for H1.

Under H0, D(n)&10~14 for the curves observed on

the stroke crossings, which is not surprising because thenumber of parameters of the false models is higher thanthe number of parameters of the true model. So we canneglect this term and, under this hypothesis, if we com-pare both loglikelihood, there is a systematic bias onaccount of H

1, whose law is !1

2s2(d).

Under H1

(2 curves), the quadratic distance D(n) be-tween the true curves and the one estimated with H

0is

a measure of the error between both models. Under H1,

the bias on account of H1

is the sum of two terms, whoselaw are 1

2s2(d) and N(D(n), D(n)).

Thus the choice of a penalty term depends closely onthe quadratic distance between both models: if underH

1D(n) has a low value, which shows that the observa-

tions are well described by the model with one curve, wehave to choose this hypothesis, to favour the model withless parameters. So we turn the hypothesis H

1into H

1D,

which is the model H1

with D(n)"n.D.*/

, where D.*/

isthe average distance for which under H

1, H

0cannot be

chosen. Thus let LCi(X) be the penalized loglikelihood,

i"0, 1:

¸C0(X)"¸

0(X)!pen(d, n, D

.*/), (37)

¸C1(X)"¸

1(X)!pen(2d, n, D

.*/), (38)

where C is such that:

P(¸C0(X)!¸C

1(X)(0DH

0)

"P(¸C1(X)!¸C

0(X)(0DH

1). (39)

The error rates are the same under both hypothesis.Giving n and D

.*/, such function pen( ) can be easily

estimated using simulation of the loglikelihood bias.The same process is used to compensate for the log-

likelihood bias of the stroke crossing observations, as thissimple case corresponds to the model of one edge withtwo branches. So we just have to make an empiricalchoice of a `limita crossing for which two branches,which are not linked, can be described as one stroke, todetermine D

.*/.

References

[1] L. Lam, C.Y. Suen, An Evaluation of parallel thinningalgorithms for character recognition, IEEE Trans. PatternAnal. Mach. Intell. 17 (9) (1995) 914}919.

[2] Z. Guo, R.W. Hall, Fast fully parallel thinning algorithms,CVGIP: Image Understanding 55 (3) (1992) 317}328.

[3] M. Roth, H. Bunke, E.G. Schukat-Talamazzini, O!-line.cursive handwriting recognition using hidden MarkovModels, Pattern Recognition 28 (9) (1995) 1399}1413.

[4] S. Iliescu, R. Shinghal, R.Y.-M. Teo, Proposed heuristicprocedures to preprocess character patterns using lineadjacency graphs, Pattern Recognition 29 (6) (1996)951}975.

[5] M.J.J. Holt, I.S.I. Abuhaiba, S. Datta, Processing of binaryimages of handwritten text documents, Pattern Recogni-tion 29 (7) (1996) 1161}1177.

[6] D.S. Yeung, H.S. Fong, A fuzzy substroke extractor forhandwritten chinese characters, Pattern Recognition 29(12) (1996) 1963}1980.

[7] J.-C. Simon, O!-line cursive word recognition, Proc. IEEE80 (7) (1992) 1150}1161.

[8] J.-C. Pettier, J. Camillerapp, Segmentation et representa-tion d'images de traits, Publication interne, 756, IRISA.Campus de Beaulieu 35042 Rennes Cedex France, 1993.

[9] T. Suzuki, H. Nishida, S. Mori, Thin line representationfrom contour representation of handprinted characters,From Pixels to Features III: Frontiers in HandwritingRecognition, Elsevier, Amsterdam, 1992, pp. 29}40.

[10] D.S. Doermann, A. Rosenfeld, The interpretation and re-construction of infering strokes, International Workshopon Frontiers in Handwriting Recognition, 1993, pp. 41}50.

[11] D.S. Doermann, A. Rosenfeld, Recovery of temporal in-formation from static images of handwriting, Int. J.Comput. Vision 15 (1}2) (1995) 143}164.

E. L'Homer / Pattern Recognition 33 (2000) 1147}1160 1159

About the Author*ERIC L'HOMER received a Ph.D. degree in applied mathematics from the University of Orsay, France, in 1998.Since 1993, he has been on the CMLA, Ecole Normale Superieure de Cachan. He is currently a research scientist at Paris 13 University.His research interests include pattern recognition, stochastic neural networks and Gaussian mixture distribution.

1160 E. L'Homer / Pattern Recognition 33 (2000) 1147}1160