Licence

A License Plate Localization using Morphologyand Recognition

c. Nelson Kennedy Babu, Member IEEEProfessor and Head, CMS College of Engineering and Technology, Namakkal

II. MORPHOLOGICAL PLATE LOCALIZATION ALGORITHM -

MPLA

This plate localization algorithm MPLA is based oncombining morphological operation sensitive to specificshapes in the input image with a good threshold value bywhich the license plate is located. A fine percentage oflocalization of License plates is achieved by this algorithm.This is a better performing algorithm for License Plate Imageswith complicated background. The steps involved in thedevelopment of the algorithm are given below in the flowdiagram. See Figure 2.1

License Plate consists of many vertical edges because itconsists of Borders, Characters, and Digits. Sobel mask is usedto detect vertical edges in the input image.

The resultant image is converted into a binary image and thenumber of ones in each row are calculated and stored in anarray. The row corresponding to the license plate usually hashighest values. The next step is to find number of rowscontaining highest number of ones. This is identified to be acandidate region.

Consider 15 rows before and after the row selected withhighest value. This region is extracted from the input image.The same region is also extracted from the vertical edge

Krishnan Nallaperumal, Senior Member IEEEProfessor and Head, Centre for Information Technology and Engineering

Manonmaniam Sundaranar University, Tirunelveli - 627 012

and it is not invariant to scaling [5]. Region growing [3]algorithm requires more memory as it involves recursion andperforms satisfactorily only if the images are taken in goodambient lighting conditions. In this paper, a morphology basedmethod to extract the licenses plate from the car imagefollowed by segmentation and Recognition of contents in it,has been proposed. An approach using Morphology has beenreported by Martin et al [4]. This approach consists of mainlytwo parts namely License Plate Localization and Recognition.Characteristic features of license plate decide the size of thestructuring element in morphological operations [6]. Sequenceof closing and opening operations are performed on binarizededge image to eliminate the some of the candidate regions notcontaining the license plate. The morphological PlateLocalization Algorithm is presented in the section 2. Thesection 3 deals the segmentation and recognition. Theexperimental results and conclusions are discussed in Sections4 and 5 respectively.

Abstract- Locating the car license plate in an image a car is animportant step in car license plate recognition/identificationapplications. This problem poses many challenges like location ofLicense plate from images taken in poor illumination and badweather condition; plates that are partly obscured by dirt andimages that have low contrast. This paper presents a morphologybased method for license plate extraction from car imagesfollowed by the segmentation of characters and reorganization.This algorithm uses morphological operations on thepreprocessed, edge images of the vehicles. Characteristic featuressuch as license plate width and height, character height andspacing are considered for defining structural elements formorphological operations. The recognition of the contents of theLicense Plate is performed using cross correlation followed byNeural Network. The experimental results with a reasonablylarge set of car images are very encouraging.

I. INTRODUCTION

THE car license plate recognition! identification is animportant application of Intelligent Transport System(ITS). The objective is to extract and recognize/ identifyvehicle registration numbers from car images without anyhuman intervention. The main advantage is the ability tocapture information in the plates, at high traffic flow andspeeds, under conditions and speeds that human observers mayfind it difficult to manually record. Some of applications ofsuch license plate recognition! identification include vehiculartraffic logging, traffic control and law enforcements. Thecaptured registration number can also be subsequently checkedagainst a database of vehicle numbers, containing numbers ofvehicles of entirely local interest such as identifying vehicleswith access privileges to specified areas, lost or stolenvehicles. These systems typically have two parts. The first partis to correctly locate and extract the license plate from carimages and the next is to recognize/identify the license plate.There are various approaches to extract license plate such asuse of Hough transform, template matching and regiongrowing. However these approaches have certain limitations.

The candidate regions that can contain the license plate, asidentified using Hough transform [1], often include regionsother than those containing just the license plate. Alsodetecting actual vertical lines using Hough transform is moredifficult as they are more prone to noise than horizontal lines.Template matching [2] usually takes more computation time

978-1-4244-2746-8/08/$25.00 2008 IEEE

detected image for further processing. This is the requirednumber plate region called candidate region.

Vertical Edge Detection

Binarization of the image andidentifying the Candidate Region

Extracting the Candidate Region

2

maximum and minimum values of the (x, y) of the surroundingrectangle is taken in to account. Experimentally, xmin isdecreased by 0.5*f and increased xmax by 1.5*f. ymin andymax are decreased and increased by 1.1 *frespectively.

III. NN AND TEMPLATE MATCHING BASED LICENSE PLATE RECOGNITIONALGORITHM - NNTM-LPRA

This recognition algorithm is combining the features ofneural network and template of the characters to berecognized. The segmentation of the characters in the numberplate image is a method based on a technique named peak-to-valley, is used. The method searches for valleys in the verticalprojection of the binary image.

Iscandidate

region

Morphological Dilation

Reject

Character SegmentationCharacter segmentation is an important step in license plate

recognition system. The segmentation of characters in alicense plate is performed by using the following steps.

PreprocessingPreprocessing is very important for the good performance of

character segmentation. The preprocessing consists of thedetermination of plate kind. There are two kinds of licenseplate in India. One is black characters in yellow backgroundand the other is black characters in white background. Thecolor image is transformed into gray scale image.

Morphological Erosion

Plate Extraction

Figure 2.1 Design of MPLA algorithms

The image is dilated horizontally and vertically. Commonbright pixels between two dilated images are considered andonce again a horizontal dilation is applied. The structuringelements of dilation operator are a 7 by 7 matrix. This processfills the holes.

The morphological operations are given below Equation (2-1) and Equation (2-2).

{

A[i-r,j-C]+ }

Dilation: S =A ffi B[i,j] =max B[r,c],(i -r,j -c) (2-1)c A,[r,c] c B

{A[i-r,j-C]-B[r,C], } (22)

Erosion: S =AOB[i j] =min -, (i-r,j-c)cA,[r,c]cB

On applying erosion operator, the extra regions which donot belong to the plate are excluded. The mask of the erosionoperator is formed with 5 x 40 matrix. The Noise is removedusing median filter with 7 by 7 matrix.

In the step relating to the extracting the License plate, thesurrounding rectangle of the filtered image is found. Byenlarging this rectangle, the plate is located in the image. The

Enhancement ofCharacter RegionsThe quality of images varies much indifferent capture

conditions. Illumination and noise make it difficult forcharacter segmentation. It is known that some of methods likehistogram equalization and gray level scaling are used toenhance the images. Here contrast stretching method is used toenhance the license plate. This has two steps. Firstly thedensity of each gray level is calculated and the threshold of thedensity is obtained by using the Equations (3-1) and (3-2).

grayA(I,A(i,j) + 1) =grayA(I,A(i,j) + 1) + 1 (3-1)treshold={m*n)/256 (3-2)

Where A is the image, m and n are the height and width ofthe license plate. The contrast stretching transformation is oneof the simplest piecewise linear function. The idea behind thecontrast stretching is to increase the dynamic range of the graylevels in the image being processed. Firstly, the minimum andmaximum gray level, with whose density is larger than thethreshold, are found in the Figure of the density gray level.The gray level with bigger density of the maximum are set tobe 255 and the gray level with smaller values of density thanthe minimum are set to be zero and others are performed bythe following Equation (3-3).

B(i,j)=((A(i,j) - min)/(max- min)) x 255 (3-3)

Edge detectionEdge is the basic feature of an image. Edge detection plays

an important role in image processing, which can definite the

boundary of two regions with different gray levels. HereLaplacian Transformation is used to detect the edge of thecharacters. The main principle of Laplacian transformationusing the Laplacian mask to multiply a 3*3 region obtainedfrom the license plate image. The result of the multiplication issaved as the value of the pixel in the middle of the 3*3 region.

Location ofthe Candidate Regions ofCharactersRegion growing is a procedure that groups pixels or sub

region based on predefined criteria. The basic approach is tostart with the seed point and form this grow regions byappending to each seed those neighboring pixels that haveproperties similar to that seed.

Determining the Character Segmentation RegionsAccording to the prior knowledge of the license plate the

characters are segmented by finding the ratio of the characterwidth and character height. The prior knowledge is neededsince in the process of region growing more than one charactermay be combining together in a region or a character may besplit into some regions, and in the process of selectingcandidate regions some real character regions may be deleted,such as the character of "1".

The accurate positions of character segmentation regions aregot. According to these positions, characters from originallicense plate are extracted accurately. For characters extractedfrom original license plate are with very low contrast, the graylevel of characters should be increased in order to get highercontrast, using the below given Equation (3-4).

. {255 char(x,y,i) ~ threschar(x,y,l) =

char(x,y,i) char(x,y,i) < thres(3-4)

In above Equation thres is the threshold of the value ofcharacter segmentation image pixel, char(x,y,i) representscharacter segmentation image with location number of i, andchar'(x,y,i) represents the binarized character segmentationimage with location number.

Character RecognitionThe Character Recognition algorithm here uses the Cross

Correlation Combined with Neural Network. After stage twosuch as License Plate Localization and Segmentation ofcontents in it, the image is ready for matching with the imagesin the database. A hardcore matching doesn't give appropriateresults and it also doesn't prove to be efficient. By hardcore itis meant matching the actual pixels in the set of two images(plate and the database image). The method used made use ofcross-correlation operator.

Cross Correlation:In order to calculate the similarities between two images the

most common method is to use cross correlation. Crosscorrelation is based on a squared Euclidean distance measurein the form:

3

d 2r,t(U, v) = L L[!(x,y)-t(x-u,y _V)]2x y

(3-5)Equation (3-5) is an expression for the sum of squared

distances between the pixels in the template and the pixels inthe image covered by the template. The pixels near the edgesare ignored as the correlation is only carried out when theentire template can be correlated. A correlation near the edgeswould mean that a pseudo-pixel value in the image for the areaunder the template exceeding the image would have to beassigned. This could for instance be the average value of therest of the region. This has not been done, because a licenseplate exceeding the image is useless in terms of recognition, sothere is no point in spending computing power on theseregions.

Expanding the expression for the Euclidean distance, d2,produces:

d 2f ,,(U, v) = L L[!2(X,y)-2!(x- y),t(x,u,y-v)+t2(x-u,y-v)]x y

(3-6)Examining the expansion, it is noted that the term

LxL/2(x-u,y-v) is constant, since it is the value ofthe sum ofpixels in the entire template squared.

Assuming that the term LF(x,y) can be regarded as constantas well, means that it is assumed that the light intensity of theimage does not vary in regions the size of the template over theentire image. This is useful, because based on this assumption,the remaining term as expressed in Equation (3-7) becomes ameasure for the similarity between the image and the template.This measure for similarity is called the cross correlation.

c(u, v) =LL!(x,y).t(x-u,y-v)x y

(3-7)The assumption for which the validity of the measure is

based, is however somewhat frail. The term LxLyF(x,y) is onlyapproximately constant for images in which the image energyonly varies slightly. In most images this is not the case. Theeffect of this is that the correlation value might be higher inbright areas, than in areas where the template is actuallymatched. Also the range of the measure is totally dependent onthe size of the template. These issues are addressed in thenormalized cross correlation.

Normalized cross correlationThe expression for calculating the normalized cross

correlation coefficient, y(u, v), is shown in Equation (4-18).The way it handles the issue of bright areas is by normalizingthe measure, by dividing by the sum of the mean deviationssquared. Without this normalization the range is dependent onthe size of the template, which seriously restricts the areas inwhich template matching can be used. Also, the mean value ofthe image regions are subtracted, corresponding to the cross

covariance of the images. Often the template will need to bescaled to a specification input image, and withoutnormalization the output can take on any size depending on thescaling factor.

4

The Experimental results obtained by the LPL algorithmsare compared with other existing algorithms as given in thefollowing tables 4.1, 4.2, 4.3 and 4.4.

Table 4.1: Plate Localization of Class 1 - Comparison with othermethods


AlgorithmsTotal Correct False Location

Detections Detections Detections rates (%)MPLA 100 96 4 96[7] 100 94 6 94[8] 100 84 16 84[9] 100 95 5 95



Detections Detections Detections rates (%)MPLA 98 90 8 91.8[7] 98 (Not Reported)[8] 98 83 13 84.6[9] 98 87 11 88.7



Detections Detections Detections rates (%)MPLA 80 71 9 88.7[7] 80 69 11 86.2[8] 80 67 13 83.7[9] 80 64 11 80.0

In the case of Recognition algorithms, the way to measurethe overall success is to calculate the percentage of licenseplates that have been correctly identified by the machine andthen verified by a person supervising the test sample. Thispercentage is estimated using the Equation (4-1):

A =(rxl'flo (4-1)where A denotes the total system accuracy, T is per centsuccessful plate segmentation and I is the percentage ofsuccessful interpretation of the plate content (characters).

Based on the observation from the experiments on discussedalgorithms, it is found that for a car image with complexbackground, the system may find hard to locate the licenseplate. The system would possibly combine a backgroundtechnology to subtract the image that only has car in a fixedsituation. This may make the location simpler. Moreover, thesystem could be possible to combine other operators toimprove the recognition rate. Some problems remain to beresolved.

L L [!(x,y)-! t,v][t(x-u,y-v)-t]r(u, v) x y

~LxLJ(X- y)-f .,.fLxL)t(x-u,y-v)-tf(3-8)

f u, v is the mean value of the image pixels in the regioncovered by the template, and t is the mean value of thetemplate. The value of 0 lies between -1 and 1, where -1 is thevalue of a reversed match, 1 when a perfect match occurs. Thevalue approaches 0 when there is no match.

As it can be seen in the expression for the cross correlationcoefficient (Equation (3-8), it is a computationally expensivetask. For each pixel in the output image, the coefficient has tobe calculated. Assuming an image of size M2, a template ofsize N2 and not including the normalization (only thenumerator of Equation (3-8), the calculations involves

approximately N 2(M - N +1)2 multiplications and thesame number of additions.

In Recognition process, each character is presented to abank of neural networks, each looking for a specific characteror number on which it has been previously trained. Asconfidence estimation, the final decision for each letter ischosen by a winner takes all comparison of the networkoutputs. If however the absolute value of the winner's output istoo low, this flagged as a failure to recognize and therecognition of the plate is aborted.

IV. EXPERIMENTAL RESULTS AND ANALYSIS

The algorithms of License Plate Localization are developedbased on Morphological operations combined with goodthresholding and efficient structuring elements. The Databasecontaining 353 images different in size, background, cameraangle, distance, and illumination conditions is considered. Thedistance between camera and car considered is not fixed. Theclass of images is based the time and place where they areacquired. The Class 1 and Class 2 contain the images acquiredfrom Tirunelveli and Tuticorin during day and night times.Class 3 and Class 4 images are acquired from those sameplaces during day and night time.

The license plates consist of characters of variable size. Thealgorithms consider features like minimum and maximumcharacter heights, license plate width and height based onwhich structuring elements for morphological opening andclosing are designed. The robustness of the algorithms andfalse detection minimization can be enhanced to detectadditional features present in the license plate region.

Algorithms

MPLA[7][8][9]

TotalDetections

75757575

CorrectDetections

65606157

FalseDetections

10151418

Locationrates (%)

86.680.081.376.0

5

(a) (b)

Figure 4.1 Results ofMorphological Plate Localization Algorithm - MPLA.

(a) Original Images (b) Candidate Region (c) Extracted Plates

(c)

These problems include ambiguities between similarly shapedcharacters such as (B - 8), (0 - 0 - 0), (I - 1), (A - 4) andconfusion errors from the character pairs like (C - G), (D - 0)and (K - X). The comparison of Recognition rate with amongall the three algorithms are given below in the Table 4.5.

Table: 4.5 Comparison based on Recognition Rate

Recognition RateImages Category

NNTM MHTNN TWSNN-LPRA -LPRA - LPRA [10] [11] [60]

Character 98.5 97.8 95.996. 87. 97.

Class} 0 3 0

Number 97.4 97 96.897. 88. 84.3 7 3

Character 88.0 87.0 85.084. 78. 83.

Class28 3 0

Number 81.0 79.0 82.078. 79. 82.0 0 0

Character 97.8 96.0 95.496. 88. 97.

} 2 0Class3

94. 87. 88.Number 95.4 96.8 91.0

3 6 0

Character 90.1 89.7 88.385. 81. 92.4 0 8

Class477. 78. 93.

Number 79.9 85.0 84.0 } 6 4

v. CONCLUSIONThe developed algorithm for License Plate Localization and

Recognition is presented. The performance of the developed ofalgorithms for License Plate Localization and License PlateRecognition is found reasonable competing with the existingclass of algorithms discussed in the respective earlier chapters.The developed algorithms accurately localize and recognizethe License Plates given as input.

REFERENCES

[1] Y. Yanamura, M. Goto, D. Nishiyama, M. Soga, Ht. Nakatani and H.Saji, "Extraction and Tracking of the License Plate using HoughTransform and Voted Block Matching," In proceedings of IEEEIntelligent Vehicles Symposium, pp. 243-246, 2003.

[2] Y. Huang, S.Y. Lai and W. P. Chuang, "A Template Based Model forLicense Plate Recognition," In proceedings of IEEE InternationalConference on Networking, Sensing and Control, vol.2, pp. 737-742,2004.

[3] H. Hansen, A.W. Kristensen, M.P. Kohler, A.W. Mikkelsen, J.M.Pedersen and M. Trangeled, "Automatic Recognition of NumberPlates," Institute of Electronic Systems, Aalborg University.

[4] F. Martin, M. Garcia and J.L. Alba, "New Methods for AutomaticReading of VLP's," In proceedings of lASTED International conferenceon Signal Processing, Pattern Recognition, and Applications (SPPRA),2002.

[5] R.C. Gonzalez, R.E. Woods, Digital Image Processing, Addison-Wesley, 1993.

[6] J. Serra, Image Analysis and Mathematical Morphology, AcademicPress, London, 1982.

[7] S. Z. Wang and H. M. Lee, "Detection and recognition of license platecharacters with different appearances," in Proc. Conf. Intell. Transp.Syst., 2003, vol. 2, pp. 979-984.

6

[8] Faradji, F.; Rezaie, A.H.; Ziaratban, M.;, "A Morphological-BasedLicense Plate Location", IEEE International Conference on ImageProcessing, 2007, Volume 1,2007 pp:1 - 57 - 1-60.

[9] Suryanarayana, P.V.; Mitra, S.K.; Banerjee, A.; Roy, A.K.;, "AMorphology Based Approach for Car License Plate Extraction", IEEEINDICON, 2005, pp:24 - 27.

[10] Qian Gao, Xinnian Wang, Gongfu Xie, "License Plate RecognitionBased On Prior Knowledge", IEEE International Conference onAutomation and Logistics, pp: 2964 - 2968.

[11] Paliy, I.; Turchenko, V.; Koval, V.; Sachenko, A.; Markowsky, G.,"Approach to recognition of license plate numbers using neuralnetworks", IEEE International Joint Conference on Neural Networks,2004, vol.4, pp 2965 - 2970.

Documents

Licence