4965 Abdulrahman Al-Juraifani

8/17/2019 4965 Abdulrahman Al-Juraifani

1/17

A Real-Time Recognition System for

Saudi License Plates using LabVIEW

By:

Abdulrahman Al-Juraifani

Yazeed Al-Audah

Ahmad Al-Zuhair

Level:

Undergraduate

Advisor:

Dr. Mohamed Deriche

13/2/2012


2/17

Abstract ––

Automatic License Plate Recognition (ALPR) technology covers

numerous applications in traffic monitoring and access control. Several LPR

methods have been presented over the last few years, many of which exhibit

large computational load making them slow and impractical for real-time

applications. In this work, we propose a real-time license plate recognition

system that we simulated using LabVIEW. The system discussed here has

been designed to work with Saudi Arabian license plates and achieves real-

time performance. To reduce complexity, the proposed system uses a hybrid

approach for locating the license plate region and recognizing the characters.

The system utilizes low resolution images of 640x480 pixels in order to

further enhance the processing speed and reduce the size of the stored data.

The success rate can reach, under optimum conditions1, up to 94%, with an

average success rate of 84%; with a processing speed of less than 40

ms/plate which is less than half the processing time achieved by other

authors. The system is fully functional and ready for deployment in

commercial systems.

1 Optimum conditions: good lighting, nice weather and clean plate.


3/17

3

A Real-Time Recognition System for Saudi

License Plates using LabVIEW

Table

of

Contents

1. Introduction .......................................................................................................................... 4

2.

Background .......................................................................................................................... 5

3.

Proposed System ................................................................................................................. 6

3.1 License Plate Localization .............................................................................................. 7

3.2

LP Segmentation & OCR ........................................................................................ 11

3.2.1 Preprocessing ........................................................................................................... 11

3.2.2 Character Segmentation & Recognition ............................................................ 12

4.

Results ................................................................................................................................. 15

5. Conclusion .......................................................................................................................... 16

References ................................................................................................................................... 17

List of Figures

Figure 1: A typical Saudi Arabian license plate .................................................................... 5

Figure 2: Localization phase block diagrams......................................................................... 7

Figure 3: Original acquired image and its masked greyscale image .................. ............... 8

Figure 4: Result of : a) Background Correction Local Threshold ..................................... 8

Figure 5: Result of Particle Filter and Centroid .................................................................... 9 Figure 6: LabVIEW block diagrams generated form Vision Assistant that gives LicensePlate Center .................................................................................................................................. 10 Figure 7: processing the image before the OCR phase ...................................................... 11

Figure 8: Results of: a) Niblack Local Threshold, b) Remove Border Objectsc) Dilation, d) Gaussian filter & Remove Small Objects .................. ... 12

Figure 9: The OCR Training Session ................................................................................... 12

Figure 10: Complete Representation of Center, Masking operation and the OCR block

diagrams of LabView ................................................................................................................ 13

Figure 11 : User interface showing final results during the day and night .................... 14


4/17

4

1.

Introduction

The field of Image processing has allowed many applications to arise

since its beginnings in the 1960s. One of these applications is the

recognition of license plates. License plate recognition (LPR) plays an

important role in many applications such as traffic monitoring and

controlled parking, as well as other applications such as automated toll

collection and security of restricted areas, and so on. Many of these

applications require real-time performance to be practical.

Unfortunately, most existing systems exhibit complex high

computational load, making them inefficient and slow. Other systems use

cloud computing in order to process the images. Using slow and

complicated systems can increase the operational costs of such systems

greatly.

This brings up the need for efficient and powerful digital image

processing techniques and algorithms to be developed.

Numerous approaches to LPR have been proposed in the literature.

However, only a few papers have studied the performance of a real-time

LPR system, and even fewer papers study the recognition of bilingual

license plates such as most GCC license plates. In addition, existing

commercial systems that are able to recognize Saudi or GCC license

plates are proprietary. Thus, small businesses and individuals are unable

to access this technology.

It is important that an open-source, fast, and efficient system be

developed to recognize Saudi license plates.In this work we approach the problem of real-time LPR by utilizing

the NI LabVIEW software. Our method has proven to be surprisingly fast

compared to other systems, with an average processing speed


5/17

5

of 38 msec/image1, and an average success rate of 84% under different

conditions.

2.

Background

After the initial stage of image acquisition, typical LPR systems

undergo three main phases. Namely, License plate localization,

segmentation, and recognition. This is generally true regardless of the

type of the plate or the alphabet used. License plate localization is a very

essential part in any LPR system.

A typical Saudi Arabian license plate is shown in figure 1. Each license plate contains three Arabic letters and at most four Indian numerals, and

their corresponding English letters and Arabic numerals.

Figure 1: A typical Saudi Arabian license plate

Many approaches to license plate extraction have been proposed, Hao

Chen et al. [1] have suggested a method based on the texture and edge

information of the plate. Chen et al. [2] and Faradji et al. [3] Proposed

different methods based on morphological techniques. In our system, we

use a basic image processing technique called Local Threshold, as well as

two different morphological techniques to locate the plate. Then we use

1 The processing speed depends on the specifications of the PC running the software. The

specifications under which these results were found are shown in the results section.


6/17

6

LabVIEW’s built-in Optical Character Recognition (OCR) tool to

segment and recognize the characters of the license plate.

After the license plate has been located in the image, the next phase is

to segment the characters of the plate, and later, recognize them. For

GCC license plates, the Arabic letters, as in English letters, are in their

separated form, this makes the segmentation phase easier. In our

proposed system, we focus more on segmenting and recognizing the

English letters, since they are more widely used. In this phase, a special

OCR tool in LabVIEW, when set properly, allows us to segment the

characters and recognize them easily and efficiently. This has allowed us

to achieve a very high recognition rate of 99% while still having a very

fast processing speed.

There are many different approaches to character recognition. For

example, Lu et al. [4] proposed a system that uses a word image

matching technique based on an Adaptive Weighted Hausdorff Distance

(AWHD) algorithm to achieve recognition rates up to 93%. In addition,

Liying and Xiaofang [5] used a phase correlation algorithm based on

FFT.

3.

Proposed System

The license plate recognition (LPR) system proposed is based on two

main phases; License plate localization, and segmentation with OCR. As

an initial step, the license plate must be located in order to read the

characters. After locating the plate, masking is performed to extract only

the license plate region out of the whole image. The extracted portion,

which is the license plate, is then forwarded to the segmentation and OCR

phase.


7/17

7

The license plate localization phase is a very crucial stage. Failing to

achieve this task will not allow the reading of the plate to proceed. The

following subsections explain in detail the two main phases of the

system.

3.1 License Plate Localization

The localization phase plays a key role in the overall system. In this

phase, the image will undergo a number of operations as shown in figure

2.2

Figure 2: Localization phase block diagrams

Each block diagram has a specific task and they are cascaded to

produce the license plate’s center coordinates. First, the block Image

Mask 1, is used to discard the borders of the image that are statistically

unlikely to contain the license plate, it is assumed the license plate is not

on the border of the image. This is done to speed up the overall

processing of the image. Next, Color Plane Extraction 1 is used to

convert the image into a gray-scale one (see figure 3).

2 To make the design easy, these processes were designed using NI Vision Assistant, and later

transferred to LabVIEW.


8/17

8

The threshold 1 block is used to convert the gray-scale image into a

binary image in order to easily distinguish between the foreground (plate

characters) and the background (plate background). A background-

correction local threshold is used, since it minimizes the non-uniform

lighting effects, and minimizes the noise in the resulting binary image.

The advanced morphology 1 block removes objects based on their

connectivity to the border (background) of the image (see figure 4).

Next, particle filter 1 is used to eliminate any objects that have very

small or very large areas compared to the license plate. This usually

removes all objects in the image except the license plate. Finally, the

Figure 3: Original acquired image and its masked greyscale image

a) b)

Figure 4: Result of : a) Background Correction Local Threshold

b) Removing connected objects using Advanced Morphology block


9/17

9

centroid 1 block diagram returns the center coordinates of the license

plate (see figure 5 & 6). It is worth mentioning that in this preliminary

stage, we do not care much about the clarity of the characters since we

are only after the coordinates of the license plate.

Figure 5: Result of Particle Filter and Centroid


10/17

10

Figure 6: LabVIEW block diagrams that determines the license plate’s center


11/17

11

After obtaining the coordinates of the center, the original image is

automatically masked into a rectangular shaped region around the center.

The mask is designed such that only the English characters and Arabic

numerals are visible.

3.2 LP Segmentation & OCR

OCR is the process by which the system reads characters in an image

after separating them in blocks. The separation process is called

Segmentation. However, before proceeding to the segmentation and

recognition processes, the original image must be prepared or

preprocessed.

3.2.1 Preprocessing

The masked image is passed through the set of processes, shown in figure

7, to prepare it for the segmentation and OCR phase. We call this

preprocessing.

Threshold1 is a Niblack local threshold. Advanced Morphology 1 is used

to remove border objects. Basic Morphology 1 has been used here for

dilation. Dilation is used to fill any gaps in the characters and connect

them. The Dilated image is then fed to a Gaussian filter, Filter 1, for

smoothing and noise reduction prior to the particle filter. Advanced

Morphology 2 is used here to eliminate small objects, relative to the

characters, that are considered as noise. The results of these steps are

shown in figure 8.

Figure 7: processing the image before the OCR phase


12/17

12

3.2.2 Character Segmentation & Recognition

Now that the image is ready, it is forwarded to LabVIEW’s OCR tool.

The tool was previously trained on a set of characters and is designed for

the size and spacing of the characters in the plate (see figure 9).

The OCR tool extracts unique features from each segmented object in the

image and compares them to each character stored in the character set. It

then returns the closest character from the character set that best matches

the object and returns a nonzero classification score. The character would

be accepted if its value is higher than the acceptance level [6]. LabVIEW

Figure 8: Results of: a) Niblack Local Threshold,

b) Remove Border Objects c) Dilation,

d) Gaussian filter & Remove Small Objects

Figure 9: The OCR Training Session

c)d)

b)a)


13/17

13

block diagrams that were utilized to perform the previous processes are

shown in figure 10.

Figure 10: Complete Representation of the masking operations, the preprocessing,

and the OCR block diagrams of LabVIEW


14/17

14

Results of three samples of the proposed system and their intermediate phases are shown in figure 11.

Figure 11 : User interface showing final results during the day and night


15/17

15

4.

Results

The images used to test the system are all of size 640x480 pixels, and

it is assumed that the license plate is clean, and that the lighting

conditions are good. A sample of 112 images has been used to test the

system. Of these images, 95 where successfully located. This amounts to

a localization success rate 84.8%.

Of the 95 successfully located plates, 94 plates were recognized

correctly. A plate is said to be unsuccessfully recognized if at least one of

its characters was not recognized correctly. Therefore, the success rate for

the recognition with our set of 95 images reached 99%. Combining this

rate with the localization success rate, the overall success rate is around

84%.

It is important to note that we used a set of images to train the

recognition system. Each character was trained 8 times, on average.

Our testing was done on an Intel Xeon CPU running at 2.67 GHz with

4 GB of RAM. Our results show a very fast recognition speed (an

average of 38 ms/plate) which has proven to be much faster than the

results mentioned in table 1.

Table 1: Summary Of Final Results

SystemLocalization

Success Rate

Recognition

Success Rate

Overall

Success Rate

Average Speed

(s/image)3

Proposed 85% 99% 84% 0.038

[4] N/A 93% N/A 0.089

[7] 91.7% 90.9% 83% 1.1

[8] 95.6% 93.7% 87% N/A

3 The processing speed depends heavily on the system used to run the testing. The numbers mentioned

here are based on different systems. They are mentioned to give an idea about the relative speeds, not

for the sake of comparison.


16/17

16

5.

Conclusion

In this paper, we have developed and implemented a system for

license plate recognition using the NI’s LabVIEW software. The system

is developed under NI environment so that seamless deployment on other

commercial platforms is possible. The system starts with thresholding

and other morphological techniques to locate the license plate in the

image. A built-in OCR tool for character segmentation and recognition is

then used to identify the LP. A database of more than 100 images was

used to test the system. A success rate of 94% was achieved under

optimum conditions and an overall success rate of 84% with an average

processing time of 38 msec/image. The simplicity of the system and itslow computational load makes it very attractive for real-time

applications. Future work considers improving the localization technique,

and the extension of the system to other GCC license plates.


17/17

17

References

[1] H. Chen, J. Ren, H. Tan and J. Wang, "A Novel Method for License Plate

Localization," in Fourth International Conference on Image and Graphics , 2007.

[2] X.-F. Chen, B.-C. Pan and S.-L. Zheng, "A License Plate Localization Mehtod Basedon Region Narrowing," in Proceedings of the Seventh International Conference on MachineLearning and Cybernetics , Kunming, 2008.

[3] F. Faradji, A. H. Rezaie and M. Ziaratban, "A Morphological Based License PlateLocation," in IEEE International Conference on Image Processing , San Antonio, 2007.

[4] S. Lu, Z. Liu, Y. Chen and L. Liu, "AWHD for License Plate Character," in ICESS ,2008.

[5] L. Liying and Z. Xiaofang, "Application of Phase Correlation Algorithm in VehicleLicense Plate," in ICICTA, 2008.

[6] National Instruments, "NI Vision 2010 Concepts Help," June 2010. [Online]. Available: http://zone.ni.com/reference/en-XX/help/372916J-01/.

[7] P. Comelli, P. Ferragina, M. N. Granieri and F. Stablie, "Optical recognition of motor vehicle license plates," IEEE Trans. Veh. Technol., vol. 44, no. 2, pp. 790-799, 1995.

[8] H. A. Hegt, R. J. Dela Haye and N. A. Khan, "A high performance license platerecognition system," IEEE Int. Conf. Syst. Man. Cybern., vol. 5, pp. 4357-4362, 1998.

Documents

4965 Abdulrahman Al-Juraifani