A ROBUSTIC AUTOMATIC VEHICLE NUMBER PLATE RECOGNITION SYSTEM USINGN IMAGEPROCESSING AND DEEP LEARNING

TECHNOLOGIES

1S SWATHI,R SARADA2,P MAMATHA3, 123 ASSISTANT PROFESSOR,DEPT OF ECE ,SRI INDU COLLEGE OF ENGG AND TECHNOLOGY ,HYDERABAD,TELANGANA

1EMAIL ID :swathi.singanaboina@gmail.com

2EMAIL ID :saradajspn@gmail.com 3EMAIL ID :mamathadokuri@gmail.com

ABSTRACT: This article describes an online, highly accurate system for automated number plate recognition

(ANPR) that may be utilised as the foundation for a variety of real-world ITS applications. The technology is intended to cope with illegible car plates, changing weather and lighting conditions, various traffic scenarios, and high-speed vehicles. This article tackles a variety of problems by providing appropriate hardware platforms as well as real-time, robust, and novel algorithms. We gathered massive and comprehensive data sets of Persian licence plates for evaluations, comparisons, and optimization of different related algorithms. Images taken from crossroads, streets, and highways throughout the day and night, in various weather situations, and with varying plate clarities are included in the data sets. Our system obtains 98.7 percent, 99.2 percent, and 97.6 percent accuracies for plate detection, character segmentation, and plate identification, respectively, across various data sets. In plate detection, the false alarm rate is less than 0.5 percent. The total accuracy of our data sets' dirty plates component is 91.4 percent. Our AVNPRsystem has been deployed in various places and has been thoroughly tested for over a year. The suggested algorithms for each component of the system are very resistant to changes in illumination, size differences, plate clarity, and plate skewness. In addition, the method is unaffected by the number of plates in recorded pictures. This method has also been tested on three additional Iranian data sets and has achieved 100% accuracy in both detection and identification. To demonstrate that our AVNPR is not language dependent, we tested our system using a data set of available English plates and obtained 97 percent overall accuracy.

Keywords: Automatic Number Plate Recognition, CNN, Character Recognition,

I. INTRODUCTION:

Automatic platform recognition has become an integral part of our lives and promises to remain integrated with the planned transport technologies in future. The idea of autonomous vehicles offers numerous opportunities for altering basic transport networks. AVNPR technology already contributes to smart transport networks and eliminates the need for human involvement. It is not only the camera at the side of the road or at the car park fence. With the passing of years, it has become mobile, initially in cars, but now, more recently with the development of smartphone technology, many AVNPR systems have now become manual systems. Because of reduced supply costs, AVNPR is frequently an option in the toll and parking companies. The primary reason for this is because, contrary to Ultra High Frequency RFID systems (UHF-RFID), the AVNPR system detects the number plate recorded without any extra transponder needs. The fast development of nations is a major step forward in our contemporary world. People move from rural regions and most of them prefer to reside in cities. As traffic increases in these regions, local governments frequently

fail to understand the current and future mobility

requirements of residents and tourists. AVNPRis are increasingly employed to check free traffic flows and to facilitate smart transport[1]. Modern AVNPR cameras can not only scan plates, but can also give valuable supplementary information such as counting, direction, vehicle groupings and speed. The capacity to identify and read huge quantities of fast-moving automobiles has led to many elements of today's digital environment being covered by the AVNPR technology. While AVNPRtechnology may be used in many various packages, they serve the same fundamental purpose as providing a very precise vehicle reading system without any human action. It is used in various ways: entry control, parking management, tolling, user billing, delivery tracking, traffic management, security and police services, customers' services and addresses, red lighting and lane enforcement, queue length estimates and many more services [2–8]. The fundamental system design for a fixed and mobile AVNPR technology is shown in Figure 1.

.

Fig 1. Typical AVNPRSystem Diagram of a Fixed AVNPR System (right) and a Mobile AVNPRSystem (left) (Source: latech.us, accessed on 5 November 2020).

Number Platform Recognition includes acquiring numeric platform pictures using a camera from the desired scenario. Whether still pictures or a photo video are taken and processed via a succession of photo processing algorithms to achieve an alpha-numerical translation of the recorded images into a text input. After acquiring a decent picture of the scene/vehicle, any AVNPR system's fundamental reliance on its algorithms' reliability. These algorithms need extremely careful thought and thousands of software code lines to get desired outcomes and cover all intricacies of the system. In all, a number of main algorithms are needed for the efficiency of intelligent vehicle technologies and AVNPR. Figure 2 shows the general procedures involved in AVNPRsystems.

A common AVNPSystem includes the basic image acquisition process (system input), the removal of the

number plate (NPE), the segmentation of the character (CS) and the recognition of the character (CR) (system output) [9]. After the vehicle has successfully been recognised the data may be retrieved and utilised as necessary for post-processing procedures. The data for cars are transmitted to the linked back office software, which is the main repository of all data along with data analysis, queries and reporting capabilities. These gathered data may be used for many additional smart transport applications, since AVNPRsystems not only capture vehicle pictures visually but also store information on their central repository. This may involve identifying the vehicle via date, time and precise location, while maintaining a complete traffic traffic record. These data may be useful in modelling and analysing various transport networks.

Fig 2. General processes of number plate recognition system.

Depending on the kind of camera used, its resolution, the lightening/illumination aids, mounting position, areas/lanes coverage capacity, complicated sceneries, shutter speed and other environmental/system limitations, the picture captured from the scene may encounter various complications. Figure 3 illustrates the variation of styles, colours, typefaces, sizes and physical conditions in the licence plate, which may influence the accuracy of identification. When a vehicle is detected in a scene/picture, the system utilises plate positioning features to extract a licence plate from the image of the vehicle. Characters are then segmented on the retrieved number plate before identification. Character segmentation is an algorithm locating the numerical alpha characters on a number plate. The segmented characters are subsequently converted into an alpha numeric text input using the OCR methods. Algorithms such as template matching or neural network classifiers are utilised for character recognition. The performative impact of an AVNPR system depends on the efficiency of each step. The performance or success rate is a metric used to quantify the whole process. This is the ratio of number plate

number successfully identified to the total number of input pictures. The rate of performance includes all three phases of recognition, number plate extraction, segmentation and character identification. AVNPRSystem gathers the basic form of information, including the pictures and related metadata, from AVNPRSoftware. It offers automation and security aspects for the transport system. Its ITS connection enables the system to be automated by offering services in mail collections, analysing traffic, enhancing legal enforcement and establishing an extensive traffic movements database. Another helpful aspect of the technology is the integration of AVNPR with information communication technology (ICT) technologies. AVNPR systems can be used for modelling and implementing various transport systems such as the Passengers Mobility Systems [10] model, traffic flow analysis and road network control strategies using Fun-damental Diagram (NFD) Network models[11], the choice model for route and travel chosen cars[12] and travel demand patterns Data from AVNPR systems can be used.

Fig 3. License plate diversity in styles, colors, fonts, sizes, and physical conditions. (Source: Plate Recognizer ALPR [14]-a division of ParkPow [15]).

II. LITERATURE REVIEW

Wen et al. [3] proposed Hough Transform Circle Arc Detection technique. They used it to identify the presence of a helmet in the automatic plate monitoring system. But the disadvantage of this study was that the helmet was detected using just the geometric characteristics. The geometric characteristics are insufficient to identify the cascade; the head may be confused with the cascade several times. In Chiu et al.[4] he utilised a computer-based vision system that detects motorbikes and segments partially covered by another vehicle. The helmet detecting system has been utilised to simplify a motorbike in the presence of the helmet. In this study, the helmet margins of the potential helmet area were calculated. Chiverton et al. [5] described and tested an automated categorization method for helmet and non-helmet bikes. It has utilised (SVM) support vector machine trained on (HOG) Oriented Gradient histogram from the head area of the static pictures and the individual video data image frame. The accuracy rate was high by this technique but the number of test pictures collected was much fewer. Silva et al. [6] developed a helmet identification method that begins with a moving segmentation of objects using descriptors, and then detects the casket tracing region of (ROI) which is the heading region, then distinguishes between the helmet and non- casco. But the disadvantage was that it utilises Hough circle to distinguish between helmet and non - helmet that also leads towards misclassification between head and helmet, because both have round shape and are computationally costly. Dahiya et al. [7] presented the method for the identification of cyclists, who have been utilising video monitoring in real-time, using handmade features (HOG) Oriented Gradients histograms, invariant scale transforming features (LBP) Local binary patterns (SIFT). The detection accuracy was 93.80%, however the processing time needed was extremely sluggish 11.58 ms per picture. Half and complete helmet with hair detection are present in the Doughmala et al.[8] such as nose, ear, mouth, left eye, right eye, and circular Hough, which identify the presence of a helmet. But it worked on fixed resolution pictures in this article.

Karwal et al.[9] presented a method for the identification of the car number plate, in which a standardised cross correlation for the matching template was employed in order to solve scaling and character recognition problems at various locations. Sulaiman et al.[10] method combining image processing with (OCR) optical character recognition for identifying vehicle number plate under various backgrounds but working on static pictures, i.e. not moving images in Malaysia. Lahiri et al [11] presented a method where image processing techniques, such as edge enhancement, blunt masking to properly identify picture edges and Optical (OCR) character recognition to detect image components were utilised. But certain misalignment and varied size of letters could not be detected in the picture. Yun-Chung et al. [12] utilised the Fuzzy (OCR) optical character recognition system however, since the positioning of the module failed to identify the limits of the number plate, the disadvantage also failed to distinguish between the numerals "1" and "7." Neural Network (RBF) has been utilised by Cika et al[13] to identify characteristics, in particular for Saudi Arabian vehicles, however it was sensitive to light shine.

III. PROPOSAL SYSTEM This study aims to build a system that will identify whether or not the person wears a cask and then collect the number plate of the car and then convert it to a text for automated challan. The methods of detection include 1st and 2nd order derivative edge detection with a neural helmet detection network and optical (OCR) neural network detection with licence plate detection system. The following are the procedures to be taken while processing • Video pre-processing. • Removal of background. • Two-wheel segmentation. • Helm detection. • If the cask is not identified then the plate number is recognised.

FIG 4 BLOCK DIAGRAM FOR THE PROPOSED MODEL In this part we offer a method in monitoring films to automatically identify bikers with no helmet. And should motorcyclists not use a helmet we will track the vehicle's licence number plate in three sentences. In the first sentence, the motorbike is detected in the monitoring footage. In the second sentence, we will find the motorcyclist head to identify whether or not the motorcyclists wear a helmet. In the third sentence, if there are motorcyclists Found without a helmet then track the motorbike licence number plate. The steps are as follows: A. ROAD VIDEO INPUT VIDEO Specific road videos are captured and the video acquired then breaks into frames of a predetermined interval. The output is the input video frame sequence. On each frame we will preprocess such as picture enhancement, improvement of the image contrast level, removal of noise and Gaussian filter application. This filtered video frame is then subtracted from the backdrop. B. REMOVAL BACKGROUND As the motorbike is the primary need for our system calculation, it is a frantic job throughout the whole movie. So we perform background subtraction in order to improve the detection rate, to distinguish things in motion from static ones. Things in motion like motorcycles, people beings, vehicles are separated from static objects such as trees, highways and buildings. The Gaussian model can accomplish this. C. TWO WHEELERS SEGMENTATION The result from background subtraction comprises of moving items like motorcycles, people, vehicles, etc. But we are only interested in motors, so that the motorbike is divided into motors and non-motors by means of object segmentation. Using extraction techniques such as (HOG) oriented gradients histogram, (SIFT) scale invariant

extraction feature, (LBP) binary local pattern and edge detection algorithm in both first and second order. D. Motorcycle detection with HELMET After detecting two wheels we identify motorcyclists without casket using feature methods (HOG) Oriented gradient histogram (SIFT) Invariant functional removal scale, Local binary pattern and Neural network edge derivative technique for first and second order detection. E. LICENCE EXTRACT NUMBER PLATE. If riders are detected without a cask, we will extract the motorbike licence number plate with OCR as a template that matches neural networks. CONCLUSION: This article presents a research on the recognition of the vehicle number plate in traffic monitoring. For effective traffic monitoring, AVNPR is extremely useful and dependable. Devices with a strong image processing technology may quickly identify cars from different angles and display the owner's information as an output. The development of the smart transport network involves AVNPRsystems. Image processing techniques in conjunction with neural networks may be used to detect numerical plates for future improvements in angled or lateral view pictures, moving distance pictures, numbering system, and kind of number plate (background). Object and neural network detection is helpful for the detection of side views or inclined pictures and for remote movement of images. For prospective recognition systems, the option is to utilise high resolution cameras with an increasing number of frames for future accuracy and accuracy. REFERENCE 1 Maya Sharma , "use your head - use a helmet", Team NDTV article , Dec 12 , 2016. 2 Anisha Bhatia ,"Wearing helmets - A choice between life and death", Team NDTV article , Dec 9 , 2016.

3 C.Y.Wen, S.H.Chiu, J.J.Liaw, and C.P. Lu, "The safety helmet detection for ATM's surveillance system via the modified hough transform" in IEEE 37th Annual International Carnahan Conference on Security Technology , 2003 , pp- 364-369. 4 C.C. Chiu, M.Y. Ku, and H. -T. Chen, "Motorcycle detection and tracking system with occlusion segmentation,” in Proceeding of International Workshop on Image Analysis for Multimedia Interactive Services , Santorini , Greece, 6-8 June 2007, pp. 32-32. 5 J. Chiverton , "Helmet presence classification with motorcycle detection and tracking," IET Intelligence Transport Systems (ITS) , Volume 6 , no. 3 , pp. 259-269, 2012. 6 R. V. Silva , T. Aires , and V. Rodrigo , " Helmet Detection on Motorcyclists using image descriptors and classifiers", in Proceeding of Graphics , Patterns an Images ( SIBGRAPI) , Rio de Janeiro ,Brazil , 27-30 August 2014 , pp. 141-148. 7 K. Dahiya, D. Singh and C.K .Mohan, "Automatic detection of bike riders without helmet using surveillance videos in real - time", in Proceeding Vancouver, Canada , 24-2 July 2016,pp.3046-3051.of International Joint Conference Neural Networks (IJCNN), 8 Han, C.C., Hsieh, C.T., Chen, Y.N., Ho, G.F., Fan, K.C. and Tsai, C.L., 2007, October. License plate detection and recognition using a dual-camera module in a large space. In 2007 41st Annual IEEE International Carnahan Conference on Security Technology (pp. 307-312). IEEE. 9 Mondal, M., Mondal, P., Saha, N. and Chattopadhyay, P., 2017, December. Automatic number plate recognition using CNN based self-synthesized feature learning. In 2017 IEEE Calcutta Conference (CALCON) (pp. 378-381). IEEE. 10 Agbemenu, A.S., Yankey, J. and Addo, E.O., 2018. An automatic number plate recognition system using opencv and tesseract ocr engine. International Journal of Computer Applications, 180, pp.1-5. 11 Laroca, R., Severo, E., Zanlorensi, L.A., Oliveira, L.S., Gonçalves, G.R., Schwartz, W.R. and Menotti, D., 2018, July. A robust real-time automatic license plate recognition based on the YOLO detector. In 2018 International Joint Conference on Neural Networks (IJCNN) (pp. 1-10). IEEE. 12 Dhar, P., Guha, S., Biswas, T. and Abedin, M.Z., 2018, February. A system design for license plate recognition by using edge detection and convolution neural network. In 2018 International Conference on Computer, Communication, Chemical, Material and Electronic Engineering (IC4ME2) (pp. 1-4). IEEE. 13 Lin, C.H. and Li, Y., 2019, August. A License Plate Recognition System for Severe Tilt Angles Using Mask RCNN. In 2019 International Conference on Advanced Mechatronic Systems (ICAMechS) (pp. 229-234). IEEE. 14 Saif. N. Ahmmed, N., Pasha, S., Shahrin, M.S.K., Hasan, M.M., Islam, S. and Jameel, A.S.M.M., 2019, October. Automatic License Plate Recognition System for Bangla License Plates using Convolutional Neural Network. In TENCON 2019-2019 IEEE Region 10 Conference (TENCON) (pp. 925-930). IEEE.