License Plate Image Generation using Generative Adversarial Networks for End-To-End License Plate Character Recognition from a Small Set of Real Images

Abstract

License Plate Character Recognition (LPCR) is a technology for reading vehicle registration plates using optical character recognition from images and videos, and it has a long history due to its usefulness. While LPCR has been significantly improved with the advance of deep learning, training deep networks for LPCR module requires a large number of license plate (LP) images and their annotations. Unlike other public datasets of vehicle information, each LP has a unique combination of characters and numbers depending on the country or the region. Therefore, collecting a sufficient number of LP images is extremely difficult for normal research. In this paper, we propose LP-GAN, an LP image generation method, by applying an ensemble of generative adversarial networks (GAN), and we also propose a modified lightweight YOLOv2 model for an efficient end-to-end LPCR module. With only 159 real LP images available online, thousands of synthetic LP images were generated by using LP-GAN. The generated images not only looked similar to real ones, but they were also shown to be effective for training the LPCR module. As a result of performance tests with 22,117 real LP images, the LPCR module trained with only the generated synthetic dataset achieved 98.72% overall accuracy, which is comparable to that of training with a real LP image dataset. In addition, we improved the processing speed of LPCR about 1.7 times faster than that of the original YOLOv2 model by using the proposed lightweight model.

Automated License Plate Recognition: A Survey on Methods and Techniques

“…A study by Han et al [90] has proposed a novel approach to generate realistic Korean license plate images using a small set of real license plates. They have used 159 online available license plate images which were collected through web-scraping.…”

An All-in-One Vehicle Type and License Plate Recognition System Using YOLOv4

“…We tried to gather another open dataset, such as KarPlate dataset [ 23 ], but it was no longer available due to legal issues. There are other approaches, such as synthetically generating LPs [ 26 ] and synthetic LP dataset ( , accessed on 7 December 2021), but we only evaluated our detector with the real data. Figure 14 shows LPR results on our custom dataset.…”

“…Their findings revealed that YOLOv4 outperformed SSD and faster-RCNN in terms of F1 score, precision, recall, and mAP. To deal with the problem of data sparsity in the training stage, Han et al synthesized LPs using an ensemble of generative adversarial networks (GAN) [ 26 ]. Wang et al developed a Korean LPR approach using deep learning and KarPlate dataset (when the dataset was still available) to recognize LPs under various conditions (i.e., fog and haze) [ 27 ].…”

Recognition of Vehicle License Plates Based on Image Processing

“…To confirm the letters and numbers of the image, corrected for perspective distortion, this paper was verified using the YOLO v2 model. This model is commonly used for character and number recognition [18][19][20][21][22][23][24][25][26]. Figure 11 shows the detection result used for the verification dataset.…”