Super-Resolution Towards License Plate Recognition

• Valfride Nascimento UFPR
• Rayson Laroca UFPR
• David Menotti UFPR

DOI: https://doi.org/10.5753/ctd.2024.1999

Resumo

Recent years have seen significant developments in license plate recognition through the integration of deep learning techniques and the increasing availability of training data. Nevertheless, reconstructing license plates from low-resolution surveillance footage remains a challenge. To address this issue, we propose an attention-based super-resolution approach that incorporates sub-pixel convolution layers and an Optical Character Recognition (OCR)-based loss function. We trained the proposed architecture using synthetic images created by applying heavy Gaussian noise followed by bicubic downsampling to high-resolution license plate images. Our results show that the proposed approach for reconstructing these low-resolution images substantially outperforms existing methods in both quantitative and qualitative measures. Our source code is publicly available at https://github.com/valfride/lpr-rsr-ext/.

Referências

Gonçalves, G. R. et al. (2018). Real-time automatic license plate recognition through deep multi-task networks. In Conf. on Graphics, Patterns and Images, pages 110–117.

Gong, Y. et al. (2022). Unified Chinese license plate detection and recognition with high efficiency. Journal of Visual Communication and Image Representation, 86:103541.

Laroca, R., Cardoso, E. V., Lucio, D. R., Estevam, V., and Menotti, D. (2022a). On the cross-dataset generalization in license plate recognition. In International Conference on Computer Vision Theory and Applications (VISAPP), pages 166–178.

Laroca, R., Estevam, V., Britto Jr., A. S., Minetto, R., and Menotti, D. (2023a). Do we train on test data? The impact of near-duplicates on license plate recognition. In International Joint Conference on Neural Networks (IJCNN), pages 1–8.

Laroca, R. et al. (2022b). A first look at dataset bias in license plate recognition. In Conference on Graphics, Patterns and Images (SIBGRAPI), pages 234–239.

Laroca, R. et al. (2023b). Leveraging model fusion for improved license plate recognition. In Iberoamerican Congress on Pattern Recognition (CIARP), pages 60–75.

Li, W., Fan, L., Wang, Z., Ma, C., and Cui, X. (2021). Tackling mode collapse in multi-generator GANs with orthogonal vectors. Pattern Recognition, 110:107646.

Lin, M., Liu, L., Wang, F., Li, J., and Pan, J. (2021). License plate image reconstruction based on generative adversarial networks. Remote Sensing, 13(15):3018.

Liu, A. et al. (2023). Blind image super-resolution: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5461–5480.

Lucas, A. et al. (2019). Generative adversarial networks and perceptual losses for video super-resolution. IEEE Transactions on Image Processing, 28(7):3312–3327.

Mehri, A., Ardakani, P. B., and Sappa, A. D. (2021). MPRNet: Multi-path residual network for lightweight image super resolution. In IEEE Winter Conference on Applications of Computer Vision (WACV), pages 2703–2712.

Nascimento, V. (2023). Super-resolution towards license plate recognition. Master’s thesis, Federal University of Paraná (UFPR).

Nascimento, V. et al. (2023). Super-resolution of license plate images using attention modules and sub-pixel convolution layers. Computers & Graphics, 113:69–76.

Nascimento, V., Laroca, R., Lambert, J. A., Schwartz, W. R., and Menotti, D. (2022). Combining attention module and pixel shuffle for license plate super-resolution. In Conference on Graphics, Patterns and Images (SIBGRAPI), pages 228–233.

Qin, S. and Liu, S. (2022). Towards end-to-end car license plate location and recognition in unconstrained scenarios. Neural Computing and Applications, 34:21551–21566.

Saharia, C. et al. (2023). Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713–4726.

Santos, M. et al. (2022). Face super-resolution using stochastic differential equations. In Conference on Graphics, Patterns and Images (SIBGRAPI), pages 216–221.

Shi, W. et al. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1874–1883.

Silva, S. M. and Jung, C. R. (2022). A flexible approach for automatic license plate recognition in unconstrained scenarios. IEEE Transactions on Intelligent Transportation Systems, 23(6):5693–5703.

Wang, Y., Bian, Z.-P., Zhou, Y., and Chau, L.-P. (2022). Rethinking and designing a high-performing automatic license plate recognition approach. IEEE Transactions on Intelligent Transportation Systems, 23(7):8868–8880.

Wang, Z. et al. (2021). Deep learning for image super-resolution: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10):3365–3387.

Yuan, Y. et al. (2017). A robust and efficient approach to license plate detection. IEEE Transactions on Image Processing, 26(3):1102–1114.

Zhang, L. et al. (2021a). A robust attentional framework for license plate recognition in the wild. IEEE Transactions on Intelligent Transportation Systems, 22(11):6967–6976.

Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 586–595.

Zhang, Y. et al. (2021b). Residual dense network for image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(7):2480–2495.

Zhang, Y., Huang, Y., Wang, K., Qi, G., and Zhu, J. (2023). Single image super-resolution reconstruction with preservation of structure and texture details. Mathematics, 11:216.