Enhancing Vehicle Identification in Challenging Conditions Through Fine-Grained Classification

• Eduardo Santos Polícia Militar do Paraná / UFPR
• Gabriel E. Lima UFPR
• Rayson Laroca PUCPR / UFPR
• Eduil Nascimento Jr. Polícia Militar do Paraná
• David Menotti UFPR

DOI: https://doi.org/10.5753/sibgrapi.est.2024.31657

Resumo

Automatic License Plate Recognition (ALPR)-based Vehicle Identification Systems are vital for modern traffic management and law enforcement, yet they frequently encounter challenges in real-world scenarios, leading to recognition errors. Our ongoing research investigates a novel method to improve ALPR accuracy through the integration of Fine-grained Vehicle Classification (FGVC), with a particular focus on vehicle make identification. By cross-referencing identified vehicle brands with registered data, we aim to enhance the reliability of ALPR-based vehicle identification systems. Nonetheless, initially, our work-in-progress is concentrated on refining FGVC techniques to facilitate their integration with ALPR. We assess four deep learning models for vehicle make classification and explore methods such as selective prediction and a new class reduction approach. Preliminary results from our experiments on a modified Brazilian vehicle dataset indicate that combining these methods significantly boosts vehicle make identification accuracy. This improved classification approach is anticipated to reduce false positives and increase recognition rates under challenging conditions. Future efforts are going to be directed towards integrating these observed enhancements into ALPR-based vehicle identification systems to further improve their performance in real-world applications.

Referências

X. Ke, G. Zeng, and W. Guo, “An ultra-fast automatic license plate recognition approach for unconstrained scenarios,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, pp. 5172–5185, 2023.

R. Laroca, L. A. Zanlorensi, V. Estevam, R. Minetto, and D. Menotti, “Leveraging model fusion for improved license plate recognition,” in Iberoamerican Congress on Pattern Recognition, Nov 2023, pp. 60–75.

V. Nascimento et al., “Super-resolution of license plate images using attention modules and sub-pixel convolution layers,” Computers & Graphics, vol. 113, pp. 69–76, 2023.

C. He, D. Wang, Z. Cai, J. Zeng, and F. Fu, “A vehicle matching algorithm by maximizing travel time probability based on automatic license plate recognition data,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 8, pp. 9103–9114, 2024.

H. Wang, J. Peng, Y. Zhao, and X. Fu, “Multi-path deep CNNs for fine-grained car recognition,” IEEE Transactions on Vehicular Technology, vol. 69, no. 10, p. 10484–10493, 2020.

I. O. Oliveira et al., “Vehicle-Rear: A new dataset to explore feature fusion for vehicle identification using convolutional neural networks,” IEEE Access, vol. 9, pp. 101 065–101 077, 2021.

S. Wolf, D. Loran, and J. Beyerer, “Knowledge-distillation-based label smoothing for fine-grained open-set vehicle recognition,” in IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), 2024, pp. 330–340.

Y. Geifman and R. El-Yaniv, “Selective classification for deep neural networks,” in International Conference on Neural Information Processing Systems (NeurIPS), 2017, p. 4885–4894.

R. Laroca et al., “On the cross-dataset generalization in license plate recognition,” in International Conference on Computer Vision Theory and Applications (VISAPP), Feb 2022, pp. 166–178.

R. Laroca, M. Santos, V. Estevam, E. Luz, and D. Menotti, “A first look at dataset bias in license plate recognition,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Oct 2022, pp. 234–239.

Q. Liu, Y. Liu, S.-L. Chen, T.-H. Zhang, F. Chen, and X.-C. Yin, “Improving multi-type license plate recognition via learning globally and contrastively,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–11, 2024, early Access.

X. Ke and Y. Zhang, “Fine-grained vehicle type detection and recognition based on dense attention network,” Neurocomputing, vol. 399, pp. 247–257, 2020.

J. Sochor et al., “BoxCars: Improving fine-grained recognition of vehicles using 3-D bounding boxes in traffic surveillance,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, p. 97–108, 2018.

D. ul Khairi, F. Ayaz, N. Saeed, K. Ahsan, and S. Z. Ali, “Analysis of deep convolutional neural network models for the fine-grained classification of vehicles,” Future Transportation, vol. 3, pp. 133–149, 2023.

Y. Wang et al., “Transformer based neural network for fine-grained classification of vehicle color,” in International Conference on Multimedia Information Processing and Retrieval (MIPR), 2021, pp. 118–124.

D. Avianto, A. Harjoko, and Afiahayati, “CNN-based classification for highly similar vehicle model using multi-task learning,” Journal of Imaging, vol. 8, no. 11, p. 293, 2022.

J. He et al., “TransFG: A transformer architecture for fine-grained recognition,” AAAI Conference on Artificial Intelligence, vol. 36, no. 1, pp. 852–860, 6 2022.

Y. H. Huan Ma, “Logo recognition of vehicles based on deep convolutional generative adversarial networks,” Journal of Measurements in Engineering, vol. 12, no. 2, pp. 353–365, 2024.

G. E. Lima, R. Laroca, E. Santos, E. Nascimento Jr., and D. Menotti, “Toward enhancing vehicle color recognition in adverse conditions: A dataset and benchmark,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Sept 2024, pp. 1–6, in Press.

Ultralytics, “YOLOv10,” 2024, accessed: 2024-07-28. [Online]. Available: [link]

R. Laroca et al., “Do we train on test data? The impact of near-duplicates on license plate recognition,” in International Joint Conference on Neural Networks (IJCNN), June 2023, pp. 1–8.

M. Celestino, [link], 2021, accessed: 2024-07-30.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.

A. Howard et al., “Searching for MobileNetV3,” in IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 1314–1324.

M. Tan and Q. Le, “EfficientNetV2: Smaller models and faster training,” in International Conference on Machine Learning (ICML), 2021, pp. 10 096–10 106.

A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations (ICLR), 2021, pp. 1–12.

A. Hassan, M. Ali, N. M. Durrani, and M. A. Tahir, “An empirical analysis of deep learning architectures for vehicle make and model recognition,” IEEE Access, vol. 9, pp. 91 487–91 499, 2021.

X.-S. Wei, Y.-Z. Song, O. M. Aodha, J. Wu, Y. Peng, J. Tang, J. Yang, and S. Belongie, “Fine-grained image analysis with deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 12, pp. 8927–8948, 2022.

J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 248–255.

N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: synthetic minority over-sampling technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002.

C. Cortes, G. DeSalvo, and M. Mohri, “Learning with rejection,” in Algorithmic Learning Theory. Springer International Publishing, 2016, pp. 67–82.

C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in International Conference on Machine Learning (ICML), 2017, p. 1321–1330.