Improving Vehicle Identification Through Advanced Fine-Grained Vehicle Classification
Resumo
Vehicle identification plays a crucial role in Intelligent Transportation Systems, impacting areas such as toll collection, vehicle access control, and criminal forensics. Despite recent strides in Automatic License Plate Recognition (ALPR) research, real-world scenarios still pose significant challenges. This work explores potential enhancements in vehicle identification systems by integrating modules such as ALPR with Fine-Grained Vehicle Classification (FGVC), which categorizes vehicles based on attributes such as type, make, model, and year. Our study focuses on advancing FGVC, particularly vehicle type classification. We investigate selective prediction, a technique that allows models to discard uncertain predictions, and examine superclass methods, including a novel online superclass approach that operates solely during the test phase. We trained and evaluated four deep learning models using a dataset adapted from a widely adopted ALPR dataset. The results demonstrate that both superclass methods and selective prediction improve classification accuracy, with the combination of online superclass and selective prediction delivering the best performance. Future research will focus on integrating these enhancements into ALPR systems to determine how FGVC can further enhance their capabilities.Referências
R. Laroca, L. A. Zanlorensi, G. R. Gonçalves, E. Todt, W. R. Schwartz, and D. Menotti, “An efficient and layout-independent automatic license plate recognition system based on the YOLO detector,” IET Intelligent Transport Systems, vol. 15, no. 4, pp. 483–503, 2021.
I. O. Oliveira et al., “Vehicle-Rear: A new dataset to explore feature fusion for vehicle identification using convolutional neural networks,” IEEE Access, vol. 9, pp. 101 065–101 077, 2021.
X. Ke, G. Zeng, and W. Guo, “An ultra-fast automatic license plate recognition approach for unconstrained scenarios,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, pp. 5172–5185, 2023.
R. Laroca, L. A. Zanlorensi, V. Estevam, R. Minetto, and D. Menotti, “Leveraging model fusion for improved license plate recognition,” in Iberoamerican Congress on Pattern Recognition, Nov 2023, pp. 60–75.
V. Nascimento et al., “Combining attention module and pixel shuffle for license plate super-resolution,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Oct 2022, pp. 228–233.
V. Nascimento et al., “Super-resolution of license plate images using attention modules and sub-pixel convolution layers,” Computers & Graphics, vol. 113, pp. 69–76, 2023.
C. He, D. Wang, Z. Cai, J. Zeng, and F. Fu, “A vehicle matching algorithm by maximizing travel time probability based on automatic license plate recognition data,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 8, pp. 9103–9114, 2024.
S. Wolf, D. Loran, and J. Beyerer, “Knowledge-distillation-based label smoothing for fine-grained open-set vehicle recognition,” in IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), 2024, pp. 330–340.
F. Oliveira, A. Macena, O. Kamel, W. Souza, N. Freitas, and T. Vinuto, “Fine-grained cars recognition using deep convolutional neural networks,” in Conference on Graphics, Patterns and Images (SIBGRAPI), 2022, pp. 240–245.
D. M. Kuhn and V. P. Moreira, “BRCars: A dataset for fine-grained classification of car images,” in Conference on Graphics, Patterns and Images (SIBGRAPI), 2021, pp. 231–238.
H. Lu, M. Han, C. Wang, and J. Cheng, “AMLNet: Attention multi-branch loss CNN models for fine-grained vehicle recognition,” IEEE Transactions on Vehicular Technology, vol. 73, no. 1, pp. 375–384, 2024.
Y. Wang et al., “Transformer based neural network for fine-grained classification of vehicle color,” in International Conference on Multimedia Information Processing and Retrieval (MIPR), 2021, pp. 118–124.
K. Valev et al., “A systematic evaluation of recent deep learning architectures for fine-grained vehicle classification,” in Pattern Recognition and Tracking, vol. 10649, 2018, p. 1064902.
J. Kim, “Deep learning-based vehicle type and color classification to support safe autonomous driving,” Applied Sciences, vol. 14, no. 4, 2024.
Y. Geifman and R. El-Yaniv, “Selective classification for deep neural networks,” in International Conference on Neural Information Processing Systems (NeurIPS), 2017, p. 4885–4894.
C. Cortes et al., “Learning with rejection,” in Algorithmic Learning Theory. Springer International Publishing, 2016, pp. 67–82.
Y. Zhou, Q. Hu, and Y. Wang, “Deep super-class learning for longtail distributed image classification,” Pattern Recognition, vol. 80, pp. 118–128, 2018.
K. Hendrickx, L. Perini, D. Van der Plas, W. Meert, and J. Davis, “Machine learning with a reject option: A survey,” Machine Learning, vol. 113, no. 5, pp. 3073–3110, 2024.
Z. Gan et al., “Superclass learning with representation enhancement,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 24 060–24 069.
X.-Y. Zhang et al., “A survey on learning to reject,” Proceedings of the IEEE, vol. 111, no. 2, pp. 185–215, 2023.
R. Laroca et al., “On the cross-dataset generalization in license plate recognition,” in International Conference on Computer Vision Theory and Applications (VISAPP), Feb 2022, pp. 166–178.
J. M. Ferryman, A. D. Worrall, G. D. Sullivan, and K. D. Baker, “A generic deformable model for vehicle recognition,” in British Machine Vision Conference (BMVC), 1995, p. 127–136.
M.-P. Dubuisson Jolly, S. Lakshmanan, and A. Jain, “Vehicle segmentation and classification using deformable templates,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 3, pp. 293–308, 1996.
A. Lai, G. Fung, and N. Yung, “Vehicle type classification from visual-based dimension estimation,” in IEEE Intelligent Transportation Systems Conference (ITSC), 2001, pp. 201–206.
W. Wu, Z. QiSen, and W. Mingjun, “A method of vehicle classification using models and neural networks,” in IEEE Vehicular Technology Conference, 2001, pp. 3022–3026.
X. Ma and W. Grimson, “Edge-based rich representation for vehicle classification,” in IEEE International Conference on Computer Vision (ICCV), 2005, pp. 1185–1192.
Z. Dong, Y. Wu, M. Pei, and Y. Jia, “Vehicle type classification using a semisupervised convolutional neural network,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 4, pp. 2247–2256, 2015.
B. Hu, J.-H. Lai, and C.-C. Guo, “Location-aware fine-grained vehicle type recognition using multi-task deep networks,” Neurocomputing, vol. 243, pp. 60–68, 2017.
X. Wang et al., “Real-time vehicle type classification with deep convolutional neural networks,” Journal of Real-Time Image Processing, vol. 16, pp. 5–14, 2019.
N. Shvai, A. Hasnat, A. Meicler, and A. Nakib, “Accurate classification for automatic vehicle-type recognition based on ensemble classifiers,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 3, pp. 1288–1297, 2020.
R. Laroca, M. Santos, V. Estevam, E. Luz, and D. Menotti, “A first look at dataset bias in license plate recognition,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Oct 2022, pp. 234–239.
Q. Liu, Y. Liu, S.-L. Chen, T.-H. Zhang, F. Chen, and X.-C. Yin, “Improving multi-type license plate recognition via learning globally and contrastively,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–11, 2024, early Access.
G. E. Lima, R. Laroca, E. Santos, E. Nascimento Jr., and D. Menotti, “Toward enhancing vehicle color recognition in adverse conditions: A dataset and benchmark,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Sept 2024, pp. 1–6, in Press.
Ultralytics, “YOLOv8,” 2023, accessed: 2024-07-28. [Online]. Available: [link]
Ministério dos Transportes, “Frota nacional (junho de 2024),” [link], 2024, accessed: 2024-08-09.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
A. Howard et al., “Searching for MobileNetV3,” in IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 1314–1324.
M. Tan and Q. Le, “EfficientNetV2: Smaller models and faster training,” in International Conference on Machine Learning (ICML), 2021, pp. 10 096–10 106.
A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations (ICLR), 2021, pp. 1–12.
X.-S. Wei, Y.-Z. Song, O. Mac Aodha, J. Wu, Y. Peng, J. Tang, J. Yang, and S. Belongie, “Fine-grained image analysis with deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 12, pp. 8927–8948, 2022.
L. Lu, P. Wang, and Y. Cao, “A novel part-level feature extraction method for fine-grained vehicle recognition,” Pattern Recognition, vol. 131, no. 108869, p. 108869, 2022.
A. Hassan, M. Ali, N. M. Durrani, and M. A. Tahir, “An empirical analysis of deep learning architectures for vehicle make and model recognition,” IEEE Access, vol. 9, pp. 91 487–91 499, 2021.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: synthetic minority over-sampling technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002.
I. O. Oliveira et al., “Vehicle-Rear: A new dataset to explore feature fusion for vehicle identification using convolutional neural networks,” IEEE Access, vol. 9, pp. 101 065–101 077, 2021.
X. Ke, G. Zeng, and W. Guo, “An ultra-fast automatic license plate recognition approach for unconstrained scenarios,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, pp. 5172–5185, 2023.
R. Laroca, L. A. Zanlorensi, V. Estevam, R. Minetto, and D. Menotti, “Leveraging model fusion for improved license plate recognition,” in Iberoamerican Congress on Pattern Recognition, Nov 2023, pp. 60–75.
V. Nascimento et al., “Combining attention module and pixel shuffle for license plate super-resolution,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Oct 2022, pp. 228–233.
V. Nascimento et al., “Super-resolution of license plate images using attention modules and sub-pixel convolution layers,” Computers & Graphics, vol. 113, pp. 69–76, 2023.
C. He, D. Wang, Z. Cai, J. Zeng, and F. Fu, “A vehicle matching algorithm by maximizing travel time probability based on automatic license plate recognition data,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 8, pp. 9103–9114, 2024.
S. Wolf, D. Loran, and J. Beyerer, “Knowledge-distillation-based label smoothing for fine-grained open-set vehicle recognition,” in IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), 2024, pp. 330–340.
F. Oliveira, A. Macena, O. Kamel, W. Souza, N. Freitas, and T. Vinuto, “Fine-grained cars recognition using deep convolutional neural networks,” in Conference on Graphics, Patterns and Images (SIBGRAPI), 2022, pp. 240–245.
D. M. Kuhn and V. P. Moreira, “BRCars: A dataset for fine-grained classification of car images,” in Conference on Graphics, Patterns and Images (SIBGRAPI), 2021, pp. 231–238.
H. Lu, M. Han, C. Wang, and J. Cheng, “AMLNet: Attention multi-branch loss CNN models for fine-grained vehicle recognition,” IEEE Transactions on Vehicular Technology, vol. 73, no. 1, pp. 375–384, 2024.
Y. Wang et al., “Transformer based neural network for fine-grained classification of vehicle color,” in International Conference on Multimedia Information Processing and Retrieval (MIPR), 2021, pp. 118–124.
K. Valev et al., “A systematic evaluation of recent deep learning architectures for fine-grained vehicle classification,” in Pattern Recognition and Tracking, vol. 10649, 2018, p. 1064902.
J. Kim, “Deep learning-based vehicle type and color classification to support safe autonomous driving,” Applied Sciences, vol. 14, no. 4, 2024.
Y. Geifman and R. El-Yaniv, “Selective classification for deep neural networks,” in International Conference on Neural Information Processing Systems (NeurIPS), 2017, p. 4885–4894.
C. Cortes et al., “Learning with rejection,” in Algorithmic Learning Theory. Springer International Publishing, 2016, pp. 67–82.
Y. Zhou, Q. Hu, and Y. Wang, “Deep super-class learning for longtail distributed image classification,” Pattern Recognition, vol. 80, pp. 118–128, 2018.
K. Hendrickx, L. Perini, D. Van der Plas, W. Meert, and J. Davis, “Machine learning with a reject option: A survey,” Machine Learning, vol. 113, no. 5, pp. 3073–3110, 2024.
Z. Gan et al., “Superclass learning with representation enhancement,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 24 060–24 069.
X.-Y. Zhang et al., “A survey on learning to reject,” Proceedings of the IEEE, vol. 111, no. 2, pp. 185–215, 2023.
R. Laroca et al., “On the cross-dataset generalization in license plate recognition,” in International Conference on Computer Vision Theory and Applications (VISAPP), Feb 2022, pp. 166–178.
J. M. Ferryman, A. D. Worrall, G. D. Sullivan, and K. D. Baker, “A generic deformable model for vehicle recognition,” in British Machine Vision Conference (BMVC), 1995, p. 127–136.
M.-P. Dubuisson Jolly, S. Lakshmanan, and A. Jain, “Vehicle segmentation and classification using deformable templates,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 3, pp. 293–308, 1996.
A. Lai, G. Fung, and N. Yung, “Vehicle type classification from visual-based dimension estimation,” in IEEE Intelligent Transportation Systems Conference (ITSC), 2001, pp. 201–206.
W. Wu, Z. QiSen, and W. Mingjun, “A method of vehicle classification using models and neural networks,” in IEEE Vehicular Technology Conference, 2001, pp. 3022–3026.
X. Ma and W. Grimson, “Edge-based rich representation for vehicle classification,” in IEEE International Conference on Computer Vision (ICCV), 2005, pp. 1185–1192.
Z. Dong, Y. Wu, M. Pei, and Y. Jia, “Vehicle type classification using a semisupervised convolutional neural network,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 4, pp. 2247–2256, 2015.
B. Hu, J.-H. Lai, and C.-C. Guo, “Location-aware fine-grained vehicle type recognition using multi-task deep networks,” Neurocomputing, vol. 243, pp. 60–68, 2017.
X. Wang et al., “Real-time vehicle type classification with deep convolutional neural networks,” Journal of Real-Time Image Processing, vol. 16, pp. 5–14, 2019.
N. Shvai, A. Hasnat, A. Meicler, and A. Nakib, “Accurate classification for automatic vehicle-type recognition based on ensemble classifiers,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 3, pp. 1288–1297, 2020.
R. Laroca, M. Santos, V. Estevam, E. Luz, and D. Menotti, “A first look at dataset bias in license plate recognition,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Oct 2022, pp. 234–239.
Q. Liu, Y. Liu, S.-L. Chen, T.-H. Zhang, F. Chen, and X.-C. Yin, “Improving multi-type license plate recognition via learning globally and contrastively,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–11, 2024, early Access.
G. E. Lima, R. Laroca, E. Santos, E. Nascimento Jr., and D. Menotti, “Toward enhancing vehicle color recognition in adverse conditions: A dataset and benchmark,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Sept 2024, pp. 1–6, in Press.
Ultralytics, “YOLOv8,” 2023, accessed: 2024-07-28. [Online]. Available: [link]
Ministério dos Transportes, “Frota nacional (junho de 2024),” [link], 2024, accessed: 2024-08-09.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
A. Howard et al., “Searching for MobileNetV3,” in IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 1314–1324.
M. Tan and Q. Le, “EfficientNetV2: Smaller models and faster training,” in International Conference on Machine Learning (ICML), 2021, pp. 10 096–10 106.
A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations (ICLR), 2021, pp. 1–12.
X.-S. Wei, Y.-Z. Song, O. Mac Aodha, J. Wu, Y. Peng, J. Tang, J. Yang, and S. Belongie, “Fine-grained image analysis with deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 12, pp. 8927–8948, 2022.
L. Lu, P. Wang, and Y. Cao, “A novel part-level feature extraction method for fine-grained vehicle recognition,” Pattern Recognition, vol. 131, no. 108869, p. 108869, 2022.
A. Hassan, M. Ali, N. M. Durrani, and M. A. Tahir, “An empirical analysis of deep learning architectures for vehicle make and model recognition,” IEEE Access, vol. 9, pp. 91 487–91 499, 2021.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: synthetic minority over-sampling technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002.
Publicado
30/09/2024
Como Citar
LIMA, Gabriel E.; LAROCA, Rayson; SANTOS, Eduardo; NASCIMENTO JR., Eduil; MENOTTI, David.
Improving Vehicle Identification Through Advanced Fine-Grained Vehicle Classification. In: WORKSHOP DE TRABALHOS EM ANDAMENTO - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 37. , 2024, Manaus/AM.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 123-128.
DOI: https://doi.org/10.5753/sibgrapi.est.2024.31656.
