An End-to-End Multimodal System for Large-Scale Road Infrastructure Monitoring Using Computer Vision and Geospatial Integration
Resumo
We present a multimodal road-monitoring system for Brazil using onboard videos, telemetry, computer vision, and geospatial references. It aligns frames with Global Navigation Satellite System (GNSS) telemetry, filters them using motion evidence, applies models to pavement defects, horizontal markings, traffic signs, and roadside vegetation, and aggregates auditable evidence by segment. Trajectories undergo map matching to OpenStreetMap (OSM) with GraphHopper through mmlib, then are linked to official federal and state road references. In operation, 23.6 million telemetry points supported 6.26 million frame-level inferences, ranging from crack detection to traffic-sign type detection at 97.57% mAP@50.
Referências
Bruno, S., De Amicis, A., Loprencipe, G., Orlandini, S., Rotondi, G., and Vita, L. (2025).
Advancing road infrastructure: Cross-slope analysis of road sections using a mobile mapping system (mms). Transportation Research Procedia, 90:456–463.
Chen, Z., Zhang, Y., Luo, Y., Wang, Z., Zhong, J., and Southon, A. (2021). Roadatlas: Intelligent platform for automated road defect detection and asset management. In Proceedings of the 3rd ACM International Conference on Multimedia in Asia, pages 1–3.
Dias de Souza, J. V. (2026). mmlib: Interoperability framework for map matching. [link]. Accessed 13 May 2026.
DNIT (2024). Índice de condição da manutenção. [link]. Page last updated on 9 October 2024.
DNIT (2026). Plano nacional de viação e sistema nacional de viação. [link]. Page last updated on 26 February 2026.
Dong, S., Wang, Y., Cao, J., Ma, J., Chen, Y., and Kang, X. (2025). Advanced lightweight deep learning vision framework for efficient pavement damage identification. Scientific Reports, 15(1):12966.
Farnebäck, G. (2003). Two-frame motion estimation based on polynomial expansion. In Scandinavian Conference on Image Analysis, pages 363–370. Springer.
GraphHopper Project (2026). Graphhopper routing engine. [link]. Accessed 13 May 2026.
Han, C., Yang, S. B., and Hu, J. (2026). Diffmm: Efficient method for accurate noisy and sparse trajectory map matching via one step diffusion. arXiv preprint arXiv:2601.08482.
IDB (2025). Pavimentados: Library for detecting cracks and failures on roads. [link]. Inter-American Development Bank. Open-source software repository. Accessed 13 May 2026.
Katsamenis, I., Bakalos, N., Lappas, A., Protopapadakis, E., Martín-Portugués Montoliu, C., Doulamis, A., Doulamis, N., Rallis, I., and Kalogeras, D. (2025). Dorie: Dataset of road infrastructure elements—a benchmark of yolo architectures for real-time patrol vehicle monitoring. Sensors, 25(21):6653.
Kulawik, J. and Kuczyński, Ł. (2025). Ai-based detection and classification of horizontal road markings in digital images dedicated to driver assistance systems. Applied Sciences, 15(22):12189.
Ma, L. and Chen, M. (2025). Road damage detection based on improved yolo algorithm. Scientific Reports, 15(1):28506.
Sheikholeslami, D., Golroo, A., Khodaii, A., Daneshvar, A., and Moghadas Nejad, F. (2026). A physic-guided yolo framework for pavement deformation distress detection. Scientific Reports.
Stipancic, J., Saunier, N., Navidi, N., Racine, E. B., Miranda-Moreno, L., and Labbe, A. (2025). Evaluating map matching algorithms for smartphone gnss data: Matching vehicle trajectories to an urban road network. Transportation Research Procedia, 82:303–322.
Wang, C., Zheng, B., and Li, C. (2025). Efficient traffic sign recognition using yolo for intelligent transport systems. Scientific reports, 15(1):13657.
Zhang, H., Liang, M., and Wang, Y. (2025a). Yolo-bs: a traffic sign detection algorithm based on yolov8. Scientific Reports, 15(1):7558.
Zhang, Y., Yan, H., and Lu, X. (2025b). An enhanced hmm map matching algorithm incorporating personal road selection preferences. Scientific Reports, 15(1):35962.
Zhumadillayeva, A., Ahanger, T. A., and Matkarimov, B. (2025). An intelligent yolo and cnn-bigru framework for road infrastructure based anomaly assessment. Scientific Reports, 15(1):41193.
Advancing road infrastructure: Cross-slope analysis of road sections using a mobile mapping system (mms). Transportation Research Procedia, 90:456–463.
Chen, Z., Zhang, Y., Luo, Y., Wang, Z., Zhong, J., and Southon, A. (2021). Roadatlas: Intelligent platform for automated road defect detection and asset management. In Proceedings of the 3rd ACM International Conference on Multimedia in Asia, pages 1–3.
Dias de Souza, J. V. (2026). mmlib: Interoperability framework for map matching. [link]. Accessed 13 May 2026.
DNIT (2024). Índice de condição da manutenção. [link]. Page last updated on 9 October 2024.
DNIT (2026). Plano nacional de viação e sistema nacional de viação. [link]. Page last updated on 26 February 2026.
Dong, S., Wang, Y., Cao, J., Ma, J., Chen, Y., and Kang, X. (2025). Advanced lightweight deep learning vision framework for efficient pavement damage identification. Scientific Reports, 15(1):12966.
Farnebäck, G. (2003). Two-frame motion estimation based on polynomial expansion. In Scandinavian Conference on Image Analysis, pages 363–370. Springer.
GraphHopper Project (2026). Graphhopper routing engine. [link]. Accessed 13 May 2026.
Han, C., Yang, S. B., and Hu, J. (2026). Diffmm: Efficient method for accurate noisy and sparse trajectory map matching via one step diffusion. arXiv preprint arXiv:2601.08482.
IDB (2025). Pavimentados: Library for detecting cracks and failures on roads. [link]. Inter-American Development Bank. Open-source software repository. Accessed 13 May 2026.
Katsamenis, I., Bakalos, N., Lappas, A., Protopapadakis, E., Martín-Portugués Montoliu, C., Doulamis, A., Doulamis, N., Rallis, I., and Kalogeras, D. (2025). Dorie: Dataset of road infrastructure elements—a benchmark of yolo architectures for real-time patrol vehicle monitoring. Sensors, 25(21):6653.
Kulawik, J. and Kuczyński, Ł. (2025). Ai-based detection and classification of horizontal road markings in digital images dedicated to driver assistance systems. Applied Sciences, 15(22):12189.
Ma, L. and Chen, M. (2025). Road damage detection based on improved yolo algorithm. Scientific Reports, 15(1):28506.
Sheikholeslami, D., Golroo, A., Khodaii, A., Daneshvar, A., and Moghadas Nejad, F. (2026). A physic-guided yolo framework for pavement deformation distress detection. Scientific Reports.
Stipancic, J., Saunier, N., Navidi, N., Racine, E. B., Miranda-Moreno, L., and Labbe, A. (2025). Evaluating map matching algorithms for smartphone gnss data: Matching vehicle trajectories to an urban road network. Transportation Research Procedia, 82:303–322.
Wang, C., Zheng, B., and Li, C. (2025). Efficient traffic sign recognition using yolo for intelligent transport systems. Scientific reports, 15(1):13657.
Zhang, H., Liang, M., and Wang, Y. (2025a). Yolo-bs: a traffic sign detection algorithm based on yolov8. Scientific Reports, 15(1):7558.
Zhang, Y., Yan, H., and Lu, X. (2025b). An enhanced hmm map matching algorithm incorporating personal road selection preferences. Scientific Reports, 15(1):35962.
Zhumadillayeva, A., Ahanger, T. A., and Matkarimov, B. (2025). An intelligent yolo and cnn-bigru framework for road infrastructure based anomaly assessment. Scientific Reports, 15(1):41193.
Publicado
19/10/2026
Como Citar
FERRUFINO, Paulina Irene Velasquez; PAIVA, Matheus Gois; SOUSA, Cornélio Albuquerque de; QUEIROZ NETO, José Florêncio de; PORCINO, Wellington Clay; MACEDO, José Antônio Fernandes de.
An End-to-End Multimodal System for Large-Scale Road Infrastructure Monitoring Using Computer Vision and Geospatial Integration. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 1280-1291.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.31054.
