Intelligent Detection of Faults in Asphalt Paving with Regional Convolutional Neural Networks
Abstract
In this paper we addressed the automatic road damage inspection as a Computer Vision detection problem in benefit of solutions to help smart cities improve traffic quality and security. To do so, we considered an experimental scenario with realistic data from three different countries and four configurations of YOLO networks. When compared to related work from literature, our results have significant improvements in prediction time using a lower number of parameters, yielding an experimental mAP of 0.53. We also evaluated our solution in a case study with images from Brazil that highlights several practical challenges that need to be taken into account when proposing automatic detection models for such problem.
Keywords:
Convolutional Neural Networks, Asphalt Paving, YOLO
References
Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Mraz, A., Kashiyama, T., and Sekimoto, Y. (2021a). Deep learning-based road damage detection and classification for multiple countries. Automation in Construction, 132:103935.
Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Omata, H., Kashiyama, T., and Sekimoto, Y. (2020). Global road damage detection: State-of-the-art solutions. In 2020 IEEE International Conference on Big Data (Big Data), pages 5533–5539. IEEE.
Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., and Sekimoto, Y. (2021b). RDD2020: An annotated image dataset for automatic road damage detection using deep learning. Data in Brief, 36:107133.
Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.
Cao, M.-T., Tran, Q.-V., Nguyen, N.-M., and Chang, K.-T. (2020). Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources. Adv. Eng. Inform., 46(101182):101182.
CNT (2018). Confederação Nacional do Transporte (cnt) – conheça os 13 principais defeitos do pavimento das rodovias. Disponível em https://www.cnt.org.br/agencia-cnt/conheca-principais-defeitos-pavimento. Acesso em 19 de maio de 2022.
CNT (2019). Confederação Nacional do Transporte (cnt – piora a qualidade das rodovias brasileiras. Disponível em https://www.cnt.org.br/agencia-cnt/piora-a-qualidade-das-rodovias-brasileiras. Acesso em 19 de maio de 2022.
CNT (2020). Confederação Nacional do Transporte (cnt) – CNT lança painel com dados do transporte rodoviário no Brasil. Disponível em [link]. Acesso em 19 de maio de 2022.
Hegde, V., Trivedi, D., Alfarrarjeh, A., Deepak, A., Ho Kim, S., and Shahabi, C. (2020). Yet another deep learning approach for road damage detection using ensemble learning. In 2020 IEEE International Conference on Big Data (Big Data). IEEE.
Janani, R. P., Renuka, K., Aruna, A., and Lakshmi, N. K. (2021). IoT in smart cities: A contemporary survey. Global Transitions Proceedings. Disponível em https://doi.org/10.1016/j.gltp.2021.08.069. Acesso em 19 de maio de 2022.
Jocher, G., Stoken, A., Borovec, J., NanoCode012, ChristopherSTAN, Changyu, L., Laughing, tkianai, Hogan, A., lorenzomammana, yxNONG, AlexWang1900, Diaconu, L., Marc, wanghaoyang0106, ml5ah, Doug, Ingham, F., Frederik, Guilhen, Hatovix, Poznanski, J., Fang, J., Yu, L., changyu98, Wang, M., Gupta, N., Akhtar, O., PetrDvoracek, and Rai, P. (2020). ultralytics/yolov5: v3.1 Bug Fixes and Performance Improvements. Disponível em https://doi.org/10.5281/zenodo.4154370. Acesso em 19 de maio de 2022.
Maeda, H., Arya, D., Sekimoto, Y., Ghosh, S. K., Toshniwal, D., Seto, T., Kashiyama, T., and Omata, H. (2020). Global road damage detection challenge 2020. IEEE BigData 2020. Disponível em https://rdd2020.sekilab.global/. Acesso em 19 de maio de 2022.
Michelucci, U. (2019). Advanced applied deep learning: convolutional neural networks and object detection. Springer.
Miller, T. R. and Zaloshnja, E. (2009). On a crash course: The dangers and health costs of deficient roadways. Technical Report 01135610. Disponível em [link]. Acesso em 19 de maio de 2022.
ONU (2018). The World’s Cities in 2018, volume 1. Department of Economical and Social Affairs – Population Dynamics. Disponível em https://population.un.org/wup/Publications/. Acesso em 19 de maio de 2022.
Padilla, R., Netto, S. L., and da Silva, E. A. B. (2020). A Survey on Performance Metrics for Object-Detection Algorithms. In 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pages 237–242, Niterói, Brasil.
Pellicer, S., Santa, G., Bleda, A. L., Maestre, R., Jara, A. J., and Skarmeta, A. G. (2013). A global perspective of smart cities: A survey. In 2013 Seventh International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, pages 439–444. Disponível em https://10.1109/IMIS.2013.79. Acesso em 19 de maio de 2022.
Rateke, T., Justen, K. A., and Von Wangenheim, A. (2019). Road surface classification with images captured from low-cost camera Road Traversing Knowledge (RTK) dataset. Rev. Inform. Teór. Apl., 26(3):50–64.
Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You only look once: Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, Estados Unidos. IEEE.
Staniek, M. (2021). Road pavement condition diagnostics using smartphone-based data crowdsourcing in smart cities. J. Traffic Transp. Eng. (Engl. Ed.), 8(4):554–567.
Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Omata, H., Kashiyama, T., and Sekimoto, Y. (2020). Global road damage detection: State-of-the-art solutions. In 2020 IEEE International Conference on Big Data (Big Data), pages 5533–5539. IEEE.
Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., and Sekimoto, Y. (2021b). RDD2020: An annotated image dataset for automatic road damage detection using deep learning. Data in Brief, 36:107133.
Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.
Cao, M.-T., Tran, Q.-V., Nguyen, N.-M., and Chang, K.-T. (2020). Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources. Adv. Eng. Inform., 46(101182):101182.
CNT (2018). Confederação Nacional do Transporte (cnt) – conheça os 13 principais defeitos do pavimento das rodovias. Disponível em https://www.cnt.org.br/agencia-cnt/conheca-principais-defeitos-pavimento. Acesso em 19 de maio de 2022.
CNT (2019). Confederação Nacional do Transporte (cnt – piora a qualidade das rodovias brasileiras. Disponível em https://www.cnt.org.br/agencia-cnt/piora-a-qualidade-das-rodovias-brasileiras. Acesso em 19 de maio de 2022.
CNT (2020). Confederação Nacional do Transporte (cnt) – CNT lança painel com dados do transporte rodoviário no Brasil. Disponível em [link]. Acesso em 19 de maio de 2022.
Hegde, V., Trivedi, D., Alfarrarjeh, A., Deepak, A., Ho Kim, S., and Shahabi, C. (2020). Yet another deep learning approach for road damage detection using ensemble learning. In 2020 IEEE International Conference on Big Data (Big Data). IEEE.
Janani, R. P., Renuka, K., Aruna, A., and Lakshmi, N. K. (2021). IoT in smart cities: A contemporary survey. Global Transitions Proceedings. Disponível em https://doi.org/10.1016/j.gltp.2021.08.069. Acesso em 19 de maio de 2022.
Jocher, G., Stoken, A., Borovec, J., NanoCode012, ChristopherSTAN, Changyu, L., Laughing, tkianai, Hogan, A., lorenzomammana, yxNONG, AlexWang1900, Diaconu, L., Marc, wanghaoyang0106, ml5ah, Doug, Ingham, F., Frederik, Guilhen, Hatovix, Poznanski, J., Fang, J., Yu, L., changyu98, Wang, M., Gupta, N., Akhtar, O., PetrDvoracek, and Rai, P. (2020). ultralytics/yolov5: v3.1 Bug Fixes and Performance Improvements. Disponível em https://doi.org/10.5281/zenodo.4154370. Acesso em 19 de maio de 2022.
Maeda, H., Arya, D., Sekimoto, Y., Ghosh, S. K., Toshniwal, D., Seto, T., Kashiyama, T., and Omata, H. (2020). Global road damage detection challenge 2020. IEEE BigData 2020. Disponível em https://rdd2020.sekilab.global/. Acesso em 19 de maio de 2022.
Michelucci, U. (2019). Advanced applied deep learning: convolutional neural networks and object detection. Springer.
Miller, T. R. and Zaloshnja, E. (2009). On a crash course: The dangers and health costs of deficient roadways. Technical Report 01135610. Disponível em [link]. Acesso em 19 de maio de 2022.
ONU (2018). The World’s Cities in 2018, volume 1. Department of Economical and Social Affairs – Population Dynamics. Disponível em https://population.un.org/wup/Publications/. Acesso em 19 de maio de 2022.
Padilla, R., Netto, S. L., and da Silva, E. A. B. (2020). A Survey on Performance Metrics for Object-Detection Algorithms. In 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pages 237–242, Niterói, Brasil.
Pellicer, S., Santa, G., Bleda, A. L., Maestre, R., Jara, A. J., and Skarmeta, A. G. (2013). A global perspective of smart cities: A survey. In 2013 Seventh International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, pages 439–444. Disponível em https://10.1109/IMIS.2013.79. Acesso em 19 de maio de 2022.
Rateke, T., Justen, K. A., and Von Wangenheim, A. (2019). Road surface classification with images captured from low-cost camera Road Traversing Knowledge (RTK) dataset. Rev. Inform. Teór. Apl., 26(3):50–64.
Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You only look once: Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, Estados Unidos. IEEE.
Staniek, M. (2021). Road pavement condition diagnostics using smartphone-based data crowdsourcing in smart cities. J. Traffic Transp. Eng. (Engl. Ed.), 8(4):554–567.
Published
2022-07-31
How to Cite
CARVALHO, Rafael Barbosa de; GUEDES, Elloá B.; FIGUEIREDO, Carlos Maurício S..
Intelligent Detection of Faults in Asphalt Paving with Regional Convolutional Neural Networks. In: BRAZILIAN WORKSHOP ON INTELLIGENT CITIES (WBCI), 3. , 2022, Niterói.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 119-130.
DOI: https://doi.org/10.5753/wbci.2022.222795.
