Real-time Ball Detection for Robocup Soccer Using Convolutional Neural Networks
Resumo
The RoboCup Soccer is one of the largest competitions in the robotics field of research. It considers the soccer match as a challenge for the robots and aims to win a match between humans versus robots by the year of 2050. The vision module is a critical system for the robots because it needs to quickly locate and classify objects of interest for the robot in order to generate the next best action. In this paper, an approach using Convolutional Neural Networks for object detection is described. The soccer ball is the chosen object and three state-ofart convolutional neural networks architectures were trained for the experiment using data augmentation and transfer learning techniques. The models were evaluated in a test set, yielding promising results in precision and frames per second. The best model achieved an average precision of 0.972 with an intersection over union of 50% and 9.64 frames per second, running on CPU.
Palavras-chave:
RoboCup, Object Detection, Convolutional Neural Networks, MobileNetV2, Faster R-CNN
Referências
J. Canas, D. Puig, E. Perdices and T. Gonzalez. Visual Goal Detection for the RoboCup Standard Platform League.
N. Dalal, B. Triggs. Histograms of Oriented Gradients for Human Detection. International Conference on Computer Vision Pattern Recognition (CVPR '05), Jun 2005, San Diego, United States. pp. 886 - DOI: 10.1109/cvpr.2005.177
P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, Kauai, HI, USA, 2001, pp. I-I. DOI: 10.1109/cvpr.2001.990517
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV, DOI: 10.1007/s11263-015-0816-y
K. Simonyan, A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556
A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 ( NIPS '12), F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.), Vol. 1. Curran Associates Inc., USA, 1097 - 1105.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich. Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pages 1 - IEEE Computer Society, 2015.
K. He, X. Zhang, S. Ren, J. Sun. Deep residual learning for image recognition. CoRR, abs/1512.03385, DOI: 10.1109/cvpr.2016.90
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L. Chen. MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.
D. Speck, P. Barros, C. Weber, S. Wermter. Ball Localization for Robocup Soccer Using Convolutional Neural Networks, RoboCup, DOI: 10.1007/978-3-319-68792-6_2
J. Menashe, J. Kelle, K. Genter, J. Hanna, E. Liebman, S. Narvekar, R. Zhang, P. Stone. Fast and Precise Black and White Ball Detection for RoboCup Soccer, 2018. Lecture Notes in Computer Science, 45 - 48. DOI: 10.1007/978-3-030-00308-1_4
N. Cruz, K. Lobos-Tsunekawa, J. Ruiz-del-Solar. Using Convolutional Neural Networks in Robots with Limited Computational Resources: Detecting NAO Robots While Playing Soccer. RoboCup 2017: Robot World Cup XXI. Lecture Notes in Computer Science, pp. 19 - 30. DOI: 10.1007/978-3-030-00308-1_2
M. Buric, M. Pobar, M. Ivasic-Kos, Marina. Adapting YOLO Network for Ball and Player Detection, 845 - 851. DOI: 10.5220/0007582008450851
J. Redmon, S. Divvala, R. Girshick, A. Farhadi. You Only Look Once: Unified, Real-Time Object Detection, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779 - 788. DOI: 10.1109/cvpr.2016.91
R. Girshick, J. Donahue, T. Darrell, J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 580 - 587. IEEE, 2014. DOI: 10.1109/cvpr.2014.81
S. Ren, K. He, R. Girshick and J. Sun, Faster R-CNN : Towards Real-Time Object Detection with Region Proposal Networks, in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137 - 1149, 1 June 2017. DOI: 10.1109/tpami.2016.2577031
A. Ge ´ron. Hands-On Machine Learning with Scikit-Learn and TensorFlow, 2019, O 'Reilly Media.
C. Murch, S. Chalup. Combining edge detection and colour segmentation in the four-legged league. In: Australasian Conference on Robotics and Automation (ACRA 2004) (2004).
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. A. less. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. CoRR ( 2017).
J. Wu, C. Leng, Y. Wang, Q. Hu, J. Cheng. Quantized Convolutional Neural Networks for Mobile Devices. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ( 2016). DOI: 10.1109/cvpr.2016.521
X. Chen, H. Mulam. An Implementation of Faster RCNN with Study for Region Sampling. CoRR abs/02138 (2017).
D. Cook, K. Feuz, N. Krishnan. Transfer learning for activity recognition: a survey, 2013. Knowledge and Information Systems, 36, 537 - 556. DOI: 10.1007/s10115-013-0665-3
L. Abreu. Object Detection Demo, 2019, GitHub repository, https://github.com/labreu/object_detection_demo
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, R. Jozefowicz, Y. Jia, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, M. Schuster, R. Monga, S. Moore, D. Murray, C. Olah, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, X. Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, Software available from tensorflow. org.
J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, J. Murphy. Speed/accuracy tradeoffs for modern convolutional object detectors., 2017, CVPR.
T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. Zitnick. Microsoft COCO : Common Objects in Context., Lecture Notes in Computer Science: 740 - 755. DOI: 10.1007/978-3-319-10602-1_48
M. Everingham, S. Eslami, L. Gool, C. Williams, J. Winn, A. Zisserman. The Pascal Visual Object Classes (VOC) challenge - a Retrospective, IJCV. DOI: 10.1007/s11263-009-0275-4
N. Dalal, B. Triggs. Histograms of Oriented Gradients for Human Detection. International Conference on Computer Vision Pattern Recognition (CVPR '05), Jun 2005, San Diego, United States. pp. 886 - DOI: 10.1109/cvpr.2005.177
P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, Kauai, HI, USA, 2001, pp. I-I. DOI: 10.1109/cvpr.2001.990517
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV, DOI: 10.1007/s11263-015-0816-y
K. Simonyan, A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556
A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 ( NIPS '12), F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.), Vol. 1. Curran Associates Inc., USA, 1097 - 1105.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich. Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pages 1 - IEEE Computer Society, 2015.
K. He, X. Zhang, S. Ren, J. Sun. Deep residual learning for image recognition. CoRR, abs/1512.03385, DOI: 10.1109/cvpr.2016.90
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L. Chen. MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.
D. Speck, P. Barros, C. Weber, S. Wermter. Ball Localization for Robocup Soccer Using Convolutional Neural Networks, RoboCup, DOI: 10.1007/978-3-319-68792-6_2
J. Menashe, J. Kelle, K. Genter, J. Hanna, E. Liebman, S. Narvekar, R. Zhang, P. Stone. Fast and Precise Black and White Ball Detection for RoboCup Soccer, 2018. Lecture Notes in Computer Science, 45 - 48. DOI: 10.1007/978-3-030-00308-1_4
N. Cruz, K. Lobos-Tsunekawa, J. Ruiz-del-Solar. Using Convolutional Neural Networks in Robots with Limited Computational Resources: Detecting NAO Robots While Playing Soccer. RoboCup 2017: Robot World Cup XXI. Lecture Notes in Computer Science, pp. 19 - 30. DOI: 10.1007/978-3-030-00308-1_2
M. Buric, M. Pobar, M. Ivasic-Kos, Marina. Adapting YOLO Network for Ball and Player Detection, 845 - 851. DOI: 10.5220/0007582008450851
J. Redmon, S. Divvala, R. Girshick, A. Farhadi. You Only Look Once: Unified, Real-Time Object Detection, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779 - 788. DOI: 10.1109/cvpr.2016.91
R. Girshick, J. Donahue, T. Darrell, J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 580 - 587. IEEE, 2014. DOI: 10.1109/cvpr.2014.81
S. Ren, K. He, R. Girshick and J. Sun, Faster R-CNN : Towards Real-Time Object Detection with Region Proposal Networks, in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137 - 1149, 1 June 2017. DOI: 10.1109/tpami.2016.2577031
A. Ge ´ron. Hands-On Machine Learning with Scikit-Learn and TensorFlow, 2019, O 'Reilly Media.
C. Murch, S. Chalup. Combining edge detection and colour segmentation in the four-legged league. In: Australasian Conference on Robotics and Automation (ACRA 2004) (2004).
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. A. less. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. CoRR ( 2017).
J. Wu, C. Leng, Y. Wang, Q. Hu, J. Cheng. Quantized Convolutional Neural Networks for Mobile Devices. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ( 2016). DOI: 10.1109/cvpr.2016.521
X. Chen, H. Mulam. An Implementation of Faster RCNN with Study for Region Sampling. CoRR abs/02138 (2017).
D. Cook, K. Feuz, N. Krishnan. Transfer learning for activity recognition: a survey, 2013. Knowledge and Information Systems, 36, 537 - 556. DOI: 10.1007/s10115-013-0665-3
L. Abreu. Object Detection Demo, 2019, GitHub repository, https://github.com/labreu/object_detection_demo
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, R. Jozefowicz, Y. Jia, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, M. Schuster, R. Monga, S. Moore, D. Murray, C. Olah, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, X. Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, Software available from tensorflow. org.
J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, J. Murphy. Speed/accuracy tradeoffs for modern convolutional object detectors., 2017, CVPR.
T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. Zitnick. Microsoft COCO : Common Objects in Context., Lecture Notes in Computer Science: 740 - 755. DOI: 10.1007/978-3-319-10602-1_48
M. Everingham, S. Eslami, L. Gool, C. Williams, J. Winn, A. Zisserman. The Pascal Visual Object Classes (VOC) challenge - a Retrospective, IJCV. DOI: 10.1007/s11263-009-0275-4
Publicado
09/09/2019
Como Citar
DE ABREU, Lucas Ribeiro; BIANCHI, Reinaldo Augusto da Costa.
Real-time Ball Detection for Robocup Soccer Using Convolutional Neural Networks. In: WORKSHOP DE VISÃO COMPUTACIONAL (WVC), 15. , 2019, São Bernardo do Campo.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
p. 103-108.
DOI: https://doi.org/10.5753/wvc.2019.7636.
