Development of a Vision System for Object Detection Embedded in a Robotic Arm

• Igor F. Soares UEFS
• Anfranserai M. Dias UEFS

DOI: https://doi.org/10.5753/erbase.2024.4511

Abstract

The present work describes the development of a vision system for object detection and measurement to be embedded in a robotic arm, using the ESP32-CAM module for image acquisition and a distance sensor. The transfer learning technique was applied with the EfficientDet-Lite model for object detection. Additionally, an object measurement model was developed based on images, which relates pixels to the actual dimensions of the object through linear regression. The system was trained with images of lego pieces, achieving an AP of 81.92% on the validation set and 61.86% on the test set. The measurement model showed an RMSE of less than 10 mm when measuring object sizes at distances of up to 340 mm, with performance degradation at greater distances. This limitation is acceptable, as the manipulator used in the project has a maximum reach of 390 mm.

References

Ali, M. H., Aizat, K., Yerkhan, K., Zhandos, T., and Anuar, O. (2018). Vision-based robot manipulator for industrial applications. Procedia computer science, 133:205–212.

COCO (2015). Detection evaluation metrics. [link]. [Online]. Acesso em 17 de agosto de 2023.

Kamal, M. U. A., Suciati, N., and Hidayati, S. C. (2021). Calculation of handwriting mathematics expressions on mobile devices using efficientdet-lite0 and reverse polish notation. The Journal on Machine Learning and Computational Intelligence (JMLCI), 1(1).

Matarić, M. J. (2014). Introdução à robótica. Editora Blucher.

Mehendale, N. (2022). Object detection using esp 32 cam. SSRN Electronic Journal.

Rice, C., Taylor, J., Widom, J., Zegart, A., and Lin, H. (2023). The Stanford Emerging Technology Review 2023: A Report on Ten Key Technologies and Their Policy Implications. Stanford University.

Silva, G. R. (2018). Detecção de objetos em imagens utilizando técnicas de aprendizagem profunda. Trabalho de Conclusão de Curso. Universidade Federal de Santa Catarina, Florianópolis.

Song, Q., Li, S., Bai, Q., Yang, J., Zhang, X., Li, Z., and Duan, Z. (2021). Object detection method for grasping robot based on improved yolov5. Micromachines, 12(11):1273.

Szeliski, R. (2011). Computer Vision: Algorithms and Applications. Springer London, 1 edition.

Tan, M., Pang, R., and Le, Q. V. (2020). Efficientdet: Scalable and Efficient Object Detection.

Yahboom (2024). Dofbot-jetson repository. [link]. Acesso em 30 de setembro de 2024.