Computer Vision and Force Control in Robotic Grippers: An Integrated Approach

• Amanda F. N. Martins IFES
• Bruna H. Xavier IFES
• Thiago B. Pereira IFES
• Debora G. Moura IFES
• Elisa de P. P. Marinho IFES
• Thiago M. Paixão IFES
• Richard J. M. G. Tello IFES

DOI: https://doi.org/10.5753/eries.2024.244700

Abstract

In the present work, a hardware/software system was developed that uses computer vision to recognize objects and adjust the gripping force of a robotic claw. The integration of the force sensor and computer vision enabled the creation of an adaptive gripping system. In a previous study, the gripping limits for objects such as a lemon, pen, and cup were recorded using an FSR402 sensor and the Arduino UNO R3 platform. Using a ResNet-18 neural network, the system was trained with images of the objects and achieved 100% accuracy in recognition. Specific commands were sent to the Arduino, controlling the claw’s grip. This project contributes to advances in industrial robotics, rehabilitation, and assistive technologies.

References

Huang, Y., Wang, H., and Zhang, X. (2024). Tactile-sensing-based robotic grasping stability analysis. Sci. China Technol., 67(1817–1828).

Niku, S. B. (2013). Introdução à Robótica - Análise, Controle, Aplicações. LTC, 2 edition.

Romano, J. M., Hsiao, K., Niemeyer, G., Chitta, S., and Kuchenbecker, K. J. (2011). Human-inspired robotic grasp control with tactile sensing. IEEE Transactions on Robotics, 27(6):1067–1079.