Robotic Arm Teleoperation Through Recognition of Body Movements

• Marcos Barbosa Schettino UFJF
• Aline Gabriela Loiola Almeida CEFET-MG
• André Luis Marques Marcato UFJF
• Vinícius Barbosa Schettino CEFET-MG

Resumo

The teleoperation of robotic arms traditionally relies on specialized devices, such as wearable sensors or depth cameras, which increase costs and limit applicability in resource-constrained environments. This paper proposes an alternative approach to robotic arm teleoperation using a single two-dimensional RGB camera to recognize hand gestures and estimate body poses. Using the MediaPipe framework, keypoints are extracted in real time, enabling both the recognition of manual gestures for trajectory planning and execution, and the mapping of arm extension to determine movement directions and amplitudes. To validate the approach, the Panda robotic arm was simulated in the Gazebo environment. The operator interacts with the system through an intuitive interface, defining target poses and-based on real-time gesture detection-sending commands to the robot. Results show that execution times are comparable to those achieved with joystick-based teleoperation, demonstrating the feasibility and efficiency of the proposed method. This study contributes to the advancement of robotics by presenting an accessible and effective solution for robotic system teleoperation.

Palavras-chave: Hands, Trajectory planning, Robot vision systems, Education, Manuals, Gesture recognition, Manipulators, Cameras, Real-time systems, Wearable sensors, teleoperation, camera, body gestures, robotic arm, gesture recognition