Reinforcement Learning for Shoulder Control of the NAO Robot Using Q-Learning: Tests in Real and Simulated Environments - Souza, Vitor Amadeu; Sá, Hebert Azevedo (Anais do Seminário Integrado de Software e Hardware (SEMISH), 2026)
This paper presents a Q-Learning approach for controlling the NAO humanoid robot’s ShoulderPitch joint, motivated by the limitations of traditional PID controllers in handling nonlinearities and inter-joint coupling. The methodology involves training in CoppeliaSim simulation before deployment on the physical robot, with a reward function based on absolute angular error relative to a 45° target and discretized state space. Results demonstrate policy convergence in simulation and real-world implementation, with the performance gap providing insights into sim-to-real transfer challenges and establishing a foundation for future multi-joint applications.