Residual Reinforcement Learning to Generate a Closed-Loop Policy from a Keyframe Kick Motion - Passamani, Marcos Vinícius S.; Maximo, Marcos R. O. A.; Melo, Luckeciano C. (Anais do Simpósio Brasileiro de Robótica e Simpósio Latino Americano de Robótica (SBR/LARS), 2025)
The RoboCup is an international robot football competition that aims to push the boundaries of robotics and artificial intelligence to a point where a team of robotic players could beat a human World Cup champion team. One of the major categories of RoboCup is the 3D Soccer Simulation League, which consists of a full game of football executed in a physically simulated environment. In this league, keyframe interpolation is a very common practice for executing specific tasks such as getting up, diving, or kicking the ball. A primary problem with this method is its open-loop kinematic description of movement, in the sense that it lacks feedback information from the world, which makes the behavior prone to underperformance or failure in complex and noisy environments. This work contributes by providing a methodology to enhance keyframe movements using residual learning, in which we train a policy using the Proximal Policy Optimization (PPO) algorithm to improve a keyframe kick motion. The approach not only improved the reliability of the kick but also enhanced its performance metrics, including distance and precision.