Deep Reinforcement Learning Applied to IEEE Very Small Size Soccer Strategy

• Thiago Felipe de Medeiros ITA
• Marcos Máximo ITA
• Takashi Yoneyama ITA

Abstract

This work concerns a virtual learning framework to train and develop specific behaviours for robots in the context of the IEEE Very Small Size Soccer competition based on Deep Reinforcement Learning. The agents are required to learn how to intercept the ball on the player’s side of the field in the presence of an adversary robot nearby. The Proximal Policy Optimization algorithm, augmented by Curriculum Learning, is used during the training process. The performance of the trained agents was evaluated and compared to those, in the same scenario, but without the Curriculum Learning technique and also to agents following previously used heuristic behaviors. As a result, agents trained with Curriculum Learning outperformed, with an success rate of 79.90%, both agents trained without Curriculum Learning and agents that used the chosen heuristic behaviour, which attained a success rate of 72.10% and 45.50%, respectively.

Keywords: Sports, Meters, Training, Mobile robots, Games, Wheels, Navigation