Applying Reinforcement Learning for Multiple Functions in Swarm Intelligence

• André A. V. Escorel Ribeiro UPE
• Rodrigo Cesar Lira UPE
• Mariana Macedo University of Toulouse
• Hugo Valadares Siqueira UTFPR
• Carmelo Bastos-Filho UPE

Resumo

Swarm intelligence (SI) algorithms have become popular due to their self-learning characteristics and adaptability to external changes. They can find reasonable solutions to complex problems without in-depth knowledge. Much of the success of these algorithms comes from balancing the exploration and exploitation tasks. This work evaluates the application and performance of a reinforcement learning approach applied to a well-known swarm intelligence algorithm, Particle Swarm Optimization (PSO). We use the reinforcement learning agent Proximal Policy Optimization (PPO) to dynamically change the swarm communication topology according to the problem. We analyze the PSO’s behavior, influenced by the reinforcement learning agent, through methods such as interaction networks and fitness analysis. We show that the RL approach can transfer the knowledge learned from one function to other functions, and that dynamic changes of topology over time makes PSO much more efficient than setting only one specific topology, even when using a Dynamic topology. Our results then suggest that changing topologies might be more efficient than having a Dynamic topology, and that indeed Local and Global topologies have an important role in the best swarm performance. Our results take a step further on explaining the performance of SI and automatizing their use for non-experts.