Abstract
Swarm intelligence (SI) algorithms have become popular due to their self-learning characteristics and adaptability to external changes. They can find reasonable solutions to complex problems without in-depth knowledge. Much of the success of these algorithms comes from balancing the exploration and exploitation tasks. This work evaluates the application and performance of a reinforcement learning approach applied to a well-known swarm intelligence algorithm, Particle Swarm Optimization (PSO). We use the reinforcement learning agent Proximal Policy Optimization (PPO) to dynamically change the swarm communication topology according to the problem. We analyze the PSO’s behavior, influenced by the reinforcement learning agent, through methods such as interaction networks and fitness analysis. We show that the RL approach can transfer the knowledge learned from one function to other functions, and that dynamic changes of topology over time makes PSO much more efficient than setting only one specific topology, even when using a Dynamic topology. Our results then suggest that changing topologies might be more efficient than having a Dynamic topology, and that indeed Local and Global topologies have an important role in the best swarm performance. Our results take a step further on explaining the performance of SI and automatizing their use for non-experts.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bansal, J.C., Singh, P.K., Pal, N.R. (eds.): Evolutionary and Swarm Intelligence Algorithms, Studies in Computational Intelligence, vol. 779. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-319-91341-4
Dorigo, M., Maniezzo, V., Colorni, A.: Ant system: optimization by a colony of cooperating agents. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 26(1), 29–41 (1996). https://doi.org/10.1109/3477.484436
Engstrom, L., et al.: Implementation matters in deep policy gradients: a case study on PPO and TRPO (2020)
Junior, M.A.C.O., Bastos Filho, C.J.A., Menezes, R.: Using network science to define a dynamic communication topology for particle swarm optimizers. In: Menezes, R., Evsukoff, A., González, M. (eds.) Complex Networks. Studies in Computational Intelligence, vol. 424, pp. 39–47. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-30287-9_5
Karaboga, D., et al.: An idea based on honey bee swarm for numerical optimization. Technical report, Technical report-tr06, Erciyes University, Engineering faculty (2005)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN 1995 - International Conference on Neural Network, vol. 4, pp. 1942–1948 (1995). https://doi.org/10.1109/ICNN.1995.488968
Kennedy, J.: Swarm Intelligence, pp. 187–219. Springer, US, Boston, MA (2006)
Lira, R.C., Macedo, M., Siqueira, H.V., Bastos-Filho, C.: Integrating reinforcement learning and optimization task: Evaluating an agent to dynamically select PSO communication topology. In: Tan, Y., Shi, Y., Luo, W. (eds.) Advances in Swarm Intelligence. ICSI 2023. LNCS, vol. 13969, pp. 38–48. Springer, Cham (2023).https://doi.org/10.1007/978-3-031-36625-3_4
Macedo, M., et al.: Overview on binary optimization using swarm-inspired algorithms. IEEE Access 9, 149814–149858 (2021). https://doi.org/10.1109/ACCESS.2021.3124710
Oliveira, M., Bastos-Filho, C.J.A., Menezes, R.: Towards a network-based approach to analyze particle swarm optimizers. In: 2014 IEEE Symposium on Swarm Intelligence, pp. 1–8 (2014). https://doi.org/10.1109/SIS.2014.7011791
Oliveira, M., Bastos-Filho, C.J.A., Menezes, R.: Using network science to assess particle swarm optimizers. Soc. Netw. Anal. Min. 5(1), 3 (2015). https://doi.org/10.1007/s13278-015-0245-5
Oliveira, M., Pinheiro, D., Andrade, B., Bastos-Filho, C., Menezes, R.: Communication diversity in particle swarm optimizers. In: Dorigo, M., et al. (eds.) ANTS 2016. LNCS, vol. 9882, pp. 77–88. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44427-7_7
Parpinelli, R.S., Lopes, H.S.: New inspirations in swarm intelligence: a survey. Int. J. Bio-Inspired Comput. 3(1), 1–16 (2011). https://doi.org/10.1504/IJBIC.2011.038700
Pervaiz, S., Ul-Qayyum, Z., Bangyal, W.H., Gao, L., Ahmad, J.: A systematic literature review on particle swarm optimization techniques for medical diseases detection. Comput. Math. Methods Med. 2021, 1–10 (2021). https://doi.org/10.1155/2021/5990999
Plevris, V., Solorzano, G.: A collection of 30 multidimensional functions for global optimization benchmarking. Data 7(4), 46 (2022). https://doi.org/10.3390/data7040046
Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization: an overview. Swarm Intell. 1, 33–57 (2007)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). https://doi.org/10.48550/ARXIV.1707.06347
da Silveira Câmara Augusto, J.P., dos Santos Nicolau, A., Schirru, R.: PSO with dynamic topology and random keys method applied to nuclear reactor reload. Progr. Nucl. Energy. 83, 191–196 (2015). https://doi.org/10.1016/j.pnucene.2015.03.009
Wauters, T., Verbeeck, K., De Causmaecker, P., Vanden Berghe, G.: Boosting metaheuristic search using reinforcement learning. In: Talbi, EG. (eds.) Hybrid Metaheuristics. Studies in Computational Intelligence, vol 434, pp. 432–452. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-30671-6_17
Wu, D., Wang, G.G.: Employing reinforcement learning to enhance particle swarm optimization methods. Eng. Optim. 54(2), 329–348 (2022). https://doi.org/10.1080/0305215X.2020.1867120
Xu, Y., Pi, D.: A reinforcement learning-based communication topology in particle swarm optimization. Neural Comput. Appl. 32(14), 10007–10032 (2020). https://doi.org/10.1007/s00521-019-04527-9
Acknowledgements
The authors thank the Federal Institute of Pernambuco (IFPE), Brazilian National Council for Scientific and Technological Development (CNPq), processes number 40558/2018-5, 315298/2020-0, and Araucaria Foundation, process number 51497, for their financial support. Mariana Macedo was supported by the Artificial and Natural Intelligence Toulouse Institute (ANITI) - Institut 3iA: ANR-19-PI3A-0004.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ribeiro, A.A.V.E., Lira, R.C., Macedo, M., Siqueira, H.V., Bastos-Filho, C. (2023). Applying Reinforcement Learning for Multiple Functions in Swarm Intelligence. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14196. Springer, Cham. https://doi.org/10.1007/978-3-031-45389-2_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-45389-2_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45388-5
Online ISBN: 978-3-031-45389-2
eBook Packages: Computer ScienceComputer Science (R0)