A Systematic Review on Knowledge Transfer in Multi-Agent Systems using Reinforcement Learning
Resumo
Este artigo apresenta uma revisão sistemática sobre transferência de conhecimento em sistemas multiagente utilizando aprendizado por reforço. A revisão seguiu o protocolo PRISMA, analisando artigos relevantes nas bases IEEE Xplore, Scopus e Web of Science, revelando pesquisas voltadas para o aprendizado por reforço multiagente, com trabalhos buscando desenvolver algoritmos ou frameworks genéricos para a transferência de conhecimento. Abordagens recentes melhoram a eficiência e a adaptabilidade dos agentes, reduzindo o tempo de aprendizado. Porém, desafios relacionados à robustez da comunicação e à compatibilidade das representações de conhecimento ainda persistem.
Referências
Bou-Ammar, H., Eaton, E., Luna, J.-M., and Ruvolo, P. (2015). Autonomous cross-domain knowledge transfer in lifelong policy gradient reinforcement learning. In International Joint Conference on Artificial Intelligence.
Busoniu, L., Babuska, R., and De Schutter, B. (2008). A comprehensive survey of multi-agent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, 38(2):156–172.
Chen, D. and Zhang, Q. (2024). e(3)-equivariant actor-critic methods for cooperative multi-agent reinforcement learning.
Du, Y., Qi, N., Li, X., Xiao, M., Boulogeorgos, A.-A. A., Tsiftsis, T. A., and Wu, Q. (2024). Distributed multi-uav trajectory planning for downlink transmission: A gnn-enhanced drl approach. IEEE Wireless Communications Letters, 13(12):3578–3582.
Farias, G., Rodrigues, P., Adamatti, D., and Gonçalves, E. (2024). Model for knowledge transfer in agent organizations: a case study on moise+. The International FLAIRS Conference Proceedings, 37(1).
Gao, Z., Xu, K., Ding, B., and Wang, H. (2021). Knowru: Knowledge reuse via knowledge distillation in multi-agent reinforcement learning. Entropy, 23(8).
Gupta, J. K. e. a. (2017). Cooperative multi-agent control using deep reinforcement learning. International Conference on Autonomous Agents and MultiAgent Systems (AAMAS).
Kim, D., Jung, H., and Bae, J. (2024). Multi-agent network randomization method for robust knowledge transfer in deep multi-agent reinforcement learning. In 2024 24th International Conference on Control, Automation and Systems (ICCAS), pages 325–330.
Kono, H., Kamimura, A., Tomita, K., Murata, Y., and Suzuki, T. (2014). Transfer learning method using ontology for heterogeneous multi-agent reinforcement learning. International Journal of Advanced Computer Science and Applications, 5(10).
Liu, Y., Wu, X., Bo, Y., Wang, J., and Ma, L. (2024). A transfer learning framework for deep multi-agent reinforcement learning. IEEE/CAA Journal of Automatica Sinica, 11(11):2346–2348.
Long, J., Yu, D., Wen, G., Li, L., Wang, Z., and Chen, C. L. P. (2024). Game-based backstepping design for strict-feedback nonlinear multi-agent systems based on reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 35(1):817–830.
Lu, Y. and Yan, K. (2020). Algorithms in multi-agent systems: A holistic perspective from reinforcement learning and game theory. ArXiv, abs/2001.06487.
Mehboob, F., Fattouh, A., and Sahoo, S. (2024). Synergizing transfer learning and multi-agent systems for thermal parametrization in induction traction motors. Applied Sciences, 14(11).
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and PRISMA Group*, t. (2009). Preferred reporting items for systematic reviews and meta-analyses: the prisma statement. Annals of internal medicine, 151(4):264–269.
Namiki, M., Okawa, Y., Morita, N., Kakuta, J., and Ogawa, M. (2024). Transfer learning with less negative transfer for multi-agent reinforcement learning: Application and evaluation in base station control. In 2024 IEEE 35th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pages 1–7.
Panait, L. and Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Autonomous agents and multi-agent systems, 11(3):387–434.
Schillo, M., Bürckert, H.-J., Fischer, K., and Klusch, M. (2001). Towards a definition of robustness for market-style open multi-agent systems. In Proceedings of the Fifth International Conference on Autonomous Agents, AGENTS ’01, page 75–76, New York, NY, USA. Association for Computing Machinery.
Siddiqua, A., Liu, S., Siddika Nipu, A., Harris, A., and Liu, Y. (2024). Co-evolving multi-agent transfer reinforcement learning via scenario independent representation. IEEE Access, 12:99439–99451.
Silva, F. and Costa, A. (2019). A survey on transfer learning for multiagent reinforcement learning systems. Journal of Artificial Intelligence Research, 64.
Stone, P. and Veloso, M. (2000). Multiagent systems: A survey from a machine learning perspective. Autonomous Robots, 8(3):345–383.
Taylor, M. E. and Stone, P. (2009). Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10:1633–1685.
Vinyals, O. e. a. (2019). Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575:350–354.
Wang, C. and Zhu, X. (2024). Ata-maopt: Multi-agent online policy transfer using attention mechanism with time abstraction. IEEE Access, 12:158282–158291.
Wang, Y., Ma, Y., Wang, J., Yuan, H., Wang, M., and Ma, H. (EasyChair, 2024). Multi-agent air combat decision-making based on battlefield attention information. Easy-Chair Preprint 15393.
Wooldridge, M. (2009). An Introduction to MultiAgent Systems. Wiley Publishing, 2nd edition.
Zambonelli, F., Jennings, N. R., and Wooldridge, M. (2001). Organisational rules as an abstraction for the analysis and design of multi-agent systems. International Journal of Software Engineering and Knowledge Engineering, 11(03):303–328.
Zhang, C., Zeng, R., Lin, B., Zhang, Y., Xie, W., and Zhang, W. (2025). Multi-usv cooperative target encirclement through learning-based distributed transferable policy and experimental validation. Ocean Engineering, 318:120124.
Zhang, K., Yang, Z., and Başar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control.
Zhu, H. e. a. (2023). A survey of knowledge transfer for deep reinforcement learning in multi-agent systems. Neurocomputing, 522:1–21.
Zhu, T. and Yu, P. S. (2019). Applying differential privacy mechanism in artificial intelligence. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pages 1601–1609.
Busoniu, L., Babuska, R., and De Schutter, B. (2008). A comprehensive survey of multi-agent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, 38(2):156–172.
Chen, D. and Zhang, Q. (2024). e(3)-equivariant actor-critic methods for cooperative multi-agent reinforcement learning.
Du, Y., Qi, N., Li, X., Xiao, M., Boulogeorgos, A.-A. A., Tsiftsis, T. A., and Wu, Q. (2024). Distributed multi-uav trajectory planning for downlink transmission: A gnn-enhanced drl approach. IEEE Wireless Communications Letters, 13(12):3578–3582.
Farias, G., Rodrigues, P., Adamatti, D., and Gonçalves, E. (2024). Model for knowledge transfer in agent organizations: a case study on moise+. The International FLAIRS Conference Proceedings, 37(1).
Gao, Z., Xu, K., Ding, B., and Wang, H. (2021). Knowru: Knowledge reuse via knowledge distillation in multi-agent reinforcement learning. Entropy, 23(8).
Gupta, J. K. e. a. (2017). Cooperative multi-agent control using deep reinforcement learning. International Conference on Autonomous Agents and MultiAgent Systems (AAMAS).
Kim, D., Jung, H., and Bae, J. (2024). Multi-agent network randomization method for robust knowledge transfer in deep multi-agent reinforcement learning. In 2024 24th International Conference on Control, Automation and Systems (ICCAS), pages 325–330.
Kono, H., Kamimura, A., Tomita, K., Murata, Y., and Suzuki, T. (2014). Transfer learning method using ontology for heterogeneous multi-agent reinforcement learning. International Journal of Advanced Computer Science and Applications, 5(10).
Liu, Y., Wu, X., Bo, Y., Wang, J., and Ma, L. (2024). A transfer learning framework for deep multi-agent reinforcement learning. IEEE/CAA Journal of Automatica Sinica, 11(11):2346–2348.
Long, J., Yu, D., Wen, G., Li, L., Wang, Z., and Chen, C. L. P. (2024). Game-based backstepping design for strict-feedback nonlinear multi-agent systems based on reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 35(1):817–830.
Lu, Y. and Yan, K. (2020). Algorithms in multi-agent systems: A holistic perspective from reinforcement learning and game theory. ArXiv, abs/2001.06487.
Mehboob, F., Fattouh, A., and Sahoo, S. (2024). Synergizing transfer learning and multi-agent systems for thermal parametrization in induction traction motors. Applied Sciences, 14(11).
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and PRISMA Group*, t. (2009). Preferred reporting items for systematic reviews and meta-analyses: the prisma statement. Annals of internal medicine, 151(4):264–269.
Namiki, M., Okawa, Y., Morita, N., Kakuta, J., and Ogawa, M. (2024). Transfer learning with less negative transfer for multi-agent reinforcement learning: Application and evaluation in base station control. In 2024 IEEE 35th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pages 1–7.
Panait, L. and Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Autonomous agents and multi-agent systems, 11(3):387–434.
Schillo, M., Bürckert, H.-J., Fischer, K., and Klusch, M. (2001). Towards a definition of robustness for market-style open multi-agent systems. In Proceedings of the Fifth International Conference on Autonomous Agents, AGENTS ’01, page 75–76, New York, NY, USA. Association for Computing Machinery.
Siddiqua, A., Liu, S., Siddika Nipu, A., Harris, A., and Liu, Y. (2024). Co-evolving multi-agent transfer reinforcement learning via scenario independent representation. IEEE Access, 12:99439–99451.
Silva, F. and Costa, A. (2019). A survey on transfer learning for multiagent reinforcement learning systems. Journal of Artificial Intelligence Research, 64.
Stone, P. and Veloso, M. (2000). Multiagent systems: A survey from a machine learning perspective. Autonomous Robots, 8(3):345–383.
Taylor, M. E. and Stone, P. (2009). Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10:1633–1685.
Vinyals, O. e. a. (2019). Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575:350–354.
Wang, C. and Zhu, X. (2024). Ata-maopt: Multi-agent online policy transfer using attention mechanism with time abstraction. IEEE Access, 12:158282–158291.
Wang, Y., Ma, Y., Wang, J., Yuan, H., Wang, M., and Ma, H. (EasyChair, 2024). Multi-agent air combat decision-making based on battlefield attention information. Easy-Chair Preprint 15393.
Wooldridge, M. (2009). An Introduction to MultiAgent Systems. Wiley Publishing, 2nd edition.
Zambonelli, F., Jennings, N. R., and Wooldridge, M. (2001). Organisational rules as an abstraction for the analysis and design of multi-agent systems. International Journal of Software Engineering and Knowledge Engineering, 11(03):303–328.
Zhang, C., Zeng, R., Lin, B., Zhang, Y., Xie, W., and Zhang, W. (2025). Multi-usv cooperative target encirclement through learning-based distributed transferable policy and experimental validation. Ocean Engineering, 318:120124.
Zhang, K., Yang, Z., and Başar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control.
Zhu, H. e. a. (2023). A survey of knowledge transfer for deep reinforcement learning in multi-agent systems. Neurocomputing, 522:1–21.
Zhu, T. and Yu, P. S. (2019). Applying differential privacy mechanism in artificial intelligence. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pages 1601–1609.
Publicado
29/09/2025
Como Citar
MARQUES, Marcone de Freitas; FARIAS, Giovani Parente; GONÇALVES, Eder Mateus Nunes; ADAMATTI, Diana Francisca.
A Systematic Review on Knowledge Transfer in Multi-Agent Systems using Reinforcement Learning. In: WORKSHOP-ESCOLA DE SISTEMAS DE AGENTES, SEUS AMBIENTES E APLICAÇÕES (WESAAC), 19. , 2025, Fortaleza/CE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 29-38.
ISSN 2326-5434.
DOI: https://doi.org/10.5753/wesaac.2025.37526.
