Multi-Agent Systems Testing: Approaches, Tools, and Challenges

• Matusalen Costa Alves IFPI
• Iallen Gábio de Sousa Santos IFPI
• Mayllon Veras da Silva IFPI
• Ricardo Moura Sekeff Budaruiche IFPI
• Wanderson de Vasconcelos Rodrigues da Silva IFPI

DOI: https://doi.org/10.5753/codec.2025.39149

Abstract

This paper presents a systematic mapping of the literature on testing in Multi-Agent Systems, focusing on two main dimensions: (i) software engineering and test automation approaches, and (ii) model-based testing strategies and frameworks. Thirteen selected papers were qualitatively analyzed, revealing recent advances such as the use of formal methods (e.g., Colored Petri Nets), platforms like JADE, and AI-driven testing solutions with Large Language Models. Despite the progress, gaps remain regarding non-functional testing, methodological standardization, and empirical validation. The study provides an overview of current trends and highlights directions for future research.

References

Elkholy, W., El-Menshawy, M., Bentahar, J., Elqortobi, M., Laarej, A., and Dssouli, R. (2020). Model checking intelligent avionics systems for test cases generation using multi-agent systems. Expert Systems with Applications, 156:113458.

Gonçalves, E. M. N., Machado, R. A., Rodrigues, B. C., and Adamatti, D. (2022). Cpn4m: Testing multi-agent systems under organizational model moise+ using colored petri nets. Applied Sciences, 12(12):5857.

Kalache, A., Badri, M., Mokhati, F., and Babahenini, M. C. (2023). A testing framework for jade agent-based software. Multiagent and Grid Systems, 19(1):61–98.

Khoee, A. G., Yu, Y., Feldt, R., Freimanis, A., Rhodin, P. A., and Parthasarathy, D. (2025). Gonogo: An efficient llm-based multi-agent system for streamlining automotive software release decision-making. In Testing Software and Systems, pages 30–45. Springer Nature Switzerland.

Kissoum, Y. and Redjimi, M. (2022). Multi-level testing approach for multi-agent systems. International Journal of Organizational and Collective Intelligence, 12(1):1–23.

Kumazawa, T., Takimoto, M., and Kambayashi, Y. (2021). Exploration strategies for model checking with ant colony optimization. In Nguyen, N. T., Iliadis, L., Maglogiannis, I., and Trawiński, B., editors, Computational Collective Intelligence, pages 264–276. Springer International Publishing.

Machado, R. A., da Silva Zelindro Cardoso, A., Farias, G. P., Gonçalves, E. M. N., and Adamatti, D. F. (2025). A formal testing method for multi-agent systems using colored petri nets. Autonomous Agents and Multi-Agent Systems, 39(1):10.

Medvedev, D. and Aksyonov, K. (2021). The development of a simulation model for assessing the ci/cd pipeline quality in the development of information systems based on a multi-agent approach. MATEC Web of Conferences, 346:03095.

O’Neill, V. and Soh, B. (2022). Improving fault tolerance and reliability of heterogeneous multi-agent iot systems using intelligence transfer. Electronics, 11(17):2724.

Palmieri, M., Quadri, C., Fagiolini, A., and Bernardeschi, C. (2023). Co-simulated digital twin on the network edge: A vehicle platoon. Computer Communications, 212:35–47.

Schwabl, P., Haim, M., and Unkel, J. (2024). Aligning agent-based testing (abt) with the experimental research paradigm: A literature review and best practices. Journal of Computational Social Science, 7(2):1625–1644.

Yang, C., Deng, Y., Lu, R., Yao, J., Liu, J., Jabbarvand, R., and Zhang, L. (2024). White-fox: White-box compiler fuzzing empowered by large language models. Proceedings of the ACM on Programming Languages, 8(OOPSLA2):296.

Zhang, X.-Y., Liu, Y., Arcaini, P., Jiang, M., and Zheng, Z. (2024). Met-mapf: A metamorphic testing approach for multi-agent path finding algorithms. ACM Transactions on Software Engineering and Methodology, 33(8):198.