An Analytical Perspective for Performance Assessment in Federated Learning

• Francisco Airton Silva UFPI
• Allan M. de Souza UNICAMP
• Iago Almeida UFPI
• Eduardo Cerqueira UFPA
• Luiz Fernando Bittencourt UNICAMP
• Denis Rosário UFPA

DOI: https://doi.org/10.5753/sbrc.2025.5813

Abstract

Conducting Federated Learning (FL) in real-world environments poses significant challenges due to client heterogeneity, network constraints, and scalability issues, which are difficult to reproduce in experimental settings. In this context, analytical models offer a more comprehensive and efficient view of the behavior of large-scale FL systems, and thus it is important to model dense scenarios and intermittent connectivity efficiently, integrating stochastic variability in transitions and states. In this work, we introduce a Stochastic Petri Net (SPN) modeling as a complementary analytical approach to traditional FL simulators. The proposed SPN model enables detailed simulation of key FL dynamics, including asynchronous participation, device dropouts, and resource contention. The application of the SPN model is showcased using experiments from a testbed. The results provide important insights for the optimization of distributed systems in different domains.

Keywords: Federated Learning, Stochastic Petri Net, Analytical Modeling, Client Heterogeneity, Distributed Systems

References

Albelaihi, R., Sun, X., Craft, W. D., Yu, L., and Wang, C. (2021). Adaptive participant selection in heterogeneous federated learning. In IEEE Global Communications Conference (GLOBECOM), pages 1–6. IEEE.

Banabilah, S., Aloqaily, M., Alsayed, E., Malik, N., and Jararweh, Y. (2022). Federated learning review: Fundamentals, enabling technologies, and future applications. Information Processing & Management, 59(6):103061.

Beutel, A., Topcuoglu, E., Ozisik, C., and Steiger, E. (2020). Flower: A friendly federated learning research framework. In Systems and Machine Learning Conference.

Bonawitz, K. (2019). Towards federated learning at scale: System design. arXiv preprint arXiv:1902.01046.

DeGlopper, D. R. (1992). The art of computer systems performance analysis: Techniques for experimental design, measurement, simulation and modeling. By Raj Jain. New York: John Wiley and Sons, 1991. pp. 720. (Hardcover). International Journal of Legal Information, 20(1):63–64.

Du, M., Zheng, H., Feng, X., Chen, Y., and Zhao, T. (2022). Decentralized federated learning with Markov chain based consensus for industrial IoT networks. IEEE Transactions on Industrial Informatics, 19(4):6006–6015.

Gilmer, M., Sohl-Dickstein, J., Schoenholz, S. S., Bauer, M. S., and Gilmer, J. (2021). Fedjax: A scalable JAX framework for federated learning. arXiv preprint arXiv:2108.02117.

Joshi, A., Agarwal, A., Agarwal, D., Murthy, B., Chaudhary, M., Agarwal, S., Sekar, A., Choudhary, D., and Girdhar, R. (2021). Flute: A scalable federated learning simulator. arXiv preprint arXiv:2110.06203.

Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. (2021). Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210.

Kholod, I., Yanaki, E., Fomichev, D., Shalugin, E., Novikova, E., Filippov, E., and Nordlund, M. (2020). Open-source federated learning frameworks for IoT: A comparative review and analysis. Sensors, 21(1):167.

Kim, M. G., De M Bastos, C. P. M. J., Park, J., Hajiesmaili, M., Oh, S., Ma, M., and Kim, M. (2021). FLSim: Scalable and extensible federated learning simulator. arXiv preprint arXiv:2107.03309.

Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., and He, B. (2021). A survey on federated learning systems: Vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering, 35(4):3347–3366.

Li, T., Sahu, A. K., Talwalkar, A., and Smith, V. (2020). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50–60.

Liu, T., Wang, H., and Ma, M. (2024). Federated learning with efficient aggregation via Markov decision process in edge networks. Mathematics, 12(6):920.

Maciel, P. R. M. (2023). Performance, reliability, and availability evaluation of computational systems, Volume 2: Reliability, availability modeling, measuring, and data analysis. Chapman and Hall/CRC.

Pu, J., Fu, X., Dong, H., Zhang, P., and Liu, L. (2024). Dynamic adaptive federated learning on local long-tailed data. IEEE Transactions on Services Computing.

Savazzi, S., Nicoli, M., Bennis, M., Kianoush, S., and Barbieri, L. (2021). Opportunities of federated learning in connected, cooperative, and automated industrial systems. IEEE Communications Magazine, 59(2):16–21.

Silva, B., Matos, R., Callou, G., Figueiredo, J., Oliveira, D., Ferreira, J., Dantas, J., Lobo, A., Alves, V., and Maciel, P. (2015). Mercury: An integrated environment for performance and dependability evaluation of general systems. In 45th Dependable Systems and Networks Conference (DSN), pages 1–4.

TensorFlow Federated (2021). TensorFlow Federated: Machine Learning on Decentralized Data.

Turgay, S. (2022). Blockchain management and federated learning adaptation on healthcare management system. International Journal of Intelligent Systems and Applications, 14(5):1.

Wen, J., Zhang, Z., Lan, Y., Cui, Z., Cai, J., and Zhang, W. (2023). A survey on federated learning: Challenges and applications. International Journal of Machine Learning and Cybernetics, 14(2):513–535.

Wilhelmi, F., Giupponi, L., and Dini, P. (2021). Blockchain-enabled server-less federated learning. arXiv preprint arXiv:2112.07938, pages 1–14.