3FL: Faster Client Selection for Federated Learning Cross-Device Performance Enhancement

• Kaylani Bochie UFRJ
• Matteo Sammarco Independent Researcher
• Miguel Elias M. Campista UFRJ

DOI: https://doi.org/10.5753/sbrc.2024.1546

Abstract

This paper proposes a new technique to increase federated learning performance and decrease its overall training latency. The proposal, called Fastest-First Federated Learning (3FL), is centered on selecting faster participants at the outset of training to minimize training latency and mitigate the impact of straggler devices. The proposal’s assessment involves simulations using realistic data distributions and client configurations for horizontal cross-device federated learning scenarios. The obtained results demonstrate the potential to achieve reductions in training latency of up to 35% compared with traditional federated learning. Moreover, the experiments confirm that the model accuracy reaches similar or even superior results, achieving values of up to 97% for image classification problems.

References

Asad, M., Otoum, S. e Shaukat, S. (2022). Resource and heterogeneity-aware clients eligibility protocol in federated learning. Em GLOBECOM 2022 - 2022 IEEE Global Communications Conference, p. 1140–1145.

Beutel, D. J., Topal, T., Mathur, A., Qiu, X., Fernandez-Marques, J., Gao, Y., Sani, L., Kwing, H. L., Parcollet, T., Gusmão, P. P. d. e Lane, N. D. (2020). Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390.

Bochie, K., Gilbert, M. S., Gantert, L., Barbosa, M. S., Medeiros, D. S. e Campista, M. E. M. (2021a). A survey on deep learning for challenged networks: Applications and trends. Journal of Network and Computer Applications, 194:103213.

Bochie, K., Sammarco, M., Detyniecki, M. e Campista, M. (2021b). Análise do aprendizado federado em redes móveis. Em Anais do XXXIX Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, p. 71–84, Porto Alegre, RS, Brasil. SBC.

Cao, Y., Maghsudi, S. e Ohtsuki, T. (2021). Mobility-aware routing and caching: A federated learning assisted approach. Em ICC 2021 - IEEE International Conference on Communications, p. 1–6.

Chen, Y., Ning, Y., Slawski, M. e Rangwala, H. (2020). Asynchronous online federated learning for edge devices with non-iid data. Em 2020 IEEE International Conference on Big Data (Big Data), p. 15–24.

Hsieh, K., Phanishayee, A., Mutlu, O. e Gibbons, P. (2020). The non-IID data quagmire of decentralized machine learning. Em III, H. D. e Singh, A., editores, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, p. 4387–4398. PMLR.

Hsu, H., Qi, H. e Brown, M. (2019). Measuring the effects of non-identical data distribution for federated visual classification. Em Neurips Workshop on Federated Learning.

Kairouz et al. (2021). Advances and open problems in federated learning.

Krizhevsky, A. (2012). Learning multiple layers of features from tiny images. University of Toronto.

Kumar, S., Schlegel, R., Rosnes, E. e Amat, A. G. i. (2022). Coding for straggler mitigation in federated learning. Em ICC 2022 - IEEE International Conference on Communications, p. 4962–4967.

Lecun, Y., Bottou, L., Bengio, Y. e Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324.

McMahan, B., Moore, E., Ramage, D., Hampson, S. e y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. Em Singh, A. e Zhu, J., editores, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, p. 1273–1282, Fort Lauderdale, FL, USA. PMLR.

Naeem, A., Anees, T., Naqvi, R. A. e Loh, W.-K. (2022a). A comprehensive analysis of recent deep and federated-learning-based methodologies for brain tumor diagnosis. Journal of Personalized Medicine, 12(2).

Naeem, M., Jamal, T., Diaz-Martinez, J., Butt, S. A., Montesano, N., Tariq, M. I., De-la Hoz-Franco, E. e De-La-Hoz-Valdiris, E. (2022b). Trends and future perspective challenges in big data. Em Pan, J.-S., Balas, V. E. e Chen, C.-M., editores, Advances in Intelligent Data Analysis and Applications, p. 309–325, Singapore. Springer Singapore.

Neto, H. C., Mattos, D. e Fernandes, N. (2021). Fedsa: Arrefecimento simulado federado para a aceleração da detecção de intrusão em ambientes colaborativos. Em Anais do XXXIX Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, p. 280–293, Porto Alegre, RS, Brasil. SBC.

Qu, Y., Uddin, M. P., Gan, C., Xiang, Y., Gao, L. e Yearwood, J. (2022). Blockchain-enabled federated learning: A survey. ACM Comput. Surv., 55(4).

Reisizadeh, A., Tziotis, I., Hassani, H., Mokhtari, A. e Pedarsani, R. (2022). Straggler-resilient federated learning: Leveraging the interplay between statistical accuracy and system heterogeneity. IEEE Journal on Selected Areas in Information Theory, 3(2):197–205.

Rimol, M. (2022). Gartner identifies top five trends in privacy through 2024. Gartner. Sivek, G. e Riley, M. (2022). Spatial model personalization in gboard. Proc. ACM Hum.Comput. Interact., 6(MHCI).

Souza, L., Camilo, G., Sammarco, M., Campista, M. e Costa, L. (2022). Aprendizado federado com agrupamento hierárquico de clientes para aumento da acurácia. Em Anais do XL Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, p. 545–558, Porto Alegre, RS, Brasil. SBC.

Xiang, W. T., Shao, M., Fu, Y., Jia, R., Lin, F. e Zheng, Z. (2022). FEDERATED LEARNING FRAMEWORK BASED ON TRIMMED MEAN AGGREGATION RULES.

Zhang, Q., Palacharla, P., Sekiya, M., Suga, J. e Katagiri, T. (2020). Demo: A blockchain based protocol for federated learning. Em 2020 IEEE 28th International Conference on Network Protocols (ICNP), p. 1–2.