Dispositivos, Eu Escolho Vocês: Seleção de Clientes Adaptativa para Comunicação Eficiente em Aprendizado Federado
Abstract
Federated Learning (FL) is a distributed approach to collaboratively training machine learning models. FL requires a high level of communication between the devices and a central server, thus creating several challenges, including communication bottlenecks and network scalability. This work introduces DEEV, a solution to lower overall communication and computation costs for training a model in the FL environment. DEEV employs a client selection strategy that dynamically adapts the number of devices training the model and the number of rounds required to achieve convergence. A use case in the human activity recognition dataset is performed to evaluate DEEV and compare it to other state-of-the-art approaches. Experimental evaluations show that DEEV efficiently reduces the overall communication and computation overhead to train a model and promote its convergence. In particular, DEEV reduces up to 60% communication and 90% computational overhead compared to literature approaches while providing good convergence even in scenarios where data is distributed differently, non-independent and identical way between client devices.
References
Abdulrahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., and Guizani, M. (2021). A survey on federated learning: The journey from centralized to distributed on-site learning and beyond. IEEE Internet of Things Journal, 8(7):5476-5497.
Amiri, M. M., Gündüz, D., Kulkarni, S. R., and Poor, H. V. (2020). Federated learning with quantized global model updates. ArXiv, abs/2006.10672.
Bernstein, J., Wang, Y.-X., Azizzadenesheli, K., and Anandkumar, A. (2018). Signsgd: compressed optimisation for non-convex problems.
Beutel, D. J., Topal, T., Mathur, A., Qiu, X., Parcollet, T., and Lane, N. D. (2020). Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390.
Cho, Y. J., Wang, J., , and Joshi, G. (2020). Client selection in federated learning: Convergence analysis and power-of-choice selection strategies.
Haykin, S. (1998). Neural Networks: A Comprehensive Foundation. Prentice Hall PTR, USA, 2nd edition.
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., and Miao, C. (2020). Federated learning in mobile edge networks: A comprehensive survey. IEEE Communications Surveys Tutorials, 22(3):2031-2063.
Lin, Y., Han, S., Mao, H., Wang, Y., and Dally, B. (2018). Deep gradient compression: Reducing the communication bandwidth for distributed training. In International Conference on Learning Representations.
Malekzadeh, M., Clegg, R. G., Cavallaro, A., and Haddadi, H. (2019). Mobile sensor data anonymization. In Proceedings of the International Conference on Internet of Things Design and Implementation, IoTDI '19, pages 49-58, New York, NY, USA. ACM.
McMahan, B., Moore, E., Ramage, D., Hampson, S., , and y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data.
Porambage, P., Okwuibe, J., Liyanage, M., Ylianttila, M., and Taleb, T. (2018). Survey on multi-access edge computing for internet of things realization. IEEE Communications Surveys & Tutorials, 20(4):2961-2991.
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., and Pedarsani, R. (2020). Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization.
Sattler, F., Wiedemann, S., Müller, K.-R., and Samek, W. (2020). Robust and communication-efficient federated learning from non-i.i.d. data. IEEE Transactions on Neural Networks and Learning Systems, 31(9):3400-3413.
Shah, S. M. and Lau, V. K. N. (2021). Model compression for communication efficient federated learning. IEEE Transactions on Neural Networks and Learning Systems, pages 1-15.
Shahid, O., Pouriyeh, S., Parizi, R. M., Sheng, Q. Z., Srivastava, G., and Zhao, L. (2021). Communication efficiency in federated learning: Achievements and challenges.
Ström, N. (2015). Scalable distributed dnn training using commodity gpu cloud computing. In Interspeech 2015.
Vaizman, Y., Ellis, K., and Lanckriet, G. (2017). Recognizing detailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Computing, 16(4):62-74.
Wahab, O. A., Mourad, A., Otrok, H., and Taleb, T. (2021). Federated machine learning: Survey, multi-level classification, desirable criteria and future directions in communication and networking systems. IEEE Communications Surveys Tutorials, 23(2):1342-1397.
Zhang, X., Zhu, X., Wang, J., Yan, H., Chen, H., and Bao, W. (2020). Federated learning with adaptive communication compression under dynamic bandwidth and unreliable networks. Information Sciences, 540:242-262.
