Adaptive Client Selection Based on Differential Privacy for Federated Learning

• Vinícius R. M. Alves UNICAMP
• Joahannes B. D. da Costa UNICAMP
• Luis F. G. Gonzalez UNICAMP
• Allan M. de Souza UNICAMP
• Leandro A. Villas UNICAMP

DOI: https://doi.org/10.5753/sbrc_estendido.2024.3334

Abstract

Federated Learning (FL) is a distributed technique to training machine learning models, where data is processed locally and only local parameters are shared with an aggregation server. Despite client’s data being kept locally, it’s still possible for an adversary to conduct a model reconstruction attacks, for example. Therefore, this work presents PEGASUS, which leverages the guarantees of Differential Privacy (DP) to mitigate adversarial attacks in the FL environment. Additionally, PEGASUS employs a client selection strategy that dynamically adapts the number of devices training the model with the aim of dealing with the increasing loss of privacy (ϵ parameter of DP) over the course of communication rounds. Experimental evaluations show that PEGASUS significantly reduces the privacy loss (58%) of participating clients in training and maintains high levels of accuracy (97%).

References

Beutel, D. J., Topal, T., Mathur, A., Qiu, X., Fernandez-Marques, J., Gao, Y., Sani, L., Li, K. H., Parcollet, T., de Gusmão, P. P. B., and Lane, N. D. (2022). Flower: A friendly federated learning research framework.

de Souza, A. M., Bittencourt, L. F., Cerqueira, E., Loureiro, A. A., and Villas, L. A. (2023). Dispositivos, eu escolho vocês: Seleção de clientes adaptativa para comunicação eficiente em aprendizado federado. In Anais do XLI Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, pages 1–14. SBC.

de Souza, A. M., Maciel, F., da Costa, J. B. D., Bittencourt, L. F., Cerqueira, E., Loureiro, A. A., and Villas, L. A. (2024). Adaptive client selection with personalization for communication efficient federated learning. Ad Hoc Networks, page 103462.

Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I., and Naor, M. (2006). Our data, ourselves: Privacy via distributed noise generation. Proc. 24th Annu. Int. Conf. The Theory Appl. Cryptography, pages 486–503.

Dwork, C. and Roth, A. (2014). The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9:221–407.

Ficco, M., Guerriero, A., Milite, E., Palmieri, F., Pietrantuono, R., and Russo, S. (2024). Federated learning for iot devices: Enhancing tinyml with on-board training. Information Fusion, 104:102189.

Fu, J., Chen, Z., and Han, X. (2022). Adap dp-fl: Differentially private federated learning with adaptive noise.

LeCun, Y., Cortes, C., and Burges, C. (2010). Mnist handwritten digit database. ATT Labs [Online]. Available: [link], 2.

Ling, X., Fu, J., Wang, K., Liu, H., and Chen, Z. (2024). Ali-dpfl: Differentially private federated learning with adaptive local iterations.

McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR.

McMahan, H. B., Andrew, G., Erlingsson, U., Chien, S., Mironov, I., Papernot, N., and Kairouz, P. (2019). A general approach to adding differential privacy to iterative training procedures.

Ouadrhiri, A. E. and Abdelhadi, A. (2022). Differential privacy for deep and federated learning: A survey. IEEE Access, 10:22359–22380.

Ren, H., Deng, J., and Xie, X. (2022). Grnn: Generative regression neural network - a data leakage attack for federated learning. ACM Transactions on Intelligent and Technology, 1.

Talaei, M. and Izadi, I. (2024). Adaptive differential privacy in federated learning: A priority-based approach.

Wei, K., Li, J., Ding, M., Ma, C., Yang, H. H., Farokhi, F., Jin, S., Quek, T. Q. S., and Vincent Poor, H. (2020). Federated learning with differential privacy: Algorithms and performance analysis. IEEE Transactions on Information Forensics and Security, 15:3454–3469.