Privacy and Efficient Communication in Federated Learning: An Approach Using Probabilistic Data Structures and Client Selection

• Eduardo M. M. Sarmento UFES
• Vinícius F. S. Mota UFES
• Rodolfo S. Villaça UFES

DOI: https://doi.org/10.5753/sbrc.2024.1264

Abstract

To mitigate inference attacks and improve communication efficiency in federated learning, this article proposes a dual approach: i) FedSketch, utilizing sketches to enhance privacy and communication efficiency, applying differential privacy and compression to trained models, and ii) the MetricBasedSelection algorithm, a client selection algorithm based on customized metrics. The proposed solution reduced communication costs, up to 73 times, while maintaining similar accuracy to conventional federated learning with a high level of differential privacy (ϵ ≈ 10⁻⁶). This represents an effective approach to addressing privacy and cost challenges associated with federated learning.

References

Abdel-Basset, M., Hawash, H., and Moustafa, N. (2022). Toward privacy preserving federated learning in internet of vehicular things: Challenges and future directions. IEEE Consumer Electronics Magazine, 11(6):56–66.

Dwork, C. (2006). Differential privacy. In Bugliesi, M., Preneel, B., Sassone, V., and Wegener, I., editors, Automata, Languages and Programming, pages 1–12, Berlin, Heidelberg. Springer Berlin Heidelberg.

Fu, L. et al. (2022). Client selection in federated learning: Principles, challenges, and opportunities. ArXiv, abs/2211.01549.

Haddadpour, F. et al. (2020). Fedsketch: Communication-efficient and private federated learning via sketching. arXiv, abs/2008.04975.

Husnoo, M. A. et al. (2022). FedREP: Towards horizontal federated load forecasting for retail energy providers. In Asia-Pacific Power and Energy Engineering Conference (APPEEC). IEEE.

Isik-Polat, E., Polat, G., and Kocyigit, A. (2023). Arfed: Attack-resistant federated averaging based on outlier elimination. Future Generation Computer Systems, 141:626–650.

Joseph Near, D. D. (2023). Guidelines for evaluating differential privacy guarantees. Technical Report NIST SP 800-226 ipd, National Institute of Standards and Technology, Gaithersburg, MD.

Konečnỳ, J. et al. (2016). Federated learning: Strategies for improving communication efficiency. arXiv:1610.05492.

Lecun, Y. et al. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324.

Li, T., Liu, Z., Sekar, V., and Smith, V. (2019). Privacy for free: Communication-efficient learning with differential privacy using sketches. ArXiv, abs/1911.00972.

Liu, P., Xu, X., and Wang, W. (2022). Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives. Cybersecurity, 5.

McMahan, H. B. et al. (2016). Communication-efficient learning of deep networks from decentralized data. In International Conference on Artificial Intelligence and Statistics.

Nishio, T. and Yonetani, R. (2019). Client selection for federated learning with heterogeneous resources in mobile edge. In International Conference on Communications (ICC), pages 1–7.

Smith, A., Song, S., and Guha Thakurta, A. (2020). The flajolet-martin sketch itself preserves differential privacy: Private counting with minimal space. Advances in Neural Information Processing Systems, 33:19561–19572.

Souza, A. et al. (2023). Dispositivos, eu escolho vocês: Seleção de clientes adaptativa para comunicação eficiente em aprendizado federado. In XLI Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, Porto Alegre, RS, Brasil. SBC.

Wang, H. et al. (2022). Federated spatio-temporal traffic flow prediction based on graph convolutional network. In 2022 14th International Conference on Wireless Communications and Signal Processing (WCSP), pages 221–225.

Wei, K. et al. (2020). Federated learning with differential privacy: Algorithms and performance analysis. IEEE Trans on Information Forensics and Security, 15:3454–3469.

Zhang, T. et al. (2022). Federated learning for the internet of things: Applications, challenges, and opportunities. IEEE Internet of Things Magazine, 5(1):24–29.