What Is Not Seen, Is Not Remembered: Efficient Federated Learning with Homomorphic Encryption
Abstract
Cross-silo federated learning (FL) allows multiple institutions to collaborate in training global models without directly sharing sensitive data. However, even without explicit data sharing, there are significant security risks, such as the possibility of gradient inversion attacks, which allow the reconstruction of private data from local client updates. To mitigate these risks, homomorphic encryption (HE) techniques have been widely explored, allowing operations on encrypted data without the need for decryption. However, the use of HE is accompanied by high computational and communication overhead, making its practical application difficult. In this context, this work proposes NVNL-FL, an efficient technique that combines packet packing and packet sparsification methods using a sliding window-based selection method, substantially reducing overhead, ensuring privacy, and maintaining model accuracy even in scenarios with heterogeneous (non-IID) data. Experimental results demonstrate that NVNL-FL overcomes limitations of existing methods, balancing efficiency, security, and performance. The full code for NVNL-FL is available on GitHub.
References
Cheon, J. H., Kim, A., Kim, M., and Song, Y. (2017). Homomorphic encryption for arithmetic of approximate numbers. In T. Takagi & T. Peyrin (Eds.), Advances in Cryptology – ASIACRYPT 2017 (pp. 409–437). Cham: Springer International Publishing.
Choi, S., Patel, D., Zad Tootaghaj, D., Cao, L., Ahmed, F., and Sharma, P. (2024). Fednic: Enhancing privacy-preserving federated learning via homomorphic encryption offload on SmartNIC. Frontiers in Computer Science, 6.
de Souza, A. M., Maciel, F., da Costa, J. B., Bittencourt, L. F., Cerqueira, E., Loureiro, A. A., and Villas, L. A. (2024). Adaptive client selection with personalization for communication efficient federated learning. Ad Hoc Networks, 157, 103462.
Fan, J. and Vercauteren, F. (2012). Somewhat practical fully homomorphic encryption. Cryptology ePrint Archive, Paper 2012/144.
Geiping, J., Bauermeister, H., Dröge, H., and Moeller, M. (2020). Inverting gradients – how easy is it to break privacy in federated learning?
Hsu, T.-M. H., Qi, H., and Brown, M. (2019). Measuring the effects of non-identical data distribution for federated visual classification.
Jiang, Z., Wang, W., and Liu, Y. (2021). Flashe: Additively symmetric homomorphic encryption for cross-silo federated learning.
Li, A., Peng, H., Zhang, L., Huang, J., Guo, Q., Yu, H., and Liu, Y. (2023). Fedsdg-fs: Efficient and secure feature selection for vertical federated learning.
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. (2023). Communication-efficient learning of deep networks from decentralized data.
Paillier, P. (1999). Public-key cryptosystems based on composite degree residuosity classes. In J. Stern (Ed.), Advances in Cryptology — EUROCRYPT ’99 (pp. 223–238). Berlin, Heidelberg: Springer Berlin Heidelberg.
Phong, L. T., Aono, Y., Hayashi, T., Wang, L., and Moriai, S. (2018). Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security, 13(5), 1333–1345.
Souza, A., Bittencourt, L., Cerqueira, E., Loureiro, A., and Villas, L. (2023). Dispositivos, eu escolho vocês: Seleção de clientes adaptativa para comunicação eficiente em aprendizado federado. In Anais do XLI Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos (pp. 1–14). Porto Alegre, RS, Brasil: SBC.
Tenison, I., Sreeramadas, S. A., Mugunthan, V., Oyallon, E., Rish, I., and Belilovsky, E. (2023). Gradient masked averaging for federated learning.
Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms.
Xu, W., Fan, H., Li, K., and Yang, K. (2021). Efficient batch homomorphic encryption for vertically federated XGBoost.
Yan, N., Li, Y., Chen, J., Wang, X., Hong, J., He, K., and Wang, W. (2024). Efficient and straggler-resistant homomorphic encryption for heterogeneous federated learning. In IEEE INFOCOM 2024 - IEEE Conference on Computer Communications (pp. 791–800).
Yang, Q., Liu, Y., Chen, T., and Tong, Y. (2019). Federated machine learning: Concept and applications.
Zhang, C., Li, S., Xia, J., Wang, W., Yan, F., and Liu, Y. (2020). BatchCrypt: Efficient homomorphic encryption for cross-silo federated learning. In 2020 USENIX Annual Technical Conference (USENIX ATC 20) (pp. 493–506). USENIX Association.
Zhang, S., Li, Z., Chen, Q., Zheng, W., Leng, J., and Guo, M. (2021). Dubhe: Towards data unbiasedness with homomorphic encryption in federated learning client selection.
