ALFadapt: Mitigating Catastrophic Forgetting in Federated Learning

• John Lucas R. P. de Sousa UFPA
• Eduardo Ribeiro UFPA
• Lucas Bastos UFPA
• Denis Rosário UFPA
• Eduardo Cerqueira UFPA

DOI: https://doi.org/10.5753/sbrc.2025.6339

Resumo

Catastrophic forgetting, driven by concept drift, challenges federated learning (FL), especially in dynamic environments. Existing layer-freezing methods focus on communication efficiency but often overlook concept drift’s impact on individual neural network layers. We propose ALFadapt (Automatic and Adaptive Layer Freezing) to mitigate catastrophic forgetting by dynamically freezing stable layers and allowing trainable layers to adapt to evolving data. Experimental results show that ALFadapt significantly reduces accuracy loss during scenario transitions and improves model resilience when revisiting previous scenarios. This method offers a robust solution for environments characterized by concept drift and non-IID data distributions.

Palavras-chave: Federated Learning, Catastrophic Forgetting, Machine Learning

Referências

Babakniya, S., Fabian, Z., He, C., Soltanolkotabi, M., and Avestimehr, S. (2023). A data-free approach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks. Advances in Neural Information Processing Systems, 36(NeurIPS 2023).

Babendererde, N., Fuchs, M., Gonzalez, C., Tolkach, Y., and Mukhopadhyay, A. (2023). Jointly exploring client drift and catastrophic forgetting in dynamic learning.

Bai, J., Sajjanhar, A., Xiang, Y., Tong, X., and Zeng, S. (2022). FedEWA: Federated learning with elastic weighted averaging. Proceedings of the International Joint Conference on Neural Networks, 2022-July:1–8.

Chellapandi, V. P., Yuan, L., Brinton, C. G., Zak, S. H., and Wang, Z. (2024). Federated learning for connected and automated vehicles: A survey of existing approaches and challenges. IEEE Transactions on Intelligent Vehicles, 9(1):119–137.

Chow, T., Raza, U., Mavromatis, I., and Khan, A. (2023). FLARE: Detection and mitigation of concept drift for federated learning based IoT deployments. 2023 International Wireless Communications and Mobile Computing, IWCMC 2023, pages 989–995.

Ding, J., Tramel, E., Sahu, A. K., Wu, S., Avestimehr, S., and Zhang, T. (2022). Federated learning challenges and opportunities: An outlook. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8752–8756.

Jothimurugesan, E., Hsieh, K., Wang, J., Joshi, G., and Gibbons, P. B. (2023). Federated learning under distributed concept drift. Proceedings of Machine Learning Research, 206:5834–5853.

Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., D’Oliveira, R. G. L., Eichner, H., Rouayheb, S. E., Evans, D., Gardner, J., Garrett, Z., Gascón, A., Ghazi, B., Gibbons, P. B., Gruteser, M., Harchaoui, Z., He, C., Hea, L., Huo, Z., Hutchinson, B., Hsu, J., Jaggi, M., Javidi, T., Joshi, G., Khodak, M., Konečný, J., Korolova, A., Koushanfar, F., Koyejo, S., Lepoint, T., Liu, Y., Mittal, P., Mohri, M., Nock, R., Özgür, A., Pagh, R., Raykova, M., Qi, H., Ramage, D., Raskar, R., Song, D., Song, W., Stich, S. U., Sun, Z., Suresh, A. T., Tramèr, F., Vepakomma, P., Wang, J., Xiong, L., Xu, Z., Yang, Q., Yu, F. X., Yu, H., and Zhao, S. (2021). Advances and open problems in federated learning.

Li, M., Avdiukhin, D., Shahout, R., Ivkin, N., Braverman, V., and Yu, M. (2024). Federated learning clients clustering with adaptation to data drifts.

Malan, E., Peluso, V., Calimera, A., and MacIi, E. (2023). Communication-efficient federated learning with gradual layer freezing. IEEE Embedded Systems Letters, 15(1):25–28.

Malan, E., Peluso, V., Calimera, A., Macii, E., and Montuschi, P. (2024). Automatic layer freezing for communication efficiency in cross-device federated learning. IEEE Internet of Things Journal, 11(4):6072–6083.

Sorrenti, A., Bellitto, G., Salanitri, F. P., Pennisi, M., Spampinato, C., and Palazzo, S. (2023). Selective freezing for efficient continual learning. Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023, pages 3542–3551.

Yang, G., Chen, X., Zhang, T., Wang, S., and Yang, Y. (2023). An impact study of concept drift in federated learning. Proceedings - IEEE International Conference on Data Mining, ICDM, (Icdm):1457–1462.

Yang, R., Yang, T., Yan, Z., Krajnik, T., and Ruichek, Y. (2024). Preventing catastrophic forgetting in continuous online learning for autonomous driving. 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5505–5512.