Treine Menos, Preveja Mais: plugin de Aprendizado Federado habilita alta eficiência em dados heterogêneos
Resumo
O aprendizado federado (FL) surgiu como uma técnica onde diversos dispositivos (também chamados de clientes) podem aprender de forma colaborativa a partir da orquestração de um servidor central, proporcionando escalabilidade, privacidade e baixo custo de comunicação. A maioria das pesquisas sobre este tema apresenta propostas para a etapa do treinamento de modelos no aprendizado federado, para endereçar diversos problemas como a heterogeneidade estatística de dados, o que muitas vezes representa aumento de custos (e.g., computacional, armazenamento e comunicação). No entanto, recentemente foi proposta a solução FedPredict, um plugin que opera na etapa de predição do aprendizado federado, que quando adicionado pode melhorar significativamente o desempenho de diversas soluções tradicionais em cenários de heterogeneidade de dados, sem requerer qualquer modificação na sua estrutura original ou adição de treinamento. Nesta direção, este trabalho apresenta experimentos sobre uma nova descoberta: quanto mais heterogêneos são os dados, menos treinamento é necessário quando o FedPredict é adicionado, tornando o processo de aprendizado altamente eficiente.Referências
Arivazhagan, M. G., Aggarwal, V., Singh, A. K., and Choudhary, S. (2019). Federated learning with personalization layers. arXiv preprint arXiv:1912.00818.
Capanema, C. G., de Oliveira, G. S., Silva, F. A., Silva, T. R., and Loureiro, A. A. (2023a). Combining recurrent and graph neural networks to predict the next place’s category. Ad Hoc Networks, 138:103016.
Capanema, C. G., de Souza, A. M., Silva, F. A., Villas, L. A., and Loureiro, A. A. (2023b). Fedpredict: Combining global and local parameters in the prediction step of federated learning. In 2023 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT), pages 17–24. IEEE.
Capanema, C. G., de Souza, Joahannes B D da Costa, F. A., Villas, L. A., and Loureiro, A. A. L. (2024). A modular plugin for concept drift in federated learning. In 2024 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT). IEEE.
Capanema, C. G., Silva, F. A., Silva, T. R., and Loureiro, A. A. (2021). Poi-rgnn: Using recurrent and graph neural networks to predict the category of the next point of interest. In Proceedings of the 18th acm symposium on performance evaluation of wireless ad hoc, sensor, & ubiquitous networks, pages 49–56.
Capanema, C. G. S., Silva, F. A., and Silva, T. R. d. M. B. (2020). Mfa-rnn: Uma rede neural recorrente para predição de próximo local de visita com base em dados esparsos. In Anais do XXXVIII Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuidos, pages 127–140. SBC.
Cho, Y. J., Wang, J., and Joshi, G. (2022). Towards understanding biased client selection in federated learning. In International Conference on Artificial Intelligence and Statistics, pages 10351–10375. PMLR.
Fu, L., Zhang, H., Gao, G., Zhang, M., and Liu, X. (2023). Client selection in federated learning: Principles, challenges, and opportunities. IEEE Internet of Things Journal.
Hsu, T.-M. H., Qi, H., and Brown, M. (2019). Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335.
Jang, J., Ha, H., Jung, D., and Yoon, S. (2022). Fedclassavg: Local representation learning for personalized federated learning on heterogeneous neural networks. In Proceedings of the 51st International Conference on Parallel Processing, pages 1–10.
Krizhevsky, A., Hinton, G., et al. (2009). Learning multiple layers of features from tiny images.
Lin, T., Kong, L., Stich, S. U., and Jaggi, M. (2020). Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, 33:2351–2363.
Maciel, F., De Souza, A. M., Bittencourt, L. F., and Villas, L. A. (2023). Resource aware client selection for federated learning in iot scenarios. In 2023 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT), pages 1–8. IEEE.
McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR.
Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečnỳ, J., Kumar, S., and McMahan, H. B. (2020). Adaptive federated optimization. arXiv preprint arXiv:2003.00295.
Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (2011). The German Traffic Sign Recognition Benchmark: A multi-class classification competition. In IEEE International Joint Conference on Neural Networks, pages 1453–1460.
Tan, A. Z., Yu, H., Cui, L., and Yang, Q. (2022a). Towards personalized federated learning. IEEE Trans. on Neural Networks and Learning Systems.
Tan, Y., Long, G., Liu, L., Zhou, T., Lu, Q., Jiang, J., and Zhang, C. (2022b). Fedproto: Federated prototype learning across heterogeneous clients. In AAAI Conference on Artificial Intelligence, volume 1, page 3.
Wu, C., Wu, F., Lyu, L., Huang, Y., and Xie, X. (2022). Communication-efficient federated learning via knowledge distillation. Nature communications, 13(1):2032.
Capanema, C. G., de Oliveira, G. S., Silva, F. A., Silva, T. R., and Loureiro, A. A. (2023a). Combining recurrent and graph neural networks to predict the next place’s category. Ad Hoc Networks, 138:103016.
Capanema, C. G., de Souza, A. M., Silva, F. A., Villas, L. A., and Loureiro, A. A. (2023b). Fedpredict: Combining global and local parameters in the prediction step of federated learning. In 2023 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT), pages 17–24. IEEE.
Capanema, C. G., de Souza, Joahannes B D da Costa, F. A., Villas, L. A., and Loureiro, A. A. L. (2024). A modular plugin for concept drift in federated learning. In 2024 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT). IEEE.
Capanema, C. G., Silva, F. A., Silva, T. R., and Loureiro, A. A. (2021). Poi-rgnn: Using recurrent and graph neural networks to predict the category of the next point of interest. In Proceedings of the 18th acm symposium on performance evaluation of wireless ad hoc, sensor, & ubiquitous networks, pages 49–56.
Capanema, C. G. S., Silva, F. A., and Silva, T. R. d. M. B. (2020). Mfa-rnn: Uma rede neural recorrente para predição de próximo local de visita com base em dados esparsos. In Anais do XXXVIII Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuidos, pages 127–140. SBC.
Cho, Y. J., Wang, J., and Joshi, G. (2022). Towards understanding biased client selection in federated learning. In International Conference on Artificial Intelligence and Statistics, pages 10351–10375. PMLR.
Fu, L., Zhang, H., Gao, G., Zhang, M., and Liu, X. (2023). Client selection in federated learning: Principles, challenges, and opportunities. IEEE Internet of Things Journal.
Hsu, T.-M. H., Qi, H., and Brown, M. (2019). Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335.
Jang, J., Ha, H., Jung, D., and Yoon, S. (2022). Fedclassavg: Local representation learning for personalized federated learning on heterogeneous neural networks. In Proceedings of the 51st International Conference on Parallel Processing, pages 1–10.
Krizhevsky, A., Hinton, G., et al. (2009). Learning multiple layers of features from tiny images.
Lin, T., Kong, L., Stich, S. U., and Jaggi, M. (2020). Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, 33:2351–2363.
Maciel, F., De Souza, A. M., Bittencourt, L. F., and Villas, L. A. (2023). Resource aware client selection for federated learning in iot scenarios. In 2023 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT), pages 1–8. IEEE.
McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR.
Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečnỳ, J., Kumar, S., and McMahan, H. B. (2020). Adaptive federated optimization. arXiv preprint arXiv:2003.00295.
Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (2011). The German Traffic Sign Recognition Benchmark: A multi-class classification competition. In IEEE International Joint Conference on Neural Networks, pages 1453–1460.
Tan, A. Z., Yu, H., Cui, L., and Yang, Q. (2022a). Towards personalized federated learning. IEEE Trans. on Neural Networks and Learning Systems.
Tan, Y., Long, G., Liu, L., Zhou, T., Lu, Q., Jiang, J., and Zhang, C. (2022b). Fedproto: Federated prototype learning across heterogeneous clients. In AAAI Conference on Artificial Intelligence, volume 1, page 3.
Wu, C., Wu, F., Lyu, L., Huang, Y., and Xie, X. (2022). Communication-efficient federated learning via knowledge distillation. Nature communications, 13(1):2032.
Publicado
20/05/2024
Como Citar
CAPANEMA, Cláudio G. S.; COSTA, Joahannes B. D. da; SILVA, Fabrício A.; VILLAS, Leandro A.; LOUREIRO, Antonio A. F..
Treine Menos, Preveja Mais: plugin de Aprendizado Federado habilita alta eficiência em dados heterogêneos. In: WORKSHOP DE COMPUTAÇÃO URBANA (COURB), 8. , 2024, Niterói/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 85-98.
ISSN 2595-2706.
DOI: https://doi.org/10.5753/courb.2024.3243.
