Federated Learning Model Selection Based on Search and Pruning for Industrial Defect Detection

• Luana Gantert UFRJ
• Miguel Elias M. Campista UFRJ

DOI: https://doi.org/10.5753/sbrc.2024.1548

Abstract

Federated learning emerges as an alternative to traditional machine learning by decentralizing model training. Client devices communicate with a central server and train the defined model iteratively. The training model definition in advance, however, is a challenge that is rarely discussed. This work proposes selecting the best neural network based on a search procedure involving multiple networks and subsequent pruning of those that appear less promising during training. To do this, in a given evaluation round, the nodes send the performance results to a comparator node using the AUC-ROC metric (Area Under The Receiver Operating Characteristics Curve) for best-model selection. Our experiments use an application for detecting malfunctions in industrial equipment using the emitted sounds. The results demonstrate that the models found when performing pruning in the 5 and 10 round of training reach final AUC-ROC values of approximately 0.95 and 0.91 in the best-case scenario, respectively. These values represent an increase of up to 6.25% over federated learning without pruning and 18.75% over traditional centralized learning.

References

Alhalabi, B., Basurra, S., e Gaber, M. M. (2023). Fednets: Federated learning on edge devices using ensembles of pruned deep neural networks. IEEE Access, 11:30726–30738.

Beutel, D. J., Topal, T., Mathur, A., Qiu, X., Parcollet, T., de Gusmão, P. P., e Lane, N. D. (2020). Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390.

Bochie, K., Gilbert, M. S., Gantert, L., Barbosa, M. S., Medeiros, D. S., e Campista, M. E. M. (2021). A survey on deep learning for challenged networks: Applications and trends. Journal of Network and Computer Applications, 194:103213.

Chawla, N. V., Bowyer, K. W., Hall, L. O., e Kegelmeyer, W. P. (2002). Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357.

de Souza, L. A. C., Camilo, G. F., Sammarco, M., Campista, M. E. M., e Costa, L. H. M. (2022). Aprendizado federado com agrupamento hierárquico de clientes para aumento da acurácia. In Anais do XL Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, pages 545–558. SBC.

Fu, L., Zhang, H., Gao, G., Zhang, M., e Liu, X. (2023). Client selection in federated learning: Principles, challenges, and opportunities. IEEE Internet of Things Journal.

Gantert, L., Sammarco, M., Detyniecki, M., e Campista, M. E. M. (2021). A supervised approach for corrective maintenance using spectral features from industrial sounds. In 2021 IEEE 7th World Forum on Internet of Things (WF-IoT), pages 723–728. IEEE.

Gantert, L., Sammarco, M., Detyniecki, M., e Campista, M. E. M. (2022). Super learner ensemble for sound classification using spectral features. In 2022 IEEE Latin-American Conference on Communications (LATINCOM), pages 1–6. IEEE.

Jiang, Y., Wang, S., Valls, V., Ko, B. J., Lee, W.-H., Leung, K. K., e Tassiulas, L. (2022). Model pruning enables efficient federated learning on edge devices. IEEE Transactions on Neural Networks and Learning Systems.

Li, L., Fan, Y., Tse, M., e Lin, K.-Y. (2020). A review of applications in federated learning. Computers & Industrial Engineering, 149:106854.

Lin, R., Xiao, Y., Yang, T.-J., Zhao, D., Xiong, L., Motta, G., e Beaufays, F. (2022). Federated pruning: improving neural network efficiency with federated learning. arXiv preprint arXiv:2209.06359.

McFee, B., Raffel, C., Liang, D., Ellis, D. P., McVicar, M., Battenberg, E., e Nieto, O. (2015). librosa: Audio and music signal analysis in python. In Proceedings of the 14th python in science conference, volume 8, pages 18–25.

McMahan, B., Moore, E., Ramage, D., Hampson, S., e y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR.

Natesha, B. e Guddeti, R. M. R. (2021). Fog-based intelligent machine malfunction monitoring system for industry 4.0. IEEE Transactions on Industrial Informatics, 17(12):7923–7932.

Neto, H. N., Mattos, D. M., e Fernandes, N. C. (2020). Privacidade do usuário em aprendizado colaborativo: Federated learning, da teoria à prática. Sociedade Brasileira de Computação.

Purohit, H., Tanabe, R., Ichige, K., Endo, T., Nikaido, Y., Suefusa, K., e Kawaguchi, Y. (2019). Mimii dataset: Sound dataset for malfunctioning industrial machine investigation and inspection. arXiv preprint arXiv:1909.09347.

Ramos, H. S., Maia, G., Papa, G. L., Alvim, M. S., Loureiro, A. A., Cardoso-Pereira, I., Campos, D. H., Filipakis, G., Riquetti, G., Chagas, E. T., et al. (2021). Aprendizado federado aplicado à internet das coisas. Sociedade Brasileira de Computação.

Suefusa, K., Nishida, T., Purohit, H., Tanabe, R., Endo, T., e Kawaguchi, Y. (2020). Anomalous sound detection based on interpolation deep neural network. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 271–275. IEEE.

Tao, B., Chen, C., e Chen, H. (2023). Communication efficient federated learning via channel-wise dynamic pruning. In 2023 IEEE Wireless Communications and Networking Conference (WCNC), pages 1–6. IEEE.