Risk-Sensitive Federated Learning in Ranking Models

• Gestefane Rabbi Magalhães Federal University of Minas Gerais (UFMG)
• Marcos André Gonçalves Federal University of Minas Gerais (UFMG)
• Daniel Xavier de Sousa Federal Institute of Goiás
• Celso França Federal University of Minas Gerais (UFMG)

DOI: https://doi.org/10.5753/sbbd_estendido.2023.233698

Abstract

This dissertation explores the use of Federated Learning to Rank (FL2R), a technique employed in search systems that considers the privacy of data from various clients. FL2R involves building a ranking model executed in a distributed manner across multiple devices. After training, the neural network parameters of the clients are combined, resulting in a new neural model that will be distributed to the clients. Considered the state of the art in Federated Learning (FL), the Federated Averaging (FedAvg) method calculates the average of parameters to construct the aggregated model. However, low-performing clients can skew the average in a biased manner, resulting in a reduction in the effectiveness of the global model. To contribute to solving this issue, we propose the study of aggregation techniques that go beyond the simple arithmetic mean of weights, in addition to applying metrics in the area of Risk Sensitivity, trying to mitigate the variance of models on the client side. Although the work is in its initial phase, in this paper we were able to show some experiments using Factorial Design to evaluate factors that may impact the effectiveness of federated models. The results show that combining parameter values is not a trivial task, but considering the proposed research questions, we believe this work has strong potential for contribution in the Information Retrieval field.

Keywords: Federated learning to rank, Search systems, Data privacy

References

Ai, Q., Bi, K., Guo, J., and Croft, W. B. (2018). Learning a deep listwise context model for ranking refinement. In ACM SIGIR conference, pages 135–144.

Deng, L. (2012). The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141–142.

Divi, S., Lin, Y.-S., Farrukh, H., and Celik, Z. B. (2021). New metrics to evaluate the performance and fairness of personalized federated learning.

Jain, R. (1991). The Art of Systems Performance Analysis: Techniques for experimental design, Measurement, simulation, and modeling. John Wiley amp; Sons.

Jiang, J. C., Kantarci, B., Oktug, S., and Soyata, T. (2020). Federated learning in smart city sensing: Challenges and opportunities. Sensors, 20(21):6230.

Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, et al. (2021). Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210.

McMahan, H. B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. (2023). Communication-efficient learning of deep networks from decentralized data.

Mukut, S., Kakoli, G., and Jyotika, B. (2012). Federated search: An information retrieval strategy for scholarly literature.

Silva Rodrigues, P. H., Xavier Sousa, D., Couto Rosa, T., and Gonçalves, M. A. (2022). Risk-sensitive deep neural learning to rank. In ACM SIGIR Conference, SIGIR ’22, page 803–813.

Wang, S. and Zuccon, G. (2022). Is non-iid data a threat in federated online learning to rank? In ACM SIGIR Conference, SIGIR ’22, page 2801–2813.

Ye, Y., Li, S., Liu, F., Tang, Y., and Hu,W. (2020). Edgefed: Optimized federated learning based on edge computing. IEEE Access, 8:209191–209198.