Incremental and Risk-Sensitive Federated Learning for Ranking Models in Scenarios with Heterogeneous Data Distributions

• Gestefane Rabbi UFMG
• Celso França UFMG
• Thierson Couto Rosa UFG
• Jussara M. Almeida UFMG
• Daniel Xavier de Sousa IFG
• Marcos André Gonçalves UFMG

DOI: https://doi.org/10.5753/sbsi_estendido.2026.249072

Abstract

This dissertation introduces FedRisk, a novel Federated Learning to Rank (FLTR) approach designed to address one of the most critical challenges in intelligent distributed information systems: model aggregation under heterogeneous and non-IID data distributions. By integrating a risk-aware aggregation mechanism—capable of weighting client updates according to prediction error variability—with a historical global parameter stabilization strategy, FedRisk promotes both robustness and convergence stability in federated ranking environments. Extensive experiments on the MSLR-WEB10K benchmark demonstrate that FedRisk significantly outperforms strong federated baselines such as FedProx, achieving a 15.6% improvement in nDCG@5 and matching centralized performance in nDCG@10, while substantially reducing performance variance across communication rounds. Beyond empirical gains, the work provides a principled formulation of risk-sensitive aggregation for FLTR, a systematic analysis of heterogeneity effects in federated ranking, and a comprehensive experimental validation under realistic distributed scenarios. By bridging Information Retrieval, Distributed Machine Learning, and privacy-aware system design, this dissertation advances the development of scalable, intelligent, and regulation-aligned Information Systems. Its scientific impact was recognized with the Honorable Mention Award for Best Full Paper at SBBD 2025, underscoring the originality and methodological rigor of the proposed approach.

References

Ads, Z. et al. (2024). Risk-aware accelerated federated learning over heterogeneous wireless networks. arXiv preprint arXiv:2401.09267.

Ai, Q., Bi, K., Guo, J., and Croft, W. B. (2018). Learning a deep listwise context model for ranking refinement. In ACM SIGIR Conference. ACM.

Araujo, R. M. d., Maciel, R. S. P., and Boscarioli, C. (2017). I GranDSI-BR: Grandes Desafios de Pesquisa em Sistemas de Informação no Brasil (2016–2026). Technical report, Comissão Especial de Sistemas de Informação (CE-SI), Sociedade Brasileira de Computação (SBC). Relatório Técnico.

Berrar, D. (2018). Cross-Validation.

Beutel, D. J., Topal, T., Mathur, A., Qiu, X., Fernandez-Marques, J., Gao, Y., Sani, L., Kwing, H. L., Parcollet, T., Gusmão, P. P. d., and Lane, N. D. (2020). Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390.

Brownlee, J. (2018). Statistical Methods for Machine Learning. Machine Learning Mastery.

Chen, S. et al. (2021). Risk-aware federated learning in crowdsensing systems. arXiv preprint arXiv:2101.01266.

Dincer, B., Zhu, Y., Craswell, N., and Zhang, M. (2016). Risk-sensitive evaluation and learning to rank using multiple baselines. In ACM SIGIR, pages 483–492.

Divi, S., Lin, Y.-S., Farrukh, H., and Celik, Z. B. (2021). New metrics to evaluate the performance and fairness of personalized federated learning.

Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics. SAGE Publications, 4 edition.

Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer.

Hu, C. (2024). Improving federated learning accuracy with the incremental averaging method: A comparative analysis of model aggregation techniques. In Applied and Computational Engineering, pages 150–157. EWA Publishing.

Järvelin, K. and Kekäläinen, J. (2002). Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4):422–446.

Jiménez-Gutiérrez, D. M., Hassanzadeh, M., Anagnostopoulos, A., et al. (2025). A thorough assessment of the non-iid data impact in federated learning. Available at: [link].

Jiménez-Gutiérrez, D. M., Solans, D., Heikkilä, M., et al. (2024). Non-iid data in federated learning: A survey with taxonomy, metrics, methods, frameworks and future directions. Available at: [link].

Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. (2021). Advances and open problems in federated learning. Foundations and Trends in ML, 14(1–2):1–210.

Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S., Stich, S. U., and Suresh, A. T. (2020). Scaffold: Stochastic controlled averaging for federated learning. In ICML.

Köppel, M., Segner, A., Wagener, M., Pensel, L., Karwath, A., and Kramer, S. (2019). Pairwise learning to rank by neural networks revisited: Reconstruction, theoretical analysis and practical performance. arXiv preprint arXiv:1909.02768.

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and anovas. Frontiers in Psychology, Volume 4 - 2013.

Lee, G., Jeong, M., Shin, Y., Bae, S., and Yun, S.-Y. (2022). Preservation of the global knowledge by not-true distillation in federated learning.

Li, C. and Ouyang, H. (2021). Federated unbiased learning to rank.

Li, Q., He, B., and Song, D. (2021). Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

Li, T., Sahu, A. K., Talwalkar, A., and Smith, V. (2020). Federated optimization in heterogeneous networks. In Proceedings of Machine Learning and Systems, pages 429–450.

Liu, J., Huang, J., et al. (2021). From distributed machine learning to federated learning: A survey. Available at: [link].

Liu, T.-Y. (2010). Learning to rank for information retrieval. In ACM SIGIR, page 904.

Lv, Y., Ding, H., Wu, H., Zhao, Y., and Zhang, L. (2023). Fedrds: Federated learning on non-iid data via regularization and data sharing. Applied Sciences, 13(23).

McMahan, H. B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. (2023). Communication-efficient learning of deep networks from decentralized data.

Mukut, S., Kakoli, G., and Jyotika, B. (2012). Federated search: An information retrieval strategy for scholarly literature.

Neto, H. N. C., Mattos, D. M. F., and Fernandes, N. C. (2020). Privacidade do usuário em aprendizado colaborativo: Federated learning, da teoria à prática. In Simpósio Brasileiro de Segurança da Informação e de Sistemas Computacionais (SBSEG).

Qin, T. and Liu, T. (2013). Introducing LETOR 4.0 datasets. CoRR, abs/1306.2597.

Rodrigues, P. H., Danielde Sousa, França, C., Rabbi, G., Rosa, T., and Gonçalves, M. A. (2025). Risk-sensitive optimization of neural deep learning ranking models with applications in ad-hoc retrieval and recommender systems. IP&M, 62(4):104126.

Rodrigues, P. H. S., Xavier Sousa, D., Couto Rosa, T., and Gonçalves, M. A. (2022). Risk-sensitive deep neural learning to rank. In ACM SIGIR, page 803–813.

Spiegelhalter, D. (2024). The Art of Uncertainty: How to Navigate Chance, Ignorance, Risk and Luck. Pelican Books.

Voorhees, E. M. (1999). The trec-8 question answering track report. In TREC-8. National Institute of Standards and Technology (NIST).

Wang, H., Xu, H., Li, Y., Xu, Y., Li, R., and Zhang, T. (2024). FedCDA: Federated learning with cross-rounds divergence-aware aggregation. In ICLR.

Wang, J. and Liu, M. (2020). Tackling the objective inconsistency problem in heterogeneous federated optimization. In NeurIPS.

Wang, S. (2024). Effective and secure federated online learning to rank. arXiv preprint arXiv:2412.19069.

Wang, S. and Zuccon, G. (2022). Is non-iid data a threat in federated online learning to rank? In ACM SIGIR Conference, SIGIR ’22, page 2801–2813.

Wang, Y., Li, T.-Y., Wang, D., and Zhu, M. (2013). A theoretical analysis of ndcg type ranking measures. Journal of Machine Learning Research, 14:25–54.

Yu, T., Bagdasaryan, E., and Shmatikov, V. (2022). Salvaging federated learning by local adaptation.

Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K., Hoang, T. N., and Khazaeni, Y. (2019). Bayesian nonparametric federated learning of neural networks.

Zhao, S. et al. (2024). Federated risk-aware learning with central sensitivity estimation. arXiv preprint arXiv:2502.17694.

Zhu, H., Jin, B., Li, H., and Liang, X. (2021). Federated learning on non-iid data: A survey. Neurocomputing, 465:371–390.