Integrating Confidence into Embedding-Based Models for Learn-to-Rank in Recommender Systems

• Joel Machado Pires Universidade Federal da Bahia (UFBA)
• Frederico Araújo Durão Universidade Federal da Bahia (UFBA)

DOI: https://doi.org/10.5753/sbbd_estendido.2026.249497

Resumo

Recommender systems produce uncertain predictions due to sparse and noisy user interactions, yet existing confidence estimation methods are mostly limited to matrix factorization models and remain underexplored for graph neural networks and learning-to-rank. This work presents a unified study of confidence estimation in recommender systems by benchmarking prior distribution-based approaches, analyzing their calibration and impact on predictive performance, and proposing probabilistic graph-based models and an uncertainty-aware pairwise ranking formulation based on the Gaussian cumulative distribution function. Experiments on multiple public datasets show that the proposed methods preserve or improve recommendation quality while producing confidence estimates that strongly correlate with prediction errors, providing reliable uncertainty quantification and a general framework for confidence-aware recommendation.

Palavras-chave: Recommender system, learno-to-rank, guassian, probabilistic graph-based model

Referências

D’Amico, E., Gabbolini, G., Bernardis, C., and Cremonesi, P. (2022). Analyzing and improving stability of matrix factorization for recommender systems. Journal of Intelligent Information Systems, 58(2):255–285.

Feng, C., Liu, Z., Lin, S., and Quek, T. Q. (2019). Attention-based graph convolutional network for recommendation system. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7560–7564.

Gohari, F. S., Aliee, F. S., and Haghighi, H. (2018). A new confidence-based recommendation approach: Combining trust and certainty. Information Sciences, 422:21–50.

Himabindu, T. V., Padmanabhan, V., and Pujari, A. K. (2018). Conformal matrix factorization based recommender system. Information Sciences, 467:685–707.

Knyazev, N. and Oosterhuis, H. (2023). A lightweight method for modeling confidence in recommendations with learned beta distributions. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 306–317.

Koren, Y. and Sill, J. (2011). Ordrec: an ordinal model for predicting personalized item rating distributions. In Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys ’11, page 117–124, New York, NY, USA. Association for Computing Machinery.

Mesas, R. M. and Bellogín, A. (2020). Exploiting recommendation confidence in decision-aware recommender systems. Journal of Intelligent Information Systems, 54(1):45–78.

Papadopoulos, G., Edwards, P., and Murray, A. (2001). Confidence estimation methods for neural networks: a practical comparison. IEEE Transactions on Neural Networks, 12(6):1278–1287.

Saia, R., Boratto, L., and Carta, S. (2016). A semantic approach to remove incoherent items from a user profile and improve the accuracy of a recommender system. Journal of Intelligent Information Systems, 47(1):111–134.

Tao, Z., Wei, Y., Wang, X., He, X., Huang, X., and Chua, T.-S. (2020). Mgat: Multimodal graph attention network for recommendation. Information Processing Management, 57(5):102277.

Wang, C., Liu, Q., Wu, R., Chen, E., Liu, C., Huang, X., and Huang, Z. (2018a). Confidence-aware matrix factorization for recommender systems. In Proceedings of the AAAI Conference on artificial intelligence, volume 32.

Wang, H., Zhang, F., Wang, J., Zhao, M., Li, W., Xie, X., and Guo, M. (2018b). Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM ’18, page 417–426, New York, NY, USA. Association for Computing Machinery.

Wang, X., He, X., Cao, Y., Liu, M., and Chua, T.-S. (2019). Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, page 950–958, New York, NY, USA. Association for Computing Machinery.

Wang, X. and Kadıoğlu, S. (2023). Modeling uncertainty to improve personalized recommendations via bayesian deep learning. International Journal of Data Science and Analytics, 16(2):191–201.

Watt, T. A., McCleery, R. H., and Hart, T. (2007). Introduction to statistics for biology. CRC Press.

Zhang, M., Guo, X., and Chen, G. (2016). Prediction uncertainty in collaborative filtering: Enhancing personalized online product ranking. Decision Support Systems, 83:10–21.