Calibrated News Recommendation under Dynamic Feedback Loops: A Simulation Study
Resumo
Calibrated recommendations, a class of techniques designed to align recommendation lists with users’ historical preference distribution, are largely underexplored under the dynamic feedback loop scenario. In this work, we use a dynamic recommendation simulation framework to study how calibration affects news recommendation over time, measuring miscalibration, diversity and fragmentation. We evaluate calibrated and uncalibrated recommendations on semi-synthetic simulation environments derived from Globo.com, a large-scale Brazilian news portal dataset. Our results show that calibration consistently increases fragmentation over time, while slightly reducing catalog coverage. Filtering out previously interacted items (exposure filtering), a common practice in recommender systems, also plays a role: calibrated recommendations start out more diverse, but diversity decreases over time as top-ranked articles get filtered out, while uncalibrated recommendations become more diverse overall, despite cyclical fluctuations. The same pattern holds for miscalibration, which increases over time for calibrated variants but decreases for uncalibrated ones.
Referências
Himan Abdollahpouri, Masoud Mansoury, Robin Burke, and Bamshad Mobasher. 2019. The impact of popularity bias on fairness and calibration in recommendation. arXiv preprint arXiv:1910.05755 (2019).
Himan Abdollahpouri, Masoud Mansoury, Robin Burke, and Bamshad Mobasher. 2019. The unfairness of popularity bias in recommendation. arXiv preprint arXiv:1907.13286 (2019).
Gabrielle Alves, Dietmar Jannach, Rodrigo Ferrari De Souza, and Marcelo Garcia Manzato. 2024. User Perception of Fairness-Calibrated Recommendations. In Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization (Cagliari, Italy) (UMAP ’24). Association for Computing Machinery, New York, NY, USA, 78–88. DOI: 10.1145/3627043.3659558
Guy Aridor, Duarte Goncalves, and Shan Sikdar. 2020. Deconstructing the filter bubble: User decision-making and recommender systems. In Proceedings of the 14th ACM conference on recommender systems. 82–91.
Nabiha Asghar. 2016. Yelp dataset challenge: Review rating prediction. arXiv preprint arXiv:1605.05362 (2016).
James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of machine learning research 13, 2 (2012).
Dimitrios Bountouridis, Jaron Harambam, Mykola Makhortykh, Mónica Marrero, Nava Tintarev, and Claudia Hauff. 2019. SIREN: A Simulation Framework for Understanding the Effects of Recommender Systems in Online News Environments. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* ’19). Association for Computing Machinery, New York, NY, USA, 150–159. DOI: 10.1145/3287560.3287583
Jaime Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval. 335–336.
Micah D Carroll, Anca Dragan, Stuart Russell, and Dylan Hadfield-Menell. 2022. Estimating and penalizing induced preference shifts in recommender systems. In International Conference on Machine Learning. PMLR, 2686–2708.
Hongqi Chen, Zhiyong Feng, Shizhan Chen, Hongyue Wu, Yingchao Sun, Jingyu Li, Qinghang Gao, Lu Zhang, and Xiao Xue. 2025. Incorporating Forgetting Curve and Memory Replay for Evolving Socially-aware Recommendation. Information Processing & Management 62, 3 (2025), 104070. DOI: 10.1016/j.ipm.2025.104070
Jiawei Chen, Hande Dong, XiangWang, Fuli Feng, MengWang, and Xiangnan He. 2023. Bias and Debias in Recommender System: A Survey and Future Directions. ACM Trans. Inf. Syst. 41, 3, Article 67 (Feb. 2023), 39 pages. DOI: 10.1145/3564284
Jun Chen, Chaokun Wang, and Jianmin Wang. 2014. Modeling the Interest-Forgetting Curve for Music Recommendation. In Proceedings of the 22nd ACM International Conference on Multimedia (Orlando, Florida, USA) (MM ’14). Association for Computing Machinery, New York, NY, USA, 921–924. DOI: 10.1145/2647868.2654954
Andrew Collins, Dominika Tkaczyk, Akiko Aizawa, and Joeran Beel. 2018. A study of position bias in digital library recommender systems.
Zhihong Cui, Xiangguo Sun, Hongxu Chen, Li Pan, Li zhen Cui, Shijun Liu, and Guandong Xu. 2024. Dynamic Recommendation Based on Graph Diffusion and Ebbinghaus Curve. IEEE Transactions on Computational Social Systems 11 (2024), 2755–2764. [link]
Diego da Silva and Dietmar Jannach. 2026. Calibrated Recommendations: Survey and Future Directions. ACM Trans. Recomm. Syst. 4, 3, Article 43 (March 2026), 32 pages. DOI: 10.1145/3789266
Diego Corrêa da Silva, Dietmar Jannach, and Frederico Araújo Durão. 2025. Considering Time and Feature Entropy in Calibrated Recommendations. ACM Transactions on Intelligent Systems and Technology 16, 3 (2025), 1–29.
Diego Corrêa da Silva, Marcelo GarciaManzato, and Frederico Araújo Durão. 2021. Exploiting personalized calibration and metrics for fairness recommendation. Expert Systems with Applications 181 (2021), 115112. DOI: 10.1016/j.eswa.2021.115112
Rodrigo Ferrari de Souza and Marcelo Garcia Manzato. 2024. Uma abordagem em etapa de processamento para redução do viés de popularidade. In Brazilian Symposium on Multimedia and the Web (WebMedia). SBC, 310–317.
Yashar Deldjoo, Dietmar Jannach, Alejandro Bellogin, Alessandro Difonzo, and Dario Zanzonelli. 2024. Fairness in recommender systems: research landscape and future directions. User Modeling and User-Adapted Interaction 34, 1 (2024), 59–108.
P Moreira Gabriel De Souza, Dietmar Jannach, and Adilson Marques Da Cunha. 2019. Contextual hybrid session-based news recommendation with recurrent neural networks. IEEE access 7 (2019), 169185–169203.
F. Maxwell Harper and Joseph A. Konstan. 2015. TheMovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5, 4, Article 19 (Dec. 2015), 19 pages. DOI: 10.1145/2827872
Nicolas Hug. 2020. Surprise: A Python library for recommender systems. Journal of Open Source Software 5, 52 (2020), 2174.
Dietmar Jannach and Gediminas Adomavicius. 2016. Recommendations with a purpose. In Proceedings of the 10th ACM conference on recommender systems. 7–10.
Dietmar Jannach, Alan Said, Marko Tkalcic, and Markus Zanker. 2025. Recommender systems for good (RS4Good): survey of use cases and a call to action for research that matters. ACM Transactions on Recommender Systems (2025).
Dietmar Jannach and Markus Zanker. 2012. Value and impact of recommender systems. In Recommender systems handbook. Springer, 519–546.
Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. 2007. Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search. ACM Trans. Inf. Syst. 25, 2 (April 2007), 7–es. DOI: 10.1145/1229179.1229181
Mesut Kaya and Derek Bridge. 2019. A comparison of calibrated and intent-aware recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems (Copenhagen, Denmark) (RecSys ’19). Association for Computing Machinery, New York, NY, USA, 151–159. DOI: 10.1145/3298689.3347045
Anastasiia Klimashevskaia, Mehdi Elahi, Dietmar Jannach, Christoph Trattner, and Lars Skjærven. 2022. Mitigating popularity bias in recommendation: potential and limits of calibration approaches. In International Workshop on Algorithmic Bias in Search and Recommendation. Springer, 82–90.
Bart P. Knijnenburg, Saadhika Sivakumar, and Daricia Wilkinson. 2016. Recommender Systems for Self-Actualization. In Proceedings of the 10th ACM Conference on Recommender Systems (Boston, Massachusetts, USA) (RecSys ’16). Association for Computing Machinery, New York, NY, USA, 11–14. DOI: 10.1145/2959100.2959189
Yehuda Koren. 2010. Collaborative filtering with temporal dynamics. Commun. ACM 53, 4 (April 2010), 89–97. DOI: 10.1145/1721654.1721677
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. Computer 42, 8 (Aug. 2009), 30–37. DOI: 10.1109/MC.2009.263
Joshua Krook and Jan Blockx. 2023. Recommender systems, autonomy and user engagement. In Proceedings of the first international symposium on trustworthy autonomous systems. 1–9.
S. Kullback and R. A. Leibler. 1951. On Information and Sufficiency. The Annals of Mathematical Statistics 22, 1 (1951), 79 – 86. DOI: 10.1214/aoms/1177729694
Kun Lin. 2026. Calibration in Dynamic Recommender Systems. In Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining (USA) (WSDM ’26). Association for Computing Machinery, New York, NY, USA, 1296–1298. DOI: 10.1145/3773966.3784983
Kun Lin, Masoud Mansoury, Farzad Eskandanian, Milad Sabouri, and Bamshad Mobasher. 2024. Beyond Static Calibration: The Impact of User Preference Dynamics on Calibrated Recommendation. In Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization. 86–91.
Kun Lin, Nasim Sonboli, Bamshad Mobasher, and Robin Burke. 2020. Calibration in Collaborative Filtering Recommender Systems: a User-Centered Analysis. In Proceedings of the 31st ACM Conference on Hypertext and Social Media (Virtual Event, USA) (HT ’20). Association for Computing Machinery, New York, NY, USA, 197–206. DOI: 10.1145/3372923.3404793
Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2020. Feedback Loop and Bias Amplification in Recommender Systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (Virtual Event, Ireland) (CIKM ’20). Association for Computing Machinery, New York, NY, USA, 2145–2148. DOI: 10.1145/3340531.3412152
Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 2020. Controlling fairness and bias in dynamic learning-to-rank. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval. 429–438.
Jaap MJ Murre and Joeri Dros. 2015. Replication and analysis of Ebbinghaus’ forgetting curve. PloS one 10, 7 (2015), e0120644.
Derek O’Callaghan, Derek Greene, Maura Conway, Joe Carthy, and Pádraig Cunningham. 2015. Down the (white) rabbit hole: The extreme right and online recommender systems. Social Science Computer Review 33, 4 (2015), 459–478.
Eli Pariser. 2011. The filter bubble: What the Internet is hiding from you. penguin UK.
Bruno Laporais Pereira. 2017. Recomendação online de músicas usando feedback implícito. In "Masters dissertation". Universidade Federal de Minas Gerais.
Erasmo Purificato, Ludovico Boratto, and Ernesto William De Luca. 2024. User modeling and user profiling: A comprehensive survey. arXiv preprint arXiv:2402.09660 (2024).
Chhavi Rana and Sanjay Jain. 2012. A study of the dynamic features of recommender systems. Artificial Intelligence Review 43 (11 2012). DOI: 10.1007/s10462-012-9359-6
Shaina Raza and Chen Ding. 2022. News recommender system: a review of recent progress, challenges, and opportunities. Artificial Intelligence Review 55, 1 (2022), 749–800.
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).
Francesco Ricci, Lior Rokach, and Bracha Shapira. 2010. Introduction to recommender systems handbook. In Recommender systems handbook. Springer, 1–35.
Andre Sacilotti, Rodrigo Ferrari de Souza, and Marcelo Garcia Manzato. 2023. Counteracting popularity-bias and improving diversity through calibrated recommendations. In Proceedings of the 25th International Conference on Enterprise Information Systems (ICEIS).
Alan Said, Maria Soledad Pera, and Michael D Ekstrand. 2025. We’re Still Doing It (All) Wrong: Recommender Systems, Fifteen Years Later. arXiv preprint arXiv:2509.09414 (2025).
Diego Silva and Frederico Durão. 2022. Explorando Justiça em Sistemas de Recomendação. In Anais Estendidos do XXVIII Simpósio Brasileiro de Sistemas Multimídia e Web (Curitiba). SBC, Porto Alegre, RS, Brasil, 11–14. DOI: 10.5753/webmedia_estendido.2022.225303
Harald Steck. 2018. Calibrated recommendations. In Proceedings of the 12th ACM conference on recommender systems. 154–162.
Célina Treuillier, Sylvain Castagnos, Özlem Özgöbek, and Armelle Brun. 2024. Beyond Trade-offs: Unveiling Fairness-Constrained Diversity in News Recommender Systems. In Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization (Cagliari, Italy) (UMAP ’24). Association for Computing Machinery, New York, NY, USA, 143–148. DOI: 10.1145/3627043.3659571
Sanne Vrijenhoek, Mesut Kaya, Nadia Metoui, Judith Möller, Daan Odijk, and Natali Helberger. 2021. Recommenders with amission: assessing diversity in news recommendations. In Proceedings of the 2021 conference on human information interaction and retrieval. 173–183.
Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, et al. 2020. Mind: A large-scale dataset for news recommendation. In Proceedings of the 58th annual meeting of the association for computational linguistics. 3597–3606.
Sirui Yao, Yoni Halpern, Nithum Thain, Xuezhi Wang, Kang Lee, Flavien Prost, Ed Chi, Jilin Chen, and Alex Beutel. 2021. Measuring Recommender System Effects with Simulated Users. DOI: 10.48550/arXiv.2101.04526
Xing Zhao, Ziwei Zhu, and James Caverlee. 2021. Rabbit holes and taste distortion: Distribution-aware recommendation with evolving interests. In Proceedings of the Web Conference 2021. 888–899.
Ziwei Zhu, Yun He, Xing Zhao, and James Caverlee. 2021. Popularity Bias in Dynamic Recommendation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (Virtual Event, Singapore) (KDD ’21). Association for Computing Machinery, New York, NY, USA, 2439–2449. DOI: 10.1145/3447548.3467376,
