Leveraging LLMs for Explainable Recommender Systems: Exploring User Perceptions and Faithfulness in Generated Explanations
Resumo
Sistemas de Recomendação (SRs) são amplamente utilizados para auxiliar usuários na descoberta de itens relevantes, mas sua complexidade levanta preocupações quanto à transparência e interpretabilidade. Este trabalho investiga o uso de Modelos de Linguagem de Grande Escala (LLMs) para gerar explicações automatizadas e centradas no usuário em SRs, avaliando também se tais explicações são fiéis ao processo decisório dos modelos. Inicialmente, conduzimos um estudo com usuários para avaliar recomendações e explicações personalizadas de filmes geradas pelo GPT-3.5 Turbo, mensurando percepção de efetividade, personalização e persuasão. Em seguida, analisamos a fidelidade das explicações nos domínios de filmes, músicas e livros utilizando quatro LLMs (GPT-4o, Llama3, Gemma2 e Mixtral 8x7B). Os resultados indicam que recomendações geradas por LLMs aumentam a satisfação dos usuários em comparação a recomendações aleatórias; entretanto, as explicações frequentemente não refletem adequadamente os fatores que influenciaram a recomendação. Além disso, explicações baseadas em preferências do usuário não foram consistentemente percebidas como superiores às genéricas.
Referências
Balog, K. and Radlinski, F. (2020). Measuring recommendation explanation quality: The conflicting goals of explanations. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 329–338, New York, NY, USA. Association for Computing Machinery.
Chang, S., Harper, F. M., and Terveen, L. G. (2016). Crowd-based personalized natural language explanations for recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, page 175–182, New York, NY, USA. Association for Computing Machinery.
Jacovi, A. and Goldberg, Y. (2020). Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J., editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4198–4205, Online. Association for Computational Linguistics.
Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., and Newell, C. (2012). Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22:441–504.
Lu, H., Ma, W., Wang, Y., Zhang, M., Wang, X., Liu, Y., Chua, T.-S., and Ma, S. (2023). User perception of recommendation explanation: Are your explanations what users need? ACM Trans. Inf. Syst., 41(2).
Lyu, H., Jiang, S., Zeng, H., Wang, Q., Zhang, S., Chen, R., Leung, C., Tang, J., Xia, Y., and Luo, J. (2023). Llm-rec: Personalized recommendation via prompting large language models.
Parcalabescu, L. and Frank, A. (2024). On measuring faithfulness or self-consistency of natural language explanations. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6048–6089, Bangkok, Thailand. Association for Computational Linguistics.
Silva, I. and Marinho, L. (2026). Faithfulness in black-box llmbased recommender systems’ explanations. In Anais da XXXVI Brazilian Conference on Intelligent Systems. Accepted for publication.
Silva, I., Marinho, L., Said, A., and Willemsen, M. C. (2024). Leveraging chatgpt for automated human-centered explanations in recommender systems. In Proceedings of the 29th International Conference on Intelligent User Interfaces, IUI ’24, page 597–608, New York, NY, USA. ACM.
Silva, I., Said, A., and Marinho, L. B. (2023). Leveraging large language models for recommendation and explanation. In Proceedings of the 10th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems (IntRS 2023). CEUR Workshop Proceedings.
Silva, I. D. S., Balby, L., Said, A., and Willemsen, M. (2025). User perceptions of personalized and generic explanations in llm-driven recommender systems. ACM Trans. Interact. Intell. Syst. Just Accepted.
Tintarev, N. and Masthoff, J. (2007). A survey of explanations in recommender systems. In 2007 IEEE 23rd International Conference on Data Engineering Workshop, pages 801–810.
Turpin, M., Michael, J., Perez, E., and Bowman, S. R. (2024). Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
Wagemans, J. (2021). Periodic table of arguments 2.5 a.
Wiegreffe, S., Marasović, A., and Smith, N. A. (2021). Measuring association between labels and free-text rationales. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t., editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10266–10284, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Wu, L., Zheng, Z., Qiu, Z., Wang, H., Gu, H., Shen, T., Qin, C., Zhu, C., Zhu, H., Liu, Q., Xiong, H., and Chen, E. (2023). A survey on large language models for recommendation. CoRR, abs/2305.19860.
Zhang, Y. and Chen, X. (2020). Explainable recommendation: A survey and new perspectives. Found. Trends Inf. Retr., 14(1):1–101.
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., and Du, M. (2024). Explainability for large language models: A survey. ACM Trans. Intell. Syst. Technol. Just Accepted.
Zhou, J. and Joachims, T. (2023). Gpt as a baseline for recommendation explanation texts. In Proceedings of the IntRS’23 Workshop.
Chang, S., Harper, F. M., and Terveen, L. G. (2016). Crowd-based personalized natural language explanations for recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, page 175–182, New York, NY, USA. Association for Computing Machinery.
Jacovi, A. and Goldberg, Y. (2020). Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J., editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4198–4205, Online. Association for Computational Linguistics.
Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., and Newell, C. (2012). Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22:441–504.
Lu, H., Ma, W., Wang, Y., Zhang, M., Wang, X., Liu, Y., Chua, T.-S., and Ma, S. (2023). User perception of recommendation explanation: Are your explanations what users need? ACM Trans. Inf. Syst., 41(2).
Lyu, H., Jiang, S., Zeng, H., Wang, Q., Zhang, S., Chen, R., Leung, C., Tang, J., Xia, Y., and Luo, J. (2023). Llm-rec: Personalized recommendation via prompting large language models.
Parcalabescu, L. and Frank, A. (2024). On measuring faithfulness or self-consistency of natural language explanations. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6048–6089, Bangkok, Thailand. Association for Computational Linguistics.
Silva, I. and Marinho, L. (2026). Faithfulness in black-box llmbased recommender systems’ explanations. In Anais da XXXVI Brazilian Conference on Intelligent Systems. Accepted for publication.
Silva, I., Marinho, L., Said, A., and Willemsen, M. C. (2024). Leveraging chatgpt for automated human-centered explanations in recommender systems. In Proceedings of the 29th International Conference on Intelligent User Interfaces, IUI ’24, page 597–608, New York, NY, USA. ACM.
Silva, I., Said, A., and Marinho, L. B. (2023). Leveraging large language models for recommendation and explanation. In Proceedings of the 10th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems (IntRS 2023). CEUR Workshop Proceedings.
Silva, I. D. S., Balby, L., Said, A., and Willemsen, M. (2025). User perceptions of personalized and generic explanations in llm-driven recommender systems. ACM Trans. Interact. Intell. Syst. Just Accepted.
Tintarev, N. and Masthoff, J. (2007). A survey of explanations in recommender systems. In 2007 IEEE 23rd International Conference on Data Engineering Workshop, pages 801–810.
Turpin, M., Michael, J., Perez, E., and Bowman, S. R. (2024). Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
Wagemans, J. (2021). Periodic table of arguments 2.5 a.
Wiegreffe, S., Marasović, A., and Smith, N. A. (2021). Measuring association between labels and free-text rationales. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t., editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10266–10284, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Wu, L., Zheng, Z., Qiu, Z., Wang, H., Gu, H., Shen, T., Qin, C., Zhu, C., Zhu, H., Liu, Q., Xiong, H., and Chen, E. (2023). A survey on large language models for recommendation. CoRR, abs/2305.19860.
Zhang, Y. and Chen, X. (2020). Explainable recommendation: A survey and new perspectives. Found. Trends Inf. Retr., 14(1):1–101.
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., and Du, M. (2024). Explainability for large language models: A survey. ACM Trans. Intell. Syst. Technol. Just Accepted.
Zhou, J. and Joachims, T. (2023). Gpt as a baseline for recommendation explanation texts. In Proceedings of the IntRS’23 Workshop.
Publicado
19/10/2026
Como Citar
SILVA, Itallo de Sousa.
Leveraging LLMs for Explainable Recommender Systems: Exploring User Perceptions and Faithfulness in Generated Explanations. In: CONCURSO DE TESES E DISSERTAÇÕES EM INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (CTDIAC), 15. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 37-51.
DOI: https://doi.org/10.5753/ctdiac.2026.32877.
