Generate or Rerank? Where to Invest an LLM’s Budget in Cold-Start E-Commerce Recommender Systems

• Christian Freitas UNIFESP
• Lilian Berton UNIFESP

DOI: https://doi.org/10.5753/eniac.2026.31461

Resumo

E-commerce recommenders retrieve candidates cheaply, then rerank them at higher cost, and large language models (LLMs) increasingly occupy that second stage, expected to help customers with little purchase history (user cold-start). We show the second stage has little left to work with: in an Amazon category where 65.6% of customers have at most three purchases, the item bought next reaches the top ten candidates in only ≈2% of cases, capping any reranker at ≈3%. The cause is behavioural, as the next purchase usually falls in a different subcategory. Off-the-shelf LLM generation does not help either, losing to classical retrieval at every cutoff for two providers (paired McNemar, p < 0.02). What helps is free: reciprocal rank fusion of four cheap signals recovers ≈40% more coverage (p < 0.001). The twofold residual gap to an oracle reflects intent the purchase history never records.

Referências

Carraro, D. and Bridge, D. (2024). Enhancing recommendation diversity by re-ranking with large language models. ACM Transactions on Recommender Systems.

Cormack, G. V., Clarke, C. L. A., and Büttcher, S. (2009). Reciprocal rank fusion outperforms Condorcet and individual rank learning methods. In Proc. ACM SIGIR, pages 758–759.

Gao, C., Lei, W., He, X., de Rijke, M., and Chua, T.-S. (2021). Advances and challenges in conversational recommender systems: A survey. AI Open, 2:100–126.

Harte, J., Zorgdrager, W., Louridas, P., Katsifodimos, A., Jannach, D., and Fragkoulis, M. (2023). Leveraging large language models for sequential recommendation. In Proc. 17th ACM Conference on Recommender Systems (RecSys), pages 1096–1102.

Hou, Y., Li, J., Fu, X., He, Z., Yan, A., Chen, X., and McAuley, J. (2026). Bridging language and items for retrieval and recommendation: Benchmarking LLMs as semantic encoders. In Proc. 64th Annual Meeting of the Association for Computational Linguistics (ACL), Volume 1: Long Papers, pages 3251–3265. Amazon Reviews 2023 dataset.

Hu, Y., Koren, Y., and Volinsky, C. (2008). Collaborative filtering for implicit feedback datasets. IEEE International Conference on Data Mining (ICDM), pages 263–272.

Jannach, D., Manzoor, A., Cai, W., and Chen, L. (2021). A survey on conversational recommender systems. ACM Computing Surveys, 54(5):1–36.

Johnson, J., Douze, M., and Jégou, H. (2019). Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547.

Lemdiasova, E. and Zmanovskii, N. (2026). Diagnosing LLM-based rerankers in cold-start recommender systems: Coverage, exposure and practical mitigations. arXiv:2604.16318.

Li, S., Wang, Y., Wang, J., Li, Y., Ghosh, J., and Cocos, A. (2026). LLM reasoning for cold-start item recommendation. In Proc. ACM Web Conference (WWW).

Reimers, N. and Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Proc. EMNLP-IJCNLP, pages 3982–3992.

Santos Silva, F., Soares, J. L., Neto, S. F. G., dos Reis, J. C., and Reis, M. S. (2026). ARARA: A LLM-based multi-agent development framework for conversational recommender systems. In Proc. 14th Int. Workshop on Engineering Multi-Agent Systems (EMAS), co-located with AAMAS.

Shehmir, S. and Kashef, R. F. (2025). LLM4Rec: A comprehensive survey on the integration of large language models in recommender systems — approaches, applications and challenges. Future Internet, 17(6):252.

Zhao, Z., Fan, W., Li, J., Liu, Y., Mei, X., Wang, Y., Wen, Z., Wang, F., Zhao, X., Tang, J., and Li, Q. (2024). Recommender systems in the era of large language models (LLMs). IEEE Transactions on Knowledge and Data Engineering, 36(11):6889–6907.