Evaluating the Impacts of Combining Embedding Extraction Techniques with Transformer Models for Music Recommendation

• Vinicius S. Simm UEM https://orcid.org/0009-0005-2541-938X
• Matheus H. C. Leme UEM https://orcid.org/0000-0002-0874-8780
• Douglas R. Tanno UEM https://orcid.org/0000-0002-9523-1615
• Marcos A. Domingues UEM https://orcid.org/0000-0001-7195-0714

DOI: https://doi.org/10.5753/semish.2026.20889

Resumo

This paper investigates the impact of different embedding strategies on Transformer-based models for session-based music recommendation. We compared two approaches: a standard Transformer model that learns item embeddings internally, and a second model that uses pre-trained embeddings from Word2Vec (CBOW and Skip-Gram), generating recommendations via cosine similarity. We evaluated on two music datasets, Music4All and Xiami, and our results highlight a trade-off between accuracy and diversity. The Transformer model achieves a significantly higher Hit Rate but shows a stronger popularity bias and lower catalog coverage. On the other hand, the approach using pre-trained embeddings shows greater coverage and diversity but lower accuracy.

Referências

Abdollahpouri, H., Mansoury, M., Burke, R., and Mobasher, B. (2019). The unfairness of popularity bias in recommendation. CoRR, abs/1907.13286.

Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019). Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD’19), page 2623–2631.

Jannach, D., Quadrana, M., and Cremonesi, P. (2022). Session-based recommender systems. In Recommender Systems Handbook, pages 301–334. Springer.

Jannach, D. and Zanker, M. (2022). Value and Impact of Recommender Systems, pages 519–546. Springer US, New York, NY.

Kang, W.-C. and McAuley, J. (2018). Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), pages 197–206. IEEE.

Koren, Y., Bell, R., and Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8):30–37.

Latifi, S., Mauro, N., and Jannach, D. (2021). Session-aware recommendation: A surprising quest for the state-of-the-art. Information Sciences, 573:291–315.

Levy, O. and Goldberg, Y. (2014). Neural word embedding as implicit matrix factorization. Advances in neural information processing systems, 27.

Lorenz, M. O. (1905). Methods of measuring the concentration of wealth. Publications of the American statistical association, 9(70):209–219.

Ludewig, M. and Jannach, D. (2018). Evaluation of session-based recommendation algorithms. CoRR, abs/1803.09587.

Ludewig, M., Mauro, N., Latifi, S., and Jannach, D. (2021). Empirical analysis of session-based recommendation algorithms. User Modeling and User-Adapted Interaction, 31(1):149–181.

Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013a). Efficient estimation of word representations in vector space. arXiv preprint, arXiv:1301.3781.

Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. (2013b). Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26.

Santana, I. A. P. (2020). Exploração de redes neurais recorrentes na recomendação sensível ao contexto de músicas. Dissertação (mestrado em ciência da computação), Universidade Estadual de Maringá, Maringá, PR.

Santana, I. A. P., Pinhelli, F., Donini, J., Catharin, L., Mangolin, R. B., Feltrim, V. D., Domingues, M. A., et al. (2020). Music4all: A new music database and its applications. In 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pages 399–404.

Schedl, M. (2019). Deep learning in music recommendation systems. Frontiers in Applied Mathematics and Statistics, 5.

Simm, V. S., Tanno, D. R., and Domingues, M. (2025). Exploring transformers in embedding-based music recommendation. In Anais do XVII Congresso Brasileiro de Inteligência Computacional (CBIC 2025), pages 1–8.

van den Oord, A., Dieleman, S., and Schrauwen, B. (2013). Deep content-based music recommendation. In Advances in Neural Information Processing Systems, volume 26.

Vasile, F., Smirnova, E., and Conneau, A. (2016). Meta-prod2vec: Product embeddings using side-information for recommendation. In Proceedings of the 10th ACM conference on recommender systems, pages 225–232.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

Wang, D., Deng, S., and Xu, G. (2017). Sequence-based context-aware music recommendation. Information Retrieval Journal, 21(2-3):230–252.

Wang, D., Deng, S., Zhang, X., and Xu, G. (2016). Learning music embedding with metadata for context aware recommendation. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval, pages 249–253.

Wang, S., Cao, L., Wang, Y., Sheng, Q. Z., Orgun, M. A., and Lian, D. (2021). A survey on session-based recommender systems. ACM Computing Surveys, 54(7):1–38.