Improving Search Quality with Automatic Ranking Evaluation and Tuning

  • Larícia Cavalcante Universidade Federal de Pernambuco / VTEX
  • Ullayne Lima Universidade Federal de Pernambuco / VTEX
  • Luciano Barbosa Universidade Federal de Pernambuco
  • Ana Luiza Gomes VTEX
  • Éden Santana Universidade Federal de Pernambuco
  • Thiago Martins Universidade Federal de Pernambuco

Resumo


Search is a common feature available in document-based applications. It allows users to find information of interest easier. Two essential aspects for building an effective search is to evaluate the ranking quality and be able to efficiently tune it based on this evaluation. In this paper, we present our Automatic Ranking Tuning and Evaluation System (ARTES) that measures the ranking performance based on users’ clicks on search resulting pages and automatically tunes the search ranking function by applying a Bayesian Optimization algorithm. Our system is integrated with Elasticsearch, a widely-used search engine, which provides the search functionality. The whole solution is currently used by our customer support platform to help users effectively find relevant information, as our experimental evaluation confirms.

Palavras-chave: ranking evaluation, rank tuning, Bayesiam optimization

Referências

Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019). Optuna: A next- generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD, pages 2623–2631.

Baeza-Yates, R., Ribeiro-Neto, B., et al. (1999). Modern information retrieval, volume 463. ACM press New York.

Bergstra, J.S., Bardenet, R., Bengio, Y., and Kégl,B.(2011). Algorithms for hyper-parameter optimization. In NeurIPS, pages 2546–2554.

Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and De Freitas, N. (2015). Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):148–175.

Sprent, P. and Smeeton, N. C. (2016). Applied nonparametric statistical methods. CRC press.
Publicado
28/09/2020
CAVALCANTE, Larícia; LIMA, Ullayne; BARBOSA, Luciano; GOMES, Ana Luiza; SANTANA, Éden; MARTINS, Thiago. Improving Search Quality with Automatic Ranking Evaluation and Tuning. In: SIMPÓSIO BRASILEIRO DE BANCO DE DADOS (SBBD), 35. , 2020, Evento Online. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2020 . p. 157-162. ISSN 2763-8979. DOI: https://doi.org/10.5753/sbbd.2020.13634.