Análise de Eficácia baseada em Diversidade para Sistemas de Recuperação de Informação
Abstract
Information retrieval systems are generally evaluated considering ranking relevance only. However, this does not always allow us to assess how good the result was, considering the user's intent. Thus, it is necessary to incorporate diversity measures into the assessment process. In this work we present extensions incorporated to AnalyzIR, which is a practical tool for analyzing the effectiveness of information retrieval systems. By default, the tool only provided relevance analysis, but by aggregating frameworks and diversity measures, it became an integrated and intuitive platform that considers both the relevance and the diversity of the results.References
Agrawal, R., Gollapudi, S., Halverson, A., and Ieong, S. (2009). Diversifying search results. In International Conference on WSDM, page 5–14, New York, NY, USA.
Clarke, C. L., Kolla, M., Cormack, G. V., Vechtomova, O., Ashkan, A., Büttcher, S., and MacKinnon, I. (2008). Novelty and diversity in information retrieval evaluation. In ACM SIGIR, page 659–666, New York, NY, USA.
Rodrygo, L. S., Macdonald, C., and Ounis, I. (2015). Search result diversification. Foundations and Trends in Information Retrieval, 9(1):1–90.
Sakai, T. (2011). Ntcireval: A generic toolkit for information access evaluation. In Proceedings of the forum on information technology, volume 2, pages 23–30.
Sakai, T., Craswell, N., Song, R., Robertson, S., Dou, Z., and yew Lin, C. (2010). Simple evaluation metrics for diversified search results. In EVIA, pages 42–50.
Xu, J., Xia, L., Lan, Y., Guo, J., and Cheng, X. (2017). Directly optimize diversity evaluation measures: A new approach to search result diversification. ACM Transactions on Intelligent Systems and Technology, 8(3).
Zhai, C., Cohen, W. W., and Lafferty, J. D. (2015). Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. SIGIR Forum, 49(1):2–9.
Clarke, C. L., Kolla, M., Cormack, G. V., Vechtomova, O., Ashkan, A., Büttcher, S., and MacKinnon, I. (2008). Novelty and diversity in information retrieval evaluation. In ACM SIGIR, page 659–666, New York, NY, USA.
Rodrygo, L. S., Macdonald, C., and Ounis, I. (2015). Search result diversification. Foundations and Trends in Information Retrieval, 9(1):1–90.
Sakai, T. (2011). Ntcireval: A generic toolkit for information access evaluation. In Proceedings of the forum on information technology, volume 2, pages 23–30.
Sakai, T., Craswell, N., Song, R., Robertson, S., Dou, Z., and yew Lin, C. (2010). Simple evaluation metrics for diversified search results. In EVIA, pages 42–50.
Xu, J., Xia, L., Lan, Y., Guo, J., and Cheng, X. (2017). Directly optimize diversity evaluation measures: A new approach to search result diversification. ACM Transactions on Intelligent Systems and Technology, 8(3).
Zhai, C., Cohen, W. W., and Lafferty, J. D. (2015). Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. SIGIR Forum, 49(1):2–9.
Published
2021-10-25
How to Cite
PEIXOTO, Raul do Carmo; FIGUERÊDO, José Solenir L.; CALUMBY, Rodrigo Tripodi.
Análise de Eficácia baseada em Diversidade para Sistemas de Recuperação de Informação. In: REGIONAL SCHOOL ON COMPUTING OF BAHIA, ALAGOAS, AND SERGIPE (ERBASE), 21. , 2021, Maceió.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 134-139.
DOI: https://doi.org/10.5753/erbase.2021.20065.
