Avaliação qualitativa da consulta Espaço-Textual Top-k
Resumo
O número de pesquisas relacionas à consulta Espaço-Textual Top-k aumentou nos últimos anos. Isso se deve ao volume de dados com informação espacial (latitude e longitude) na Internet, o que gera necessidade de criação de métodos de busca eficientes. A maioria dos artigos que tratam este problema focam na eficiência dos métodos de busca, no entanto, é necessário avaliar também a eficácia da consulta no que se refere à relevância dos documentos recuperados para o usuário. Este artigo descreve uma metodologia para avaliar qualitativamente a consulta Espaço-Textual Top-k, e para criar coleções de referência espaço-textuais adaptadas de coleções tradicionais existentes, como a coleção Reuters-21578. Os testes realizados avaliaram as duas funções de ranqueamento existentes e indicaram que o balanceamento entre a relevância textual e a distância espacial é crucial para atingir melhores resultados.
Palavras-chave:
Ranking funcional, Avaliação Ecacy, Query Espaço-textual
Referências
R. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval - the concepts and technology behind search, Second edition. Pearson Education Ltd., Harlow, England, 2011.
R. Baeza-Yates and B. Ribeiro-Neto. Modern information retrieval. ACM Press. New York, 2013.
X. C. Cao, G. Cong, C. S. Jensen, Q. Qu, A. Skovsgaard, D. Wu, and M. L. Yiu. Spatial keyword querying. Er, 7532(1):16–29, 2012.
B. Carterette and J. Allan. Incremental test collections. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management, CIKM ’05, pages 680–687, New York, NY, USA, 2005. ACM.
O. Chapelle, T. Joachims, F. Radlinski, and Y. Yue. Large-scale validation and analysis of interleaved search evaluation. ACM Trans. Inf. Syst., 30(1):6:1–6:41, Mar. 2012.
O. Chapelle, D. Metlzer, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM ’09, pages 621–630, New York, NY, USA, 2009. ACM.
L. Chen, G. Cong, C. S. Jensen, and D. Wu. Spatial keyword query processing: An experimental evaluation. Proceedings of the VLDB Endowment, 6(3):217–228, 2013.
C. W. Cleverdon. The significance of the cranfield tests on index languages. Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, pages 3–12, 1991.
G. Cong, C. S. Jesen, and D. Wu. Ecient retrieval of the top-k most relevant spatial web objects. Proceedings of the VLDB Endowment, 2(1):337–348, 2009.
A. GAűker and H. Myrhaug. Evaluation of a mobile ˜ information system in context. Pergamon Press, 44(1):39–65, 2008.
K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst., 20(4):422–446, Oct. 2002.
K. S. Jones. Information Retrieval Experiment. Butterworth-Heinemann Newton, MA, USA, 1981.
G. Kazai, N. GÂűvert, M. Lalmas, and N. Fuhr. The inex evaluation initiative. INtelligent Search on XML Data, 2818(3):279–293, 2003.
D. D. Lewis. Representation and learning in information retrieval. PhD thesis, Department of Computer and Information Science, University of Massachusetts, Amherst, 1992. UMI Order No. GAX92-19460.
D. D. Lewis. Reuters-21578 text categorization text collection, 2004. http://www.daviddlewis.com/resources/testcollections/reuters21578/. [Online; acessado 19-maio-2016].
C. D. Manning, P. Raghavan, and H. Shutze. Introduction to Information Retrieval. Cambridge University Press., 2008.
Reuters-21578. Reuters-21578 test collection 2006, 2006. [Online; acessado 23-setembro-2016].
C. J. V. Rijsbergen. Information Retrieval. Butterworth-Heinemann Newton, MA, USA, 1979.
J. B. Rocha-Junior and K. NÃÿrvag. Top-k spatial keyword queries on road networks. Proceedings of the 15th International Conference on Extending Database Technology - EDBT 12, Norwegian University of Science and Technology, 2012.
G. Salton and M. J. McGill. Introduction to Modern Information Retrieval. McGraw Hill Book Co., 1986.
M. Sanderson. Reuters test collection. Technical report, Proceedings of the Sixteenth Research Colloquium of the British Computer Society Information Retrieval Specialist Group, Drymen, 1994.
M. Sanderson and H. Joho. Forming test collections with no system pooling. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’04, pages 33–40, New York, NY, USA, 2004. ACM.
E. M. Voorhees and D. K. Harman. TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing). The MIT Press, 2005.
R. Wilkinson and M. Wu. Evaluation experiments and experience from the perspective of interactive information retrieval. the Proceedings of the Third Workshop on Empirical of Adaptive Systems, pages 23–26, 2004.
E. Yilmaz, M. Shokouhi, N. Craswell, and S. Robertson. Expected browsing utility for web search evaluation. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM ’10, pages 1561–1564, New York, NY, USA, 2010. ACM.
J. Zobel and A. Moffat. Inverted files for text search engines. ACM Comput. Surv., 38(2), July 2006.
R. Baeza-Yates and B. Ribeiro-Neto. Modern information retrieval. ACM Press. New York, 2013.
X. C. Cao, G. Cong, C. S. Jensen, Q. Qu, A. Skovsgaard, D. Wu, and M. L. Yiu. Spatial keyword querying. Er, 7532(1):16–29, 2012.
B. Carterette and J. Allan. Incremental test collections. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management, CIKM ’05, pages 680–687, New York, NY, USA, 2005. ACM.
O. Chapelle, T. Joachims, F. Radlinski, and Y. Yue. Large-scale validation and analysis of interleaved search evaluation. ACM Trans. Inf. Syst., 30(1):6:1–6:41, Mar. 2012.
O. Chapelle, D. Metlzer, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM ’09, pages 621–630, New York, NY, USA, 2009. ACM.
L. Chen, G. Cong, C. S. Jensen, and D. Wu. Spatial keyword query processing: An experimental evaluation. Proceedings of the VLDB Endowment, 6(3):217–228, 2013.
C. W. Cleverdon. The significance of the cranfield tests on index languages. Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, pages 3–12, 1991.
G. Cong, C. S. Jesen, and D. Wu. Ecient retrieval of the top-k most relevant spatial web objects. Proceedings of the VLDB Endowment, 2(1):337–348, 2009.
A. GAűker and H. Myrhaug. Evaluation of a mobile ˜ information system in context. Pergamon Press, 44(1):39–65, 2008.
K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst., 20(4):422–446, Oct. 2002.
K. S. Jones. Information Retrieval Experiment. Butterworth-Heinemann Newton, MA, USA, 1981.
G. Kazai, N. GÂűvert, M. Lalmas, and N. Fuhr. The inex evaluation initiative. INtelligent Search on XML Data, 2818(3):279–293, 2003.
D. D. Lewis. Representation and learning in information retrieval. PhD thesis, Department of Computer and Information Science, University of Massachusetts, Amherst, 1992. UMI Order No. GAX92-19460.
D. D. Lewis. Reuters-21578 text categorization text collection, 2004. http://www.daviddlewis.com/resources/testcollections/reuters21578/. [Online; acessado 19-maio-2016].
C. D. Manning, P. Raghavan, and H. Shutze. Introduction to Information Retrieval. Cambridge University Press., 2008.
Reuters-21578. Reuters-21578 test collection 2006, 2006. [Online; acessado 23-setembro-2016].
C. J. V. Rijsbergen. Information Retrieval. Butterworth-Heinemann Newton, MA, USA, 1979.
J. B. Rocha-Junior and K. NÃÿrvag. Top-k spatial keyword queries on road networks. Proceedings of the 15th International Conference on Extending Database Technology - EDBT 12, Norwegian University of Science and Technology, 2012.
G. Salton and M. J. McGill. Introduction to Modern Information Retrieval. McGraw Hill Book Co., 1986.
M. Sanderson. Reuters test collection. Technical report, Proceedings of the Sixteenth Research Colloquium of the British Computer Society Information Retrieval Specialist Group, Drymen, 1994.
M. Sanderson and H. Joho. Forming test collections with no system pooling. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’04, pages 33–40, New York, NY, USA, 2004. ACM.
E. M. Voorhees and D. K. Harman. TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing). The MIT Press, 2005.
R. Wilkinson and M. Wu. Evaluation experiments and experience from the perspective of interactive information retrieval. the Proceedings of the Third Workshop on Empirical of Adaptive Systems, pages 23–26, 2004.
E. Yilmaz, M. Shokouhi, N. Craswell, and S. Robertson. Expected browsing utility for web search evaluation. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM ’10, pages 1561–1564, New York, NY, USA, 2010. ACM.
J. Zobel and A. Moffat. Inverted files for text search engines. ACM Comput. Surv., 38(2), July 2006.
Publicado
17/05/2017
Como Citar
NETO, Luiz; ROCHA-JUNIOR, João; CALUMBY, Rodrigo.
Avaliação qualitativa da consulta Espaço-Textual Top-k. In: SIMPÓSIO BRASILEIRO DE SISTEMAS DE INFORMAÇÃO (SBSI), 13. , 2017, Lavras.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2017
.
p. 80-87.
DOI: https://doi.org/10.5753/sbsi.2017.6029.
