TY - JOUR AU - Veloso, Adriano AU - Gonçalves, Marcos AU - Meira Jr., Wagner PY - 2011/10/04 Y2 - 2024/03/29 TI - Competence-Conscious Associative Rank Aggregation JF - Journal of Information and Data Management JA - JIDM VL - 2 IS - 3 SE - SBBD Articles DO - 10.5753/jidm.2011.1413 UR - https://sol.sbc.org.br/journals/index.php/jidm/article/view/1413 SP - 337 AB - The ultimate goal of ranking methods is to achieve the best possible ranking performance for the problem at<br />hand. Recently, a body of empirical evidence has emerged suggesting that methods that learn to rank offer substantial<br />improvements in enough situations to be regarded as a relevant advance for applications that depend on ranking.<br />Previous studies have shown that different (learning to rank) methods may produce conflicting ranked lists. Rank<br />aggregation is based on the idea that combining such lists may provide complementary information that can be used<br />to improve ranking performance. In this paper we investigate learning to rank methods that uncover, from the training<br />data, associations between document features and relevance levels in order to estimate the relevance of documents with<br />regard to a given query. There is a variety of statistic measures or metrics that provide a different interpretation for an<br />association. Interestingly, we observed that each association metric has a specific domain for which it is most competent<br />(that is, there is a specific set of documents for which a specific metric consistently produces better ranked lists). We<br />employ a second-stage meta-learning approach, which describes the domain of competence of each metric, enabling a<br />more sensible aggregation of the ranked lists produced by different metrics. We call this new aggregation paradigm<br />competence-conscious associative rank aggregation. We conducted a systematic evaluation of competence-conscious<br />aggregation methods using the LETOR 3.0 benchmark collections. We demonstrate that the proposed aggregation<br />methods outperform the constituent learning to rank methods not only when they are considered in isolation, but also<br />when they are combined using existing aggregation approaches. ER -