WCL2R : a benchmark collection for Learning to rank research with clickthrough data.
No Thumbnail Available
Date
2010
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
WCL2R: A benchmark collection for Learning to rank research with clickthrough data In this paper we present WCL2R, a benchmark collection for supporting research in learning to rank (L2R) algorithms which exploit clickthrough features. Differently from other L2R benchmark collections, such as LETOR and the recently released Yahoo!’s collection for a L2R competition, in WCL2R we focus on defining a significant (and new) set of features over clickthrough data extracted from the logs of a real-world search engine. In this paper, we describe the WCL2R collection by providing details about how the corpora, queries and relevance judgments were obtained, how the learning features were constructed and how the process of splitting the collection in folds for representative learning was performed. We also analyze the discriminative power of the WCL2R collection using traditional feature selection algorithms and show that the most discriminative features are, in fact, those based on clickthrough data. We then compare several L2R algorithms on WCL2R, showing that all of them obtain significant gains by exploiting clickthrough information over using traditional ranking approaches.
Description
Keywords
Benchmark, Clicktrough, Learning to rank
Citation
ALCÂNTARA, O. D. A. WCL2R : a benchmark collection for Learning to rank research with clickthrough data. Journal of Information and Data Management, v. 1, n. 3, p. 551-566, 2010. Disponível em: <http://seer.lcc.ufmg.br/index.php/jidm/article/viewFile/83/49>. Acesso em: 11 out. 2012.