Zhu, J and Wang, J and Cox, I and Vinay, V (2009) Topic (Query) Selection for IR Evaluation. In: Sanderson, M and Zhai, CX and Zobel, J and Allan, J and Aslam, JA, (eds.) PROCEEDINGS 32ND ANNUAL INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL. (pp. 802 - 803). ASSOC COMPUTING MACHINERY
Full text not available from this repository.
Abstract
The need for evaluating large amounts of topics (queries) makes IR evaluation an uneasy task. In this paper, we study a topic selection problem for IR evaluation. The selection criterion is based on the overall difficulty of the chosen set, as well as the uncertainty of the final IR metric applied to the systems. Our preliminary experiments demonstrate that our approach helps to identify a set of topics that provides confident estimates of systems' performance while keeping the requirement of the query difficulty.
| Type: | Proceedings paper |
|---|---|
| Title: | Topic (Query) Selection for IR Evaluation |
| Event: | 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval |
| Location: | Boston, MA |
| Dates: | 2009-07-19 - 2009-07-23 |
| ISBN-13: | 978-1-60558-483-6 |
| UCL classification: | UCL > School of BEAMS > Faculty of Engineering Science > Computer Science |
Archive Staff Only: edit this record