UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Fixed budget pooling strategies based on fusion methods

Lipani, A; Lupu, M; Palotti, J; Zuccon, G; Hanbury, A; (2017) Fixed budget pooling strategies based on fusion methods. In: Shin, SY and Shin, D and Lencastre, M, (eds.) SAC '17: Proceedings of the Symposium on Applied Computing. (pp. pp. 919-924). ACM Green open access

[img]
Preview
Text
paper.pdf - Accepted version

Download (382kB) | Preview

Abstract

The empirical nature of Information Retrieval (IR) mandates strong experimental practices. The Cranfield/TREC evaluation paradigm represents a keystone of such experimental practices. Within this paradigm, the generation of relevance judgments has been the subject of intense scientific investigation. This is because, on one hand, consistent, precise and numerous judgements are key to reduce evaluation uncertainty and test collection bias; on the other hand, however, relevance judgements are costly to collect. The selection of which documents to judge for relevance (known as pooling) has therefore great impact in IR evaluation. In this paper, we contribute a set of 8 novel pooling strategies based on retrieval fusion methods. We show that the choice of the pooling strategy has significant effects on the cost needed to obtain an unbiased test collection; we also identify the best performing pooling strategy according to three evaluation measure.

Type: Proceedings paper
Title: Fixed budget pooling strategies based on fusion methods
Event: SAC 2017, 32nd ACM SIGAPP Symposium On Applied Computing, 3-7 April 2017, Marrakech, Morocco
ISBN-13: 9781450344869
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/3019612.3019692
Publisher version: https://doi.org/10.1145/3019612.3019692
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Pooling Method, Pooling Strategies, Pool Bias
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Civil, Environ and Geomatic Eng
URI: https://discovery.ucl.ac.uk/id/eprint/10059982
Downloads since deposit
51Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item