Min, Sewon;
Boyd-Graber, Jordan;
Alberti, Chris;
Chen, Danqi;
Choi, Eunsol;
Collins, Michael;
Guu, Kelvin;
... Yih, Wen-tau; + view all
(2021)
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned.
Proceedings of Machine Learning Research
, 133
pp. 86-111.
Preview |
Text
min21a.pdf - Published Version Download (2MB) | Preview |
Abstract
We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and return natural language answers. The aim of the competition was to build systems that can predict correct answers while also satisfying strict on-disk memory budgets. These memory budgets were designed to encourage contestants to explore the trade-off between storing retrieval corpora or the parameters of learned models. In this report, we describe the motivation and organization of the competition, review the best submissions, and analyze system predictions to inform a discussion of evaluation for open-domain QA.
| Type: | Article |
|---|---|
| Title: | NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.mlr.press/v133/min21a.html |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher's terms and conditions. |
| Keywords: | Question answering, Memory efficiency, Knowledge representation |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10151983 |
Archive Staff Only
 |
View Item |
