Wu, Y;
Minervini, P;
Stenetorp, P;
Riedel, S;
(2021)
Training Adaptive Computation for Open-Domain Question Answering with Computational Constraints.
In:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers).
(pp. pp. 447-453).
Association for Computational Linguistics
Preview |
Text
2021.acl-short.57.pdf - Published Version Download (295kB) | Preview |
Abstract
Adaptive Computation (AC) has been shown to be effective in improving the efficiency of Open-Domain Question Answering (ODQA) systems. However, the current AC approaches require tuning of all model parameters, and training state-of-the-art ODQA models requires significant computational resources that may not be available for most researchers. We propose Adaptive Passage Encoder, an AC method that can be applied to an existing ODQA model and can be trained efficiently on a single GPU. It keeps the parameters of the base ODQA model fixed, but it overrides the default layer-by-layer computation of the encoder with an AC policy that is trained to optimise the computational efficiency of the model. Our experimental results show that our method improves upon a state-of-the-art model on two datasets, and is also more accurate than previous AC methods due to the stronger base ODQA model. All source code and datasets are available at https://github.com/uclnlp/APE.
| Type: | Proceedings paper |
|---|---|
| Title: | Training Adaptive Computation for Open-Domain Question Answering with Computational Constraints |
| Event: | Joint Conference of 59th Annual Meeting of the Association-for-Computational-Linguistics (ACL) / 11th International Joint Conference on Natural Language Processing (IJCNLP) / 6th Workshop on Representation Learning for NLP (RepL4NLP) |
| Location: | ELECTR NETWORK |
| Dates: | 01 August 2021 - 06 August 2021 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.18653/v1/2021.acl-short.57 |
| Publisher version: | https://aclanthology.org/2021.acl-short.57/ |
| Language: | English |
| Additional information: | ACL materials are Copyright © 1963–2021 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License. |
| Keywords: | Science & Technology, Social Sciences, Technology, Computer Science, Artificial Intelligence, Computer Science, Interdisciplinary Applications, Linguistics, Computer Science |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10136198 |
Archive Staff Only
 |
View Item |
