Jang, M;
Majumder, BP;
McAuley, J;
Lukasiewicz, T;
Camburu, OM;
(2023)
KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations.
In:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).
(pp. pp. 540-553).
Association for Computational Linguistics (ACL): Toronto, Canada.
Preview |
Text
Camburu_2023.acl-short.47.pdf Download (238kB) | Preview |
Abstract
While recent works have been considerably improving the quality of the natural language explanations (NLEs) generated by a model to justify its predictions, there is very limited research in detecting and alleviating inconsistencies among generated NLEs. In this work, we leverage external knowledge bases to significantly improve on an existing adversarial attack for detecting inconsistent NLEs. We apply our attack to high-performing NLE models and show that models with higher NLE quality do not necessarily generate fewer inconsistencies. Moreover, we propose an off-the-shelf mitigation method to alleviate inconsistencies by grounding the model into external background knowledge. Our method decreases the inconsistencies of previous high-performing NLE models as detected by our attack.
| Type: | Proceedings paper |
|---|---|
| Title: | KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations |
| Event: | 61st Annual Meeting of the Association for Computational Linguistics |
| ISBN-13: | 9781959429715 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.18653/v1/2023.acl-short.47 |
| Publisher version: | http://dx.doi.org/10.18653/v1/2023.acl-short.47 |
| Language: | English |
| Additional information: | ACL materials are Copyright © 1963–2023 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10178603 |
Archive Staff Only
 |
View Item |
