Zhuo, Terry Yue; Xu, Qiongkai; He, Xuanli; Cohn, Trevor; (2023) Rethinking Round-Trip Translation for Machine Translation Evaluation. In: Rogers, Anna and Boyd-Graber, Jordan and Okazaki, Naoaki, (eds.) Proceedings of the Annual Meeting of the Association for Computational Linguistics: ACL 2023. (pp. pp. 319-337). Association for Computational Linguistics: Toronto, Canada.

Preview

PDF 2023.findings-acl.22.pdf - Published Version Download (788kB) | Preview

Abstract

Automatic evaluation methods for translation often require model training, and thus the availability of parallel corpora limits their applicability to low-resource settings. Round-trip translation is a potential workaround, which can reframe bilingual evaluation into a much simpler monolingual task. Early results from the era of statistical machine translation (SMT) raised fundamental concerns about the utility of this approach, based on poor correlation with human translation quality judgments. In this paper, we revisit this technique with modern neural translation (NMT) and show that round-trip translation does allow for accurate automatic evaluation without the need for reference translations. These opposite findings can be explained through the copy mechanism in SMT that is absent in NMT. We demonstrate that round-trip translation benefits multiple machine translation evaluation tasks: i) predicting forward translation scores; ii) improving the performance of a quality estimation model; and iii) identifying adversarial competitors in shared tasks via cross-system verification.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item