Shi, Zhengxiang; Lipani, Aldo; (2023) Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysis. In: ESANN 2023 proceedings: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. (pp. pp. 169-174). ESANN (In press).

Preview

Text Lipani_ES2023-42.pdf Download (1MB) | Preview

Abstract

In recent years, language models (LMs) have made remarkable progress in advancing the field of natural language processing (NLP). However, the impact of data augmentation (DA) techniques on the fine-tuning (FT) performance of these LMs has been a topic of ongoing debate. In this study, we evaluate the effectiveness of three different FT methods in conjugation with back-translation across an array of 7 diverse NLP tasks, including classification and regression types, covering single-sentence and sentence-pair tasks. Contrary to prior assumptions that DA does not contribute to the enhancement of LMs' FT performance, our findings reveal that continued pre-training on augmented data can effectively improve the FT performance of the downstream tasks. In the most favourable case, continued pre-training improves the performance of FT by more than 10% in the few-shot learning setting. Our finding highlights the potential of DA as a powerful tool for bolstering LMs' performance.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item