UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

The Transient Nature of Emergent In-context Learning in Transformers

Singh, aaditya; Chan, Stephanie CY; Moskovitz, Ted; Grant, erin; Saxe, Andrew; Hill, Felix; (2023) The Transient Nature of Emergent In-context Learning in Transformers. In: Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023). NeurIPS (In press). Green open access

[thumbnail of 8706_the_transient_nature_of_emerge.pdf]
Preview
Text
8706_the_transient_nature_of_emerge.pdf - Accepted Version

Download (4MB) | Preview

Abstract

Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL), despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e.g., through the lens of mechanistic interpretability, Bayesian inference, or by examining the distributional properties of training data. However, in each of these cases, ICL is treated largely as a persistent phenomenon; namely, once ICL emerges, it is assumed to persist asymptotically. Here, we show that the emergence of ICL during transformer training is, in fact, often transient. We train transformers on synthetic data designed so that both ICL or in-weights learning (IWL) strategies can lead to correct predictions. We find that ICL first emerges, then disappears and gives way to IWL, all while the training loss decreases, indicating an asymptotic preference for IWL. The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to “overtrain” transformers when seeking compact, cheaper-to-run models. We find that L2 regularization may offer a path to more persistent ICL that removes the need for early stopping based on ICL-style validation tasks.

Type: Proceedings paper
Title: The Transient Nature of Emergent In-context Learning in Transformers
Event: 37th Conference on Neural Information Processing Systems (NeurIPS 2023)
Location: New Orleans, LA, USA
Dates: 10th-16th December 2023
Open access status: An open access version is available from UCL Discovery
Publisher version: https://openreview.net/forum?id=Of0GBzow8P
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions.
Keywords: in-context learning, transformers, emergence, transience
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10181228
Downloads since deposit
90Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item