Riveland, Reidar;
Pouget, Alexandre;
(2024)
Natural language instructions induce compositional generalization in networks of neurons.
Nature Neuroscience
, 27
pp. 988-999.
10.1038/s41593-024-01607-5.
Preview |
Text
s41593-024-01607-5.pdf - Published Version Download (5MB) | Preview |
Abstract
A fundamental human cognitive feat is to interpret linguistic instructions in order to perform novel tasks without explicit task experience. Yet, the neural computations that might be used to accomplish this remain poorly understood. We use advances in natural language processing to create a neural model of generalization based on linguistic instructions. Models are trained on a set of common psychophysical tasks, and receive instructions embedded by a pretrained language model. Our best models can perform a previously unseen task with an average performance of 83% correct based solely on linguistic instructions (that is, zero-shot learning). We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic representations of instructions, allowing language to cue the proper composition of practiced skills in unseen settings. We show how this model generates a linguistic description of a novel task it has identified using only motor feedback, which can subsequently guide a partner model to perform the task. Our models offer several experimentally testable predictions outlining how linguistic information must be represented to facilitate flexible and general cognition in the human brain.
Type: | Article |
---|---|
Title: | Natural language instructions induce compositional generalization in networks of neurons |
Open access status: | An open access version is available from UCL Discovery |
DOI: | 10.1038/s41593-024-01607-5 |
Publisher version: | https://doi.org/10.1038/s41593-024-01607-5 |
Language: | English |
Additional information: | © The Author(s), 2025. This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. https://creativecommons.org/licenses/by/4.0/ |
Keywords: | Intelligence, Language, Network models |
UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit |
URI: | https://discovery.ucl.ac.uk/id/eprint/10211884 |
Archive Staff Only
![]() |
View Item |