UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Natural language instructions induce compositional generalization in networks of neurons

Riveland, Reidar; Pouget, Alexandre; (2024) Natural language instructions induce compositional generalization in networks of neurons. Nature Neuroscience , 27 pp. 988-999. 10.1038/s41593-024-01607-5. Green open access

[thumbnail of s41593-024-01607-5.pdf]
Preview
Text
s41593-024-01607-5.pdf - Published Version

Download (5MB) | Preview

Abstract

A fundamental human cognitive feat is to interpret linguistic instructions in order to perform novel tasks without explicit task experience. Yet, the neural computations that might be used to accomplish this remain poorly understood. We use advances in natural language processing to create a neural model of generalization based on linguistic instructions. Models are trained on a set of common psychophysical tasks, and receive instructions embedded by a pretrained language model. Our best models can perform a previously unseen task with an average performance of 83% correct based solely on linguistic instructions (that is, zero-shot learning). We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic representations of instructions, allowing language to cue the proper composition of practiced skills in unseen settings. We show how this model generates a linguistic description of a novel task it has identified using only motor feedback, which can subsequently guide a partner model to perform the task. Our models offer several experimentally testable predictions outlining how linguistic information must be represented to facilitate flexible and general cognition in the human brain.

Type: Article
Title: Natural language instructions induce compositional generalization in networks of neurons
Open access status: An open access version is available from UCL Discovery
DOI: 10.1038/s41593-024-01607-5
Publisher version: https://doi.org/10.1038/s41593-024-01607-5
Language: English
Additional information: © The Author(s), 2025. This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. https://creativecommons.org/licenses/by/4.0/
Keywords: Intelligence, Language, Network models
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10211884
Downloads since deposit
2Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item