UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Differential Geometry Methods for Constructing Manifold-Targeted Recurrent Neural Networks

Claudi, F; Branco, T; (2022) Differential Geometry Methods for Constructing Manifold-Targeted Recurrent Neural Networks. Neural Computation , 34 (8) pp. 1790-1811. 10.1162/neco_a_01511. Green open access

[thumbnail of neco_a_01511.pdf]
Preview
Text
neco_a_01511.pdf - Published Version

Download (2MB) | Preview

Abstract

Neural computations can be framed as dynamical processes, whereby the structure of the dynamics within a neural network is a direct reflection of the computations that the network performs. A key step in generating mechanistic interpretations within this computation through dynamics framework is to establish the link among network connectivity, dynamics, and computation. This link is only partly understood. Recent work has focused on producing algorithms for engineering artificial recurrent neural networks (RNN) with dynamics targeted to a specific goal manifold. Some of these algorithms require only a set of vectors tangent to the target manifold to be computed and thus provide a general method that can be applied to a diverse set of problems. Nevertheless, computing such vectors for an arbitrary manifold in a high-dimensional state space remains highly challenging, which in practice limits the applicability of this approach. Here we demonstrate how topology and differential geometry can be leveraged to simplify this task by first computing tangent vectors on a low-dimensional topological manifold and then embedding these in state space. The simplicity of this procedure greatly facilitates the creation of manifold-targeted RNNs, as well as the process of designing task-solving, on-manifold dynamics. This new method should enable the application of network engineering–based approaches to a wide set of problems in neuroscience and machine learning. Our description of how fundamental concepts from differential geometry can be mapped onto different aspects of neural dynamics is a further demonstration of how the language of differential geometry can enrich the conceptual framework for describing neural dynamics and computation.

Type: Article
Title: Differential Geometry Methods for Constructing Manifold-Targeted Recurrent Neural Networks
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1162/neco_a_01511
Publisher version: https://doi.org/10.1162/neco_a_01511
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Algorithms, Language, Machine Learning, Neural Networks, Computer
UCL classification: UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > The Sainsbury Wellcome Centre
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL
URI: https://discovery.ucl.ac.uk/id/eprint/10153385
Downloads since deposit
171Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item