Claudi, F;
Branco, T;
(2022)
Differential Geometry Methods for Constructing Manifold-Targeted Recurrent Neural Networks.
Neural Computation
, 34
(8)
pp. 1790-1811.
10.1162/neco_a_01511.
Preview |
Text
neco_a_01511.pdf - Published Version Download (2MB) | Preview |
Abstract
Neural computations can be framed as dynamical processes, whereby the structure of the dynamics within a neural network is a direct reflection of the computations that the network performs. A key step in generating mechanistic interpretations within this computation through dynamics framework is to establish the link among network connectivity, dynamics, and computation. This link is only partly understood. Recent work has focused on producing algorithms for engineering artificial recurrent neural networks (RNN) with dynamics targeted to a specific goal manifold. Some of these algorithms require only a set of vectors tangent to the target manifold to be computed and thus provide a general method that can be applied to a diverse set of problems. Nevertheless, computing such vectors for an arbitrary manifold in a high-dimensional state space remains highly challenging, which in practice limits the applicability of this approach. Here we demonstrate how topology and differential geometry can be leveraged to simplify this task by first computing tangent vectors on a low-dimensional topological manifold and then embedding these in state space. The simplicity of this procedure greatly facilitates the creation of manifold-targeted RNNs, as well as the process of designing task-solving, on-manifold dynamics. This new method should enable the application of network engineering–based approaches to a wide set of problems in neuroscience and machine learning. Our description of how fundamental concepts from differential geometry can be mapped onto different aspects of neural dynamics is a further demonstration of how the language of differential geometry can enrich the conceptual framework for describing neural dynamics and computation.
| Type: | Article |
|---|---|
| Title: | Differential Geometry Methods for Constructing Manifold-Targeted Recurrent Neural Networks |
| Location: | United States |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1162/neco_a_01511 |
| Publisher version: | https://doi.org/10.1162/neco_a_01511 |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Algorithms, Language, Machine Learning, Neural Networks, Computer |
| UCL classification: | UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > The Sainsbury Wellcome Centre UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10153385 |
Archive Staff Only
 |
View Item |
