UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

A Geometrical Analysis of Global Stability in Trained Feedback Networks

Mastrogiuseppe, F; Ostojic, S; (2019) A Geometrical Analysis of Global Stability in Trained Feedback Networks. Neural Computation , 31 (6) pp. 1139-1182. 10.1162/neco_a_01187. Green open access

[thumbnail of NECO-09-18-3251R2-source.pdf]
Preview
Text
NECO-09-18-3251R2-source.pdf - Accepted version

Download (5MB) | Preview

Abstract

Recurrent neural networks have been extensively studied in the context of neuroscience and machine learning due to their ability to implement complex computations. While substantial progress in designing effective learning algorithms has been achieved in the last years, a full understanding of trained recurrent networks is still lacking. Specifically, the mechanisms that allow computations to emerge from the underlying recurrent dynamics are largely unknown. Here we focus on a simple, yet underexplored computational setup: a feedback architecture trained to associate a stationary output to a stationary input. As a starting point, we derive an approximate analytical description of global dynamics in trained networks which assumes uncorrelated connectivity weights in the feedback and in the random bulk. The resulting mean-field theory suggests that the task admits several classes of solutions, which imply different stability properties. Different classes are characterized in terms of the geometrical arrangement of the readout with respect to the input vectors, defined in the high-dimensional space spanned by the network population. We find that such approximate theoretical approach can be used to understand how standard training techniques implement the input-output task in finite-size feedback networks. In particular, our simplified description captures the local and the global stability properties of the target solution, and thus predicts training performance.

Type: Article
Title: A Geometrical Analysis of Global Stability in Trained Feedback Networks
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1162/neco_a_01187
Publisher version: https://doi.org/10.1162/neco_a_01187
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10072993
Downloads since deposit
72Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item