UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

On the Stability and Scalability of Node Perturbation Learning

Hiratani, naoki; Mehta, Yash; Lillicrap, Timothy; Latham, Peter; (2023) On the Stability and Scalability of Node Perturbation Learning. In: Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Neural Information Processing Systems (NeurIPS) (In press). Green open access

[thumbnail of 10935_on_the_stability_and_scalabili.pdf]
Preview
Text
10935_on_the_stability_and_scalabili.pdf - Accepted Version

Download (2MB) | Preview

Abstract

To survive, animals must adapt synaptic weights based on external stimuli and rewards. And they must do so using local, biologically plausible, learning rules – a highly nontrivial constraint. One possible approach is to perturb neural activity (or use intrinsic, ongoing noise to perturb it), determine whether performance increases or decreases, and use that information to adjust the weights. This algorithm – known as node perturbation – has been shown to work on simple problems, but little is known about either its stability or its scalability with respect to network size. We investigate these issues both analytically, in deep linear networks, and numerically, in deep nonlinear ones. We show analytically that in deep linear networks with one hidden layer, both learning time and performance depend very weakly on hidden layer size. However, unlike stochastic gradient descent, when there is model mismatch between the student and teacher networks, node perturbation is always unstable. The instability is triggered by weight diffusion, which eventually leads to very large weights. This instability can be suppressed by weight normalization, at the cost of bias in the learning rule. We confirm numerically that a similar instability, and to a lesser extent scalability, exist in deep nonlinear networks trained on both a motor control task and image classification tasks. Our study highlights the limitations and potential of node perturbation as a biologically plausible learning rule in the brain.

Type: Proceedings paper
Title: On the Stability and Scalability of Node Perturbation Learning
Event: Thirty-Sixth Conference on Neural Information Processing Systems
Location: New Orleans, US
Dates: 28 Nov 2022 - 9 Dec 2022
Open access status: An open access version is available from UCL Discovery
Publisher version: https://nips.cc/
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10166318
Downloads since deposit
553Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item