UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Deep Learning Models of Learning in the Brain

Pogodin, Roman; (2023) Deep Learning Models of Learning in the Brain. Doctoral thesis (Ph.D), UCL (University College London). Green open access

[thumbnail of PhD_Thesis-FINAL.pdf]
Preview
Text
PhD_Thesis-FINAL.pdf - Other

Download (5MB) | Preview

Abstract

This thesis considers deep learning theories of brain function, and in particular biologically plausible deep learning. The idea is to treat a standard deep network as a high-level model of a neural circuit (e.g., the visual stream), adding biological constraints to some clearly artificial features. Two big questions are possible. First, how to train deep networks in a biologically realistic manner? The standard approach, supervised training via backpropagation, needs overly complicated machinery for backpropagation and precise labels (that are somewhat scarce in the real world). The first result in this thesis approaches the first problem, backpropagation, by avoiding it completely. A layer-wise objective is proposed, which results in local, Hebbian weight updates that use a global error signal. The second result approaches the need for precise labels. It is focused on a principled approach to self-supervised learning, framing the problem as dependence maximisation using kernel methods. Although this is a deep learning study, it is relevant to neuroscience: self-supervised learning appears to be a suitable learning paradigm for the brain as it only requires binary (same source or not) teaching signals for pairs of inputs. Second, how realistic is the architecture itself? For instance, most well-performing networks have some form of weight sharing - having the same weights for different neurons at all times. Convolutional networks share filter weights among neurons, and transformers do so for matrix-matrix products. While the operation is biologically implausible, the third result of this thesis shows that it can be successfully approximated with a separate phase of weight-sharing-inducing Hebbian learning.

Type: Thesis (Doctoral)
Qualification: Ph.D
Title: Deep Learning Models of Learning in the Brain
Open access status: An open access version is available from UCL Discovery
Language: English
Additional information: Copyright © The Author 2022. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10163891
Downloads since deposit
283Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item