Adams, Reuben J.;
(2025)
Neural network generalisation in the overparameterised regime.
Doctoral thesis (Ph.D), UCL (University College London).
Preview |
Text
PhD_Thesis_with_Corrections.pdf - Submitted Version Download (9MB) | Preview |
Abstract
The generalisation mystery is the gap in our understanding of why commonly used Deep Learning algorithms produce neural networks that generalise to unseen data, even using large architectures with capacity far greater than that required to fit their training data exactly. Solving this puzzle would theoretically ground the astonishing empirical success of neural networks, potentially enabling them to be used with greater understanding and with quantitative performance guarantees. We make three contributions towards answering this question. First, we extend a classic PAC-Bayesian generalisation bound to provide more information-rich test time guarantees. Second, we demonstrate that PAC-Bayesian bounds on deterministic networks can be tightened by relating their performance to compressible neighbouring networks. Finally, we take a more empirical approach and show that the generalisation ability of a network is connected to its compressibility via distillation.
| Type: | Thesis (Doctoral) |
|---|---|
| Qualification: | Ph.D |
| Title: | Neural network generalisation in the overparameterised regime |
| Open access status: | An open access version is available from UCL Discovery |
| Language: | English |
| Additional information: | Copyright © The Author 2025. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10217499 |
Archive Staff Only
 |
View Item |
