Biggs, Felix;
(2024)
Exploring Generalisation Performance through PAC-Bayes.
Doctoral thesis (Ph.D), UCL (University College London).
Preview |
Text
thesis-main.pdf - Other Download (4MB) | Preview |
Abstract
Generalisation in machine learning refers to the ability of a predictor learned on some dataset to perform accurately on new, unseen data. Without generalisation, we might be able to memorise the training data perfectly while predicting poorly on new data, a pathology known as over-fitting. Despite its centrality, the generalisation behaviour of many methods remains poorly understood, particularly in complex domains such as deep learning. Indeed, some models that should over-fit according to traditional theoretical bounds do not. This thesis addresses these issues, particularly in the context of classification, and introduces innovative methods for producing non-vacuous generalisation bounds. The primary thrust is in the development and application of PAC-Bayesian bounds, which are usually used as a method for studying the generalisation of randomised predictors. We begin with an introduction to previous work on the problem of generalisation and to PAC-Bayesian ideas, before applying these in work based on a series of five papers (referenced a-e below). Firstly in (a), we provide lower-variance methods for training stochastic neural networks methods, improving the use of these PAC-Bayes bounds as training objectives. Then, we use PAC-Bayes as a stepping stone to provide non-randomised bounds: (b) using margins, both in general and for several different classifiers; (c) for a specific class of deterministic shallow neural networks (where our bounds are the first to be non-vacuous on real-world data using standard training methods); (d) for majority voting on finite ensembles of classifiers, providing state-of-the-art (and sometimes sharp) guarantees. Lastly in (e), we introduce a PAC-Bayes bound for a modified excess risk, using information about the relative hardness of data examples to reduce variance and tighten a general bound.
| Type: | Thesis (Doctoral) |
|---|---|
| Qualification: | Ph.D |
| Title: | Exploring Generalisation Performance through PAC-Bayes |
| Open access status: | An open access version is available from UCL Discovery |
| Language: | English |
| Additional information: | Copyright © The Author 2023. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10193715 |
Archive Staff Only
 |
View Item |
