Biggs, Felix;
Guedj, Benjamin;
(2022)
Non-Vacuous Generalisation Bounds for Shallow Neural Networks.
In:
Proceedings of the 39th International Conference on Machine Learning.
(pp. pp. 1963-1981).
MLResearchPress
Preview |
Text
Guedj_biggs22a.pdf Download (562kB) | Preview |
Abstract
We focus on a specific class of shallow neural networks with a single hidden layer, namely those with L2-normalised data and either a sigmoid-shaped Gaussian error function (“erf”) activation or a Gaussian Error Linear Unit (GELU) activation. For these networks, we derive new generalisation bounds through the PAC-Bayesian theory; unlike most existing such bounds they apply to neural networks with deterministic rather than randomised parameters. Our bounds are empirically non-vacuous when the network is trained with vanilla stochastic gradient descent on MNIST and Fashion-MNIST.
| Type: | Proceedings paper |
|---|---|
| Title: | Non-Vacuous Generalisation Bounds for Shallow Neural Networks |
| Event: | 39th International Conference on Machine Learning |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.mlr.press/v162/biggs22a.html |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10143073 |
Archive Staff Only
 |
View Item |
