Biggs, F;
Guedj, B;
(2023)
Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty.
In:
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics.
(pp. pp. 8165-8182).
PMLR 206: Palau de Congressos, Valencia, Spain.
Preview |
Text
biggs23a.pdf - Published Version Download (440kB) | Preview |
Abstract
We introduce a modified version of the excess risk, which can be used to obtain empirically tighter, faster-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data examples to reduce the variance of its empirical counterpart, tightening the bound. We combine this with a new bound for [-1, 1]-valued (and potentially non-independent) signed losses, which is more favourable when they empirically have low variance around 0. The primary new technical tool is a novel result for sequences of interdependent random vectors which may be of independent interest. We empirically evaluate these new bounds on a number of real-world datasets.
| Type: | Proceedings paper |
|---|---|
| Title: | Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty |
| Event: | International Conference on Artificial Intelligence and Statistics |
| Location: | Valencia, Spain |
| Dates: | 25 Apr 2023 - 27 Apr 2023 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.mlr.press/v206/biggs23a.html |
| Language: | English |
| Additional information: | This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10175465 |
Archive Staff Only
 |
View Item |
