Haddouche, M;
Guedj, B;
Rivasplata, O;
Shawe-Taylor, J;
(2021)
PAC-Bayes Unleashed: Generalisation Bounds with Unbounded Losses.
Entropy
, 23
(10)
, Article 1330. 10.3390/e23101330.
Preview |
Text
Rivasplata_entropy-23-01330.pdf - Published Version Download (434kB) | Preview |
Abstract
We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems with a bounded loss function (typically assumed to take values in the interval [0;1]). In order to relax this classical assumption, we propose to allow the range of the loss to depend on each predictor. This relaxation is captured by our new notion of HYPothesis-dependent rangE (HYPE). Based on this, we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions.
| Type: | Article |
|---|---|
| Title: | PAC-Bayes Unleashed: Generalisation Bounds with Unbounded Losses |
| Location: | Switzerland |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.3390/e23101330 |
| Publisher version: | https://doi.org/10.3390/e23101330 |
| Language: | English |
| Additional information: | This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
| Keywords: | PAC-Bayes, generalisation bounds, statistical learning theory |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Mathematics |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10137498 |
Archive Staff Only
 |
View Item |
