Abbas, Alhabib;
Andreopoulos, Yiannis;
(2022)
PAC-Bayesian Bounds on Rate-Efficient Classifiers.
In: Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan, (eds.)
Proceedings of Machine Learning Research.
Proceedings of Machine Learning Research (PMLR): Baltimore, MA, USA.
Preview |
Text
Andreopoulos_abbas22a.pdf Download (1MB) | Preview |
Abstract
We derive analytic bounds on the noise invariance of majority vote classifiers operating on compressed inputs. Specifically, starting from recent bounds on the true risk of majority vote classifiers, we extend the applicability of PAC-Bayesian theory to quantify the resilience of majority votes to input noise stemming from compression. The derived bounds are intuitive in binary classification settings, where they can be measured as expressions of voter differentials and voter pair agreement. By combining measures of input distortion with analytic guarantees on noise invariance, we prescribe rate-efficient machines to compress inputs without affecting subsequent classification. Our validation shows how bounding noise invariance can inform the compression stage for any majority vote classifier such that worst-case implications of bad input reconstructions are known, and inputs can be compressed to the minimum amount of information needed prior to inference.
| Type: | Proceedings paper |
|---|---|
| Title: | PAC-Bayesian Bounds on Rate-Efficient Classifiers |
| Event: | 38th International Conference on Machine Learning (ICML) |
| Location: | Baltimore, MD |
| Dates: | 17 Jul 2022 - 23 Jul 2022 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.mlr.press/v162/abbas22a.html |
| Language: | English |
| Additional information: | © The Author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. https://creativecommons.org/licenses/by/4.0/ |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10170823 |
Archive Staff Only
 |
View Item |
