Zantedeschi, V;
Viallard, P;
Morvant, E;
Emonet, R;
Habrard, A;
Germain, P;
Guedj, B;
(2021)
Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound.
In:
Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021).
Advances in Neural Information Processing Systems (NeurIPS 2021)
Preview |
Text
Guedj_Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound_VoR.pdf - Published Version Download (1MB) | Preview |
Abstract
We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective.The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives -- both with uninformed (data-independent) and informed (data-dependent) priors.
| Type: | Proceedings paper |
|---|---|
| Title: | Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound |
| Event: | 35th Conference on Neural Information Processing Systems (NeurIPS 2021) |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.neurips.cc/paper/2021/hash/041... |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher's terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10138174 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
