Zantedeschi, V; Viallard, P; Morvant, E; Emonet, R; Habrard, A; Germain, P; Guedj, B; (2021) Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound. In: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Advances in Neural Information Processing Systems (NeurIPS 2021)

Preview

Text Guedj_Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound_VoR.pdf - Published Version Download (1MB) | Preview

Abstract

We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective.The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives -- both with uninformed (data-independent) and informed (data-dependent) priors.

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item