UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Adversarial Interpretation of Bayesian Inference

Husain, H; Knoblauch, J; (2022) Adversarial Interpretation of Bayesian Inference. In: Proceedings of The 33rd International Conference on Algorithmic Learning Theory. (pp. pp. 553-572). Proceedings of Machine Learning Research (PMLR): Paris, France. Green open access

[thumbnail of husain22a.pdf]
Preview
Text
husain22a.pdf - Published Version

Download (291kB) | Preview

Abstract

We build on the optimization-centric view on Bayesian inference advocated by Knoblauch et al. (2019). Thinking about Bayesian and generalized Bayesian posteriors as the solutions to a regularized minimization problem allows us to answer an intriguing question: If minimization is the primal problem, then what is its dual? By deriving the Fenchel dual of the problem, we demonstrate that this dual corresponds to an adversarial game: In the dual space, the prior becomes the cost function for an adversary that seeks to perturb the likelihood [loss] function targeted by standard [generalized] Bayesian inference. This implies that Bayes-like procedures are adversarially robust—providing another firm theoretical foundation for their empirical performance. Our contributions are foundational, and apply to a wide-ranging set of Machine Learning methods. This includes standard Bayesian inference, generalized Bayesian and Gibbs posteriors (Bissiri et al., 2016), as well as a diverse set of other methods including Generalized Variational Inference (Knoblauch et al., 2019) and the Wasserstein Autoencoder (Tolstikhin et al., 2017).

Type: Proceedings paper
Title: Adversarial Interpretation of Bayesian Inference
Event: 33rd International Conference on Algorithmic Learning Theory (ALT 2022)
Open access status: An open access version is available from UCL Discovery
Publisher version: https://proceedings.mlr.press/v167/husain22a.html
Language: English
Additional information: This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.
Keywords: Bayesian Inference, Fenchel Duality, f-divergences, Integral Probability Metric
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science
URI: https://discovery.ucl.ac.uk/id/eprint/10174466
Downloads since deposit
4Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item