Hirt, Marcel;
Kreouzis, Vasileios;
Dellaportas, Petros;
(2024)
Learning variational autoencoders via MCMC speed measures.
Statistics and Computing
, 34
(5)
, Article 164. 10.1007/s11222-024-10481-x.
Preview |
Text
s11222-024-10481-x.pdf - Published Version Download (1MB) | Preview |
Abstract
Variational autoencoders (VAEs) are popular likelihood-based generative models which can be efficiently trained by maximising an evidence lower bound. There has been much progress in improving the expressiveness of the variational distribution to obtain tighter variational bounds and increased generative performance. Whilst previous work has leveraged Markov chain Monte Carlo methods for constructing variational densities, gradient-based methods for adapting the proposal distributions for deep latent variable models have received less attention. This work suggests an entropy-based adaptation for a short-run metropolis-adjusted Langevin or Hamiltonian Monte Carlo (HMC) chain while optimising a tighter variational bound to the log-evidence. Experiments show that this approach yields higher held-out log-likelihoods as well as improved generative metrics. Our implicit variational density can adapt to complicated posterior geometries of latent hierarchical representations arising in hierarchical VAEs.
| Type: | Article |
|---|---|
| Title: | Learning variational autoencoders via MCMC speed measures |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1007/s11222-024-10481-x |
| Publisher version: | https://doi.org/10.1007/s11222-024-10481-x |
| Language: | English |
| Additional information: | This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. |
| Keywords: | Adaptive MCMC, Generative models, Hierarchical models, HMC, MALA |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10195741 |
Archive Staff Only
 |
View Item |
