Tran, L;
Pantic, M;
Deisenroth, MP;
(2022)
Cauchy-Schwarz Regularized Autoencoder.
Journal of Machine Learning Research
, 23
, Article 115.
Preview |
Text
21-0681.pdf - Published Version Download (7MB) | Preview |
Abstract
Recent work in unsupervised learning has focused on efficient inference and learning in latent variables models. Training these models by maximizing the evidence (marginal likelihood) is typically intractable. Thus, a common approximation is to maximize the Evidence Lower BOund (ELBO) instead. Variational autoencoders (VAE) are a powerful and widely-used class of generative models that optimize the ELBO efficiently for large datasets. However, the VAE's default Gaussian choice for the prior imposes a strong constraint on its ability to represent the true posterior, thereby degrading overall performance. A Gaussian mixture model (GMM) would be a richer prior but cannot be handled efficiently within the VAE framework because of the intractability of the Kullback{Leibler divergence for GMMs. We deviate from the common VAE framework in favor of one with an analytical solution for Gaussian mixture prior. To perform efficient inference for GMM priors, we introduce a new constrained objective based on the Cauchy{Schwarz divergence, which can be computed analytically for GMMs. This new objective allows us to incorporate richer, multi-modal priors into the autoencoding framework. We provide empirical studies on a range of datasets and show that our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
Type: | Article |
---|---|
Title: | Cauchy-Schwarz Regularized Autoencoder |
Open access status: | An open access version is available from UCL Discovery |
Publisher version: | https://jmlr.org/papers/v23/21-0681.html |
Language: | English |
Additional information: | © 2022 Linh Tran, Maja Pantic, Marc Peter Deisenroth. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v23/21-0681.html. |
Keywords: | Generative models, Cauchy–Schwarz divergence, constrained optimization, auto-encoding models, face analysis |
UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
URI: | https://discovery.ucl.ac.uk/id/eprint/10150716 |
Archive Staff Only
![]() |
View Item |