Shah, H;
Barber, D;
Botev, A;
(2017)
Overdispersed Variational Autoencoders.
In:
2017 International Joint Conference on Neural Networks (IJCNN).
(pp. pp. 1109-1116).
IEEE
Preview |
Text
OVAE_verified.pdf - Accepted Version Download (936kB) | Preview |
Abstract
The ability to fit complex generative probabilistic models to data is a key challenge in AI. Currently, variational methods are popular, but remain difficult to train due to high variance of the sampling methods employed. We introduce the overdispersed variational autoencoder and overdispersed importance weighted autoencoder, which combine overdispersed black box variational inference with the variational autoencoder and importance weighted autoencoder respectively. We use the log likelihood lower bounds and reparametrisation trick from the variational and importance weighted autoencoders, but rather than drawing samples from the variational distribution itself, we use importance sampling to draw samples from an overdispersed (i.e. heavier-tailed) proposal in the same family as the variational distribution. We run experiments on two different datasets, and show that this technique produces a lower variance estimate of the gradients, and reaches a higher bound on the log likelihood of the observed data.
| Type: | Proceedings paper |
|---|---|
| Title: | Overdispersed Variational Autoencoders |
| Event: | International Joint Conference on Neural Networks (IJCNN), 14-19 May 2017, Anchorage, AK, USA |
| Location: | Anchorage, AK |
| Dates: | 14 May 2017 - 19 May 2017 |
| ISBN-13: | 978-1-5090-6182-2 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/IJCNN.2017.7965976 |
| Publisher version: | https://doi.org/10.1109/IJCNN.2017.7965976 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10059973 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
