Shi, Y;
Siddharth, N;
Paige, B;
Torr, PHS;
(2019)
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models.
In:
Advances in Neural Information Processing Systems 32 (NeurIPS 2019).
Neural Information Processing Systems (NeurIPS): Vancouver, BC, Canada.
Preview |
Text
NeurIPS-2019-variational-mixture-of-experts-autoencoders-for-multi-modal-deep-generative-models-Paper.pdf - Published Version Download (6MB) | Preview |
Abstract
Learning generative models that span multiple data modalities, such as vision and language, is often motivated by the desire to learn more useful, generalisable representations that faithfully capture common underlying factors between the modalities. In this work, we characterise successful learning of such models as the fulfillment of four criteria: i) implicit latent decomposition into shared and private subspaces, ii) coherent joint generation over all modalities, iii) coherent cross-generation across individual modalities, and iv) improved model learning for individual modalities through multi-modal integration. Here, we propose a mixture-of-experts multimodal variational autoencoder (MMVAE) to learn generative models on different sets of modalities, including a challenging image-language dataset, and demonstrate its ability to satisfy all four criteria, both qualitatively and quantitatively.
Type: | Proceedings paper |
---|---|
Title: | Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models |
Event: | 33rd Conference on Neural Information Processing Systems (NeurIPS 2019) |
Open access status: | An open access version is available from UCL Discovery |
Publisher version: | https://papers.nips.cc/paper/2019/hash/0ae775a8cb3... |
Language: | English |
Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
UCL classification: | UCL UCL > Provost and Vice Provost Offices UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
URI: | https://discovery.ucl.ac.uk/id/eprint/10115926 |
Archive Staff Only
![]() |
View Item |