UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Demystifying MMD GANs

Bińkowski, M; Sutherland, DJ; Arbel, M; Gretton, A; (2018) Demystifying MMD GANs. In: Bengio, Yoshua and LeCun, Yann, (eds.) Proceedings of ICLR 2018 : International Conference on Learning Representations. ICLR: Vancouver, BC, Canada,. Green open access

[thumbnail of 1801.01401v4.pdf]
Preview
Text
1801.01401v4.pdf - Published version

Download (10MB) | Preview

Abstract

We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs. As our main theoretical contribution, we clarify the situation with bias in GAN loss functions raised by recent work: we show that gradient estimators used in the optimization process for both MMD GANs and Wasserstein GANs are unbiased, but learning a discriminator based on samples leads to biased gradients for the generator parameters. We also discuss the issue of kernel choice for the MMD critic, and characterize the kernel corresponding to the energy distance used for the Cramer GAN critic. Being an integral probability metric, the MMD benefits from training strategies recently developed for Wasserstein GANs. In experiments, the MMD GAN is able to employ a smaller critic network than the Wasserstein GAN, resulting in a simpler and faster-training algorithm with matching performance. We also propose an improved measure of GAN convergence, the Kernel Inception Distance, and show how to use it to dynamically adapt learning rates during GAN training.

Type: Proceedings paper
Title: Demystifying MMD GANs
Event: ICLR 2018 : International Conference on Learning Representations
Open access status: An open access version is available from UCL Discovery
Publisher version: https://openreview.net/pdf?id=r1lUOzWCW
Language: English
Additional information: Published at ICLR 2018: https://openreview.net/forum?id=r1lUOzWCW . v4: actually-final version: non-existence of unbiased estimators for IPMs and FID; clarity edits to the main proof.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10062884
Downloads since deposit
13Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item