Yu, L;
Zhang, W;
Wang, J;
Yu, Y;
(2017)
Seqgan: sequence generative adversarial nets with policy gradient.
In:
Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17).
(pp. pp. 2852-2858).
Association for the Advancement of Artificial Intelligence (AAAI)
(In press).
Preview |
Text
1609.05473.pdf - Published Version Download (823kB) | Preview |
Abstract
As a new way of training generative models, Generative Adversarial Net (GAN) that uses a discriminative model to guide the training of the generative model has enjoyed considerable success in generating real-valued data. However, it has limitations when the goal is for generating sequences of discrete tokens. A major reason lies in that the discrete outputs from the generative model make it difficult to pass the gradient update from the discriminative model to the generative model. Also, the discriminative model can only assess a complete sequence, while for a partially generated sequence, it is nontrivial to balance its current score and the future one once the entire sequence has been generated. In this paper, we propose a sequence generation framework, called SeqGAN, to solve the problems. Modeling the data generator as a stochastic policy in reinforcement learning (RL), SeqGAN bypasses the generator differentiation problem by directly performing gradient policy update. The RL reward signal comes from the GAN discriminator judged on a complete sequence, and is passed back to the intermediate state-action steps using Monte Carlo search. Extensive experiments on synthetic data and real-world tasks demonstrate significant improvements over strong baselines.
Type: | Proceedings paper |
---|---|
Title: | Seqgan: sequence generative adversarial nets with policy gradient |
Event: | AAAI-17: Thirty-first AAAI Conference on Artificial Intelligence, 4-9 February 2017, San Francisco, California, USA |
Dates: | 04 February 2017 - 09 February 2017 |
Open access status: | An open access version is available from UCL Discovery |
Publisher version: | https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/v... |
Language: | English |
Additional information: | Copyright © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. |
Keywords: | Generative Adversarial Nets; Deep Learning; Unsupervised Learning; Reinforcement Learning |
UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
URI: | https://discovery.ucl.ac.uk/id/eprint/1533005 |
Archive Staff Only
![]() |
View Item |