UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Long text generation via adversarial training with leaked information

Guo, J; Lu, S; Cai, H; Zhang, W; Yu, Y; Wang, J; (2018) Long text generation via adversarial training with leaked information. In: (Proceedings) 32nd AAAI Conference on Artificial Intelligence, AAAI 2018. (pp. pp. 5141-5148). AAAI: New Orleans, LA, USA. Green open access

[thumbnail of 16360-76823-1-PB.pdf]
Preview
Text
16360-76823-1-PB.pdf - Published version

Download (1MB) | Preview

Abstract

Automatically generating coherent and semantically meaningful text has many applications in machine translation, dialogue systems, image captioning, etc. Recently, by combining with policy gradient, Generative Adversarial Nets (GAN) that use a discriminative model to guide the training of the generative model as a reinforcement learning policy has shown promising results in text generation. However, the scalar guiding signal is only available after the entire text has been generated and lacks intermediate information about text structure during the generative process. As such, it limits its success when the length of the generated text samples is long (more than 20 words). In this paper, we propose a new framework, called LeakGAN, to address the problem for long text generation. We allow the discriminative net to leak its own high-level extracted features to the generative net to further help the guidance. The generator incorporates such informative signals into all generation steps through an additional MANAGER module, which takes the extracted features of current generated words and outputs a latent vector to guide the WORKER module for next-word generation. Our extensive experiments on synthetic data and various real-world tasks with Turing test demonstrate that LeakGAN is highly effective in long text generation and also improves the performance in short text generation scenarios. More importantly, without any supervision, LeakGAN would be able to implicitly learn sentence structures only through the interaction between MANAGER and WORKER.

Type: Proceedings paper
Title: Long text generation via adversarial training with leaked information
Event: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018
ISBN-13: 9781577358008
Open access status: An open access version is available from UCL Discovery
Publisher version: https://www.aaai.org/ocs/index.php/AAAI/AAAI18/pap...
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Text Generation; GAN; RL
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10070681
Downloads since deposit
23Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item