UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Learning a generative model for validity in complex discrete structures

Janz, D; Van Der Westhuizen, J; Paige, B; Kusner, MJ; Hernández-Lobato, JM; (2018) Learning a generative model for validity in complex discrete structures. In: Proceedings of the Sixth International Conference on Learning Representations (ICLR 2018). International Conference on Learning Representations (ICLR): Vancouver, Canada. Green open access

[img]
Preview
Text
1712.01664v4.pdf - Accepted version

Download (718kB) | Preview

Abstract

Deep generative models have been successfully used to learn representations for high-dimensional discrete spaces by representing discrete objects as sequences and employing powerful sequence-based deep models. Unfortunately, these sequence-based models often produce invalid sequences: sequences which do not represent any underlying discrete structure; invalid sequences hinder the utility of such models. As a step towards solving this problem, we propose to learn a deep recurrent validator model, which can estimate whether a partial sequence can function as the beginning of a full, valid sequence. This validator provides insight as to how individual sequence elements influence the validity of the overall sequence, and can be used to constrain sequence based models to generate valid sequences – and thus faithfully model discrete objects. Our approach is inspired by reinforcement learning, where an oracle which can evaluate validity of complete sequences provides a sparse reward signal. We demonstrate its effectiveness as a generative model of Python 3 source code for mathematical expressions, and in improving the ability of a variational autoencoder trained on SMILES strings to decode valid molecular structures.

Type: Proceedings paper
Title: Learning a generative model for validity in complex discrete structures
Event: Sixth International Conference on Learning Representations (ICLR 2018)
Open access status: An open access version is available from UCL Discovery
Publisher version: https://iclr.cc/Conferences/2018/Schedule?showEven...
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10088322
Downloads since deposit
0Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item