UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

SCAN: Learning Hierarchical Compositional Visual Concepts

Higgins, I; Sonnerat, N; Matthey, L; Pal, A; Burgess, CP; Bošnjak, M; Shanahan, M; ... Lerchner, A; + view all (2018) SCAN: Learning Hierarchical Compositional Visual Concepts. In: Bengio, Y and LeCun, Y, (eds.) Proceedings of the Sixth International Conference on Learning Representations (ICLR 2018). International Conference on Learning Representations (ICLR): Vancouver, Canada. Green open access

[thumbnail of Bosjnak_scan_learning_hierarchical_compositional_visual_concepts.pdf]
Preview
Text
Bosjnak_scan_learning_hierarchical_compositional_visual_concepts.pdf - Published Version

Download (22MB) | Preview

Abstract

The seemingly infinite diversity of the natural world arises from a relatively small set of coherent rules, such as the laws of physics or chemistry. We conjecture that these rules give rise to regularities that can be discovered through primarily unsupervised experiences and represented as abstract concepts. If such representations are compositional and hierarchical, they can be recombined into an exponentially large set of new concepts. This paper describes SCAN (Symbol-Concept Association Network), a new framework for learning such abstractions in the visual domain. SCAN learns concepts through fast symbol association, grounding them in disentangled visual primitives that are discovered in an unsupervised manner. Unlike state of the art multimodal generative model baselines, our approach requires very few pairings between symbols and images and makes no assumptions about the form of symbol representations. Once trained, SCAN is capable of multimodal bi-directional inference, generating a diverse set of image samples from symbolic descriptions and vice versa. It also allows for traversal and manipulation of the implicit hierarchy of visual concepts through symbolic instructions and learnt logical recombination operations. Such manipulations enable SCAN to break away from its training data distribution and imagine novel visual concepts through symbolically instructed recombination of previously learnt concepts.

Type: Proceedings paper
Title: SCAN: Learning Hierarchical Compositional Visual Concepts
Event: Sixth International Conference on Learning Representations (ICLR 2018)
Open access status: An open access version is available from UCL Discovery
Publisher version: https://openreview.net/forum?id=rkN2Il-RZ
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: grounded visual concepts, compositional representation, concept hierarchy, disentangling, beta-VAE, variational autoencoder, deep learning, generative model
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10124080
Downloads since deposit
21Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item