UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

A neurally plausible model learns successor representations in partially observable environments

Vertes, E; Sahani, M; (2019) A neurally plausible model learns successor representations in partially observable environments. In: Proceedings of 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). NIPS Proceedings: Vancouver, Canada. Green open access

[thumbnail of neurally_plausible_model.pdf]
Preview
Text
neurally_plausible_model.pdf - Published version

Download (786kB) | Preview

Abstract

Animals need to devise strategies to maximize returns while interacting with their environment based on incoming noisy sensory observations. Task-relevant states, such as the agent's location within an environment or the presence of a predator, are often not directly observable but must be inferred using available sensory information. Successor representations (SR) have been proposed as a middle-ground between model-based and model-free reinforcement learning strategies, allowing for fast value computation and rapid adaptation to changes in the reward function or goal locations. Indeed, recent studies suggest that features of neural responses are consistent with the SR framework. However, it is not clear how such representations might be learned and computed in partially observed, noisy environments. Here, we introduce a neurally plausible model using \emph{distributional successor features}, which builds on the distributed distributional code for the representation and computation of uncertainty, and which allows for efficient value function computation in partially observed environments via the successor representation. We show that distributional successor features can support reinforcement learning in noisy environments in which direct learning of successful policies is infeasible.

Type: Proceedings paper
Title: A neurally plausible model learns successor representations in partially observable environments
Event: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Open access status: An open access version is available from UCL Discovery
Publisher version: https://papers.nips.cc/paper/9522-a-neurally-plaus...
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10090063
Downloads since deposit
33Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item