UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Successor features for transfer in reinforcement learning

Barreto, A; Munos, R; Schaul, T; Silver, D; (2016) Successor features for transfer in reinforcement learning. ArXiv: Ithaca, NY, USA. Green open access

[img]
Preview
Text
Barreto_Successor_features_transfer.pdf - Accepted version

Download (429kB) | Preview

Abstract

Transfer in reinforcement learning refers to the notion that generalization should occur not only within a task but also across tasks. Our focus is on transfer where the reward functions vary across tasks while the environment's dynamics remain the same. The method we propose rests on two key ideas: "successor features," a value function representation that decouples the dynamics of the environment from the rewards, and "generalized policy improvement," a generalization of dynamic programming's policy improvement step that considers a set of policies rather than a single one. Put together, the two ideas lead to an approach that integrates seamlessly within the reinforcement learning framework and allows transfer to take place between tasks without any restriction. The proposed method also provides performance guarantees for the transferred policy even before any learning has taken place. We derive two theorems that set our approach in firm theoretical ground and present experiments that show that it successfully promotes transfer in practice.

Type: Working / discussion paper
Title: Successor features for transfer in reinforcement learning
Open access status: An open access version is available from UCL Discovery
Publisher version: https://arxiv.org/abs/1606.05312#
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/1523480
Downloads since deposit
16Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item