UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control

Kamthe, S; Deisenroth, MP; (2018) Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control. In: Storkey, AJ and Pérez-Cruz, F, (eds.) Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS 2018). (pp. pp. 1701-1710). Proceedings of Machine Learning (PMLR): Lanzarote, Canary Islands, Spain. Green open access

[thumbnail of kamthe18a.pdf]
Preview
Text
kamthe18a.pdf - Published Version

Download (527kB) | Preview

Abstract

Trial-and-error based reinforcement learning (RL) has seen rapid advancements in recent times, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. A large number of interactions may be impractical in many real-world applications, such as robotics, and many practical systems have to obey limitations in the form of state space or control constraints. To reduce the number of system interactions while simultaneously handling constraints, we propose a modelbased RL framework based on probabilistic Model Predictive Control (MPC). In particular, we propose to learn a probabilistic transition model using Gaussian Processes (GPs) to incorporate model uncertainty into longterm predictions, thereby, reducing the impact of model errors. We then use MPC to find a control sequence that minimises the expected long-term cost. We provide theoretical guarantees for first-order optimality in the GP-based transition models with deterministic approximate inference for long-term planning. We demonstrate that our approach does not only achieve state-of-the-art data efficiency, but also is a principled way for RL in constrained environments.

Type: Proceedings paper
Title: Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control
Event: 21st International Conference on Artificial Intelligence and Statistics (AISTATS 2018), 9-11 April 2018, Lanzarote, Canary Islands, Spain
Open access status: An open access version is available from UCL Discovery
Publisher version: http://proceedings.mlr.press/v84/kamthe18a/kamthe1...
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10083563
Downloads since deposit
28Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item