Kamthe, S; Deisenroth, MP; (2018) Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control. In: Storkey, AJ and Pérez-Cruz, F, (eds.) Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS 2018). (pp. pp. 1701-1710). Proceedings of Machine Learning (PMLR): Lanzarote, Canary Islands, Spain.

Preview

Text kamthe18a.pdf - Published Version Download (527kB) | Preview

Abstract

Trial-and-error based reinforcement learning (RL) has seen rapid advancements in recent times, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. A large number of interactions may be impractical in many real-world applications, such as robotics, and many practical systems have to obey limitations in the form of state space or control constraints. To reduce the number of system interactions while simultaneously handling constraints, we propose a modelbased RL framework based on probabilistic Model Predictive Control (MPC). In particular, we propose to learn a probabilistic transition model using Gaussian Processes (GPs) to incorporate model uncertainty into longterm predictions, thereby, reducing the impact of model errors. We then use MPC to find a control sequence that minimises the expected long-term cost. We provide theoretical guarantees for first-order optimality in the GP-based transition models with deterministic approximate inference for long-term planning. We demonstrate that our approach does not only achieve state-of-the-art data efficiency, but also is a principled way for RL in constrained environments.

Downloads since deposit

Loading...

71Downloads

Download activity - last month

Export as XMLCSVJSON

Loading...

Download activity - last 12 months

Export as CSVXMLJSON

Loading...

Downloads by country - last 12 months

Export as JSONXMLCSV

Loading...

Archive Staff Only

View Item