Cowen-Rivers, Alexander;
Palenicek, Daniel;
Moens, Vincent;
Abdullah, Mohammed Amin;
Sootla, Aivar;
Wang, Jun;
Bou-Ammar, Haitham;
(2022)
SAMBA: safe model-based & active reinforcement learning.
Machine Learning
, 111
(1)
pp. 173-203.
10.1007/s10994-021-06103-6.
Preview |
Text
2006.09436v1.pdf - Accepted Version Download (4MB) | Preview |
Abstract
In this paper, we propose SAMBA, a novel framework for safe reinforcement learning that combines aspects from probabilistic modelling, information theory, and statistics. Our method builds upon PILCO to enable active exploration using novel acquisition functions for out-of-sample Gaussian process evaluation optimised through a multi-objective problem that supports conditional-value-at-risk constraints. We evaluate our algorithm on a variety of safe dynamical system benchmarks involving both low and high-dimensional state representations. Our results show orders of magnitude reductions in samples and violations compared to state-of-the-art methods. Lastly, we provide intuition as to the effectiveness of the framework by a detailed analysis of our acquisition functions and safety constraints.
| Type: | Article |
|---|---|
| Title: | SAMBA: safe model-based & active reinforcement learning |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1007/s10994-021-06103-6 |
| Publisher version: | https://doi.org/10.1007/s10994-021-06103-6 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10217139 |
Archive Staff Only
 |
View Item |
