De Lellis, Francesco;
Coraggio, Marco;
Russo, Giovanni;
Musolesi, Mirco;
di Bernardo, Mario;
(2024)
Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning.
IEEE Transactions on Control Systems Technology
10.1109/TCST.2024.3393210.
(In press).
Preview |
Text
Musolesi_Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning_AAM.pdf Download (9MB) | Preview |
Abstract
In addressing control problems such as regulation and tracking through reinforcement learning (RL), it is often required to guarantee that the acquired policy meets essential performance and stability criteria such as a desired settling time and steady-state error before deployment. Motivated by this, we present a set of results and a systematic reward-shaping procedure that: 1) ensures the optimal policy generates trajectories that align with specified control requirements and 2) allows to assess whether any given policy satisfies them. We validate our approach through comprehensive numerical experiments conducted in two representative environments from OpenAI Gym: the Pendulum swing-up problem and the Lunar Lander. Utilizing both tabular and deep RL methods, our experiments consistently affirm the efficacy of our proposed framework, highlighting its effectiveness in ensuring policy adherence to the prescribed control requirements.
| Type: | Article |
|---|---|
| Title: | Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/TCST.2024.3393210 |
| Publisher version: | http://dx.doi.org/10.1109/tcst.2024.3393210 |
| Language: | English |
| Additional information: | © 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see (https://creativecommons.org/licenses/by/4.0/). |
| Keywords: | Computational control, deep reinforcement learning (RL), learning-based control, policy validation, reward shaping |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10194559 |
Archive Staff Only
 |
View Item |
