eprintid: 10194559
rev_number: 9
eprint_status: archive
userid: 699
dir: disk0/10/19/45/59
datestamp: 2024-07-16 09:07:58
lastmod: 2024-07-16 09:07:58
status_changed: 2024-07-16 09:07:58
type: article
metadata_visibility: show
sword_depositor: 699
creators_name: De Lellis, Francesco
creators_name: Coraggio, Marco
creators_name: Russo, Giovanni
creators_name: Musolesi, Mirco
creators_name: di Bernardo, Mario
title: Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning
ispublished: inpress
divisions: UCL
divisions: B04
divisions: F48
keywords: Computational control, deep reinforcement learning (RL), learning-based control, policy validation, reward shaping
note: © 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License.
For more information, see (https://creativecommons.org/licenses/by/4.0/).
abstract: In addressing control problems such as regulation and tracking through reinforcement learning (RL), it is often required to guarantee that the acquired policy meets essential performance and stability criteria such as a desired settling time and steady-state error before deployment. Motivated by this, we present a set of results and a systematic reward-shaping procedure that: 1) ensures the optimal policy generates trajectories that align with specified control requirements and 2) allows to assess whether any given policy satisfies them. We validate our approach through comprehensive numerical experiments conducted in two representative environments from OpenAI Gym: the Pendulum swing-up problem and the Lunar Lander. Utilizing both tabular and deep RL methods, our experiments consistently affirm the efficacy of our proposed framework, highlighting its effectiveness in ensuring policy adherence to the prescribed control requirements.
date: 2024-05-17
date_type: published
publisher: Institute of Electrical and Electronics Engineers
official_url: http://dx.doi.org/10.1109/tcst.2024.3393210
oa_status: green
full_text_type: other
language: eng
primo: open
primo_central: open_green
verified: verified_manual
elements_id: 2115444
doi: 10.1109/TCST.2024.3393210
lyricists_name: Musolesi, Mirco
lyricists_id: MMUSO05
actors_name: Musolesi, Mirco
actors_id: MMUSO05
actors_role: owner
funding_acknowledgements: [European Union]
full_text_status: public
publication: IEEE Transactions on Control Systems Technology
pages: 12
citation:        De Lellis, Francesco;    Coraggio, Marco;    Russo, Giovanni;    Musolesi, Mirco;    di Bernardo, Mario;      (2024)    Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning.                   IEEE Transactions on Control Systems Technology        10.1109/TCST.2024.3393210 <https://doi.org/10.1109/TCST.2024.3393210>.    (In press).    Green open access   
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10194559/2/Musolesi_Guaranteeing%20Control%20Requirements%20via%20Reward%20Shaping%20in%20Reinforcement%20Learning_AAM.pdf