%J IEEE Transactions on Control Systems Technology
%K Computational control, deep reinforcement learning (RL), learning-based control, policy validation, reward shaping
%L discovery10194559
%I Institute of Electrical and Electronics Engineers
%O © 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License.
For more information, see (https://creativecommons.org/licenses/by/4.0/).
%X In addressing control problems such as regulation and tracking through reinforcement learning (RL), it is often required to guarantee that the acquired policy meets essential performance and stability criteria such as a desired settling time and steady-state error before deployment. Motivated by this, we present a set of results and a systematic reward-shaping procedure that: 1) ensures the optimal policy generates trajectories that align with specified control requirements and 2) allows to assess whether any given policy satisfies them. We validate our approach through comprehensive numerical experiments conducted in two representative environments from OpenAI Gym: the Pendulum swing-up problem and the Lunar Lander. Utilizing both tabular and deep RL methods, our experiments consistently affirm the efficacy of our proposed framework, highlighting its effectiveness in ensuring policy adherence to the prescribed control requirements.
%T Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning
%A Francesco De Lellis
%A Marco Coraggio
%A Giovanni Russo
%A Mirco Musolesi
%A Mario di Bernardo
%D 2024