De Lellis, Francesco;
Coraggio, Marco;
Russo, Giovanni;
Musolesi, Mirco;
Di Bernardo, Mario;
(2022)
Control-Tutored Reinforcement Learning: Towards the Integration of Data-Driven and Model-Based Control.
In: Firoozi, Roya and Mehr, Negar and Yel, Esen and Antonova, Rika and Bohg, Jeannette and Schwager, Mac and Kochenderfer, Mykel, (eds.)
Proceedings of The 4th Annual Learning for Dynamics and Control Conference.
(pp. pp. 1048-1059).
MIT Press: Stanford, CA, USA.
Preview |
Text
Musolesi_L4DC_22.pdf - Accepted Version Download (1MB) | Preview |
Abstract
We present an architecture where a feedback controller derived on an approximate model of the environment assists the learning process to enhance its data efficiency. This architecture, which we term as Control-Tutored Q-learning (CTQL), is presented in two alternative flavours. The former is based on defining the reward function so that a Boolean condition can be used to determine when the control tutor policy is adopted, while the latter, termed as probabilistic CTQL (pCTQL), is instead based on executing calls to the tutor with a certain probability during learning. Both approaches are validated, and thoroughly benchmarked against Q-Learning, by considering the stabilization of an inverted pendulum as defined in OpenAI Gym as a representative problem.
| Type: | Proceedings paper |
|---|---|
| Title: | Control-Tutored Reinforcement Learning: Towards the Integration of Data-Driven and Model-Based Control |
| Event: | 4th Annual Learning for Dynamics and Control Conference |
| Location: | Stanford |
| Dates: | 23 Jun 2022 - 24 Jun 2022 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.mlr.press/v168/lellis22a |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Reinforcement learning based control, data-driven control, feedback control |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10150547 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
