Wu, Ziyan;
Zhang, Wenhao;
Tang, Rui;
Wang, Huilong;
Korolija, Ivan;
(2024)
Reinforcement learning in building controls: A comparative study of algorithms considering model availability and policy representation.
Journal of Building Engineering
, 90
, Article 109497. 10.1016/j.jobe.2024.109497.
Preview |
PDF
1-s2.0-S2352710224010659-main.pdf - Published Version Download (6MB) | Preview |
Abstract
Reinforcement Learning (RL) has presented considerable potential as an advanced control technique in building controls to enable buildings operating more energy-efficient. As various types of RL algorithms have been studied on their performance of building controls, benchmarking these algorithms across the entire spectrum of features is essential to provide an overview and deepen the understanding of RL applications. Therefore, this study aims to compare and analyze the effectiveness of various RL algorithms, encompassing the entire RL categories featured by value-based, policy gradient, actor-critic and model-based RL considering model availability and policy representation. To provide a comprehensive analysis, in addition to the control performance quantified by the cumulative rewards based on the cost function of RL, data demand and robustness of hyperparameter tuning were investigated. The open-source Gym-Eplus framework was selected as the virtual environment to train and test different RL agents. The results showed that both model-free and model-based RL agents outperformed the baseline rule-based control in terms of energy consumption and thermal comfort, and RL agents were capable of evaluating both short-term and long-term rewards to achieve adaptive control optimization continuously along with the online control process. Model-based RL agent improved the data sampling efficiency but presented a relatively sacrificed control performance during the tested summer days.
| Type: | Article |
|---|---|
| Title: | Reinforcement learning in building controls: A comparative study of algorithms considering model availability and policy representation |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.jobe.2024.109497 |
| Publisher version: | http://dx.doi.org/10.1016/j.jobe.2024.109497 |
| Language: | English |
| Additional information: | © 2024 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). |
| Keywords: | Reinforcement learning, Model-free RL, Model-based RL, Building energy efficiency, Control optimization |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of the Built Environment UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of the Built Environment > Bartlett School Env, Energy and Resources |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10191638 |
Archive Staff Only
 |
View Item |
