UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Applying reinforcement learning and tree search to the unit commitment problem

de Mars, P; O'Sullivan, A; (2021) Applying reinforcement learning and tree search to the unit commitment problem. Applied Energy , 302 , Article 117519. 10.1016/j.apenergy.2021.117519. Green open access

[thumbnail of 1-s2.0-S0306261921008990-main.pdf]
Preview
Text
1-s2.0-S0306261921008990-main.pdf - Published Version

Download (1MB) | Preview

Abstract

Recent advances in artificial intelligence have demonstrated the capability of reinforcement learning (RL) methods to outperform the state of the art in decision-making problems under uncertainty. Day-ahead unit commitment (UC), scheduling power generation based on forecasts, is a complex power systems task that is becoming more challenging in light of increasing uncertainty. While RL is a promising framework for solving the UC problem, the space of possible actions from a given state is exponential in the number of generators and it is infeasible to apply existing RL methods in power systems larger than a few generators. Here we present a novel RL algorithm, guided tree search, which does not suffer from an exponential explosion in the action space with increasing number of generators. The method augments a tree search algorithm with a policy that intelligently reduces the branching factor. Using data from the GB power system, we demonstrate that guided tree search outperforms an unguided method in terms of computational complexity, while producing solutions that show no performance loss in terms of operating costs. We compare solutions against mixed-integer linear programming (MILP) and find that guided tree search outperforms a solution using reserve constraints, the current industry approach. The RL solutions exhibit complex behaviours that differ qualitatively from MILP, demonstrating its potential as a decision support tool for human operators.

Type: Article
Title: Applying reinforcement learning and tree search to the unit commitment problem
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.apenergy.2021.117519
Publisher version: https://doi.org/10.1016/j.apenergy.2021.117519
Language: English
Additional information: © 2021 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Keywords: Unit commitment, Reinforcement learning, Tree search, Deep learning, Power systems
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of the Built Environment
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of the Built Environment > Bartlett School Env, Energy and Resources
URI: https://discovery.ucl.ac.uk/id/eprint/10133018
Downloads since deposit
100Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item