Johnn, Syu-Ning;
Charitopoulos, Vasileios M;
(2026)
A hybrid deep Q-learning approach to online planning and rescheduling of single-stage multi-product continuous processes.
Computers and Chemical Engineering
, 204
, Article 109415. 10.1016/j.compchemeng.2025.109415.
(In press).
Preview |
Text
Charitopoulos_1-s2.0-S0098135425004181-main.pdf Download (3MB) | Preview |
Abstract
Optimisation-based process scheduling methods lie at the core of the process supply chains, facilitating the efficient allocation of limited resources and ensuring profitable operations. The efficiency and adaptability of these methods are of paramount importance, especially when dealing with frequent modifications to existing scheduling plans, caused by uncertainties and unforeseen real-world disturbances. Compared to heuristic methods that heavily rely on instance-specific manual customisation and fine-tuning, reinforcement learning (RL) has the advantages of learning from existing experiments and generalising to unknown scenarios, thus automating the process with higher flexibility and adaptability. In this work, we propose an RL-based method that transforms a single-stage multi-product process scheduling problem, originally framed as a mixed-integer linear programming (MILP) problem, into a Markov decision process and trains the RL agent to identify the optimal production sequence. The trained agent is subsequently integrated into a simplified planning and scheduling linear programming (LP) framework to enable efficient decision-making for the re-optimised production sequence and time length. Results show that our proposed learning-based integrated decision-making framework demonstrates strong computational efficiency and adaptability, outperforming both the benchmark random agent and heuristic approaches with minimal deviation from the optimal solution achieved by the state-of-the-art MILP solvers.
| Type: | Article |
|---|---|
| Title: | A hybrid deep Q-learning approach to online planning and rescheduling of single-stage multi-product continuous processes |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.compchemeng.2025.109415 |
| Publisher version: | https://doi.org/10.1016/j.compchemeng.2025.109415 |
| Language: | English |
| Additional information: | © The Author(s), 2025. This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. https://creativecommons.org/licenses/by/4.0/ |
| Keywords: | Autonomous online scheduling, Reinforcement learning, Deep neural networks, Continuous processes, Process scheduling, Q-learning |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Chemical Engineering |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10216918 |
Archive Staff Only
 |
View Item |
