Mguni, D; Jafferjee, T; Wang, J; Perez-Nieves, N; Song, W; Tong, F; Taylor, ME; ... Yang, Y; + view all Mguni, D; Jafferjee, T; Wang, J; Perez-Nieves, N; Song, W; Tong, F; Taylor, ME; Yang, T; Dai, Z; Chen, H; Zhu, J; Shao, K; Wang, J; Yang, Y; - view fewer (2023) Learning to Shape Rewards Using a Game of Two Partners. In: Williams, B and Chen, Y and Neville, J, (eds.) Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI 2023). (pp. pp. 11604-11612). Association for the Advancement of Artifcial Intelligence: Washington, D.C., USA.

Preview

Text 2103.09159.pdf - Accepted Version Download (4MB) | Preview

Abstract

Reward shaping (RS) is a powerful method in reinforcement learning (RL) for overcoming the problem of sparse or uninformative rewards. However, RS typically relies on manually engineered shaping-reward functions whose construction is time-consuming and error-prone. It also requires domain knowledge which runs contrary to the goal of autonomous learning. We introduce Reinforcement Learning Optimising Shaping Algorithm (ROSA), an automated reward shaping framework in which the shaping-reward function is constructed in a Markov game between two agents. A reward-shaping agent (Shaper) uses switching controls to determine which states to add shaping rewards for more efficient learning while the other agent (Controller) learns the optimal policy for the task using these shaped rewards. We prove that ROSA, which adopts existing RL algorithms, learns to construct a shaping-reward function that is beneficial to the task thus ensuring efficient convergence to high-performance policies. We demonstrate ROSA’s properties in three didactic experiments and show its superior performance against state-of-the-art RS algorithms in challenging sparse reward environments.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item