Tirumala Bukkapatnam, Dhruva; (2024) Teaching Robots to Play: The Power of Prior Knowledge for Practical Reinforcement Learning. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Tirumala Bukkapatnam_10194432_thesis.pdf Download (20MB) | Preview

Abstract

The field of Artifical Intelligence has seen rapid advancements in recent years with powerful generative models like Chat-GPT and Gemini. A key element that enabled these advances was the availability of large corpuses of structured training data. This requirement is difficult to meet for many real world domains like Robotics. Designing controllers for these domains remains a challenging open problem that could unlock further progress. This thesis explores how Reinforcement Learning (RL) when combined with sources of prior knowledge can help alleviate this issue. We begin by proposing a framework to reason about priors in the RL paradigm using tools from probabilistic modeling. We demonstrate how contemporary algorithms fit into this framework and how we can leverage insights from this view to improve learning efficiency on a range of simulated multi-task control domains. Building upon this foundation, we introduce Skills, an algorithm that learns to combine previously acquired sub-policies into flexible strategies. This "mix-and-match" approach enables robots to adapt to diverse tasks by assembling relevant skill combinations, significantly outperforming traditional single-policy RL methods. To further enhance performance, we present RaE, a simple and surprisingly effective technique to reuse previously collected data stored in replay buffers across different experiments. RaE leads to considerably improved asymptotic performance and achieves state-of-the-art results on a number of simulated environments. Finally, we bridge the gap between simulation and real-world applications by demonstrating the combined power of these techniques. We train a policy to play soccer on a 2-legged humanoid robot platform (OP3) purely using onboard sensors and compute, showcasing the remarkable capabilities that can be achieved through these methods. This thesis contributes to the development of robots that learn faster, adapt better, and achieve superior performance. We hope these insights will be useful in order to generate robust low level controllers through learning based methods.

Download activity - last month

Export as XMLJSONCSV

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item