Jiang, Z;
Minervini, P;
Jiang, M;
Rocktäschel, T;
(2021)
Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learning.
In: Dignum, F and Lomuscio, A and Endriss, U and Nowé, A, (eds.)
AAMAS '21: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems.
(pp. pp. 674-682).
ACM
Preview |
Text
Rocktäschel_2102.04220.pdf - Accepted Version Download (2MB) | Preview |
Abstract
Although reinforcement learning has been successfully applied in many domains in recent years, we still lack agents that can systematically generalize. While relational inductive biases that fit a task can improve generalization of RL agents, these biases are commonly hard-coded directly in the agent's neural architecture. In this work, we show that we can incorporate relational inductive biases, encoded in the form of relational graphs, into agents. Based on this insight, we propose Grid-to-Graph (GTG), a mapping from grid structures to relational graphs that carry useful spatial relational inductive biases when processed through a Relational Graph Convolution Network (R-GCN). We show that, with GTG, R-GCNs generalize better both in terms of in-distribution and out-of-distribution compared to baselines based on Convolutional Neural Networks and Neural Logic Machines on challenging procedurally generated environments and MinAtar. Furthermore, we show that GTG produces agents that can jointly reason over observations and environment dynamics encoded in knowledge bases.
| Type: | Proceedings paper |
|---|---|
| Title: | Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learning. |
| Event: | 20th International Conference on Autonomous Agents and MultiAgent Systems |
| ISBN-13: | 978-1-4503-8307-3 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.5555/3463952.3464034 |
| Publisher version: | https://dl.acm.org/doi/10.5555/3463952.3464034 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10129950 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
