UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control

Kurin, V; Igl, M; Rocktäschel, T; Boehmer, W; Whiteson, S; (2020) My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control. In: (Proceedings) ICLR 2021: Ninth International Conference on Learning Representations. OpenReview.net Green open access

[thumbnail of my_body_is_a_cage_the_role_of_.pdf] Text
my_body_is_a_cage_the_role_of_.pdf - Published Version
Available under License : See the attached licence file.

Download (3MB)

Abstract

Multitask Reinforcement Learning is a promising way to obtain models with better performance, generalisation, data efficiency, and robustness. Most existing work is limited to compatible settings, where the state and action space dimensions are the same across tasks. Graph Neural Networks (GNN) are one way to address incompatible environments, because they can process graphs of arbitrary size. They also allow practitioners to inject biases encoded in the structure of the input graph. Existing work in graph-based continuous control uses the physical morphology of the agent to construct the input graph, i.e., encoding limb features as node labels and using edges to connect the nodes if their corresponded limbs are physically connected. In this work, we present a series of ablations on existing methods that show that morphological information encoded in the graph does not improve their performance. Motivated by the hypothesis that any benefits GNNs extract from the graph structure are outweighed by difficulties they create for message passing, we also propose Amorpheus, a transformer-based approach. Further results show that, while Amorpheus ignores the morphological information that GNNs encode, it nonetheless substantially outperforms GNN-based methods.

Type: Proceedings paper
Title: My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control
Event: ICLR 2021: Ninth International Conference on Learning Representations
Open access status: An open access version is available from UCL Discovery
Publisher version: https://openreview.net/forum?id=N3zUDGN5lO
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Deep Reinforcement Learning, Multitask Reinforcement Learning, Graph Neural Networks, Continuous Control, Incompatible Environments
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10112090
Downloads since deposit
19Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item