Banarse, D;
Bachrach, Y;
Liu, S;
Lever, G;
Heess, N;
Fernando, C;
Kohli, P;
(2019)
The Body is Not a Given: Joint Agent Policy Learning and Morphology Evolution.
In: Elkind, Edith and Veloso, Manuela, (eds.)
Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019).
(pp. pp. 1134-1142).
IFAAMAS (International Foundation of Autonomous Agents and MultiAgent Systems): Montreal, Canada.
Preview |
Text
The body is not a given AMAAS Banarse 2019.pdf - Published Version Download (6MB) | Preview |
Abstract
Reinforcement learning (RL) has proven to be a powerful paradigm for deriving complex behaviors from simple reward signals in a wide range of environments. When applying RL to continuous control agents in simulated physics environments, the body is usually considered to be part of the environment. However, during evolution the physical body of biological organisms and their controlling brains are co-evolved, thus exploring a much larger space of actuator/controller configurations. Put differently, the intelligence does not reside only in the agent’s mind, but also in the design of their body. We propose a method for uncovering strong agents, consisting of a good combination of a body and policy, based on combining RL with an evolutionary procedure. Given the resulting agent, we also propose an approach for identifying the body changes that contributed the most to the agent performance. We use the Shapley value from cooperative game theory to find the fair contribution of individual components, taking into account synergies between components. We evaluate our methods in an environment similar to the the recently proposed Robo-Sumo task, where agents in a software physics simulator compete in tipping over their opponent or pushing them out of the arena. Our results show that the proposed methods are indeed capable of generating strong agents, significantly outperforming baselines that focus on optimizing the agent policy alone.
| Type: | Proceedings paper |
|---|---|
| Title: | The Body is Not a Given: Joint Agent Policy Learning and Morphology Evolution |
| Event: | 18th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2019), 13-17 May 2019, Montreal, Canada |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | http://www.ifaamas.org/Proceedings/aamas2019/pdfs/... |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Reinforcement Learning; Evolutionary Computation |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10081611 |
Archive Staff Only
 |
View Item |
