Samvelyan, M; Khan, A; Dennis, M; Jiang, M; Parker-Holder, J; Foerster, J; Raileanu, R; Samvelyan, M; Khan, A; Dennis, M; Jiang, M; Parker-Holder, J; Foerster, J; Raileanu, R; Rocktäschel, T; - view fewer (2023) MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning. In: Proceedings of the Eleventh International Conference on Learning Representations. : Kigali, Rwanda.

Preview

Text 2303.03376v1.pdf - Accepted Version Download (3MB) | Preview

Abstract

Open-ended learning methods that automatically generate a curriculum of increasingly challenging tasks serve as a promising avenue toward generally capable reinforcement learning agents. Existing methods adapt curricula independently over either environment parameters (in single-agent settings) or co-player policies (in multi-agent settings). However, the strengths and weaknesses of co-players can manifest themselves differently depending on environmental features. It is thus crucial to consider the dependency between the environment and co-player when shaping a curriculum in multi-agent domains. In this work, we use this insight and extend Unsupervised Environment Design (UED) to multi-agent environments. We then introduce Multi-Agent Environment Design Strategist for Open-Ended Learning (MAESTRO), the first multi-agent UED approach for two-player zero-sum settings. MAESTRO efficiently produces adversarial, joint curricula over both environments and co-players and attains minimax-regret guarantees at Nash equilibrium. Our experiments show that MAESTRO outperforms a number of strong baselines on competitive two-player games, spanning discrete and continuous control settings.

Download activity - last month

Export as XMLCSVJSON

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item