Anthony, TW; Eccles, T; Tacchetti, A; Kramár, J; Gemp, IM; Hudson, TC; Porcel, N; ... Bachrach, Y; + view all Anthony, TW; Eccles, T; Tacchetti, A; Kramár, J; Gemp, IM; Hudson, TC; Porcel, N; Lanctot, M; Pérolat, J; Everett, R; Singh, S; Graepel, T; Bachrach, Y; - view fewer (2020) Learning to Play No-Press Diplomacy with Best Response Policy Iteration. In: Advances in Neural Information Processing Systems 33 pre-proceedings (NeurIPS 2020). NeurIPS (In press).

Preview

Text Learning to Play No-Press Diplomacy with Best Response Policy Iteration.pdf - Published Version Download (1MB) | Preview

Abstract

Recent advances in deep reinforcement learning (RL) have led to considerable progress in many 2-player zero-sum games, such as Go, Poker and Starcraft. The purely adversarial nature of such games allows for conceptually simple and principled application of RL methods. However real-world settings are many-agent, and agent interactions are complex mixtures of common-interest and competitive aspects. We consider Diplomacy, a 7-player board game designed to accentuate dilemmas resulting from many-agent interactions. It also features a large combinatorial action space and simultaneous moves, which are challenging for RL algorithms. We propose a simple yet effective approximate best response operator, designed to handle large combinatorial action spaces and simultaneous moves. We also introduce a family of policy iteration methods that approximate fictitious play. With these methods, we successfully apply RL to Diplomacy: we show that our agents convincingly outperform the previous state-of-the-art, and game theoretic equilibrium analysis shows that the new process yields consistent improvements.

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as JSONCSVXML

Archive Staff Only

View Item