Lanctot, M;
Zambaldi, V;
Gruslys, A;
Lazaridou, A;
Tuyls, K;
Perolat, J;
Silver, D;
(2017)
A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning.
In: Guyon, I and Luxburg, UV and Bengio, S and Wallach, H and Fergus, R and Vishwanathan, S and Garnett, R, (eds.)
Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017).
Neural Information Processing Systems (NIPS): Long Beach, CA, USA.
Preview |
Text
Graepel_7007-a-unified-game-theoretic-approach-to-multiagent-reinforcement-learning.pdf - Published Version Download (512kB) | Preview |
Abstract
To achieve general intelligence, agents must learn how to interact with others in a shared environment: this is the challenge of multiagent reinforcement learning (MARL). The simplest form is independent reinforcement learning (InRL), where each agent treats its experience as part of its (non-stationary) environment. In this paper, we first observe that policies learned using InRL can overfit to the other agents’ policies during training, failing to sufficiently generalize duringn execution. We introduce a new metric, joint-policy correlation, to quantify this effect. We describe an algorithm for general MARL, based on approximate best responses to mixtures of policies generated using deep reinforcement learning, and empirical game-theoretic analysis to compute meta-strategies for policy selection. The algorithm generalizes previous ones such as InRL, iterated best response, double oracle, and fictitious play. Then, we present a scalable implementation which reduces the memory requirement using decoupled meta-solvers. Finally, we demonstrate the generality of the resulting policies in two partially observable settings: gridworld coordination games and poker.
| Type: | Proceedings paper |
|---|---|
| Title: | A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning |
| Event: | 31st Conference on Neural Information Processing Systems (NIPS), 4-9 December 2017, Long Beach, CA, USA |
| Location: | Long Beach, CA |
| Dates: | 2017 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | http://papers.nips.cc/paper/7007-a-unified-game-th... |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10069051 |
Loading...
Loading...Loading...| 1. |  United States | 4 |
| 2. |  China | 2 |
| 3. |  Germany | 2 |
| 4. |  South Africa | 1 |
| 5. |  Russian Federation | 1 |
Archive Staff Only
 |
View Item |
