UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning

Lanctot, M; Zambaldi, V; Gruslys, A; Lazaridou, A; Tuyls, K; Perolat, J; Silver, D; (2017) A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning. In: Guyon, I and Luxburg, UV and Bengio, S and Wallach, H and Fergus, R and Vishwanathan, S and Garnett, R, (eds.) Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017). Neural Information Processing Systems (NIPS): Long Beach, CA, USA. Green open access

[thumbnail of Graepel_7007-a-unified-game-theoretic-approach-to-multiagent-reinforcement-learning.pdf]
Preview
Text
Graepel_7007-a-unified-game-theoretic-approach-to-multiagent-reinforcement-learning.pdf - Published Version

Download (512kB) | Preview

Abstract

To achieve general intelligence, agents must learn how to interact with others in a shared environment: this is the challenge of multiagent reinforcement learning (MARL). The simplest form is independent reinforcement learning (InRL), where each agent treats its experience as part of its (non-stationary) environment. In this paper, we first observe that policies learned using InRL can overfit to the other agents’ policies during training, failing to sufficiently generalize duringn execution. We introduce a new metric, joint-policy correlation, to quantify this effect. We describe an algorithm for general MARL, based on approximate best responses to mixtures of policies generated using deep reinforcement learning, and empirical game-theoretic analysis to compute meta-strategies for policy selection. The algorithm generalizes previous ones such as InRL, iterated best response, double oracle, and fictitious play. Then, we present a scalable implementation which reduces the memory requirement using decoupled meta-solvers. Finally, we demonstrate the generality of the resulting policies in two partially observable settings: gridworld coordination games and poker.

Type: Proceedings paper
Title: A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning
Event: 31st Conference on Neural Information Processing Systems (NIPS), 4-9 December 2017, Long Beach, CA, USA
Location: Long Beach, CA
Dates: 2017
Open access status: An open access version is available from UCL Discovery
Publisher version: http://papers.nips.cc/paper/7007-a-unified-game-th...
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10069051
Downloads since deposit
0Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item