UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Differentiable Game Mechanics

Letcher, A; Balduzzi, D; Racaniere, S; Martens, J; Foerster, J; Tuyls, K; Graepel, T; (2019) Differentiable Game Mechanics. Journal of Machine Learning Research , 20 (84) pp. 1-40. Green open access

[thumbnail of Graepel_Inequity aversion improves cooperation in intertemporal social dilemmas_AOP.pdf]
Preview
Text
Graepel_Inequity aversion improves cooperation in intertemporal social dilemmas_AOP.pdf - Published Version

Download (2MB) | Preview

Abstract

Deep learning is built on the foundational guarantee that gradient descent on an objective function converges to local minima. Unfortunately, this guarantee fails in settings, such as generative adversarial nets, that exhibit multiple interacting losses. The behavior of gradient-based methods in games is not well understood and is becoming increasingly important as adversarial and multi-objective architectures proliferate. In this paper, we develop new tools to understand and control the dynamics in n-player differentiable games. The key result is to decompose the game Jacobian into two components. The first, symmetric component, is related to potential games, which reduce to gradient descent on an implicit function. The second, antisymmetric component, relates to Hamiltonian games, a new class of games that obey a conservation law akin to conservation laws in classical mechanical systems. The decomposition motivates Symplectic Gradient Adjustment (SGA), a new algorithm for finding stable fixed points in differentiable games. Basic experiments show SGA is competitive with recently proposed algorithms for finding stable fixed points in GANs – while at the same time being applicable to, and having guarantees in, much more general cases.

Type: Article
Title: Differentiable Game Mechanics
Open access status: An open access version is available from UCL Discovery
Publisher version: http://jmlr.org/papers/v20/19-008.html
Language: English
Additional information: © The Authors. License: CC-BY 4.0, see (https://creativecommons.org/licenses/by/4.0/).
Keywords: game theory, generative adversarial networks, deep learning, classical mechanics, hamiltonian mechanics, gradient descent, dynamical systems
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10076314
Downloads since deposit
19Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item