Nieves, Nicolas Perez;
Yang, Yaodong;
Slumbers, Oliver;
Mguni, David Henry;
Wen, Ying;
Wang, Jun;
(2021)
Modelling Behavioural Diversity for Learning in Open-Ended Games.
In: Meila, M and Zhang, T, (eds.)
Proceedings of Machine Learning Research (PMLR).
MLResearchPress
Preview |
PDF
perez-nieves21a.pdf - Published Version Download (1MB) | Preview |
Abstract
Promoting behavioural diversity is critical for solving games with non-transitive dynamics where strategic cycles exist, and there is no consistent winner (e.g., Rock-Paper-Scissors). Yet, there is a lack of rigorous treatment for defining diversity and constructing diversity-aware learning dynamics. In this work, we offer a geometric interpretation of behavioural diversity in games and introduce a novel diversity metric based on determinantal point processes (DPP). By incorporating the diversity metric into best-response dynamics, we develop diverse fictitious play and diverse policy-space response oracle for solving normal-form games and open-ended games. We prove the uniqueness of the diverse best response and the convergence of our algorithms on two-player games. Importantly, we show that maximising the DPP-based diversity metric guarantees to enlarge the gamescape - convex polytopes spanned by agents' mixtures of strategies. To validate our diversity-aware solvers, we test on tens of games that show strong non-transitivity. Results suggest that our methods achieve at least the same, and in most games, lower exploitability than PSRO solvers by finding effective and diverse strategies.
| Type: | Proceedings paper |
|---|---|
| Title: | Modelling Behavioural Diversity for Learning in Open-Ended Games |
| Event: | International Conference on Machine Learning (ICML) |
| Location: | ELECTR NETWORK |
| Dates: | 18 Jul 2021 - 24 Jul 2021 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.mlr.press/v139/ |
| Language: | English |
| Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
| Keywords: | Science & Technology, Technology, Computer Science, Artificial Intelligence, Computer Science, LEVEL |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10172309 |
Archive Staff Only
 |
View Item |
