Lou, X;
Guo, J;
Zhang, J;
Wang, J;
Huang, K;
Du, Y;
(2023)
PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination.
In:
Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems.
(pp. pp. 679-688).
International Foundation for Autonomous Agents and Multiagent Systems
Preview |
Text
2301.06387.pdf - Accepted Version Download (4MB) | Preview |
Abstract
Zero-shot human-AI coordination holds the promise of collaborating with humans without human data. Prevailing methods try to train the ego agent with a population of partners via self-play. However, these methods suffer from two problems: 1) The diversity of a population with finite partners is limited, thereby limiting the capacity of the trained ego agent to collaborate with a novel human; 2) Current methods only provide a common best response for every partner in the population, which may result in poor zero-shot coordination performance with a novel partner or humans. To address these issues, we first propose the policy ensemble method to increase the diversity of partners in the population, and then develop a context-aware method enabling the ego agent to analyze and identify the partner's potential policy primitives so that it can take different actions accordingly. In this way, the ego agent is able to learn more universal cooperative behaviors for collaborating with diverse partners. We conduct experiments on the Overcooked environment, and evaluate the zero-shot human-AI coordination performance of our method with both behavior-cloned human proxies and real humans. The results demonstrate that our method significantly increases the diversity of partners and enables ego agents to learn more diverse behaviors than baselines, thus achieving state-of-the-art performance in all scenarios. We also open-source a human-AI coordination study framework on the Overcooked for the convenience of future studies. Codes and demo videos are available at https://sites.google.com/view/pecan-overcooked.
| Type: | Proceedings paper |
|---|---|
| Title: | PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination |
| Event: | AAMAS '23: 2023 International Conference on Autonomous Agents and Multiagent Systems |
| ISBN-13: | 9781450394321 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://dl.acm.org/doi/abs/10.5555/3545946.3598700 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Zero-shot Human-AI Coordination; Multi-agent; Reinforcement Learning |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10178133 |
Archive Staff Only
 |
View Item |
