Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2305.09458 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909111645372416 |
|---|---|
| author | Song, Yan Jiang, He Tian, Zheng Zhang, Haifeng Zhang, Yingping Zhu, Jiangcheng Dai, Zonghong Zhang, Weinan Wang, Jun |
| author_facet | Song, Yan Jiang, He Tian, Zheng Zhang, Haifeng Zhang, Yingping Zhu, Jiangcheng Dai, Zonghong Zhang, Weinan Wang, Jun |
| contents | Few multi-agent reinforcement learning (MARL) research on Google Research Football (GRF) focus on the 11v11 multi-agent full-game scenario and to the best of our knowledge, no open benchmark on this scenario has been released to the public. In this work, we fill the gap by providing a population-based MARL training pipeline and hyperparameter settings on multi-agent football scenario that outperforms the bot with difficulty 1.0 from scratch within 2 million steps. Our experiments serve as a reference for the expected performance of Independent Proximal Policy Optimization (IPPO), a state-of-the-art multi-agent reinforcement learning algorithm where each agent tries to maximize its own policy independently across various training configurations. Meanwhile, we open-source our training framework Light-MALib which extends the MALib codebase by distributed and asynchronized implementation with additional analytical tools for football games. Finally, we provide guidance for building strong football AI with population-based training and release diverse pretrained policies for benchmarking. The goal is to provide the community with a head start for whoever experiment their works on GRF and a simple-to-use population-based training framework for further improving their agents through self-play. The implementation is available at https://github.com/Shanghai-Digital-Brain-Laboratory/DB-Football. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_09458 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | An Empirical Study on Google Research Football Multi-agent Scenarios Song, Yan Jiang, He Tian, Zheng Zhang, Haifeng Zhang, Yingping Zhu, Jiangcheng Dai, Zonghong Zhang, Weinan Wang, Jun Machine Learning Multiagent Systems Few multi-agent reinforcement learning (MARL) research on Google Research Football (GRF) focus on the 11v11 multi-agent full-game scenario and to the best of our knowledge, no open benchmark on this scenario has been released to the public. In this work, we fill the gap by providing a population-based MARL training pipeline and hyperparameter settings on multi-agent football scenario that outperforms the bot with difficulty 1.0 from scratch within 2 million steps. Our experiments serve as a reference for the expected performance of Independent Proximal Policy Optimization (IPPO), a state-of-the-art multi-agent reinforcement learning algorithm where each agent tries to maximize its own policy independently across various training configurations. Meanwhile, we open-source our training framework Light-MALib which extends the MALib codebase by distributed and asynchronized implementation with additional analytical tools for football games. Finally, we provide guidance for building strong football AI with population-based training and release diverse pretrained policies for benchmarking. The goal is to provide the community with a head start for whoever experiment their works on GRF and a simple-to-use population-based training framework for further improving their agents through self-play. The implementation is available at https://github.com/Shanghai-Digital-Brain-Laboratory/DB-Football. |
| title | An Empirical Study on Google Research Football Multi-agent Scenarios |
| topic | Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2305.09458 |