Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910354373607424 |
|---|---|
| author | Wang, Liangzhou Zhu, Kaiwen Zhu, Fengming Yao, Xinghu Zhang, Shujie Ye, Deheng Fu, Haobo Fu, Qiang Yang, Wei |
| author_facet | Wang, Liangzhou Zhu, Kaiwen Zhu, Fengming Yao, Xinghu Zhang, Shujie Ye, Deheng Fu, Haobo Fu, Qiang Yang, Wei |
| contents | Reaching consensus is key to multi-agent coordination. To accomplish a cooperative task, agents need to coherently select optimal joint actions to maximize the team reward. However, current cooperative multi-agent reinforcement learning (MARL) methods usually do not explicitly take consensus into consideration, which may cause miscoordination problem. In this paper, we propose a model-based consensus mechanism to explicitly coordinate multiple agents. The proposed Multi-agent Goal Imagination (MAGI) framework guides agents to reach consensus with an Imagined common goal. The common goal is an achievable state with high value, which is obtained by sampling from the distribution of future states. We directly model this distribution with a self-supervised generative model, thus alleviating the "curse of dimensinality" problem induced by multi-agent multi-step policy rollout commonly used in model-based methods. We show that such efficient consensus mechanism can guide all agents cooperatively reaching valuable future states. Results on Multi-agent Particle-Environments and Google Research Football environment demonstrate the superiority of MAGI in both sample efficiency and performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_03172 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination Wang, Liangzhou Zhu, Kaiwen Zhu, Fengming Yao, Xinghu Zhang, Shujie Ye, Deheng Fu, Haobo Fu, Qiang Yang, Wei Artificial Intelligence Machine Learning Reaching consensus is key to multi-agent coordination. To accomplish a cooperative task, agents need to coherently select optimal joint actions to maximize the team reward. However, current cooperative multi-agent reinforcement learning (MARL) methods usually do not explicitly take consensus into consideration, which may cause miscoordination problem. In this paper, we propose a model-based consensus mechanism to explicitly coordinate multiple agents. The proposed Multi-agent Goal Imagination (MAGI) framework guides agents to reach consensus with an Imagined common goal. The common goal is an achievable state with high value, which is obtained by sampling from the distribution of future states. We directly model this distribution with a self-supervised generative model, thus alleviating the "curse of dimensinality" problem induced by multi-agent multi-step policy rollout commonly used in model-based methods. We show that such efficient consensus mechanism can guide all agents cooperatively reaching valuable future states. Results on Multi-agent Particle-Environments and Google Research Football environment demonstrate the superiority of MAGI in both sample efficiency and performance. |
| title | Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2403.03172 |