Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916226931884032 |
|---|---|
| author | Li, Dapeng Dong, Hang Wang, Lu Qiao, Bo Qin, Si Lin, Qingwei Zhang, Dongmei Zhang, Qi Xu, Zhiwei Zhang, Bin Fan, Guoliang |
| author_facet | Li, Dapeng Dong, Hang Wang, Lu Qiao, Bo Qin, Si Lin, Qingwei Zhang, Dongmei Zhang, Qi Xu, Zhiwei Zhang, Bin Fan, Guoliang |
| contents | In recent years, multi-agent reinforcement learning algorithms have made significant advancements in diverse gaming environments, leading to increased interest in the broader application of such techniques. To address the prevalent challenge of partial observability, communication-based algorithms have improved cooperative performance through the sharing of numerical embedding between agents. However, the understanding of the formation of collaborative mechanisms is still very limited, making designing a human-understandable communication mechanism a valuable problem to address. In this paper, we propose a novel multi-agent reinforcement learning algorithm that embeds large language models into agents, endowing them with the ability to generate human-understandable verbal communication. The entire framework has a message module and an action module. The message module is responsible for generating and sending verbal messages to other agents, effectively enhancing information sharing among agents. To further enhance the message module, we employ a teacher model to generate message labels from the global view and update the student model through Supervised Fine-Tuning (SFT). The action module receives messages from other agents and selects actions based on current local observations and received messages. Experiments conducted on the Overcooked game demonstrate our method significantly enhances the learning efficiency and performance of existing methods, while also providing an interpretable tool for humans to understand the process of multi-agent cooperation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_17780 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning Li, Dapeng Dong, Hang Wang, Lu Qiao, Bo Qin, Si Lin, Qingwei Zhang, Dongmei Zhang, Qi Xu, Zhiwei Zhang, Bin Fan, Guoliang Multiagent Systems Artificial Intelligence In recent years, multi-agent reinforcement learning algorithms have made significant advancements in diverse gaming environments, leading to increased interest in the broader application of such techniques. To address the prevalent challenge of partial observability, communication-based algorithms have improved cooperative performance through the sharing of numerical embedding between agents. However, the understanding of the formation of collaborative mechanisms is still very limited, making designing a human-understandable communication mechanism a valuable problem to address. In this paper, we propose a novel multi-agent reinforcement learning algorithm that embeds large language models into agents, endowing them with the ability to generate human-understandable verbal communication. The entire framework has a message module and an action module. The message module is responsible for generating and sending verbal messages to other agents, effectively enhancing information sharing among agents. To further enhance the message module, we employ a teacher model to generate message labels from the global view and update the student model through Supervised Fine-Tuning (SFT). The action module receives messages from other agents and selects actions based on current local observations and received messages. Experiments conducted on the Overcooked game demonstrate our method significantly enhances the learning efficiency and performance of existing methods, while also providing an interpretable tool for humans to understand the process of multi-agent cooperation. |
| title | Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning |
| topic | Multiagent Systems Artificial Intelligence |
| url | https://arxiv.org/abs/2404.17780 |