Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917940915339264 |
|---|---|
| author | Lin, Muhan Shi, Shuyang Guo, Yue Tadiparthi, Vaishnav Chalaki, Behdad Pari, Ehsan Moradi Stepputtis, Simon Kim, Woojun Campbell, Joseph Sycara, Katia |
| author_facet | Lin, Muhan Shi, Shuyang Guo, Yue Tadiparthi, Vaishnav Chalaki, Behdad Pari, Ehsan Moradi Stepputtis, Simon Kim, Woojun Campbell, Joseph Sycara, Katia |
| contents | Credit assignment, the process of attributing credit or blame to individual agents for their contributions to a team's success or failure, remains a fundamental challenge in multi-agent reinforcement learning (MARL), particularly in environments with sparse rewards. Commonly-used approaches such as value decomposition often lead to suboptimal policies in these settings, and designing dense reward functions that align with human intuition can be complex and labor-intensive. In this work, we propose a novel framework where a large language model (LLM) generates dense, agent-specific rewards based on a natural language description of the task and the overall team goal. By learning a potential-based reward function over multiple queries, our method reduces the impact of ranking errors while allowing the LLM to evaluate each agent's contribution to the overall task. Through extensive experiments, we demonstrate that our approach achieves faster convergence and higher policy returns compared to state-of-the-art MARL baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_03723 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning Lin, Muhan Shi, Shuyang Guo, Yue Tadiparthi, Vaishnav Chalaki, Behdad Pari, Ehsan Moradi Stepputtis, Simon Kim, Woojun Campbell, Joseph Sycara, Katia Multiagent Systems Credit assignment, the process of attributing credit or blame to individual agents for their contributions to a team's success or failure, remains a fundamental challenge in multi-agent reinforcement learning (MARL), particularly in environments with sparse rewards. Commonly-used approaches such as value decomposition often lead to suboptimal policies in these settings, and designing dense reward functions that align with human intuition can be complex and labor-intensive. In this work, we propose a novel framework where a large language model (LLM) generates dense, agent-specific rewards based on a natural language description of the task and the overall team goal. By learning a potential-based reward function over multiple queries, our method reduces the impact of ranking errors while allowing the LLM to evaluate each agent's contribution to the overall task. Through extensive experiments, we demonstrate that our approach achieves faster convergence and higher policy returns compared to state-of-the-art MARL baselines. |
| title | Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning |
| topic | Multiagent Systems |
| url | https://arxiv.org/abs/2502.03723 |