Cooperative Game-Theoretic Credit Assignment for Multi-Agent Policy Gradients via the Core
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918380332646400 |
|---|---|
| author | Ji, Mengda Xu, Genjiu Jia, Keke Duan, Zekun Qiu, Yong Ge, Jianjun Li, Mingqiang |
| author_facet | Ji, Mengda Xu, Genjiu Jia, Keke Duan, Zekun Qiu, Yong Ge, Jianjun Li, Mingqiang |
| contents | This work focuses on the credit assignment problem in cooperative multi-agent reinforcement learning (MARL). Sharing the global advantage among agents often leads to insufficient policy optimization, as it fails to capture the coalitional contributions of different agents. In this work, we revisit the policy update process from a coalitional perspective and propose CORA, an advantage allocation method guided by a cooperative game-theoretic core allocation. By evaluating the marginal contributions of different coalitions and combining clipped double Q-learning to mitigate overestimation bias, CORA estimates coalition-wise advantages. The core formulation enforces coalition-wise lower bounds on allocated credits, so that coalitions with higher advantages receive stronger total incentives for their participating agents, enabling the global advantage to be attributed to different coalition strategies and promoting coordinated optimal behavior. To reduce computational overhead, we employ random coalition sampling to approximate the core allocation efficiently. Experiments on matrix games, differential games, and multi-agent collaboration benchmarks demonstrate that our method outperforms baselines. These findings highlight the importance of coalition-level credit assignment and cooperative games for advancing multi-agent learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04265 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cooperative Game-Theoretic Credit Assignment for Multi-Agent Policy Gradients via the Core Ji, Mengda Xu, Genjiu Jia, Keke Duan, Zekun Qiu, Yong Ge, Jianjun Li, Mingqiang Multiagent Systems Artificial Intelligence Computer Science and Game Theory Machine Learning This work focuses on the credit assignment problem in cooperative multi-agent reinforcement learning (MARL). Sharing the global advantage among agents often leads to insufficient policy optimization, as it fails to capture the coalitional contributions of different agents. In this work, we revisit the policy update process from a coalitional perspective and propose CORA, an advantage allocation method guided by a cooperative game-theoretic core allocation. By evaluating the marginal contributions of different coalitions and combining clipped double Q-learning to mitigate overestimation bias, CORA estimates coalition-wise advantages. The core formulation enforces coalition-wise lower bounds on allocated credits, so that coalitions with higher advantages receive stronger total incentives for their participating agents, enabling the global advantage to be attributed to different coalition strategies and promoting coordinated optimal behavior. To reduce computational overhead, we employ random coalition sampling to approximate the core allocation efficiently. Experiments on matrix games, differential games, and multi-agent collaboration benchmarks demonstrate that our method outperforms baselines. These findings highlight the importance of coalition-level credit assignment and cooperative games for advancing multi-agent learning. |
| title | Cooperative Game-Theoretic Credit Assignment for Multi-Agent Policy Gradients via the Core |
| topic | Multiagent Systems Artificial Intelligence Computer Science and Game Theory Machine Learning |
| url | https://arxiv.org/abs/2506.04265 |