Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.12123 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914159844655104 |
|---|---|
| author | Liu, Zejiao Tu, Junqi Hong, Yitian Xiong, Luolin Jin, Yaochu Tang, Yang Li, Fangfei |
| author_facet | Liu, Zejiao Tu, Junqi Hong, Yitian Xiong, Luolin Jin, Yaochu Tang, Yang Li, Fangfei |
| contents | In cooperative Multi-Agent Reinforcement Learning (MARL), efficient exploration is crucial for optimizing the performance of joint policy. However, existing methods often update joint policies via independent agent exploration, without coordination among agents, which inherently constrains the expressive capacity and exploration of joint policies. To address this issue, we propose a conductor-based joint policy framework that directly enhances the expressive capacity of joint policies and coordinates exploration. In addition, we develop a Hierarchical Conductor-based Policy Optimization (HCPO) algorithm that instructs policy updates for the conductor and agents in a direction aligned with performance improvement. A rigorous theoretical guarantee further establishes the monotonicity of the joint policy optimization process. By deploying local conductors, HCPO retains centralized training benefits while eliminating inter-agent communication during execution. Finally, we evaluate HCPO on three challenging benchmarks: StarCraftII Multi-agent Challenge, Multi-agent MuJoCo, and Multi-agent Particle Environment. The results indicate that HCPO outperforms competitive MARL baselines regarding cooperative efficiency and stability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_12123 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HCPO: Hierarchical Conductor-Based Policy Optimization in Multi-Agent Reinforcement Learning Liu, Zejiao Tu, Junqi Hong, Yitian Xiong, Luolin Jin, Yaochu Tang, Yang Li, Fangfei Machine Learning In cooperative Multi-Agent Reinforcement Learning (MARL), efficient exploration is crucial for optimizing the performance of joint policy. However, existing methods often update joint policies via independent agent exploration, without coordination among agents, which inherently constrains the expressive capacity and exploration of joint policies. To address this issue, we propose a conductor-based joint policy framework that directly enhances the expressive capacity of joint policies and coordinates exploration. In addition, we develop a Hierarchical Conductor-based Policy Optimization (HCPO) algorithm that instructs policy updates for the conductor and agents in a direction aligned with performance improvement. A rigorous theoretical guarantee further establishes the monotonicity of the joint policy optimization process. By deploying local conductors, HCPO retains centralized training benefits while eliminating inter-agent communication during execution. Finally, we evaluate HCPO on three challenging benchmarks: StarCraftII Multi-agent Challenge, Multi-agent MuJoCo, and Multi-agent Particle Environment. The results indicate that HCPO outperforms competitive MARL baselines regarding cooperative efficiency and stability. |
| title | HCPO: Hierarchical Conductor-Based Policy Optimization in Multi-Agent Reinforcement Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2511.12123 |