Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911672513331200 |
|---|---|
| author | Li, Simin Mao, Zihao Yuwei, Zheng Wang, Linhao Xu, Ruixiao Ma, Chengdong Liu, Zhiqian Yu, Xin Ma, Yuqing Wang, Xin Luo, Jie An, Bo Yang, Yaodong Lv, Weifeng Liu, Xianglong |
| author_facet | Li, Simin Mao, Zihao Yuwei, Zheng Wang, Linhao Xu, Ruixiao Ma, Chengdong Liu, Zhiqian Yu, Xin Ma, Yuqing Wang, Xin Luo, Jie An, Bo Yang, Yaodong Lv, Weifeng Liu, Xianglong |
| contents | Partial agent failure becomes inevitable when systems scale up, making it crucial to identify the subset of agents whose failure causes worst-case system performance degradations. We study this Vulnerable Agent Identification (VAI) problem in large-scale multi-agent reinforcement learning (MARL). We frame VAI as a Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), where the upper level selects vulnerable agents as an NP-hard task and the lower level learns their worst-case adversarial policies via mean-field MARL. The two problems are coupled together, making HAD-MFC difficult to solve. To handle this, we first decouple the hierarchical process by Fenchel-Rockafellar transform, resulting a regularized mean-field Bellman operator for upper level that enables independent learning at each level, thus reducing computational complexity. We next reformulate the upper-level NP-hard problem as an MDP with dense rewards, allowing sequential identification of vulnerable agents via greedy and RL algorithms. This decomposition provably preserves the optimal solution. Experiments show our method effectively identifies more vulnerable agents in large-scale MARL and the rule-based system, fooling system into worse failures, and reveals the vulnerability of each agent in large systems. Code available at https://github.com/Waken-dream/VAI |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_15103 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning Li, Simin Mao, Zihao Yuwei, Zheng Wang, Linhao Xu, Ruixiao Ma, Chengdong Liu, Zhiqian Yu, Xin Ma, Yuqing Wang, Xin Luo, Jie An, Bo Yang, Yaodong Lv, Weifeng Liu, Xianglong Multiagent Systems Artificial Intelligence Partial agent failure becomes inevitable when systems scale up, making it crucial to identify the subset of agents whose failure causes worst-case system performance degradations. We study this Vulnerable Agent Identification (VAI) problem in large-scale multi-agent reinforcement learning (MARL). We frame VAI as a Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), where the upper level selects vulnerable agents as an NP-hard task and the lower level learns their worst-case adversarial policies via mean-field MARL. The two problems are coupled together, making HAD-MFC difficult to solve. To handle this, we first decouple the hierarchical process by Fenchel-Rockafellar transform, resulting a regularized mean-field Bellman operator for upper level that enables independent learning at each level, thus reducing computational complexity. We next reformulate the upper-level NP-hard problem as an MDP with dense rewards, allowing sequential identification of vulnerable agents via greedy and RL algorithms. This decomposition provably preserves the optimal solution. Experiments show our method effectively identifies more vulnerable agents in large-scale MARL and the rule-based system, fooling system into worse failures, and reveals the vulnerability of each agent in large systems. Code available at https://github.com/Waken-dream/VAI |
| title | Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning |
| topic | Multiagent Systems Artificial Intelligence |
| url | https://arxiv.org/abs/2509.15103 |