Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915572587954176 |
|---|---|
| author | Anand, Emile Karmarkar, Ishani Qu, Guannan |
| author_facet | Anand, Emile Karmarkar, Ishani Qu, Guannan |
| contents | Designing efficient algorithms for multi-agent reinforcement learning (MARL) is fundamentally challenging because the size of the joint state and action spaces grows exponentially in the number of agents. These difficulties are exacerbated when balancing sequential global decision-making with local agent interactions. In this work, we propose a new algorithm $\texttt{SUBSAMPLE-MFQ}$ ($\textbf{Subsample}$-$\textbf{M}$ean-$\textbf{F}$ield-$\textbf{Q}$-learning) and a decentralized randomized policy for a system with $n$ agents. For any $k\leq n$, our algorithm learns a policy for the system in time polynomial in $k$. We prove that this learned policy converges to the optimal policy on the order of $\tilde{O}(1/\sqrt{k})$ as the number of subsampled agents $k$ increases. In particular, this bound is independent of the number of agents $n$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_00661 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning Anand, Emile Karmarkar, Ishani Qu, Guannan Machine Learning Artificial Intelligence Multiagent Systems Systems and Control Optimization and Control 60J20, 68T99 I.2.11 Designing efficient algorithms for multi-agent reinforcement learning (MARL) is fundamentally challenging because the size of the joint state and action spaces grows exponentially in the number of agents. These difficulties are exacerbated when balancing sequential global decision-making with local agent interactions. In this work, we propose a new algorithm $\texttt{SUBSAMPLE-MFQ}$ ($\textbf{Subsample}$-$\textbf{M}$ean-$\textbf{F}$ield-$\textbf{Q}$-learning) and a decentralized randomized policy for a system with $n$ agents. For any $k\leq n$, our algorithm learns a policy for the system in time polynomial in $k$. We prove that this learned policy converges to the optimal policy on the order of $\tilde{O}(1/\sqrt{k})$ as the number of subsampled agents $k$ increases. In particular, this bound is independent of the number of agents $n$. |
| title | Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning |
| topic | Machine Learning Artificial Intelligence Multiagent Systems Systems and Control Optimization and Control 60J20, 68T99 I.2.11 |
| url | https://arxiv.org/abs/2412.00661 |