Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908552891727872 |
|---|---|
| author | Sebastián, Eduardo Keskar, Maitrayee Iqbal, Eeman Montijano, Eduardo Sagüés, Carlos Atanasov, Nikolay |
| author_facet | Sebastián, Eduardo Keskar, Maitrayee Iqbal, Eeman Montijano, Eduardo Sagüés, Carlos Atanasov, Nikolay |
| contents | Multi-agent games in dynamic nonlinear settings are challenging due to the time-varying interactions among the agents and the non-stationarity of the (potential) Nash equilibria. In this paper we consider model-free games, where agent transitions and costs are observed without knowledge of the transition and cost functions that generate them. We propose a policy gradient approach to learn distributed policies that follow the communication structure in multi-team games, with multiple agents per team. Our formulation is inspired by the structure of distributed policies in linear quadratic games, which take the form of time-varying linear feedback gains. In the nonlinear case, we model the policies as nonlinear feedback gains, parameterized by self-attention layers to account for the time-varying multi-agent communication topology. We demonstrate that our distributed policy gradient approach achieves strong performance in several settings, including distributed linear and nonlinear regulation, and simulated and real multi-robot pursuit-and-evasion games. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_18371 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games Sebastián, Eduardo Keskar, Maitrayee Iqbal, Eeman Montijano, Eduardo Sagüés, Carlos Atanasov, Nikolay Systems and Control Multiagent Systems Robotics Multi-agent games in dynamic nonlinear settings are challenging due to the time-varying interactions among the agents and the non-stationarity of the (potential) Nash equilibria. In this paper we consider model-free games, where agent transitions and costs are observed without knowledge of the transition and cost functions that generate them. We propose a policy gradient approach to learn distributed policies that follow the communication structure in multi-team games, with multiple agents per team. Our formulation is inspired by the structure of distributed policies in linear quadratic games, which take the form of time-varying linear feedback gains. In the nonlinear case, we model the policies as nonlinear feedback gains, parameterized by self-attention layers to account for the time-varying multi-agent communication topology. We demonstrate that our distributed policy gradient approach achieves strong performance in several settings, including distributed linear and nonlinear regulation, and simulated and real multi-robot pursuit-and-evasion games. |
| title | Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games |
| topic | Systems and Control Multiagent Systems Robotics |
| url | https://arxiv.org/abs/2509.18371 |