Exploiting inter-agent coupling information for efficient reinforcement learning of cooperative LQR
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918004563902464 |
|---|---|
| author | Syed, Shahbaz P Qadri Bai, He |
| author_facet | Syed, Shahbaz P Qadri Bai, He |
| contents | Developing scalable and efficient reinforcement learning algorithms for cooperative multi-agent control has received significant attention over the past years. Existing literature has proposed inexact decompositions of local Q-functions based on empirical information structures between the agents. In this paper, we exploit inter-agent coupling information and propose a systematic approach to exactly decompose the local Q-function of each agent. We develop an approximate least square policy iteration algorithm based on the proposed decomposition and identify two architectures to learn the local Q-function for each agent. We establish that the worst-case sample complexity of the decomposition is equal to the centralized case and derive necessary and sufficient graphical conditions on the inter-agent couplings to achieve better sample efficiency. We demonstrate the improved sample efficiency and computational efficiency on numerical examples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_20927 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploiting inter-agent coupling information for efficient reinforcement learning of cooperative LQR Syed, Shahbaz P Qadri Bai, He Systems and Control Machine Learning Multiagent Systems Optimization and Control Developing scalable and efficient reinforcement learning algorithms for cooperative multi-agent control has received significant attention over the past years. Existing literature has proposed inexact decompositions of local Q-functions based on empirical information structures between the agents. In this paper, we exploit inter-agent coupling information and propose a systematic approach to exactly decompose the local Q-function of each agent. We develop an approximate least square policy iteration algorithm based on the proposed decomposition and identify two architectures to learn the local Q-function for each agent. We establish that the worst-case sample complexity of the decomposition is equal to the centralized case and derive necessary and sufficient graphical conditions on the inter-agent couplings to achieve better sample efficiency. We demonstrate the improved sample efficiency and computational efficiency on numerical examples. |
| title | Exploiting inter-agent coupling information for efficient reinforcement learning of cooperative LQR |
| topic | Systems and Control Machine Learning Multiagent Systems Optimization and Control |
| url | https://arxiv.org/abs/2504.20927 |