Multi-Agent Reinforcement Learning via Distributed MPC as a Function Approximator
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917883020312576 |
|---|---|
| author | Mallick, Samuel Airaldi, Filippo Dabiri, Azita De Schutter, Bart |
| author_facet | Mallick, Samuel Airaldi, Filippo Dabiri, Azita De Schutter, Bart |
| contents | This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC scheme as a function approximator, with a structure allowing for distributed learning and deployment. We then show that Q-learning updates can be performed distributively without introducing nonstationarity, by reconstructing a centralized learning update. The effectiveness of the approach is demonstrated on two numerical examples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_05166 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Multi-Agent Reinforcement Learning via Distributed MPC as a Function Approximator Mallick, Samuel Airaldi, Filippo Dabiri, Azita De Schutter, Bart Systems and Control This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC scheme as a function approximator, with a structure allowing for distributed learning and deployment. We then show that Q-learning updates can be performed distributively without introducing nonstationarity, by reconstructing a centralized learning update. The effectiveness of the approach is demonstrated on two numerical examples. |
| title | Multi-Agent Reinforcement Learning via Distributed MPC as a Function Approximator |
| topic | Systems and Control |
| url | https://arxiv.org/abs/2312.05166 |