Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methods
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2019
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916710718636032 |
|---|---|
| author | Carmona, René Laurière, Mathieu Tan, Zongjun |
| author_facet | Carmona, René Laurière, Mathieu Tan, Zongjun |
| contents | We investigate reinforcement learning in the setting of Markov decision processes for a large number of exchangeable agents interacting in a mean field manner. Applications include, for example, the control of a large number of robots communicating through a central unit dispatching the optimal policy computed by maximizing an aggregate reward. An approximate solution is obtained by learning the optimal policy of a generic agent interacting with the statistical distribution of the states and actions of the other agents. We first provide a full analysis this discrete-time mean field control problem. We then rigorously prove the convergence of exact and model-free policy gradient methods in a mean-field linear-quadratic setting and establish bounds on the rates of convergence. We also provide graphical evidence of the convergence based on implementations of our algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1910_04295 |
| institution | arXiv |
| publishDate | 2019 |
| record_format | arxiv |
| spellingShingle | Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methods Carmona, René Laurière, Mathieu Tan, Zongjun Optimization and Control Machine Learning We investigate reinforcement learning in the setting of Markov decision processes for a large number of exchangeable agents interacting in a mean field manner. Applications include, for example, the control of a large number of robots communicating through a central unit dispatching the optimal policy computed by maximizing an aggregate reward. An approximate solution is obtained by learning the optimal policy of a generic agent interacting with the statistical distribution of the states and actions of the other agents. We first provide a full analysis this discrete-time mean field control problem. We then rigorously prove the convergence of exact and model-free policy gradient methods in a mean-field linear-quadratic setting and establish bounds on the rates of convergence. We also provide graphical evidence of the convergence based on implementations of our algorithms. |
| title | Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methods |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/1910.04295 |