Asynchronous Parallel Policy Gradient Methods for the Linear Quadratic Regulator
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913416087601152 |
|---|---|
| author | Sha, Xingyu Zhao, Feiran You, Keyou |
| author_facet | Sha, Xingyu Zhao, Feiran You, Keyou |
| contents | Learning policies in an asynchronous parallel way is essential to the numerous successes of RL for solving large-scale problems. However, their convergence performance is still not rigorously evaluated. To this end, we adopt the asynchronous parallel zero-order policy gradient (AZOPG) method to solve the continuous-time linear quadratic regulation problem. Specifically, as in the celebrated A3C algorithm, there are multiple parallel workers to asynchronously estimate PGs which are then sent to a central master for policy updates. Via quantifying its convergence rate of policy iterations, we show the linear speedup property of the AZOPG, both in theory and simulation, which clearly reveals the advantages of using parallel workers for learning policies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_03233 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Asynchronous Parallel Policy Gradient Methods for the Linear Quadratic Regulator Sha, Xingyu Zhao, Feiran You, Keyou Optimization and Control Learning policies in an asynchronous parallel way is essential to the numerous successes of RL for solving large-scale problems. However, their convergence performance is still not rigorously evaluated. To this end, we adopt the asynchronous parallel zero-order policy gradient (AZOPG) method to solve the continuous-time linear quadratic regulation problem. Specifically, as in the celebrated A3C algorithm, there are multiple parallel workers to asynchronously estimate PGs which are then sent to a central master for policy updates. Via quantifying its convergence rate of policy iterations, we show the linear speedup property of the AZOPG, both in theory and simulation, which clearly reveals the advantages of using parallel workers for learning policies. |
| title | Asynchronous Parallel Policy Gradient Methods for the Linear Quadratic Regulator |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2407.03233 |