Convergence of Policy Gradient for Stochastic Linear-Quadratic Control Problem in Infinite Horizon
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909173456830464 |
|---|---|
| author | Zhang, Xinpei Jia, Guangyan |
| author_facet | Zhang, Xinpei Jia, Guangyan |
| contents | With the outstanding performance of policy gradient (PG) method in the reinforcement learning field, the convergence theory of it has aroused more and more interest recently. Meanwhile, the significant importance and abundant theoretical researches make the stochastic linear quadratic (SLQ) control problem a starting point for studying PG in model-based learning setting. In this paper, we study the PG method for the SLQ problem in infinite horizon and take a step towards providing rigorous guarantees for gradient methods. Although the cost functional of linear-quadratic problem is typically nonconvex, we still overcome the difficulty based on gradient domination condition and L-smoothness property, and prove exponential/linear convergence of gradient flow/descent algorithm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_11382 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Convergence of Policy Gradient for Stochastic Linear-Quadratic Control Problem in Infinite Horizon Zhang, Xinpei Jia, Guangyan Optimization and Control With the outstanding performance of policy gradient (PG) method in the reinforcement learning field, the convergence theory of it has aroused more and more interest recently. Meanwhile, the significant importance and abundant theoretical researches make the stochastic linear quadratic (SLQ) control problem a starting point for studying PG in model-based learning setting. In this paper, we study the PG method for the SLQ problem in infinite horizon and take a step towards providing rigorous guarantees for gradient methods. Although the cost functional of linear-quadratic problem is typically nonconvex, we still overcome the difficulty based on gradient domination condition and L-smoothness property, and prove exponential/linear convergence of gradient flow/descent algorithm. |
| title | Convergence of Policy Gradient for Stochastic Linear-Quadratic Control Problem in Infinite Horizon |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2404.11382 |