Linear Convergence of Data-Enabled Policy Optimization for Linear Quadratic Tracking
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910638914142208 |
|---|---|
| author | Kang, Shubo Zhao, Feiran You, Keyou |
| author_facet | Kang, Shubo Zhao, Feiran You, Keyou |
| contents | Data-enabled policy optimization (DeePO) is a newly proposed method to attack the open problem of direct adaptive LQR. In this work, we extend the DeePO framework to the linear quadratic tracking (LQT) with offline data. By introducing a covariance parameterization of the LQT policy, we derive a direct data-driven formulation of the LQT problem. Then, we use gradient descent method to iteratively update the parameterized policy to find an optimal LQT policy. Moreover, by revealing the connection between DeePO and model-based policy optimization, we prove the linear convergence of the DeePO iteration. Finally, a numerical experiment is given to validate the convergence results. We hope our work paves the way to direct adaptive LQT with online closed-loop data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_05596 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Linear Convergence of Data-Enabled Policy Optimization for Linear Quadratic Tracking Kang, Shubo Zhao, Feiran You, Keyou Systems and Control Optimization and Control Data-enabled policy optimization (DeePO) is a newly proposed method to attack the open problem of direct adaptive LQR. In this work, we extend the DeePO framework to the linear quadratic tracking (LQT) with offline data. By introducing a covariance parameterization of the LQT policy, we derive a direct data-driven formulation of the LQT problem. Then, we use gradient descent method to iteratively update the parameterized policy to find an optimal LQT policy. Moreover, by revealing the connection between DeePO and model-based policy optimization, we prove the linear convergence of the DeePO iteration. Finally, a numerical experiment is given to validate the convergence results. We hope our work paves the way to direct adaptive LQT with online closed-loop data. |
| title | Linear Convergence of Data-Enabled Policy Optimization for Linear Quadratic Tracking |
| topic | Systems and Control Optimization and Control |
| url | https://arxiv.org/abs/2410.05596 |