Constrained Policy Optimization for Stochastic Optimal Control under Nonstationary Uncertainties
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2022
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909026062696448 |
|---|---|
| author | Shin, Sungho Pacaud, François Contantinescu, Emil Anitescu, Mihai |
| author_facet | Shin, Sungho Pacaud, François Contantinescu, Emil Anitescu, Mihai |
| contents | This article presents a constrained policy optimization approach for the optimal control of systems under nonstationary uncertainties. We introduce an assumption that we call Markov embeddability that allows us to cast the stochastic optimal control problem as a policy optimization problem over the augmented state space. Then, the infinite-dimensional policy optimization problem is approximated as a finite-dimensional nonlinear program by applying function approximation, deterministic sampling, and temporal truncation. The approximated problem is solved by using automatic differentiation and condensed-space interior-point methods. We formulate several conceptual and practical open questions regarding the asymptotic exactness of the approximation and the solution strategies for the approximated problem. As a proof of concept, we provide a numerical example demonstrating the performance of the control policy obtained by the proposed method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2209_13050 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Constrained Policy Optimization for Stochastic Optimal Control under Nonstationary Uncertainties Shin, Sungho Pacaud, François Contantinescu, Emil Anitescu, Mihai Optimization and Control This article presents a constrained policy optimization approach for the optimal control of systems under nonstationary uncertainties. We introduce an assumption that we call Markov embeddability that allows us to cast the stochastic optimal control problem as a policy optimization problem over the augmented state space. Then, the infinite-dimensional policy optimization problem is approximated as a finite-dimensional nonlinear program by applying function approximation, deterministic sampling, and temporal truncation. The approximated problem is solved by using automatic differentiation and condensed-space interior-point methods. We formulate several conceptual and practical open questions regarding the asymptotic exactness of the approximation and the solution strategies for the approximated problem. As a proof of concept, we provide a numerical example demonstrating the performance of the control policy obtained by the proposed method. |
| title | Constrained Policy Optimization for Stochastic Optimal Control under Nonstationary Uncertainties |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2209.13050 |