Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912422262996992 |
|---|---|
| author | Feng, Qi Wang, Gu |
| author_facet | Feng, Qi Wang, Gu |
| contents | We introduce a continuous policy-value iteration algorithm where the approximations of the value function of a stochastic control problem and the optimal control are simultaneously updated through Langevin-type dynamics. This framework applies to both the entropy-regularized relaxed control problems and the classical control problems, with infinite horizon. We establish policy improvement and demonstrate convergence to the optimal control under the monotonicity condition of the Hamiltonian. By utilizing Langevin-type stochastic differential equations for continuous updates along the policy iteration direction, our approach enables the use of distribution sampling and non-convex learning techniques in machine learning to optimize the value function and identify the optimal control simultaneously. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_08121 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence Feng, Qi Wang, Gu Optimization and Control Machine Learning 93E20, 93E35, 60H10 We introduce a continuous policy-value iteration algorithm where the approximations of the value function of a stochastic control problem and the optimal control are simultaneously updated through Langevin-type dynamics. This framework applies to both the entropy-regularized relaxed control problems and the classical control problems, with infinite horizon. We establish policy improvement and demonstrate convergence to the optimal control under the monotonicity condition of the Hamiltonian. By utilizing Langevin-type stochastic differential equations for continuous updates along the policy iteration direction, our approach enables the use of distribution sampling and non-convex learning techniques in machine learning to optimize the value function and identify the optimal control simultaneously. |
| title | Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence |
| topic | Optimization and Control Machine Learning 93E20, 93E35, 60H10 |
| url | https://arxiv.org/abs/2506.08121 |