Policy Iteration for Exploratory Hamilton--Jacobi--Bellman Equations
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912396340101120 |
|---|---|
| author | Tran, Hung Vinh Wang, Zhenhua Zhang, Yuming Paul |
| author_facet | Tran, Hung Vinh Wang, Zhenhua Zhang, Yuming Paul |
| contents | We study the policy iteration algorithm (PIA) for entropy-regularized stochastic control problems on an infinite time horizon with a large discount rate, focusing on two main scenarios. First, we analyze PIA with bounded coefficients where the controls applied to the diffusion term satisfy a smallness condition. We demonstrate the convergence of PIA based on a uniform $\mathcal{C}^{2,α}$ estimate for the value sequence generated by PIA, and provide a quantitative convergence analysis for this scenario. Second, we investigate PIA with unbounded coefficients but no control over the diffusion term. In this scenario, we first provide the well-posedness of the exploratory Hamilton--Jacobi--Bellman equation with linear growth coefficients and polynomial growth reward function. By such a well-posedess result we achieve PIA's convergence by establishing a quantitative locally uniform $\mathcal{C}^{1,α}$ estimates for the generated value sequence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00612 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Policy Iteration for Exploratory Hamilton--Jacobi--Bellman Equations Tran, Hung Vinh Wang, Zhenhua Zhang, Yuming Paul Optimization and Control Analysis of PDEs 35F21, 60J60, 68W40, 93E20 We study the policy iteration algorithm (PIA) for entropy-regularized stochastic control problems on an infinite time horizon with a large discount rate, focusing on two main scenarios. First, we analyze PIA with bounded coefficients where the controls applied to the diffusion term satisfy a smallness condition. We demonstrate the convergence of PIA based on a uniform $\mathcal{C}^{2,α}$ estimate for the value sequence generated by PIA, and provide a quantitative convergence analysis for this scenario. Second, we investigate PIA with unbounded coefficients but no control over the diffusion term. In this scenario, we first provide the well-posedness of the exploratory Hamilton--Jacobi--Bellman equation with linear growth coefficients and polynomial growth reward function. By such a well-posedess result we achieve PIA's convergence by establishing a quantitative locally uniform $\mathcal{C}^{1,α}$ estimates for the generated value sequence. |
| title | Policy Iteration for Exploratory Hamilton--Jacobi--Bellman Equations |
| topic | Optimization and Control Analysis of PDEs 35F21, 60J60, 68W40, 93E20 |
| url | https://arxiv.org/abs/2406.00612 |