Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2506.22203 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910214853230592 |
|---|---|
| author | Liang, Zongxia Luo, Xiaodong Yu, Xiang |
| author_facet | Liang, Zongxia Luo, Xiaodong Yu, Xiang |
| contents | We develop a continuous-time reinforcement learning framework for a class of singular stochastic control problems without entropy regularization. The optimal singular control is characterized as the optimal singular control law, which is a pair of regions of time and the augmented states. The goal of learning is to identify such an optimal region via the trial-and-error procedure. In this context, we generalize the existing policy evaluation theories with regular controls to learn our optimal singular control law and develop a policy improvement theorem via the region iteration. To facilitate the model-free policy iteration procedure, we further introduce the zero-order and first-order q-functions arising from singular control problems and establish the martingale characterization for the pair of q-functions together with the value function. Based on our theoretical findings, some q-learning algorithms are devised accordingly and a numerical example based on simulation experiment is presented. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_22203 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Reinforcement Learning Framework for Some Singular Stochastic Control Problems Liang, Zongxia Luo, Xiaodong Yu, Xiang Optimization and Control 93E20, 93B47, 49K45 We develop a continuous-time reinforcement learning framework for a class of singular stochastic control problems without entropy regularization. The optimal singular control is characterized as the optimal singular control law, which is a pair of regions of time and the augmented states. The goal of learning is to identify such an optimal region via the trial-and-error procedure. In this context, we generalize the existing policy evaluation theories with regular controls to learn our optimal singular control law and develop a policy improvement theorem via the region iteration. To facilitate the model-free policy iteration procedure, we further introduce the zero-order and first-order q-functions arising from singular control problems and establish the martingale characterization for the pair of q-functions together with the value function. Based on our theoretical findings, some q-learning algorithms are devised accordingly and a numerical example based on simulation experiment is presented. |
| title | A Reinforcement Learning Framework for Some Singular Stochastic Control Problems |
| topic | Optimization and Control 93E20, 93B47, 49K45 |
| url | https://arxiv.org/abs/2506.22203 |