Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.19299 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918308059545600 |
|---|---|
| author | Zhang, Minghui Li, Xun Zhang, Xin |
| author_facet | Zhang, Minghui Li, Xun Zhang, Xin |
| contents | This paper studies the continuous-time q-learning (the continuous time counterpart of Q-learing) for Markov switching system under Tsallis entropy regularization. We address the difficulty in traditional RL algorithms where the Tsallis entropy regularization leads to an optimal policy distribution not necessarily a Gibbs measure, which often complicates algorithm design. Furthermore, to address the limited universality of current continuous time regime-switching RL algorithms (often restricted to the EMV framework), this study focuses on continuous-time q-learning for Markov regime-switching systems based on Tsallis entropy, aiming for a more universally applicable continuous-time RL method. We establish the martingale characterization of the q-function under Tsallis entropy for continuous-time Markov regime-switching systems. Based on this, we design two q-learning algorithms, distinguished by whether the Lagrange multiplier can be explicitly derived. We apply these algorithms to the continuous-time exploratory Mean-Variance (EMV) portfolio optimization problem in a regime-switching market. Numerical experiments demonstrate the satisfactory performance of our q-learning algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_19299 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Continuous-time q-learning for Markov regime switching system under Tsallis entropy Zhang, Minghui Li, Xun Zhang, Xin Optimization and Control This paper studies the continuous-time q-learning (the continuous time counterpart of Q-learing) for Markov switching system under Tsallis entropy regularization. We address the difficulty in traditional RL algorithms where the Tsallis entropy regularization leads to an optimal policy distribution not necessarily a Gibbs measure, which often complicates algorithm design. Furthermore, to address the limited universality of current continuous time regime-switching RL algorithms (often restricted to the EMV framework), this study focuses on continuous-time q-learning for Markov regime-switching systems based on Tsallis entropy, aiming for a more universally applicable continuous-time RL method. We establish the martingale characterization of the q-function under Tsallis entropy for continuous-time Markov regime-switching systems. Based on this, we design two q-learning algorithms, distinguished by whether the Lagrange multiplier can be explicitly derived. We apply these algorithms to the continuous-time exploratory Mean-Variance (EMV) portfolio optimization problem in a regime-switching market. Numerical experiments demonstrate the satisfactory performance of our q-learning algorithms. |
| title | Continuous-time q-learning for Markov regime switching system under Tsallis entropy |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2601.19299 |