Generalized Fitted Q-Iteration with Clustered Data
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911191996039168 |
|---|---|
| author | Hu, Liyuan Wang, Jitao Wu, Zhenke Shi, Chengchun |
| author_facet | Hu, Liyuan Wang, Jitao Wu, Zhenke Shi, Chengchun |
| contents | This paper focuses on reinforcement learning (RL) with clustered data, which is commonly encountered in healthcare applications. We propose a generalized fitted Q-iteration (FQI) algorithm that incorporates generalized estimating equations into policy learning to handle the intra-cluster correlations. Theoretically, we demonstrate (i) the optimalities of our Q-function and policy estimators when the correlation structure is correctly specified, and (ii) their consistencies when the structure is mis-specified. Empirically, through simulations and analyses of a mobile health dataset, we find the proposed generalized FQI achieves, on average, a half reduction in regret compared to the standard FQI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03912 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Generalized Fitted Q-Iteration with Clustered Data Hu, Liyuan Wang, Jitao Wu, Zhenke Shi, Chengchun Machine Learning This paper focuses on reinforcement learning (RL) with clustered data, which is commonly encountered in healthcare applications. We propose a generalized fitted Q-iteration (FQI) algorithm that incorporates generalized estimating equations into policy learning to handle the intra-cluster correlations. Theoretically, we demonstrate (i) the optimalities of our Q-function and policy estimators when the correlation structure is correctly specified, and (ii) their consistencies when the structure is mis-specified. Empirically, through simulations and analyses of a mobile health dataset, we find the proposed generalized FQI achieves, on average, a half reduction in regret compared to the standard FQI. |
| title | Generalized Fitted Q-Iteration with Clustered Data |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.03912 |