Missing Data Multiple Imputation for Tabular Q-Learning in Online RL
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908589023559680 |
|---|---|
| author | Chasalow, Kyla Wu, Skyler Murphy, Susan |
| author_facet | Chasalow, Kyla Wu, Skyler Murphy, Susan |
| contents | Missing data in online reinforcement learning (RL) poses challenges compared to missing data in standard tabular data or in offline policy learning. The need to impute and act at each time step means that imputation cannot be put off until enough data exist to produce stable imputation models. It also means future data collection and learning depend on previous imputations. This paper proposes fully online imputation ensembles. We find that maintaining multiple imputation pathways may help balance the need to capture uncertainty under missingness and the need for efficiency in online settings. We consider multiple approaches for incorporating these pathways into learning and action selection. Using a Grid World experiment with various types of missingness, we provide preliminary evidence that multiple imputation pathways may be a useful framework for constructing simple and efficient online missing data RL methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_10709 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Missing Data Multiple Imputation for Tabular Q-Learning in Online RL Chasalow, Kyla Wu, Skyler Murphy, Susan Machine Learning Artificial Intelligence Missing data in online reinforcement learning (RL) poses challenges compared to missing data in standard tabular data or in offline policy learning. The need to impute and act at each time step means that imputation cannot be put off until enough data exist to produce stable imputation models. It also means future data collection and learning depend on previous imputations. This paper proposes fully online imputation ensembles. We find that maintaining multiple imputation pathways may help balance the need to capture uncertainty under missingness and the need for efficiency in online settings. We consider multiple approaches for incorporating these pathways into learning and action selection. Using a Grid World experiment with various types of missingness, we provide preliminary evidence that multiple imputation pathways may be a useful framework for constructing simple and efficient online missing data RL methods. |
| title | Missing Data Multiple Imputation for Tabular Q-Learning in Online RL |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2510.10709 |