Combining Experimental and Historical Data for Policy Evaluation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909214635458560 |
|---|---|
| author | Li, Ting Shi, Chengchun Wen, Qianglin Sui, Yang Qin, Yongli Lai, Chunbo Zhu, Hongtu |
| author_facet | Li, Ting Shi, Chengchun Wen, Qianglin Sui, Yang Qin, Yongli Lai, Chunbo Zhu, Hongtu |
| contents | This paper studies policy evaluation with multiple data sources, especially in scenarios that involve one experimental dataset with two arms, complemented by a historical dataset generated under a single control arm. We propose novel data integration methods that linearly integrate base policy value estimators constructed based on the experimental and historical data, with weights optimized to minimize the mean square error (MSE) of the resulting combined estimator. We further apply the pessimistic principle to obtain more robust estimators, and extend these developments to sequential decision making. Theoretically, we establish non-asymptotic error bounds for the MSEs of our proposed estimators, and derive their oracle, efficiency and robustness properties across a broad spectrum of reward shift scenarios. Numerical experiments and real-data-based analyses from a ridesharing company demonstrate the superior performance of the proposed estimators. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00317 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Combining Experimental and Historical Data for Policy Evaluation Li, Ting Shi, Chengchun Wen, Qianglin Sui, Yang Qin, Yongli Lai, Chunbo Zhu, Hongtu Machine Learning Methodology This paper studies policy evaluation with multiple data sources, especially in scenarios that involve one experimental dataset with two arms, complemented by a historical dataset generated under a single control arm. We propose novel data integration methods that linearly integrate base policy value estimators constructed based on the experimental and historical data, with weights optimized to minimize the mean square error (MSE) of the resulting combined estimator. We further apply the pessimistic principle to obtain more robust estimators, and extend these developments to sequential decision making. Theoretically, we establish non-asymptotic error bounds for the MSEs of our proposed estimators, and derive their oracle, efficiency and robustness properties across a broad spectrum of reward shift scenarios. Numerical experiments and real-data-based analyses from a ridesharing company demonstrate the superior performance of the proposed estimators. |
| title | Combining Experimental and Historical Data for Policy Evaluation |
| topic | Machine Learning Methodology |
| url | https://arxiv.org/abs/2406.00317 |