Bayesian Off-Policy Evaluation and Learning for Large Action Spaces
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917981037002752 |
|---|---|
| author | Aouali, Imad Brunel, Victor-Emmanuel Rohde, David Korba, Anna |
| author_facet | Aouali, Imad Brunel, Victor-Emmanuel Rohde, David Korba, Anna |
| contents | In interactive systems, actions are often correlated, presenting an opportunity for more sample-efficient off-policy evaluation (OPE) and learning (OPL) in large action spaces. We introduce a unified Bayesian framework to capture these correlations through structured and informative priors. In this framework, we propose sDM, a generic Bayesian approach for OPE and OPL, grounded in both algorithmic and theoretical foundations. Notably, sDM leverages action correlations without compromising computational efficiency. Moreover, inspired by online Bayesian bandits, we introduce Bayesian metrics that assess the average performance of algorithms across multiple problem instances, deviating from the conventional worst-case assessments. We analyze sDM in OPE and OPL, highlighting the benefits of leveraging action correlations. Empirical evidence showcases the strong performance of sDM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14664 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Bayesian Off-Policy Evaluation and Learning for Large Action Spaces Aouali, Imad Brunel, Victor-Emmanuel Rohde, David Korba, Anna Machine Learning Artificial Intelligence In interactive systems, actions are often correlated, presenting an opportunity for more sample-efficient off-policy evaluation (OPE) and learning (OPL) in large action spaces. We introduce a unified Bayesian framework to capture these correlations through structured and informative priors. In this framework, we propose sDM, a generic Bayesian approach for OPE and OPL, grounded in both algorithmic and theoretical foundations. Notably, sDM leverages action correlations without compromising computational efficiency. Moreover, inspired by online Bayesian bandits, we introduce Bayesian metrics that assess the average performance of algorithms across multiple problem instances, deviating from the conventional worst-case assessments. We analyze sDM in OPE and OPL, highlighting the benefits of leveraging action correlations. Empirical evidence showcases the strong performance of sDM. |
| title | Bayesian Off-Policy Evaluation and Learning for Large Action Spaces |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2402.14664 |