Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.02735 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912258481717248 |
|---|---|
| author | Weissmann, Simon Freihaut, Till Vernade, Claire Ramponi, Giorgia Döring, Leif |
| author_facet | Weissmann, Simon Freihaut, Till Vernade, Claire Ramponi, Giorgia Döring, Leif |
| contents | In the context of stochastic bandit models, this article examines how to design sample-efficient behavior policies for the importance sampling evaluation of multiple target policies. From importance sampling theory, it is well established that sample efficiency is highly sensitive to the KL divergence between the target and importance sampling distributions. We first analyze a single behavior policy defined as the KL-barycenter of the target policies. Then, we refine this approach by clustering the target policies into groups with small KL divergences and assigning each cluster its own KL-barycenter as a behavior policy. This clustered KL-based policy evaluation (CKL-PE) algorithm provides a novel perspective on optimal policy selection. We prove upper bounds on the sample complexity of our method and demonstrate its effectiveness with numerical validation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_02735 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Clustered KL-barycenter design for policy evaluation Weissmann, Simon Freihaut, Till Vernade, Claire Ramponi, Giorgia Döring, Leif Machine Learning In the context of stochastic bandit models, this article examines how to design sample-efficient behavior policies for the importance sampling evaluation of multiple target policies. From importance sampling theory, it is well established that sample efficiency is highly sensitive to the KL divergence between the target and importance sampling distributions. We first analyze a single behavior policy defined as the KL-barycenter of the target policies. Then, we refine this approach by clustering the target policies into groups with small KL divergences and assigning each cluster its own KL-barycenter as a behavior policy. This clustered KL-based policy evaluation (CKL-PE) algorithm provides a novel perspective on optimal policy selection. We prove upper bounds on the sample complexity of our method and demonstrate its effectiveness with numerical validation. |
| title | Clustered KL-barycenter design for policy evaluation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.02735 |