Efficient and Optimal Policy Gradient Algorithm for Corrupted Multi-armed Bandits
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916621611696128 |
|---|---|
| author | Liu, Jiayuan Wang, Siwei Fang, Zhixuan |
| author_facet | Liu, Jiayuan Wang, Siwei Fang, Zhixuan |
| contents | In this paper, we consider the stochastic multi-armed bandits problem with adversarial corruptions, where the random rewards of the arms are partially modified by an adversary to fool the algorithm. We apply the policy gradient algorithm SAMBA to this setting, and show that it is computationally efficient, and achieves a state-of-the-art $O(K\log T/Δ) + O(C/Δ)$ regret upper bound, where $K$ is the number of arms, $C$ is the unknown corruption level, $Δ$ is the minimum expected reward gap between the best arm and other ones, and $T$ is the time horizon. Compared with the best existing efficient algorithm (e.g., CBARBAR), whose regret upper bound is $O(K\log^2 T/Δ) + O(C)$, we show that SAMBA reduces one $\log T$ factor in the regret bound, while maintaining the corruption-dependent term to be linear with $C$. This is indeed asymptotically optimal. We also conduct simulations to demonstrate the effectiveness of SAMBA, and the results show that SAMBA outperforms existing baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_14146 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient and Optimal Policy Gradient Algorithm for Corrupted Multi-armed Bandits Liu, Jiayuan Wang, Siwei Fang, Zhixuan Machine Learning In this paper, we consider the stochastic multi-armed bandits problem with adversarial corruptions, where the random rewards of the arms are partially modified by an adversary to fool the algorithm. We apply the policy gradient algorithm SAMBA to this setting, and show that it is computationally efficient, and achieves a state-of-the-art $O(K\log T/Δ) + O(C/Δ)$ regret upper bound, where $K$ is the number of arms, $C$ is the unknown corruption level, $Δ$ is the minimum expected reward gap between the best arm and other ones, and $T$ is the time horizon. Compared with the best existing efficient algorithm (e.g., CBARBAR), whose regret upper bound is $O(K\log^2 T/Δ) + O(C)$, we show that SAMBA reduces one $\log T$ factor in the regret bound, while maintaining the corruption-dependent term to be linear with $C$. This is indeed asymptotically optimal. We also conduct simulations to demonstrate the effectiveness of SAMBA, and the results show that SAMBA outperforms existing baselines. |
| title | Efficient and Optimal Policy Gradient Algorithm for Corrupted Multi-armed Bandits |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.14146 |