Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.03752 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909014525214720 |
|---|---|
| author | Anita, Stefana-Lucia Turinici, Gabriel |
| author_facet | Anita, Stefana-Lucia Turinici, Gabriel |
| contents | Multi Armed Bandit (MAB) algorithms are a cornerstone of reinforcement learning and have been studied both theoretically and numerically. One of the most commonly used implementation uses a softmax mapping to prescribe the optimal policy and served as the foundation for downstream algorithms, including REINFORCE. Distinct from vanilla approaches, we consider here the L2 regularized softmax policy gradient where a quadratic term is subtracted from the mean reward. Previous studies exploiting convexity failed to identify a suitable theoretical framework to analyze its convergence when the regularization parameter vanishes. We prove here theoretical convergence results and confirm empirically that this regime makes the L2 regularization numerically advantageous on standard benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_03752 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Vanishing L2 regularization for the softmax Multi Armed Bandit Anita, Stefana-Lucia Turinici, Gabriel Machine Learning Statistics Theory 68T05, 90C40, 68Q25, 90C25 F.2.2; F.1.2; I.2.6 Multi Armed Bandit (MAB) algorithms are a cornerstone of reinforcement learning and have been studied both theoretically and numerically. One of the most commonly used implementation uses a softmax mapping to prescribe the optimal policy and served as the foundation for downstream algorithms, including REINFORCE. Distinct from vanilla approaches, we consider here the L2 regularized softmax policy gradient where a quadratic term is subtracted from the mean reward. Previous studies exploiting convexity failed to identify a suitable theoretical framework to analyze its convergence when the regularization parameter vanishes. We prove here theoretical convergence results and confirm empirically that this regime makes the L2 regularization numerically advantageous on standard benchmarks. |
| title | Vanishing L2 regularization for the softmax Multi Armed Bandit |
| topic | Machine Learning Statistics Theory 68T05, 90C40, 68Q25, 90C25 F.2.2; F.1.2; I.2.6 |
| url | https://arxiv.org/abs/2605.03752 |