Convergence of a L2 regularized Policy Gradient Algorithm for the Multi Armed Bandit
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914231379558400 |
|---|---|
| author | Anita, Stefana Turinici, Gabriel |
| author_facet | Anita, Stefana Turinici, Gabriel |
| contents | Although Multi Armed Bandit (MAB) on one hand and the policy gradient approach on the other hand are among the most used frameworks of Reinforcement Learning, the theoretical properties of the policy gradient algorithm used for MAB have not been given enough attention. We investigate in this work the convergence of such a procedure for the situation when a $L2$ regularization term is present jointly with the 'softmax' parametrization. We prove convergence under appropriate technical hypotheses and test numerically the procedure including situations beyond the theoretical setting. The tests show that a time dependent regularized procedure can improve over the canonical approach especially when the initial guess is far from the solution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_06388 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Convergence of a L2 regularized Policy Gradient Algorithm for the Multi Armed Bandit Anita, Stefana Turinici, Gabriel Machine Learning Artificial Intelligence Data Structures and Algorithms Numerical Analysis Although Multi Armed Bandit (MAB) on one hand and the policy gradient approach on the other hand are among the most used frameworks of Reinforcement Learning, the theoretical properties of the policy gradient algorithm used for MAB have not been given enough attention. We investigate in this work the convergence of such a procedure for the situation when a $L2$ regularization term is present jointly with the 'softmax' parametrization. We prove convergence under appropriate technical hypotheses and test numerically the procedure including situations beyond the theoretical setting. The tests show that a time dependent regularized procedure can improve over the canonical approach especially when the initial guess is far from the solution. |
| title | Convergence of a L2 regularized Policy Gradient Algorithm for the Multi Armed Bandit |
| topic | Machine Learning Artificial Intelligence Data Structures and Algorithms Numerical Analysis |
| url | https://arxiv.org/abs/2402.06388 |