Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916034554888192 |
|---|---|
| author | Ota, Kazuki Osa, Takayuki Omura, Motoki Harada, Tatsuya |
| author_facet | Ota, Kazuki Osa, Takayuki Omura, Motoki Harada, Tatsuya |
| contents | Two-player games such as board games have long been used as traditional benchmarks for reinforcement learning. This work revisits a policy optimization method with reverse Kullback-Leibler regularization and entropy regularization and analyzes this combination in two-player zero-sum settings from theoretical and empirical perspectives. From a theoretical perspective, we investigate the stability of the policy update rule in two theoretical settings: game-theoretic normal-form games and finite-length games. We provide novel convergence guarantees and verify our theoretical results through numerical experiments on synthetic games. From an empirical perspective, we derive a practical model-free reinforcement learning algorithm based on the regularized policy optimization. We validate the training efficiency of our algorithm through comprehensive experiments on five board games: Animal Shogi, Gardner Chess, Go, Hex, and Othello. Experimental results show that our agent learns more efficiently than existing methods across environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_10894 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games Ota, Kazuki Osa, Takayuki Omura, Motoki Harada, Tatsuya Machine Learning Artificial Intelligence Two-player games such as board games have long been used as traditional benchmarks for reinforcement learning. This work revisits a policy optimization method with reverse Kullback-Leibler regularization and entropy regularization and analyzes this combination in two-player zero-sum settings from theoretical and empirical perspectives. From a theoretical perspective, we investigate the stability of the policy update rule in two theoretical settings: game-theoretic normal-form games and finite-length games. We provide novel convergence guarantees and verify our theoretical results through numerical experiments on synthetic games. From an empirical perspective, we derive a practical model-free reinforcement learning algorithm based on the regularized policy optimization. We validate the training efficiency of our algorithm through comprehensive experiments on five board games: Animal Shogi, Gardner Chess, Go, Hex, and Othello. Experimental results show that our agent learns more efficiently than existing methods across environments. |
| title | Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2602.10894 |