Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.14564 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910957804978176 |
|---|---|
| author | Kadurha, David Krame Moutouo, Domini Jocema Leko Gaba, Yae Ulrich |
| author_facet | Kadurha, David Krame Moutouo, Domini Jocema Leko Gaba, Yae Ulrich |
| contents | This paper reviews the topological groundwork for the study of reinforcement learning (RL) by focusing on the structure of state, action, and policy spaces. We begin by recalling key mathematical concepts such as complete metric spaces, which form the foundation for expressing RL problems. By leveraging the Banach contraction principle, we illustrate how the Banach fixed-point theorem explains the convergence of RL algorithms and how Bellman operators, expressed as operators on Banach spaces, ensure this convergence. The work serves as a bridge between theoretical mathematics and practical algorithm design, offering new approaches to enhance the efficiency of RL. In particular, we investigate alternative formulations of Bellman operators and demonstrate their impact on improving convergence rates and performance in standard RL environments such as MountainCar, CartPole, and Acrobot. Our findings highlight how a deeper mathematical understanding of RL can lead to more effective algorithms for decision-making problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14564 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bellman operator convergence enhancements in reinforcement learning algorithms Kadurha, David Krame Moutouo, Domini Jocema Leko Gaba, Yae Ulrich Machine Learning Artificial Intelligence This paper reviews the topological groundwork for the study of reinforcement learning (RL) by focusing on the structure of state, action, and policy spaces. We begin by recalling key mathematical concepts such as complete metric spaces, which form the foundation for expressing RL problems. By leveraging the Banach contraction principle, we illustrate how the Banach fixed-point theorem explains the convergence of RL algorithms and how Bellman operators, expressed as operators on Banach spaces, ensure this convergence. The work serves as a bridge between theoretical mathematics and practical algorithm design, offering new approaches to enhance the efficiency of RL. In particular, we investigate alternative formulations of Bellman operators and demonstrate their impact on improving convergence rates and performance in standard RL environments such as MountainCar, CartPole, and Acrobot. Our findings highlight how a deeper mathematical understanding of RL can lead to more effective algorithms for decision-making problems. |
| title | Bellman operator convergence enhancements in reinforcement learning algorithms |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.14564 |