Control and Reinforcement Learning through the Lens of Optimization: An Algorithmic Perspective
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912753905565696 |
|---|---|
| author | Ok, Tolga Kolarijani, Arman Sharifi Kolarijani, Mohamad Amin Sharif Esfahani, Peyman Mohajerin |
| author_facet | Ok, Tolga Kolarijani, Arman Sharifi Kolarijani, Mohamad Amin Sharif Esfahani, Peyman Mohajerin |
| contents | The connection between control algorithms for Markov decision processes and optimization algorithms has been implicitly and explicitly exploited since the introduction of dynamic programming algorithm by Bellman in the 1950s. Recently, this connection has attracted a lot of attention for developing new control algorithms inspired by well-established optimization algorithms. In this paper, we make this analogy explicit across four problem classes with a unified solution characterization. This novel framework, in turn, allows for a systematic transformation of algorithms from one domain to the other. In particular, we identify equivalent optimization and control algorithms that have already been pointed out in the existing literature, but mostly in a scattered way. We also discuss the issues arising in providing theoretical convergence guarantees for these new control algorithms and provide simple yet effective techniques to solve them. The provided framework and techniques then lay out a concrete methodology for developing new convergent control algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_07377 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Control and Reinforcement Learning through the Lens of Optimization: An Algorithmic Perspective Ok, Tolga Kolarijani, Arman Sharifi Kolarijani, Mohamad Amin Sharif Esfahani, Peyman Mohajerin Optimization and Control The connection between control algorithms for Markov decision processes and optimization algorithms has been implicitly and explicitly exploited since the introduction of dynamic programming algorithm by Bellman in the 1950s. Recently, this connection has attracted a lot of attention for developing new control algorithms inspired by well-established optimization algorithms. In this paper, we make this analogy explicit across four problem classes with a unified solution characterization. This novel framework, in turn, allows for a systematic transformation of algorithms from one domain to the other. In particular, we identify equivalent optimization and control algorithms that have already been pointed out in the existing literature, but mostly in a scattered way. We also discuss the issues arising in providing theoretical convergence guarantees for these new control algorithms and provide simple yet effective techniques to solve them. The provided framework and techniques then lay out a concrete methodology for developing new convergent control algorithms. |
| title | Control and Reinforcement Learning through the Lens of Optimization: An Algorithmic Perspective |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2512.07377 |