Properties of Turnpike Functions for Discounted Finite MDPs
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915384791138304 |
|---|---|
| author | Feinberg, Eugene A. He, Gaojin |
| author_facet | Feinberg, Eugene A. He, Gaojin |
| contents | This paper studies discounted Markov Decision Processes (MDPs) with finite sets of states and actions. Value iteration is one of the major methods for finding optimal policies. For each discount factor, starting from a finite number of iterations, which is called the turnpike integer, value iteration algorithms always generate decision rules, which are deterministic optimal policies for the infinite-horizon problems. This fact justifies the rolling horizon approach for computing infinite-horizon optimal policies by conducting a finite number of value iterations. This paper describes properties of turnpike integers and provides their upper bounds. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_05375 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Properties of Turnpike Functions for Discounted Finite MDPs Feinberg, Eugene A. He, Gaojin Optimization and Control 90C40, 60J05 This paper studies discounted Markov Decision Processes (MDPs) with finite sets of states and actions. Value iteration is one of the major methods for finding optimal policies. For each discount factor, starting from a finite number of iterations, which is called the turnpike integer, value iteration algorithms always generate decision rules, which are deterministic optimal policies for the infinite-horizon problems. This fact justifies the rolling horizon approach for computing infinite-horizon optimal policies by conducting a finite number of value iterations. This paper describes properties of turnpike integers and provides their upper bounds. |
| title | Properties of Turnpike Functions for Discounted Finite MDPs |
| topic | Optimization and Control 90C40, 60J05 |
| url | https://arxiv.org/abs/2502.05375 |