Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.11062 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915281469702144 |
|---|---|
| author | Stachurski, John Yang, Jingni Yang, Ziyue |
| author_facet | Stachurski, John Yang, Jingni Yang, Ziyue |
| contents | In the theory of dynamic programming, an optimal policy is a policy whose lifetime value dominates that of all other policies from every possible initial condition in the state space. This raises a natural question: when does optimality from a single state imply optimality from every state? Working in a general setting, we provide sufficient conditions for this property that relate to reachability and irreducibility. Our results have significant implications for modern policy-based algorithms used to solve large-scale dynamic programs. We illustrate our findings by applying them to an optimal savings problem via an algorithm that implements gradient ascent in a policy space constructed from neural networks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11062 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dynamic Programming: From Local Optimality to Global Optimality Stachurski, John Yang, Jingni Yang, Ziyue Optimization and Control 90C39 In the theory of dynamic programming, an optimal policy is a policy whose lifetime value dominates that of all other policies from every possible initial condition in the state space. This raises a natural question: when does optimality from a single state imply optimality from every state? Working in a general setting, we provide sufficient conditions for this property that relate to reachability and irreducibility. Our results have significant implications for modern policy-based algorithms used to solve large-scale dynamic programs. We illustrate our findings by applying them to an optimal savings problem via an algorithm that implements gradient ascent in a policy space constructed from neural networks. |
| title | Dynamic Programming: From Local Optimality to Global Optimality |
| topic | Optimization and Control 90C39 |
| url | https://arxiv.org/abs/2411.11062 |