Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.00348 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916982093250560 |
|---|---|
| author | Tercan, Alperen Ozay, Necmiye |
| author_facet | Tercan, Alperen Ozay, Necmiye |
| contents | Constrained Markov Decision Processes (CMDPs) are notably more complex to solve than standard MDPs due to the absence of universally optimal policies across all initial state distributions. This necessitates re-solving the CMDP whenever the initial distribution changes. In this work, we analyze how the optimal value of CMDPs varies with different initial distributions, deriving bounds on these variations using duality analysis of CMDPs and perturbation analysis in linear programming. Moreover, we show how such bounds can be used to analyze the regret of a given policy due to unknown variations of the initial distribution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_00348 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Initial Distribution Sensitivity of Constrained Markov Decision Processes Tercan, Alperen Ozay, Necmiye Machine Learning Constrained Markov Decision Processes (CMDPs) are notably more complex to solve than standard MDPs due to the absence of universally optimal policies across all initial state distributions. This necessitates re-solving the CMDP whenever the initial distribution changes. In this work, we analyze how the optimal value of CMDPs varies with different initial distributions, deriving bounds on these variations using duality analysis of CMDPs and perturbation analysis in linear programming. Moreover, we show how such bounds can be used to analyze the regret of a given policy due to unknown variations of the initial distribution. |
| title | Initial Distribution Sensitivity of Constrained Markov Decision Processes |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.00348 |