Robustness to Model Approximation, Model Learning From Data, and Sample Complexity in Wasserstein Regular MDPs
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914376796078080 |
|---|---|
| author | Zhou, Yichen Song, Yanglei Yüksel, Serdar |
| author_facet | Zhou, Yichen Song, Yanglei Yüksel, Serdar |
| contents | The paper studies the robustness properties of discrete-time stochastic optimal control under Wasserstein model approximation for both discounted-cost and average-cost criteria. Specifically, we study the performance loss when applying an optimal policy designed for an approximate model to the true dynamics compared with the optimal cost for the true model under the sup-norm-induced metric, and relate it to the Wasserstein-1 distance between the approximate and true transition kernels. A primary motivation of this analysis is empirical model learning, as well as empirical noise distribution learning, where Wasserstein convergence holds under mild conditions but stronger convergence criteria, such as total variation, may not. We discuss applications of the results to the disturbance estimation problem, where sample complexity bounds are given, and also to a general empirical model learning approach, obtained under either Markov or i.i.d. learning settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_14116 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robustness to Model Approximation, Model Learning From Data, and Sample Complexity in Wasserstein Regular MDPs Zhou, Yichen Song, Yanglei Yüksel, Serdar Systems and Control Optimization and Control The paper studies the robustness properties of discrete-time stochastic optimal control under Wasserstein model approximation for both discounted-cost and average-cost criteria. Specifically, we study the performance loss when applying an optimal policy designed for an approximate model to the true dynamics compared with the optimal cost for the true model under the sup-norm-induced metric, and relate it to the Wasserstein-1 distance between the approximate and true transition kernels. A primary motivation of this analysis is empirical model learning, as well as empirical noise distribution learning, where Wasserstein convergence holds under mild conditions but stronger convergence criteria, such as total variation, may not. We discuss applications of the results to the disturbance estimation problem, where sample complexity bounds are given, and also to a general empirical model learning approach, obtained under either Markov or i.i.d. learning settings. |
| title | Robustness to Model Approximation, Model Learning From Data, and Sample Complexity in Wasserstein Regular MDPs |
| topic | Systems and Control Optimization and Control |
| url | https://arxiv.org/abs/2410.14116 |