Stochastic Nonlinear Control via Finite-dimensional Spectral Dynamic Embedding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909753084477440 |
|---|---|
| author | Ren, Zhaolin Ren, Tongzheng Ma, Haitong Li, Na Dai, Bo |
| author_facet | Ren, Zhaolin Ren, Tongzheng Ma, Haitong Li, Na Dai, Bo |
| contents | This paper proposes an approach, Spectral Dynamics Embedding Control (SDEC), to optimal control for nonlinear stochastic systems. This method reveals an infinite-dimensional feature representation induced by the system's nonlinear stochastic dynamics, enabling a linear representation of the state-action value function. For practical implementation, this representation is approximated using finite-dimensional truncations, specifically via two prominent kernel approximation methods: random feature truncation and Nystrom approximation. To characterize the effectiveness of these approximations, we provide an in-depth theoretical analysis to characterize the approximation error arising from the finite-dimension truncation and statistical error due to finite-sample approximation in both policy evaluation and policy optimization. Empirically, our algorithm performs favorably against existing stochastic control algorithms on several benchmark problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_03907 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Stochastic Nonlinear Control via Finite-dimensional Spectral Dynamic Embedding Ren, Zhaolin Ren, Tongzheng Ma, Haitong Li, Na Dai, Bo Machine Learning Optimization and Control This paper proposes an approach, Spectral Dynamics Embedding Control (SDEC), to optimal control for nonlinear stochastic systems. This method reveals an infinite-dimensional feature representation induced by the system's nonlinear stochastic dynamics, enabling a linear representation of the state-action value function. For practical implementation, this representation is approximated using finite-dimensional truncations, specifically via two prominent kernel approximation methods: random feature truncation and Nystrom approximation. To characterize the effectiveness of these approximations, we provide an in-depth theoretical analysis to characterize the approximation error arising from the finite-dimension truncation and statistical error due to finite-sample approximation in both policy evaluation and policy optimization. Empirically, our algorithm performs favorably against existing stochastic control algorithms on several benchmark problems. |
| title | Stochastic Nonlinear Control via Finite-dimensional Spectral Dynamic Embedding |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2304.03907 |