On the Feedback Law in Stochastic Optimal Nonlinear Control
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2020
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917798954926080 |
|---|---|
| author | Mohamed, Mohamed Naveed Gul Chakravorty, Suman Goyal, Raman Wang, Ran |
| author_facet | Mohamed, Mohamed Naveed Gul Chakravorty, Suman Goyal, Raman Wang, Ran |
| contents | We consider the problem of nonlinear stochastic optimal control. This problem is thought to be fundamentally intractable owing to Bellman's "curse of dimensionality". We present a result that shows that repeatedly solving an open-loop deterministic problem from the current state with progressively shorter horizons, similar to Model Predictive Control (MPC), results in a feedback policy that is $O(ε^4)$ near to the true global stochastic optimal policy, where $ε$ is a perturbation parameter modulating the noise. We also show that the optimal deterministic feedback problem has a perturbation structure such that higher-order terms of the feedback law do not affect lower-order terms and that this structure is lost in the optimal stochastic feedback problem. Consequently, solving the Stochastic Dynamic Programming problem is highly susceptible to noise, even in low dimensional problems, and in practice, the MPC-type feedback law offers superior performance even for high noise levels. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2004_01041 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | On the Feedback Law in Stochastic Optimal Nonlinear Control Mohamed, Mohamed Naveed Gul Chakravorty, Suman Goyal, Raman Wang, Ran Systems and Control Robotics We consider the problem of nonlinear stochastic optimal control. This problem is thought to be fundamentally intractable owing to Bellman's "curse of dimensionality". We present a result that shows that repeatedly solving an open-loop deterministic problem from the current state with progressively shorter horizons, similar to Model Predictive Control (MPC), results in a feedback policy that is $O(ε^4)$ near to the true global stochastic optimal policy, where $ε$ is a perturbation parameter modulating the noise. We also show that the optimal deterministic feedback problem has a perturbation structure such that higher-order terms of the feedback law do not affect lower-order terms and that this structure is lost in the optimal stochastic feedback problem. Consequently, solving the Stochastic Dynamic Programming problem is highly susceptible to noise, even in low dimensional problems, and in practice, the MPC-type feedback law offers superior performance even for high noise levels. |
| title | On the Feedback Law in Stochastic Optimal Nonlinear Control |
| topic | Systems and Control Robotics |
| url | https://arxiv.org/abs/2004.01041 |