Stochastic Approximation for Nonlinear Discrete Stochastic Control: Finite-Sample Bounds
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908510682349568 |
|---|---|
| author | Nguyen, Hoang Huy Maguluri, Siva Theja |
| author_facet | Nguyen, Hoang Huy Maguluri, Siva Theja |
| contents | We consider a nonlinear discrete stochastic control system, and our goal is to design a feedback control policy in order to lead the system to a prespecified state. We adopt a stochastic approximation viewpoint of this problem. It is known that by solving the corresponding continuous-time deterministic system, and using the resulting feedback control policy, one ensures almost sure convergence to the prespecified state in the discrete system. In this paper, we adopt such a control mechanism and provide its finite-sample convergence bounds whenever a Lyapunov function is known for the continuous system. In particular, we consider four cases based on whether the Lyapunov function for the continuous system gives exponential or sub-exponential rates and based on whether it is smooth or not. We provide the finite-time bounds in all cases. Our proof relies on constructing a Lyapunov function for the discrete system based on the given Lyapunov function for the continuous system. We do this by appropriately smoothing the given function using the Moreau envelope. We present numerical experiments corresponding to the various cases, which validate the rates we establish. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_11854 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Stochastic Approximation for Nonlinear Discrete Stochastic Control: Finite-Sample Bounds Nguyen, Hoang Huy Maguluri, Siva Theja Optimization and Control 62L20 (Primary) 34H05, 37N35, 65K10 (Secondary) We consider a nonlinear discrete stochastic control system, and our goal is to design a feedback control policy in order to lead the system to a prespecified state. We adopt a stochastic approximation viewpoint of this problem. It is known that by solving the corresponding continuous-time deterministic system, and using the resulting feedback control policy, one ensures almost sure convergence to the prespecified state in the discrete system. In this paper, we adopt such a control mechanism and provide its finite-sample convergence bounds whenever a Lyapunov function is known for the continuous system. In particular, we consider four cases based on whether the Lyapunov function for the continuous system gives exponential or sub-exponential rates and based on whether it is smooth or not. We provide the finite-time bounds in all cases. Our proof relies on constructing a Lyapunov function for the discrete system based on the given Lyapunov function for the continuous system. We do this by appropriately smoothing the given function using the Moreau envelope. We present numerical experiments corresponding to the various cases, which validate the rates we establish. |
| title | Stochastic Approximation for Nonlinear Discrete Stochastic Control: Finite-Sample Bounds |
| topic | Optimization and Control 62L20 (Primary) 34H05, 37N35, 65K10 (Secondary) |
| url | https://arxiv.org/abs/2304.11854 |