Finite-Time Analysis of Projected Two-Time-Scale Stochastic Approximation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914435836149760 |
|---|---|
| author | Bai, Yitao Doan, Thinh T. Romberg, Justin |
| author_facet | Bai, Yitao Doan, Thinh T. Romberg, Justin |
| contents | We study the finite-time convergence of projected linear two-time-scale stochastic approximation with constant step sizes and Polyak--Ruppert averaging. We establish an explicit mean-square error bound, decomposing it into two interpretable components, an approximation error determined by the constrained subspace and a statistical error decaying at a sublinear rate, with constants expressed through restricted stability margins and a coupling invertibility condition. These constants cleanly separate the effect of subspace choice (approximation errors) from the effect of the averaging horizon (statistical errors). We illustrate our theoretical results through a number of numerical experiments on both synthetic and reinforcement learning problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_00179 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Finite-Time Analysis of Projected Two-Time-Scale Stochastic Approximation Bai, Yitao Doan, Thinh T. Romberg, Justin Systems and Control Machine Learning 62L20, 93E35 We study the finite-time convergence of projected linear two-time-scale stochastic approximation with constant step sizes and Polyak--Ruppert averaging. We establish an explicit mean-square error bound, decomposing it into two interpretable components, an approximation error determined by the constrained subspace and a statistical error decaying at a sublinear rate, with constants expressed through restricted stability margins and a coupling invertibility condition. These constants cleanly separate the effect of subspace choice (approximation errors) from the effect of the averaging horizon (statistical errors). We illustrate our theoretical results through a number of numerical experiments on both synthetic and reinforcement learning problems. |
| title | Finite-Time Analysis of Projected Two-Time-Scale Stochastic Approximation |
| topic | Systems and Control Machine Learning 62L20, 93E35 |
| url | https://arxiv.org/abs/2604.00179 |