On the (almost) Global Exponential Convergence of the Overparameterized Policy Optimization for the LQR Problem
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916985673089024 |
|---|---|
| author | Wafi, Moh Kamalul de Oliveira, Arthur Castello B. Sontag, Eduardo D. |
| author_facet | Wafi, Moh Kamalul de Oliveira, Arthur Castello B. Sontag, Eduardo D. |
| contents | In this work we study the convergence of gradient methods for nonconvex optimization problems -- specifically the effect of the problem formulation to the convergence behavior of the solution of a gradient flow. We show through a simple example that, surprisingly, the gradient flow solution can be exponentially or asymptotically convergent, depending on how the problem is formulated. We then deepen the analysis and show that a policy optimization strategy for the continuous-time linear quadratic regulator (LQR) (which is known to present only asymptotic convergence globally) presents almost global exponential convergence if the problem is overparameterized through a linear feed-forward neural network (LFFNN). We prove this qualitative improvement always happens for a simplified version of the LQR problem and derive explicit convergence rates for the gradient flow. Finally, we show that both the qualitative improvement and the quantitative rate gains persist in the general LQR through numerical simulations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_02140 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the (almost) Global Exponential Convergence of the Overparameterized Policy Optimization for the LQR Problem Wafi, Moh Kamalul de Oliveira, Arthur Castello B. Sontag, Eduardo D. Optimization and Control Systems and Control In this work we study the convergence of gradient methods for nonconvex optimization problems -- specifically the effect of the problem formulation to the convergence behavior of the solution of a gradient flow. We show through a simple example that, surprisingly, the gradient flow solution can be exponentially or asymptotically convergent, depending on how the problem is formulated. We then deepen the analysis and show that a policy optimization strategy for the continuous-time linear quadratic regulator (LQR) (which is known to present only asymptotic convergence globally) presents almost global exponential convergence if the problem is overparameterized through a linear feed-forward neural network (LFFNN). We prove this qualitative improvement always happens for a simplified version of the LQR problem and derive explicit convergence rates for the gradient flow. Finally, we show that both the qualitative improvement and the quantitative rate gains persist in the general LQR through numerical simulations. |
| title | On the (almost) Global Exponential Convergence of the Overparameterized Policy Optimization for the LQR Problem |
| topic | Optimization and Control Systems and Control |
| url | https://arxiv.org/abs/2510.02140 |