Second-Order Bilevel Optimization with Accelerated Convergence Rates
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915989631795200 |
|---|---|
| author | Yang, Sheng Liu, Chengchang Chen, Lesi Lui, John C. S. |
| author_facet | Yang, Sheng Liu, Chengchang Chen, Lesi Lui, John C. S. |
| contents | This paper studies second-order methods for nonconvex-strongly-convex bilevel optimization. We propose a novel fully second-order bilevel approximation method (FSBA) that achieves an iteration complexity of $\tilde{\mathcal{O}}(ε^{-1.5})$ for finding the $(ε, \mathcal{O}(\sqrtε))$ second-order stationary point of the hyper-objective function. Our results demonstrate that second-order methods can achieve an accelerated convergence rate than first-order methods in bilevel optimization. To address the heavy computational cost associated with the second-order oracle, we introduce a lazy variant of FSBA, called LFSBA, which reuses second-order information across several iterations. We prove that LFSBA exhibits better computational complexity than FSBA by a factor of $\sqrt{d}$, where $d$ is the dimension of the problem. We also apply a similar idea to nonconvex strongly-concave minimax optimization and propose the lazy minimax cubic-regularized Newton (LMCN) method with better computational complexity compared to existing second-order methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_06431 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Second-Order Bilevel Optimization with Accelerated Convergence Rates Yang, Sheng Liu, Chengchang Chen, Lesi Lui, John C. S. Optimization and Control This paper studies second-order methods for nonconvex-strongly-convex bilevel optimization. We propose a novel fully second-order bilevel approximation method (FSBA) that achieves an iteration complexity of $\tilde{\mathcal{O}}(ε^{-1.5})$ for finding the $(ε, \mathcal{O}(\sqrtε))$ second-order stationary point of the hyper-objective function. Our results demonstrate that second-order methods can achieve an accelerated convergence rate than first-order methods in bilevel optimization. To address the heavy computational cost associated with the second-order oracle, we introduce a lazy variant of FSBA, called LFSBA, which reuses second-order information across several iterations. We prove that LFSBA exhibits better computational complexity than FSBA by a factor of $\sqrt{d}$, where $d$ is the dimension of the problem. We also apply a similar idea to nonconvex strongly-concave minimax optimization and propose the lazy minimax cubic-regularized Newton (LMCN) method with better computational complexity compared to existing second-order methods. |
| title | Second-Order Bilevel Optimization with Accelerated Convergence Rates |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2605.06431 |