Moduli space of optimization algorithms
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909917512728576 |
|---|---|
| author | Pasechnyuk-Vilensky, Dmitry Takáč, Martin |
| author_facet | Pasechnyuk-Vilensky, Dmitry Takáč, Martin |
| contents | We introduce a geometric and operator-theoretic formalism viewing optimization algorithms as discrete connections on a space of update operators. Each iterative method is encoded by two coupled channels-drift and diffusion-whose algebraic curvature measures the deviation from ideal reversibility and determines the attainable order of accuracy. Flat connections correspond to methods whose updates commute up to higher order and thus achieve minimal numerical dissipation while preserving stability.
The formalism recovers classical gradient, proximal, and momentum schemes as first-order flat cases and extends naturally to stochastic, preconditioned, and adaptive algorithms through perturbations controlled by curvature order. Explicit gauge corrections yield higher-order variants with guaranteed energy monotonicity and noise stability. The associated determinantal and isomonodromic formulations yield exact nonasymptotic bounds and describe acceleration effects via Painlevé-type invariants and Stokes corrections. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_18004 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Moduli space of optimization algorithms Pasechnyuk-Vilensky, Dmitry Takáč, Martin Optimization and Control Differential Geometry 65K05, 65L07, 49M27, 53C05 G.1.0; F.2.1; J.2 We introduce a geometric and operator-theoretic formalism viewing optimization algorithms as discrete connections on a space of update operators. Each iterative method is encoded by two coupled channels-drift and diffusion-whose algebraic curvature measures the deviation from ideal reversibility and determines the attainable order of accuracy. Flat connections correspond to methods whose updates commute up to higher order and thus achieve minimal numerical dissipation while preserving stability. The formalism recovers classical gradient, proximal, and momentum schemes as first-order flat cases and extends naturally to stochastic, preconditioned, and adaptive algorithms through perturbations controlled by curvature order. Explicit gauge corrections yield higher-order variants with guaranteed energy monotonicity and noise stability. The associated determinantal and isomonodromic formulations yield exact nonasymptotic bounds and describe acceleration effects via Painlevé-type invariants and Stokes corrections. |
| title | Moduli space of optimization algorithms |
| topic | Optimization and Control Differential Geometry 65K05, 65L07, 49M27, 53C05 G.1.0; F.2.1; J.2 |
| url | https://arxiv.org/abs/2511.18004 |