A General Recipe for Parameter-Free Nonconvex Optimization via Higher-Order Regularization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916064499073024 |
|---|---|
| author | Marumo, Naoki Takeda, Akiko |
| author_facet | Marumo, Naoki Takeda, Akiko |
| contents | We develop a systematic framework for constructing parameter-free algorithms for smooth nonconvex optimization. The framework is based on higher-order regularization: each step is computed from a regularized local model whose regularization exponent exceeds the order of the model error. This design makes the resulting method robust to misspecification of the regularization parameter and yields complexity bounds without backtracking or other acceptance tests. We apply the framework to gradient descent, Newton's method, the Gauss--Newton method, stochastic gradient descent, and PAGE. Without prior knowledge of problem-dependent parameters, the resulting algorithms achieve complexity bounds with optimal or best-known dependence on the target accuracy. When the problem-dependent parameters are known up to constant factors, suitable tuning also recovers the optimal or best-known dependence on those parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_30891 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A General Recipe for Parameter-Free Nonconvex Optimization via Higher-Order Regularization Marumo, Naoki Takeda, Akiko Optimization and Control 90C26, 90C30, 65K05, 49M15 We develop a systematic framework for constructing parameter-free algorithms for smooth nonconvex optimization. The framework is based on higher-order regularization: each step is computed from a regularized local model whose regularization exponent exceeds the order of the model error. This design makes the resulting method robust to misspecification of the regularization parameter and yields complexity bounds without backtracking or other acceptance tests. We apply the framework to gradient descent, Newton's method, the Gauss--Newton method, stochastic gradient descent, and PAGE. Without prior knowledge of problem-dependent parameters, the resulting algorithms achieve complexity bounds with optimal or best-known dependence on the target accuracy. When the problem-dependent parameters are known up to constant factors, suitable tuning also recovers the optimal or best-known dependence on those parameters. |
| title | A General Recipe for Parameter-Free Nonconvex Optimization via Higher-Order Regularization |
| topic | Optimization and Control 90C26, 90C30, 65K05, 49M15 |
| url | https://arxiv.org/abs/2605.30891 |