SUN-DSBO: A Structured Unified Framework for Nonconvex Decentralized Stochastic Bilevel Optimization
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918315489755136 |
|---|---|
| author | Ma, Yaoshuai Wang, Xiao Yao, Wei Zhang, Jin |
| author_facet | Ma, Yaoshuai Wang, Xiao Yao, Wei Zhang, Jin |
| contents | Decentralized stochastic bilevel optimization (DSBO) is a powerful tool for various machine learning tasks, including decentralized meta-learning and hyperparameter tuning. Existing DSBO methods primarily address problems with strongly convex lower-level objective functions. However, nonconvex objective functions are increasingly prevalent in modern deep learning. In this work, we introduce SUN-DSBO, a Structured Unified framework for Nonconvex DSBO, in which both the upper- and lower-level objective functions may be nonconvex. Notably, SUN-DSBO offers the flexibility to incorporate decentralized stochastic gradient descent or various techniques for mitigating data heterogeneity, such as gradient tracking (GT). We demonstrate that SUN-DSBO-GT, an adaptation of the GT technique within our framework, achieves a linear speedup with respect to the number of agents. This is accomplished without relying on restrictive assumptions, such as gradient boundedness or any specific assumptions regarding gradient heterogeneity. Numerical experiments validate the effectiveness of our method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_22682 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SUN-DSBO: A Structured Unified Framework for Nonconvex Decentralized Stochastic Bilevel Optimization Ma, Yaoshuai Wang, Xiao Yao, Wei Zhang, Jin Optimization and Control Decentralized stochastic bilevel optimization (DSBO) is a powerful tool for various machine learning tasks, including decentralized meta-learning and hyperparameter tuning. Existing DSBO methods primarily address problems with strongly convex lower-level objective functions. However, nonconvex objective functions are increasingly prevalent in modern deep learning. In this work, we introduce SUN-DSBO, a Structured Unified framework for Nonconvex DSBO, in which both the upper- and lower-level objective functions may be nonconvex. Notably, SUN-DSBO offers the flexibility to incorporate decentralized stochastic gradient descent or various techniques for mitigating data heterogeneity, such as gradient tracking (GT). We demonstrate that SUN-DSBO-GT, an adaptation of the GT technique within our framework, achieves a linear speedup with respect to the number of agents. This is accomplished without relying on restrictive assumptions, such as gradient boundedness or any specific assumptions regarding gradient heterogeneity. Numerical experiments validate the effectiveness of our method. |
| title | SUN-DSBO: A Structured Unified Framework for Nonconvex Decentralized Stochastic Bilevel Optimization |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2601.22682 |