Implicit Bias and Convergence of Matrix Stochastic Mirror Descent
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915820942131200 |
|---|---|
| author | Akhtiamov, Danil Ghane, Reza Pooladzandi, Omead Hassibi, Babak |
| author_facet | Akhtiamov, Danil Ghane, Reza Pooladzandi, Omead Hassibi, Babak |
| contents | We investigate Stochastic Mirror Descent (SMD) with matrix parameters and vector-valued predictions, a framework relevant to multi-class classification and matrix completion problems. Focusing on the overparameterized regime, where the total number of parameters exceeds the number of training samples, we prove that SMD with matrix mirror functions $ψ(\cdot)$ converges exponentially to a global interpolator. Furthermore, we generalize classical implicit bias results of vector SMD by demonstrating that the matrix SMD algorithm converges to the unique solution minimizing the Bregman divergence induced by $ψ(\cdot)$ from initialization subject to interpolating the data. These findings reveal how matrix mirror maps dictate inductive bias in high-dimensional, multi-output problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_18997 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Implicit Bias and Convergence of Matrix Stochastic Mirror Descent Akhtiamov, Danil Ghane, Reza Pooladzandi, Omead Hassibi, Babak Machine Learning Optimization and Control We investigate Stochastic Mirror Descent (SMD) with matrix parameters and vector-valued predictions, a framework relevant to multi-class classification and matrix completion problems. Focusing on the overparameterized regime, where the total number of parameters exceeds the number of training samples, we prove that SMD with matrix mirror functions $ψ(\cdot)$ converges exponentially to a global interpolator. Furthermore, we generalize classical implicit bias results of vector SMD by demonstrating that the matrix SMD algorithm converges to the unique solution minimizing the Bregman divergence induced by $ψ(\cdot)$ from initialization subject to interpolating the data. These findings reveal how matrix mirror maps dictate inductive bias in high-dimensional, multi-output problems. |
| title | Implicit Bias and Convergence of Matrix Stochastic Mirror Descent |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2602.18997 |