Asymptotic Dynamics of Alternating Minimization for Bilinear Regression
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929692797304832 |
|---|---|
| author | Okajima, Koki Takahashi, Takashi |
| author_facet | Okajima, Koki Takahashi, Takashi |
| contents | This study investigates the dynamics of alternating minimization applied to a bilinear regression task with normally distributed covariates, under the asymptotic system size limit where the number of parameters and observations diverge at the same rate. This is achieved by employing the replica method to a multi-temperature glassy system which unfolds the algorithm's time evolution. Our results show that the dynamics can be described effectively by a two-dimensional discrete stochastic process, where each step depends on all previous time steps, revealing the structure of the memory dependence in the evolution of alternating minimization. The theoretical framework developed in this work can be applied to the analysis of various iterative algorithms, extending beyond the scope of alternating minimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_04751 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Asymptotic Dynamics of Alternating Minimization for Bilinear Regression Okajima, Koki Takahashi, Takashi Optimization and Control Disordered Systems and Neural Networks Machine Learning This study investigates the dynamics of alternating minimization applied to a bilinear regression task with normally distributed covariates, under the asymptotic system size limit where the number of parameters and observations diverge at the same rate. This is achieved by employing the replica method to a multi-temperature glassy system which unfolds the algorithm's time evolution. Our results show that the dynamics can be described effectively by a two-dimensional discrete stochastic process, where each step depends on all previous time steps, revealing the structure of the memory dependence in the evolution of alternating minimization. The theoretical framework developed in this work can be applied to the analysis of various iterative algorithms, extending beyond the scope of alternating minimization. |
| title | Asymptotic Dynamics of Alternating Minimization for Bilinear Regression |
| topic | Optimization and Control Disordered Systems and Neural Networks Machine Learning |
| url | https://arxiv.org/abs/2402.04751 |