Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2509.21716 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910098466537472 |
|---|---|
| author | Gonzalez, Xavier Buchanan, E. Kelly Lee, Hyun Dong Liu, Jerry Weihong Wang, Ke Alexander Zoltowski, David M. Kozachkov, Leo Ré, Christopher Linderman, Scott W. |
| author_facet | Gonzalez, Xavier Buchanan, E. Kelly Lee, Hyun Dong Liu, Jerry Weihong Wang, Ke Alexander Zoltowski, David M. Kozachkov, Leo Ré, Christopher Linderman, Scott W. |
| contents | Harnessing parallelism in seemingly sequential models is a central challenge for modern machine learning. Several approaches have been proposed for evaluating sequential processes in parallel using iterative fixed-point methods, like Newton, Picard, and Jacobi iterations. In this work, we show that these methods can be understood within a common framework based on linear dynamical systems (LDSs), where different iteration schemes arise naturally as approximate linearizations of a nonlinear recursion. Moreover, we theoretically analyze the rates of convergence of these methods, and we verify the predictions of this theory with several case studies. This unifying framework highlights shared principles behind these techniques and clarifies when particular fixed-point methods are most likely to be effective. By bridging diverse algorithms through the language of LDSs, the framework provides a clearer theoretical foundation for parallelizing sequential models and points toward new opportunities for efficient and scalable computation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_21716 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Unifying Framework for Parallelizing Sequential Models with Linear Dynamical Systems Gonzalez, Xavier Buchanan, E. Kelly Lee, Hyun Dong Liu, Jerry Weihong Wang, Ke Alexander Zoltowski, David M. Kozachkov, Leo Ré, Christopher Linderman, Scott W. Machine Learning Harnessing parallelism in seemingly sequential models is a central challenge for modern machine learning. Several approaches have been proposed for evaluating sequential processes in parallel using iterative fixed-point methods, like Newton, Picard, and Jacobi iterations. In this work, we show that these methods can be understood within a common framework based on linear dynamical systems (LDSs), where different iteration schemes arise naturally as approximate linearizations of a nonlinear recursion. Moreover, we theoretically analyze the rates of convergence of these methods, and we verify the predictions of this theory with several case studies. This unifying framework highlights shared principles behind these techniques and clarifies when particular fixed-point methods are most likely to be effective. By bridging diverse algorithms through the language of LDSs, the framework provides a clearer theoretical foundation for parallelizing sequential models and points toward new opportunities for efficient and scalable computation. |
| title | A Unifying Framework for Parallelizing Sequential Models with Linear Dynamical Systems |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2509.21716 |