The Serial Scaling Hypothesis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908999224393728 |
|---|---|
| author | Liu, Yuxi Preechakul, Konpat Kuwaranancharoen, Kananart Bai, Yutong |
| author_facet | Liu, Yuxi Preechakul, Konpat Kuwaranancharoen, Kananart Bai, Yutong |
| contents | While machine learning has advanced through massive parallelization, we identify a critical blind spot: some problems are fundamentally sequential. These "inherently serial" problems-from mathematical reasoning to physical simulations to sequential decision-making-require sequentially dependent computational steps that cannot be efficiently parallelized. We formalize this distinction in complexity theory, and demonstrate that current parallel-centric architectures face fundamental limitations on such tasks. Then, we show for first time that diffusion models despite their sequential nature are incapable of solving inherently serial problems. We argue that recognizing the serial nature of computation holds profound implications on machine learning, model design, and hardware development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_12549 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Serial Scaling Hypothesis Liu, Yuxi Preechakul, Konpat Kuwaranancharoen, Kananart Bai, Yutong Machine Learning Computational Complexity 68Q15, 68Q10, 68T07 F.1.1; F.1.3; I.2.6 While machine learning has advanced through massive parallelization, we identify a critical blind spot: some problems are fundamentally sequential. These "inherently serial" problems-from mathematical reasoning to physical simulations to sequential decision-making-require sequentially dependent computational steps that cannot be efficiently parallelized. We formalize this distinction in complexity theory, and demonstrate that current parallel-centric architectures face fundamental limitations on such tasks. Then, we show for first time that diffusion models despite their sequential nature are incapable of solving inherently serial problems. We argue that recognizing the serial nature of computation holds profound implications on machine learning, model design, and hardware development. |
| title | The Serial Scaling Hypothesis |
| topic | Machine Learning Computational Complexity 68Q15, 68Q10, 68T07 F.1.1; F.1.3; I.2.6 |
| url | https://arxiv.org/abs/2507.12549 |