I've Got 99 Problems But FLOPS Ain't One
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929554796314624 |
|---|---|
| author | Gherghescu, Alexandru M. Bădoiu, Vlad-Andrei Agache, Alexandru Dumitru, Mihai-Valentin Vasilescu, Iuliu Mantu, Radu Raiciu, Costin |
| author_facet | Gherghescu, Alexandru M. Bădoiu, Vlad-Andrei Agache, Alexandru Dumitru, Mihai-Valentin Vasilescu, Iuliu Mantu, Radu Raiciu, Costin |
| contents | Hyperscalers dominate the landscape of large network deployments, yet they rarely share data or insights about the challenges they face. In light of this supremacy, what problems can we find to solve in this space? We take an unconventional approach to find relevant research directions, starting from public plans to build a $100 billion datacenter for machine learning applications. Leveraging the language models scaling laws, we discover what workloads such a datacenter might carry and explore the challenges one may encounter in doing so, with a focus on networking research. We conclude that building the datacenter and training such models is technically possible, but this requires novel wide-area transports for inter-DC communication, a multipath transport and novel datacenter topologies for intra-datacenter communication, high speed scale-up networks and transports, outlining a rich research agenda for the networking community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12819 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | I've Got 99 Problems But FLOPS Ain't One Gherghescu, Alexandru M. Bădoiu, Vlad-Andrei Agache, Alexandru Dumitru, Mihai-Valentin Vasilescu, Iuliu Mantu, Radu Raiciu, Costin Distributed, Parallel, and Cluster Computing Computation and Language Machine Learning Networking and Internet Architecture Hyperscalers dominate the landscape of large network deployments, yet they rarely share data or insights about the challenges they face. In light of this supremacy, what problems can we find to solve in this space? We take an unconventional approach to find relevant research directions, starting from public plans to build a $100 billion datacenter for machine learning applications. Leveraging the language models scaling laws, we discover what workloads such a datacenter might carry and explore the challenges one may encounter in doing so, with a focus on networking research. We conclude that building the datacenter and training such models is technically possible, but this requires novel wide-area transports for inter-DC communication, a multipath transport and novel datacenter topologies for intra-datacenter communication, high speed scale-up networks and transports, outlining a rich research agenda for the networking community. |
| title | I've Got 99 Problems But FLOPS Ain't One |
| topic | Distributed, Parallel, and Cluster Computing Computation and Language Machine Learning Networking and Internet Architecture |
| url | https://arxiv.org/abs/2407.12819 |