The Energy Cost of Execution-Idle in GPU Clusters
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913007790981120 |
|---|---|
| author | Lei, Yiran Fernandez, Jared Kypriotis, Vasilis Skarlatos, Dimitrios Strubell, Emma Sherry, Justine Vosler, Daniel |
| author_facet | Lei, Yiran Fernandez, Jared Kypriotis, Vasilis Skarlatos, Dimitrios Strubell, Emma Sherry, Justine Vosler, Daniel |
| contents | GPUs are becoming a major contributor to data center power, yet unlike CPUs, they can remain at high power even when visible activity is near zero. We call this state execution-idle. Using per-second telemetry from a large academic AI cluster, we characterize execution-idle as a recurring low-activity yet high-power state in real deployments. Across diverse workloads and multiple GPU generations, it accounts for 19.7% of in-execution time and 10.7% of energy. This suggests a need to both reduce the cost of execution-idle and reduce exposure to it. We therefore build two prototypes: one uses automatic downscaling during execution-idle, and the other uses load imbalance to reduce exposure, both with performance trade-offs. These findings suggest that future energy-efficient GPU systems should treat execution-idle as a first-class operating state. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_04745 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Energy Cost of Execution-Idle in GPU Clusters Lei, Yiran Fernandez, Jared Kypriotis, Vasilis Skarlatos, Dimitrios Strubell, Emma Sherry, Justine Vosler, Daniel Distributed, Parallel, and Cluster Computing Performance GPUs are becoming a major contributor to data center power, yet unlike CPUs, they can remain at high power even when visible activity is near zero. We call this state execution-idle. Using per-second telemetry from a large academic AI cluster, we characterize execution-idle as a recurring low-activity yet high-power state in real deployments. Across diverse workloads and multiple GPU generations, it accounts for 19.7% of in-execution time and 10.7% of energy. This suggests a need to both reduce the cost of execution-idle and reduce exposure to it. We therefore build two prototypes: one uses automatic downscaling during execution-idle, and the other uses load imbalance to reduce exposure, both with performance trade-offs. These findings suggest that future energy-efficient GPU systems should treat execution-idle as a first-class operating state. |
| title | The Energy Cost of Execution-Idle in GPU Clusters |
| topic | Distributed, Parallel, and Cluster Computing Performance |
| url | https://arxiv.org/abs/2604.04745 |