Hummingbird: SLO-Oriented GPU Preemption at Microsecond-scale
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910017324580864 |
|---|---|
| author | Hu, Tiancheng Wang, Chenxi Cao, Ting Qin, Jin Chen, Lei Xiao, Xinyu Hu, Junhao Tian, Hongliang Yan, Shoumeng Cui, Huimin Chen, Quan Xie, Tao |
| author_facet | Hu, Tiancheng Wang, Chenxi Cao, Ting Qin, Jin Chen, Lei Xiao, Xinyu Hu, Junhao Tian, Hongliang Yan, Shoumeng Cui, Huimin Chen, Quan Xie, Tao |
| contents | Existing GPU-sharing techniques, including spatial and temporal sharing, aim to improve utilization but face challenges in simultaneously ensuring SLO adherence and maximizing efficiency due to the lack of fine-grained task scheduling on closed-source GPUs. This paper presents Hummingbird, an SLO-oriented GPU scheduling system that overcomes these challenges by enabling microsecond-scale preemption on closed-source GPUs while effectively harvesting idle GPU time slices. Comprehensive evaluations across diverse GPU architectures reveal that Hummingbird improves the SLO attainment of high-priority tasks by 9.7x and 3.5x compared to the state-of-the-art spatial and temporal-sharing approaches. When compared to executing exclusively, the SLO attainment of the high-priority task, collocating with low-priority tasks on Hummingbird, only drops by less than 1%. Meanwhile, the throughput of the low-priority task outperforms the state-of-the-art temporal-sharing approaches by 2.4x. Hummingbird demonstrates significant effectiveness in ensuring the SLO while enhancing GPU utilization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_04071 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Hummingbird: SLO-Oriented GPU Preemption at Microsecond-scale Hu, Tiancheng Wang, Chenxi Cao, Ting Qin, Jin Chen, Lei Xiao, Xinyu Hu, Junhao Tian, Hongliang Yan, Shoumeng Cui, Huimin Chen, Quan Xie, Tao Distributed, Parallel, and Cluster Computing Existing GPU-sharing techniques, including spatial and temporal sharing, aim to improve utilization but face challenges in simultaneously ensuring SLO adherence and maximizing efficiency due to the lack of fine-grained task scheduling on closed-source GPUs. This paper presents Hummingbird, an SLO-oriented GPU scheduling system that overcomes these challenges by enabling microsecond-scale preemption on closed-source GPUs while effectively harvesting idle GPU time slices. Comprehensive evaluations across diverse GPU architectures reveal that Hummingbird improves the SLO attainment of high-priority tasks by 9.7x and 3.5x compared to the state-of-the-art spatial and temporal-sharing approaches. When compared to executing exclusively, the SLO attainment of the high-priority task, collocating with low-priority tasks on Hummingbird, only drops by less than 1%. Meanwhile, the throughput of the low-priority task outperforms the state-of-the-art temporal-sharing approaches by 2.4x. Hummingbird demonstrates significant effectiveness in ensuring the SLO while enhancing GPU utilization. |
| title | Hummingbird: SLO-Oriented GPU Preemption at Microsecond-scale |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2601.04071 |