GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915924154515456 |
|---|---|
| author | Ye, Fanjiang Li, Zhangke Zhong, Xinrui Ma, Ethan Chen, Russell Wang, Kaijian Zuo, Jingwei Sun, Desen Cao, Ye Cao, Triston Lee, Myungjin Krishnamurthy, Arvind Wang, Yuke |
| author_facet | Ye, Fanjiang Li, Zhangke Zhong, Xinrui Ma, Ethan Chen, Russell Wang, Kaijian Zuo, Jingwei Sun, Desen Cao, Ye Cao, Triston Lee, Myungjin Krishnamurthy, Arvind Wang, Yuke |
| contents | Diffusion models have emerged as the prevailing approach for text-to-image (T2I) and text-to-video (T2V) generation, yet production platforms must increasingly serve both modalities on shared GPU clusters while meeting stringent latency SLOs. Co-serving such heterogeneous workloads is challenging: T2I and T2V requests exhibit vastly different compute demands, parallelism characteristics, and latency requirements, leading to significant SLO violations in existing serving systems. We present GENSERVE, a co-serving system that leverages the inherent predictability of the diffusion process to optimize serving efficiency. A central insight is that diffusion inference proceeds in discrete, predictable steps and is naturally preemptible at step boundaries, opening a new design space for heterogeneity-aware resource management. GENSERVE introduces step-level resource adaptation through three coordinated mechanisms: intelligent video preemption, elastic sequence parallelism with dynamic batching, and an SLO-aware scheduler that jointly optimizes resource allocation across all concurrent requests. Experimental results show that GENSERVE improves the SLO attainment rate by up to 44% over the strongest baseline across diverse configurations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_04335 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads Ye, Fanjiang Li, Zhangke Zhong, Xinrui Ma, Ethan Chen, Russell Wang, Kaijian Zuo, Jingwei Sun, Desen Cao, Ye Cao, Triston Lee, Myungjin Krishnamurthy, Arvind Wang, Yuke Distributed, Parallel, and Cluster Computing Diffusion models have emerged as the prevailing approach for text-to-image (T2I) and text-to-video (T2V) generation, yet production platforms must increasingly serve both modalities on shared GPU clusters while meeting stringent latency SLOs. Co-serving such heterogeneous workloads is challenging: T2I and T2V requests exhibit vastly different compute demands, parallelism characteristics, and latency requirements, leading to significant SLO violations in existing serving systems. We present GENSERVE, a co-serving system that leverages the inherent predictability of the diffusion process to optimize serving efficiency. A central insight is that diffusion inference proceeds in discrete, predictable steps and is naturally preemptible at step boundaries, opening a new design space for heterogeneity-aware resource management. GENSERVE introduces step-level resource adaptation through three coordinated mechanisms: intelligent video preemption, elastic sequence parallelism with dynamic batching, and an SLO-aware scheduler that jointly optimizes resource allocation across all concurrent requests. Experimental results show that GENSERVE improves the SLO attainment rate by up to 44% over the strongest baseline across diverse configurations. |
| title | GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2604.04335 |