TridentServe: A Stage-level Serving System for Diffusion Pipelines
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916987273216000 |
|---|---|
| author | Xia, Yifei Fu, Fangcheng Yuan, Hao Zhang, Hanke Miao, Xupeng Liu, Yijun Ling, Suhan Jiang, Jie Cui, Bin |
| author_facet | Xia, Yifei Fu, Fangcheng Yuan, Hao Zhang, Hanke Miao, Xupeng Liu, Yijun Ling, Suhan Jiang, Jie Cui, Bin |
| contents | Diffusion pipelines, renowned for their powerful visual generation capabilities, have seen widespread adoption in generative vision tasks (e.g., text-to-image/video). These pipelines typically follow an encode--diffuse--decode three-stage architecture. Current serving systems deploy diffusion pipelines within a static, manual, and pipeline-level paradigm, allocating the same resources to every request and stage. However, through an in-depth analysis, we find that such a paradigm is inefficient due to the discrepancy in resource needs across the three stages of each request, as well as across different requests. Following the analysis, we propose the dynamic stage-level serving paradigm and develop TridentServe, a brand new diffusion serving system. TridentServe automatically, dynamically derives the placement plan (i.e., how each stage resides) for pipeline deployment and the dispatch plan (i.e., how the requests are routed) for request processing, co-optimizing the resource allocation for both model and requests. Extensive experiments show that TridentServe consistently improves SLO attainment and reduces average/P95 latencies by up to 2.5x and 3.6x/4.1x over existing works across a variety of workloads. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_02838 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TridentServe: A Stage-level Serving System for Diffusion Pipelines Xia, Yifei Fu, Fangcheng Yuan, Hao Zhang, Hanke Miao, Xupeng Liu, Yijun Ling, Suhan Jiang, Jie Cui, Bin Distributed, Parallel, and Cluster Computing Diffusion pipelines, renowned for their powerful visual generation capabilities, have seen widespread adoption in generative vision tasks (e.g., text-to-image/video). These pipelines typically follow an encode--diffuse--decode three-stage architecture. Current serving systems deploy diffusion pipelines within a static, manual, and pipeline-level paradigm, allocating the same resources to every request and stage. However, through an in-depth analysis, we find that such a paradigm is inefficient due to the discrepancy in resource needs across the three stages of each request, as well as across different requests. Following the analysis, we propose the dynamic stage-level serving paradigm and develop TridentServe, a brand new diffusion serving system. TridentServe automatically, dynamically derives the placement plan (i.e., how each stage resides) for pipeline deployment and the dispatch plan (i.e., how the requests are routed) for request processing, co-optimizing the resource allocation for both model and requests. Extensive experiments show that TridentServe consistently improves SLO attainment and reduces average/P95 latencies by up to 2.5x and 3.6x/4.1x over existing works across a variety of workloads. |
| title | TridentServe: A Stage-level Serving System for Diffusion Pipelines |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2510.02838 |