SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910730385620992 |
|---|---|
| author | Li, Suyi Yang, Lingyun Jiang, Xiaoxiao Lu, Hanfeng An, Dakai Di, Zhipeng Lu, Weiyi Chen, Jiawei Liu, Kan Yu, Yinghao Lan, Tao Yang, Guodong Qu, Lin Zhang, Liping Wang, Wei |
| author_facet | Li, Suyi Yang, Lingyun Jiang, Xiaoxiao Lu, Hanfeng An, Dakai Di, Zhipeng Lu, Weiyi Chen, Jiawei Liu, Kan Yu, Yinghao Lan, Tao Yang, Guodong Qu, Lin Zhang, Liping Wang, Wei |
| contents | Text-to-image (T2I) generation using diffusion models has become a blockbuster service in today's AI cloud. A production T2I service typically involves a serving workflow where a base diffusion model is augmented with various "add-on" modules, notably ControlNet and LoRA, to enhance image generation control. Compared to serving the base model alone, these add-on modules introduce significant loading and computational overhead, resulting in increased latency. In this paper, we present SwiftDiffusion, a system that efficiently serves a T2I workflow through a holistic approach. SwiftDiffusion decouples ControNet from the base model and deploys it as a separate, independently scaled service on dedicated GPUs, enabling ControlNet caching, parallelization, and sharing. To mitigate the high loading overhead of LoRA serving, SwiftDiffusion employs a bounded asynchronous LoRA loading (BAL) technique, allowing LoRA loading to overlap with the initial base model execution by up to k steps without compromising image quality. Furthermore, SwiftDiffusion optimizes base model execution with a novel latent parallelism technique. Collectively, these designs enable SwiftDiffusion to outperform the state-of-the-art T2I serving systems, achieving up to 7.8x latency reduction and 1.6x throughput improvement in serving SDXL models on H800 GPUs, without sacrificing image quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02031 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules Li, Suyi Yang, Lingyun Jiang, Xiaoxiao Lu, Hanfeng An, Dakai Di, Zhipeng Lu, Weiyi Chen, Jiawei Liu, Kan Yu, Yinghao Lan, Tao Yang, Guodong Qu, Lin Zhang, Liping Wang, Wei Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning Text-to-image (T2I) generation using diffusion models has become a blockbuster service in today's AI cloud. A production T2I service typically involves a serving workflow where a base diffusion model is augmented with various "add-on" modules, notably ControlNet and LoRA, to enhance image generation control. Compared to serving the base model alone, these add-on modules introduce significant loading and computational overhead, resulting in increased latency. In this paper, we present SwiftDiffusion, a system that efficiently serves a T2I workflow through a holistic approach. SwiftDiffusion decouples ControNet from the base model and deploys it as a separate, independently scaled service on dedicated GPUs, enabling ControlNet caching, parallelization, and sharing. To mitigate the high loading overhead of LoRA serving, SwiftDiffusion employs a bounded asynchronous LoRA loading (BAL) technique, allowing LoRA loading to overlap with the initial base model execution by up to k steps without compromising image quality. Furthermore, SwiftDiffusion optimizes base model execution with a novel latent parallelism technique. Collectively, these designs enable SwiftDiffusion to outperform the state-of-the-art T2I serving systems, achieving up to 7.8x latency reduction and 1.6x throughput improvement in serving SDXL models on H800 GPUs, without sacrificing image quality. |
| title | SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2407.02031 |