SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915182534459392 |
|---|---|
| author | Liu, Hongjian Xie, Qingsong Ye, TianXiang Deng, Zhijie Chen, Chen Tang, Shixiang Fu, Xueyang Lu, Haonan Zha, Zheng-jun |
| author_facet | Liu, Hongjian Xie, Qingsong Ye, TianXiang Deng, Zhijie Chen, Chen Tang, Shixiang Fu, Xueyang Lu, Haonan Zha, Zheng-jun |
| contents | The iterative sampling procedure employed by diffusion models (DMs) often leads to significant inference latency. To address this, we propose Stochastic Consistency Distillation (SCott) to enable accelerated text-to-image generation, where high-quality and diverse generations can be achieved within just 2-4 sampling steps. In contrast to vanilla consistency distillation (CD) which distills the ordinary differential equation solvers-based sampling process of a pre-trained teacher model into a student, SCott explores the possibility and validates the efficacy of integrating stochastic differential equation (SDE) solvers into CD to fully unleash the potential of the teacher. SCott is augmented with elaborate strategies to control the noise strength and sampling process of the SDE solver. An adversarial loss is further incorporated to strengthen the consistency constraints in rare sampling steps. Empirically, on the MSCOCO-2017 5K dataset with a Stable Diffusion-V1.5 teacher, SCott achieves an FID of 21.9 with 2 sampling steps, surpassing that of the 1-step InstaFlow (23.4) and the 4-step UFOGen (22.1). Moreover, SCott can yield more diverse samples than other consistency models for high-resolution image generation, with up to 16% improvement in a qualified metric. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01505 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation Liu, Hongjian Xie, Qingsong Ye, TianXiang Deng, Zhijie Chen, Chen Tang, Shixiang Fu, Xueyang Lu, Haonan Zha, Zheng-jun Computer Vision and Pattern Recognition The iterative sampling procedure employed by diffusion models (DMs) often leads to significant inference latency. To address this, we propose Stochastic Consistency Distillation (SCott) to enable accelerated text-to-image generation, where high-quality and diverse generations can be achieved within just 2-4 sampling steps. In contrast to vanilla consistency distillation (CD) which distills the ordinary differential equation solvers-based sampling process of a pre-trained teacher model into a student, SCott explores the possibility and validates the efficacy of integrating stochastic differential equation (SDE) solvers into CD to fully unleash the potential of the teacher. SCott is augmented with elaborate strategies to control the noise strength and sampling process of the SDE solver. An adversarial loss is further incorporated to strengthen the consistency constraints in rare sampling steps. Empirically, on the MSCOCO-2017 5K dataset with a Stable Diffusion-V1.5 teacher, SCott achieves an FID of 21.9 with 2 sampling steps, surpassing that of the 1-step InstaFlow (23.4) and the 4-step UFOGen (22.1). Moreover, SCott can yield more diverse samples than other consistency models for high-resolution image generation, with up to 16% improvement in a qualified metric. |
| title | SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.01505 |