EdgeFusion: On-Device Text-to-Image Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929318661193728 |
|---|---|
| author | Castells, Thibault Song, Hyoung-Kyu Piao, Tairen Choi, Shinkook Kim, Bo-Kyeong Yim, Hanyoung Lee, Changgwun Kim, Jae Gon Kim, Tae-Ho |
| author_facet | Castells, Thibault Song, Hyoung-Kyu Piao, Tairen Choi, Shinkook Kim, Bo-Kyeong Yim, Hanyoung Lee, Changgwun Kim, Jae Gon Kim, Tae-Ho |
| contents | The intensive computational burden of Stable Diffusion (SD) for text-to-image generation poses a significant hurdle for its practical application. To tackle this challenge, recent research focuses on methods to reduce sampling steps, such as Latent Consistency Model (LCM), and on employing architectural optimizations, including pruning and knowledge distillation. Diverging from existing approaches, we uniquely start with a compact SD variant, BK-SDM. We observe that directly applying LCM to BK-SDM with commonly used crawled datasets yields unsatisfactory results. It leads us to develop two strategies: (1) leveraging high-quality image-text pairs from leading generative models and (2) designing an advanced distillation process tailored for LCM. Through our thorough exploration of quantization, profiling, and on-device deployment, we achieve rapid generation of photo-realistic, text-aligned images in just two steps, with latency under one second on resource-limited edge devices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_11925 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EdgeFusion: On-Device Text-to-Image Generation Castells, Thibault Song, Hyoung-Kyu Piao, Tairen Choi, Shinkook Kim, Bo-Kyeong Yim, Hanyoung Lee, Changgwun Kim, Jae Gon Kim, Tae-Ho Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition The intensive computational burden of Stable Diffusion (SD) for text-to-image generation poses a significant hurdle for its practical application. To tackle this challenge, recent research focuses on methods to reduce sampling steps, such as Latent Consistency Model (LCM), and on employing architectural optimizations, including pruning and knowledge distillation. Diverging from existing approaches, we uniquely start with a compact SD variant, BK-SDM. We observe that directly applying LCM to BK-SDM with commonly used crawled datasets yields unsatisfactory results. It leads us to develop two strategies: (1) leveraging high-quality image-text pairs from leading generative models and (2) designing an advanced distillation process tailored for LCM. Through our thorough exploration of quantization, profiling, and on-device deployment, we achieve rapid generation of photo-realistic, text-aligned images in just two steps, with latency under one second on resource-limited edge devices. |
| title | EdgeFusion: On-Device Text-to-Image Generation |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.11925 |