BitsFusion: 1.99 bits Weight Quantization of Diffusion Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929560945164288 |
|---|---|
| author | Sui, Yang Li, Yanyu Kag, Anil Idelbayev, Yerlan Cao, Junli Hu, Ju Sagar, Dhritiman Yuan, Bo Tulyakov, Sergey Ren, Jian |
| author_facet | Sui, Yang Li, Yanyu Kag, Anil Idelbayev, Yerlan Cao, Junli Hu, Ju Sagar, Dhritiman Yuan, Bo Tulyakov, Sergey Ren, Jian |
| contents | Diffusion-based image generation models have achieved great success in recent years by showing the capability of synthesizing high-quality content. However, these models contain a huge number of parameters, resulting in a significantly large model size. Saving and transferring them is a major bottleneck for various applications, especially those running on resource-constrained devices. In this work, we develop a novel weight quantization method that quantizes the UNet from Stable Diffusion v1.5 to 1.99 bits, achieving a model with 7.9X smaller size while exhibiting even better generation quality than the original one. Our approach includes several novel techniques, such as assigning optimal bits to each layer, initializing the quantized model for better performance, and improving the training strategy to dramatically reduce quantization error. Furthermore, we extensively evaluate our quantized model across various benchmark datasets and through human evaluation to demonstrate its superior generation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_04333 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BitsFusion: 1.99 bits Weight Quantization of Diffusion Model Sui, Yang Li, Yanyu Kag, Anil Idelbayev, Yerlan Cao, Junli Hu, Ju Sagar, Dhritiman Yuan, Bo Tulyakov, Sergey Ren, Jian Computer Vision and Pattern Recognition Diffusion-based image generation models have achieved great success in recent years by showing the capability of synthesizing high-quality content. However, these models contain a huge number of parameters, resulting in a significantly large model size. Saving and transferring them is a major bottleneck for various applications, especially those running on resource-constrained devices. In this work, we develop a novel weight quantization method that quantizes the UNet from Stable Diffusion v1.5 to 1.99 bits, achieving a model with 7.9X smaller size while exhibiting even better generation quality than the original one. Our approach includes several novel techniques, such as assigning optimal bits to each layer, initializing the quantized model for better performance, and improving the training strategy to dramatically reduce quantization error. Furthermore, we extensively evaluate our quantized model across various benchmark datasets and through human evaluation to demonstrate its superior generation quality. |
| title | BitsFusion: 1.99 bits Weight Quantization of Diffusion Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.04333 |