StableGarment: Garment-Centric Generation via Stable Diffusion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916161252229120 |
|---|---|
| author | Wang, Rui Guo, Hailong Liu, Jiaming Li, Huaxia Zhao, Haibo Tang, Xu Hu, Yao Tang, Hao Li, Peipei |
| author_facet | Wang, Rui Guo, Hailong Liu, Jiaming Li, Huaxia Zhao, Haibo Tang, Xu Hu, Yao Tang, Hao Li, Peipei |
| contents | In this paper, we introduce StableGarment, a unified framework to tackle garment-centric(GC) generation tasks, including GC text-to-image, controllable GC text-to-image, stylized GC text-to-image, and robust virtual try-on. The main challenge lies in retaining the intricate textures of the garment while maintaining the flexibility of pre-trained Stable Diffusion. Our solution involves the development of a garment encoder, a trainable copy of the denoising UNet equipped with additive self-attention (ASA) layers. These ASA layers are specifically devised to transfer detailed garment textures, also facilitating the integration of stylized base models for the creation of stylized images. Furthermore, the incorporation of a dedicated try-on ControlNet enables StableGarment to execute virtual try-on tasks with precision. We also build a novel data engine that produces high-quality synthesized data to preserve the model's ability to follow prompts. Extensive experiments demonstrate that our approach delivers state-of-the-art (SOTA) results among existing virtual try-on methods and exhibits high flexibility with broad potential applications in various garment-centric image generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10783 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | StableGarment: Garment-Centric Generation via Stable Diffusion Wang, Rui Guo, Hailong Liu, Jiaming Li, Huaxia Zhao, Haibo Tang, Xu Hu, Yao Tang, Hao Li, Peipei Computer Vision and Pattern Recognition In this paper, we introduce StableGarment, a unified framework to tackle garment-centric(GC) generation tasks, including GC text-to-image, controllable GC text-to-image, stylized GC text-to-image, and robust virtual try-on. The main challenge lies in retaining the intricate textures of the garment while maintaining the flexibility of pre-trained Stable Diffusion. Our solution involves the development of a garment encoder, a trainable copy of the denoising UNet equipped with additive self-attention (ASA) layers. These ASA layers are specifically devised to transfer detailed garment textures, also facilitating the integration of stylized base models for the creation of stylized images. Furthermore, the incorporation of a dedicated try-on ControlNet enables StableGarment to execute virtual try-on tasks with precision. We also build a novel data engine that produces high-quality synthesized data to preserve the model's ability to follow prompts. Extensive experiments demonstrate that our approach delivers state-of-the-art (SOTA) results among existing virtual try-on methods and exhibits high flexibility with broad potential applications in various garment-centric image generation. |
| title | StableGarment: Garment-Centric Generation via Stable Diffusion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.10783 |