FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-on
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913583172943872 |
|---|---|
| author | Jiang, Boyuan Hu, Xiaobin Luo, Donghao He, Qingdong Xu, Chengming Peng, Jinlong Zhang, Jiangning Wang, Chengjie Wu, Yunsheng Fu, Yanwei |
| author_facet | Jiang, Boyuan Hu, Xiaobin Luo, Donghao He, Qingdong Xu, Chengming Peng, Jinlong Zhang, Jiangning Wang, Chengjie Wu, Yunsheng Fu, Yanwei |
| contents | Although image-based virtual try-on has made considerable progress, emerging approaches still encounter challenges in producing high-fidelity and robust fitting images across diverse scenarios. These methods often struggle with issues such as texture-aware maintenance and size-aware fitting, which hinder their overall effectiveness. To address these limitations, we propose a novel garment perception enhancement technique, termed FitDiT, designed for high-fidelity virtual try-on using Diffusion Transformers (DiT) allocating more parameters and attention to high-resolution features. First, to further improve texture-aware maintenance, we introduce a garment texture extractor that incorporates garment priors evolution to fine-tune garment feature, facilitating to better capture rich details such as stripes, patterns, and text. Additionally, we introduce frequency-domain learning by customizing a frequency distance loss to enhance high-frequency garment details. To tackle the size-aware fitting issue, we employ a dilated-relaxed mask strategy that adapts to the correct length of garments, preventing the generation of garments that fill the entire mask area during cross-category try-on. Equipped with the above design, FitDiT surpasses all baselines in both qualitative and quantitative evaluations. It excels in producing well-fitting garments with photorealistic and intricate details, while also achieving competitive inference times of 4.57 seconds for a single 1024x768 image after DiT structure slimming, outperforming existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_10499 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-on Jiang, Boyuan Hu, Xiaobin Luo, Donghao He, Qingdong Xu, Chengming Peng, Jinlong Zhang, Jiangning Wang, Chengjie Wu, Yunsheng Fu, Yanwei Computer Vision and Pattern Recognition Although image-based virtual try-on has made considerable progress, emerging approaches still encounter challenges in producing high-fidelity and robust fitting images across diverse scenarios. These methods often struggle with issues such as texture-aware maintenance and size-aware fitting, which hinder their overall effectiveness. To address these limitations, we propose a novel garment perception enhancement technique, termed FitDiT, designed for high-fidelity virtual try-on using Diffusion Transformers (DiT) allocating more parameters and attention to high-resolution features. First, to further improve texture-aware maintenance, we introduce a garment texture extractor that incorporates garment priors evolution to fine-tune garment feature, facilitating to better capture rich details such as stripes, patterns, and text. Additionally, we introduce frequency-domain learning by customizing a frequency distance loss to enhance high-frequency garment details. To tackle the size-aware fitting issue, we employ a dilated-relaxed mask strategy that adapts to the correct length of garments, preventing the generation of garments that fill the entire mask area during cross-category try-on. Equipped with the above design, FitDiT surpasses all baselines in both qualitative and quantitative evaluations. It excels in producing well-fitting garments with photorealistic and intricate details, while also achieving competitive inference times of 4.57 seconds for a single 1024x768 image after DiT structure slimming, outperforming existing methods. |
| title | FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-on |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.10499 |