Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.10821 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917591341072384 |
|---|---|
| author | Yan, Divin Qi, Lu Hu, Vincent Tao Yang, Ming-Hsuan Tang, Meng |
| author_facet | Yan, Divin Qi, Lu Hu, Vincent Tao Yang, Ming-Hsuan Tang, Meng |
| contents | Diffusion models have made significant advances recently in high-quality image synthesis and related tasks. However, diffusion models trained on real-world datasets, which often follow long-tailed distributions, yield inferior fidelity for tail classes. Deep generative models, including diffusion models, are biased towards classes with abundant training images. To address the observed appearance overlap between synthesized images of rare classes and tail classes, we propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes. We show variants of our probabilistic contrastive learning method can be applied to any class conditional diffusion model. We show significant improvement in image synthesis using our loss for multiple datasets with long-tailed distribution. Extensive experimental results demonstrate that the proposed method can effectively handle imbalanced data for diffusion-based generation and classification models. Our code and datasets will be publicly available at https://github.com/yanliang3612/DiffROP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10821 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Training Class-Imbalanced Diffusion Model Via Overlap Optimization Yan, Divin Qi, Lu Hu, Vincent Tao Yang, Ming-Hsuan Tang, Meng Computer Vision and Pattern Recognition Diffusion models have made significant advances recently in high-quality image synthesis and related tasks. However, diffusion models trained on real-world datasets, which often follow long-tailed distributions, yield inferior fidelity for tail classes. Deep generative models, including diffusion models, are biased towards classes with abundant training images. To address the observed appearance overlap between synthesized images of rare classes and tail classes, we propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes. We show variants of our probabilistic contrastive learning method can be applied to any class conditional diffusion model. We show significant improvement in image synthesis using our loss for multiple datasets with long-tailed distribution. Extensive experimental results demonstrate that the proposed method can effectively handle imbalanced data for diffusion-based generation and classification models. Our code and datasets will be publicly available at https://github.com/yanliang3612/DiffROP. |
| title | Training Class-Imbalanced Diffusion Model Via Overlap Optimization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.10821 |