SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912130834366464 |
|---|---|
| author | Wang, Chaoyang Li, Xiangtai Qi, Lu Ding, Henghui Tong, Yunhai Yang, Ming-Hsuan |
| author_facet | Wang, Chaoyang Li, Xiangtai Qi, Lu Ding, Henghui Tong, Yunhai Yang, Ming-Hsuan |
| contents | Semantic segmentation and semantic image synthesis are two representative tasks in visual perception and generation. While existing methods consider them as two distinct tasks, we propose a unified framework (SemFlow) and model them as a pair of reverse problems. Specifically, motivated by rectified flow theory, we train an ordinary differential equation (ODE) model to transport between the distributions of real images and semantic masks. As the training object is symmetric, samples belonging to the two distributions, images and semantic masks, can be effortlessly transferred reversibly. For semantic segmentation, our approach solves the contradiction between the randomness of diffusion outputs and the uniqueness of segmentation results. For image synthesis, we propose a finite perturbation approach to enhance the diversity of generated results without changing the semantic categories. Experiments show that our SemFlow achieves competitive results on semantic segmentation and semantic image synthesis tasks. We hope this simple framework will motivate people to rethink the unification of low-level and high-level vision. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_20282 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow Wang, Chaoyang Li, Xiangtai Qi, Lu Ding, Henghui Tong, Yunhai Yang, Ming-Hsuan Computer Vision and Pattern Recognition Semantic segmentation and semantic image synthesis are two representative tasks in visual perception and generation. While existing methods consider them as two distinct tasks, we propose a unified framework (SemFlow) and model them as a pair of reverse problems. Specifically, motivated by rectified flow theory, we train an ordinary differential equation (ODE) model to transport between the distributions of real images and semantic masks. As the training object is symmetric, samples belonging to the two distributions, images and semantic masks, can be effortlessly transferred reversibly. For semantic segmentation, our approach solves the contradiction between the randomness of diffusion outputs and the uniqueness of segmentation results. For image synthesis, we propose a finite perturbation approach to enhance the diversity of generated results without changing the semantic categories. Experiments show that our SemFlow achieves competitive results on semantic segmentation and semantic image synthesis tasks. We hope this simple framework will motivate people to rethink the unification of low-level and high-level vision. |
| title | SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.20282 |