SyncTweedies: A General Generative Framework Based on Synchronized Diffusions
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909374889328640 |
|---|---|
| author | Kim, Jaihoon Koo, Juil Yeo, Kyeongmin Sung, Minhyuk |
| author_facet | Kim, Jaihoon Koo, Juil Yeo, Kyeongmin Sung, Minhyuk |
| contents | We introduce a general framework for generating diverse visual content, including ambiguous images, panorama images, mesh textures, and Gaussian splat textures, by synchronizing multiple diffusion processes. We present exhaustive investigation into all possible scenarios for synchronizing multiple diffusion processes through a canonical space and analyze their characteristics across applications. In doing so, we reveal a previously unexplored case: averaging the outputs of Tweedie's formula while conducting denoising in multiple instance spaces. This case also provides the best quality with the widest applicability to downstream tasks. We name this case SyncTweedies. In our experiments generating visual content aforementioned, we demonstrate the superior quality of generation by SyncTweedies compared to other synchronization methods, optimization-based and iterative-update-based methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14370 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SyncTweedies: A General Generative Framework Based on Synchronized Diffusions Kim, Jaihoon Koo, Juil Yeo, Kyeongmin Sung, Minhyuk Computer Vision and Pattern Recognition We introduce a general framework for generating diverse visual content, including ambiguous images, panorama images, mesh textures, and Gaussian splat textures, by synchronizing multiple diffusion processes. We present exhaustive investigation into all possible scenarios for synchronizing multiple diffusion processes through a canonical space and analyze their characteristics across applications. In doing so, we reveal a previously unexplored case: averaging the outputs of Tweedie's formula while conducting denoising in multiple instance spaces. This case also provides the best quality with the widest applicability to downstream tasks. We name this case SyncTweedies. In our experiments generating visual content aforementioned, we demonstrate the superior quality of generation by SyncTweedies compared to other synchronization methods, optimization-based and iterative-update-based methods. |
| title | SyncTweedies: A General Generative Framework Based on Synchronized Diffusions |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.14370 |