$\infty$-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913439181438976 |
|---|---|
| author | Le, Minh-Quan Graikos, Alexandros Yellapragada, Srikar Gupta, Rajarsi Saltz, Joel Samaras, Dimitris |
| author_facet | Le, Minh-Quan Graikos, Alexandros Yellapragada, Srikar Gupta, Rajarsi Saltz, Joel Samaras, Dimitris |
| contents | Synthesizing high-resolution images from intricate, domain-specific information remains a significant challenge in generative modeling, particularly for applications in large-image domains such as digital histopathology and remote sensing. Existing methods face critical limitations: conditional diffusion models in pixel or latent space cannot exceed the resolution on which they were trained without losing fidelity, and computational demands increase significantly for larger image sizes. Patch-based methods offer computational efficiency but fail to capture long-range spatial relationships due to their overreliance on local information. In this paper, we introduce a novel conditional diffusion model in infinite dimensions, $\infty$-Brush for controllable large image synthesis. We propose a cross-attention neural operator to enable conditioning in function space. Our model overcomes the constraints of traditional finite-dimensional diffusion models and patch-based methods, offering scalability and superior capability in preserving global image structures while maintaining fine details. To our best knowledge, $\infty$-Brush is the first conditional diffusion model in function space, that can controllably synthesize images at arbitrary resolutions of up to $4096\times4096$ pixels. The code is available at https://github.com/cvlab-stonybrook/infinity-brush. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_14709 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | $\infty$-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions Le, Minh-Quan Graikos, Alexandros Yellapragada, Srikar Gupta, Rajarsi Saltz, Joel Samaras, Dimitris Computer Vision and Pattern Recognition Synthesizing high-resolution images from intricate, domain-specific information remains a significant challenge in generative modeling, particularly for applications in large-image domains such as digital histopathology and remote sensing. Existing methods face critical limitations: conditional diffusion models in pixel or latent space cannot exceed the resolution on which they were trained without losing fidelity, and computational demands increase significantly for larger image sizes. Patch-based methods offer computational efficiency but fail to capture long-range spatial relationships due to their overreliance on local information. In this paper, we introduce a novel conditional diffusion model in infinite dimensions, $\infty$-Brush for controllable large image synthesis. We propose a cross-attention neural operator to enable conditioning in function space. Our model overcomes the constraints of traditional finite-dimensional diffusion models and patch-based methods, offering scalability and superior capability in preserving global image structures while maintaining fine details. To our best knowledge, $\infty$-Brush is the first conditional diffusion model in function space, that can controllably synthesize images at arbitrary resolutions of up to $4096\times4096$ pixels. The code is available at https://github.com/cvlab-stonybrook/infinity-brush. |
| title | $\infty$-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.14709 |