Unified Editing of Panorama, 3D Scenes, and Videos Through Disentangled Self-Attention Injection
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909211197177856 |
|---|---|
| author | Kwon, Gihyun Park, Jangho Ye, Jong Chul |
| author_facet | Kwon, Gihyun Park, Jangho Ye, Jong Chul |
| contents | While text-to-image models have achieved impressive capabilities in image generation and editing, their application across various modalities often necessitates training separate models. Inspired by existing method of single image editing with self attention injection and video editing with shared attention, we propose a novel unified editing framework that combines the strengths of both approaches by utilizing only a basic 2D image text-to-image (T2I) diffusion model. Specifically, we design a sampling method that facilitates editing consecutive images while maintaining semantic consistency utilizing shared self-attention features during both reference and consecutive image sampling processes. Experimental results confirm that our method enables editing across diverse modalities including 3D scenes, videos, and panorama images. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16823 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unified Editing of Panorama, 3D Scenes, and Videos Through Disentangled Self-Attention Injection Kwon, Gihyun Park, Jangho Ye, Jong Chul Computer Vision and Pattern Recognition Artificial Intelligence While text-to-image models have achieved impressive capabilities in image generation and editing, their application across various modalities often necessitates training separate models. Inspired by existing method of single image editing with self attention injection and video editing with shared attention, we propose a novel unified editing framework that combines the strengths of both approaches by utilizing only a basic 2D image text-to-image (T2I) diffusion model. Specifically, we design a sampling method that facilitates editing consecutive images while maintaining semantic consistency utilizing shared self-attention features during both reference and consecutive image sampling processes. Experimental results confirm that our method enables editing across diverse modalities including 3D scenes, videos, and panorama images. |
| title | Unified Editing of Panorama, 3D Scenes, and Videos Through Disentangled Self-Attention Injection |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2405.16823 |