Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915582148870144 |
|---|---|
| author | Na, Byeonghu Park, Minsang Sim, Gyuwon Shin, Donghyeok Bae, HeeSun Kang, Mina Kwon, Se Jung Kang, Wanmo Moon, Il-Chul |
| author_facet | Na, Byeonghu Park, Minsang Sim, Gyuwon Shin, Donghyeok Bae, HeeSun Kang, Mina Kwon, Se Jung Kang, Wanmo Moon, Il-Chul |
| contents | Text-to-image diffusion models rely on text embeddings from a pre-trained text encoder, but these embeddings remain fixed across all diffusion timesteps, limiting their adaptability to the generative process. We propose Diffusion Adaptive Text Embedding (DATE), which dynamically updates text embeddings at each diffusion timestep based on intermediate perturbed data. We formulate an optimization problem and derive an update rule that refines the text embeddings at each sampling step to improve alignment and preference between the mean predicted image and the text. This allows DATE to dynamically adapts the text conditions to the reverse-diffused images throughout diffusion sampling without requiring additional model training. Through theoretical analysis and empirical results, we show that DATE maintains the generative capability of the model while providing superior text-image alignment over fixed text embeddings across various tasks, including multi-concept generation and text-guided image editing. Our code is available at https://github.com/aailab-kaist/DATE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_23974 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models Na, Byeonghu Park, Minsang Sim, Gyuwon Shin, Donghyeok Bae, HeeSun Kang, Mina Kwon, Se Jung Kang, Wanmo Moon, Il-Chul Machine Learning Artificial Intelligence Text-to-image diffusion models rely on text embeddings from a pre-trained text encoder, but these embeddings remain fixed across all diffusion timesteps, limiting their adaptability to the generative process. We propose Diffusion Adaptive Text Embedding (DATE), which dynamically updates text embeddings at each diffusion timestep based on intermediate perturbed data. We formulate an optimization problem and derive an update rule that refines the text embeddings at each sampling step to improve alignment and preference between the mean predicted image and the text. This allows DATE to dynamically adapts the text conditions to the reverse-diffused images throughout diffusion sampling without requiring additional model training. Through theoretical analysis and empirical results, we show that DATE maintains the generative capability of the model while providing superior text-image alignment over fixed text embeddings across various tasks, including multi-concept generation and text-guided image editing. Our code is available at https://github.com/aailab-kaist/DATE. |
| title | Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2510.23974 |