Prompt Reinjection: Alleviating Prompt Forgetting in Multimodal Diffusion Transformers
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913146989445120 |
|---|---|
| author | Yao, Yuxuan Chen, Yuxuan Li, Hui Cheng, Kaihui Guo, Qipeng Sun, Yuwei Dong, Zilong Wang, Jingdong Zhu, Siyu |
| author_facet | Yao, Yuxuan Chen, Yuxuan Li, Hui Cheng, Kaihui Guo, Qipeng Sun, Yuwei Dong, Zilong Wang, Jingdong Zhu, Siyu |
| contents | Multimodal Diffusion Transformers (MMDiTs) for text-to-image generation maintain separate text and image branches, with bidirectional information flow between text tokens and visual latents throughout denoising. In this setting, we observe a prompt forgetting phenomenon: the semantics of the prompt representation in the text branch is progressively forgotten as depth increases. We further verify this effect on three representative MMDiTs--SD3, SD3.5, and FLUX.1 by probing linguistic attributes of the representations over the layers in the text branch. Motivated by these findings, we introduce a training-free approach, prompt reinjection, which reinjects prompt representations from early layers into later layers to alleviate this forgetting. Experiments on GenEval, DPG, and T2I-CompBench++ show consistent gains in instruction-following capability, along with improvements on metrics capturing preference, aesthetics, and overall text--image generation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_06886 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Prompt Reinjection: Alleviating Prompt Forgetting in Multimodal Diffusion Transformers Yao, Yuxuan Chen, Yuxuan Li, Hui Cheng, Kaihui Guo, Qipeng Sun, Yuwei Dong, Zilong Wang, Jingdong Zhu, Siyu Computer Vision and Pattern Recognition Multimodal Diffusion Transformers (MMDiTs) for text-to-image generation maintain separate text and image branches, with bidirectional information flow between text tokens and visual latents throughout denoising. In this setting, we observe a prompt forgetting phenomenon: the semantics of the prompt representation in the text branch is progressively forgotten as depth increases. We further verify this effect on three representative MMDiTs--SD3, SD3.5, and FLUX.1 by probing linguistic attributes of the representations over the layers in the text branch. Motivated by these findings, we introduce a training-free approach, prompt reinjection, which reinjects prompt representations from early layers into later layers to alleviate this forgetting. Experiments on GenEval, DPG, and T2I-CompBench++ show consistent gains in instruction-following capability, along with improvements on metrics capturing preference, aesthetics, and overall text--image generation quality. |
| title | Prompt Reinjection: Alleviating Prompt Forgetting in Multimodal Diffusion Transformers |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.06886 |