Self-Prompting Diffusion Transformer for Open-Vocabulary Scene Text Editing via In-Context Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910263042637824 |
|---|---|
| author | Li, Hongxi Wang, Tong Wu, Chengjing Liu, Tianbao Yao, Jiangtao Qu, Xiaochao Wu, Xinxiao Liu, Luoqi Liu, Ting |
| author_facet | Li, Hongxi Wang, Tong Wu, Chengjing Liu, Tianbao Yao, Jiangtao Qu, Xiaochao Wu, Xinxiao Liu, Luoqi Liu, Ting |
| contents | Scene text editing aims to modify text in a target region of an image while preserving surrounding background style and texture. Existing methods rely solely on image background information while neglecting the visual details of target regions, which discards stylistic features in the original text and essentially degrades the task to text rendering. Moreover, the conditions imposed by pre-trained glyph encoder limit the scope of editable text. To address these issues, this paper proposes a self-prompting scene text editing method that constructs style and glyph prompts directly from the original image, without introducing additional style or glyph encoders. We employ a two-stage training strategy: the diffusion transformer is first trained on large-scale self-supervised data and then refined using a small set of paired images. By leveraging the in-context learning capability of the Multi-Modal Diffusion Transformer (MM-DiT), it achieves open-vocabulary and style-consistent text editing. Experimental results on various languages demonstrate that our method achieves the state-of-the-art performance in both text accuracy and style consistency. Our project page: hongxiii.github.io/mstedit. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_15523 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Self-Prompting Diffusion Transformer for Open-Vocabulary Scene Text Editing via In-Context Learning Li, Hongxi Wang, Tong Wu, Chengjing Liu, Tianbao Yao, Jiangtao Qu, Xiaochao Wu, Xinxiao Liu, Luoqi Liu, Ting Computer Vision and Pattern Recognition Scene text editing aims to modify text in a target region of an image while preserving surrounding background style and texture. Existing methods rely solely on image background information while neglecting the visual details of target regions, which discards stylistic features in the original text and essentially degrades the task to text rendering. Moreover, the conditions imposed by pre-trained glyph encoder limit the scope of editable text. To address these issues, this paper proposes a self-prompting scene text editing method that constructs style and glyph prompts directly from the original image, without introducing additional style or glyph encoders. We employ a two-stage training strategy: the diffusion transformer is first trained on large-scale self-supervised data and then refined using a small set of paired images. By leveraging the in-context learning capability of the Multi-Modal Diffusion Transformer (MM-DiT), it achieves open-vocabulary and style-consistent text editing. Experimental results on various languages demonstrate that our method achieves the state-of-the-art performance in both text accuracy and style consistency. Our project page: hongxiii.github.io/mstedit. |
| title | Self-Prompting Diffusion Transformer for Open-Vocabulary Scene Text Editing via In-Context Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.15523 |