STRICT: Stress Test of Rendering Images Containing Text
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915493209702400 |
|---|---|
| author | Zhang, Tianyu Wang, Xinyu Li, Lu Tai, Zhenghan Chi, Jijun Tian, Jingrui He, Hailin Wang, Suyuchen |
| author_facet | Zhang, Tianyu Wang, Xinyu Li, Lu Tai, Zhenghan Chi, Jijun Tian, Jingrui He, Hailin Wang, Suyuchen |
| contents | While diffusion models have revolutionized text-to-image generation with their ability to synthesize realistic and diverse scenes, they continue to struggle to generate consistent and legible text within images. This shortcoming is commonly attributed to the locality bias inherent in diffusion-based generation, which limits their ability to model long-range spatial dependencies. In this paper, we introduce $\textbf{STRICT}$, a benchmark designed to systematically stress-test the ability of diffusion models to render coherent and instruction-aligned text in images. Our benchmark evaluates models across multiple dimensions: (1) the maximum length of readable text that can be generated; (2) the correctness and legibility of the generated text, and (3) the ratio of not following instructions for generating text. We evaluate several state-of-the-art models, including proprietary and open-source variants, and reveal persistent limitations in long-range consistency and instruction-following capabilities. Our findings provide insights into architectural bottlenecks and motivate future research directions in multimodal generative modeling. We release our entire evaluation pipeline at https://github.com/tianyu-z/STRICT-Bench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18985 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | STRICT: Stress Test of Rendering Images Containing Text Zhang, Tianyu Wang, Xinyu Li, Lu Tai, Zhenghan Chi, Jijun Tian, Jingrui He, Hailin Wang, Suyuchen Machine Learning Computation and Language Computer Vision and Pattern Recognition 68T50 I.2.7; I.4.0 While diffusion models have revolutionized text-to-image generation with their ability to synthesize realistic and diverse scenes, they continue to struggle to generate consistent and legible text within images. This shortcoming is commonly attributed to the locality bias inherent in diffusion-based generation, which limits their ability to model long-range spatial dependencies. In this paper, we introduce $\textbf{STRICT}$, a benchmark designed to systematically stress-test the ability of diffusion models to render coherent and instruction-aligned text in images. Our benchmark evaluates models across multiple dimensions: (1) the maximum length of readable text that can be generated; (2) the correctness and legibility of the generated text, and (3) the ratio of not following instructions for generating text. We evaluate several state-of-the-art models, including proprietary and open-source variants, and reveal persistent limitations in long-range consistency and instruction-following capabilities. Our findings provide insights into architectural bottlenecks and motivate future research directions in multimodal generative modeling. We release our entire evaluation pipeline at https://github.com/tianyu-z/STRICT-Bench. |
| title | STRICT: Stress Test of Rendering Images Containing Text |
| topic | Machine Learning Computation and Language Computer Vision and Pattern Recognition 68T50 I.2.7; I.4.0 |
| url | https://arxiv.org/abs/2505.18985 |