Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917538461384704 |
|---|---|
| author | Li, Niantong Hu, Guangzheng Qiao, Weixu Ba, Ying Hong, Qichen Shen, Shijun Wang, Jinlin Zhou, Fan Kang, Jianye Shang, Xin He, Ziyi Wang, Wei Li, Dalin Li, Jiahao Zhang, Jie Gao, Kaiyuan Yan, Kun Jiang, Lihan Tang, Ningyuan Yin, Shengming Wu, Tianhe Xu, Xiao Chen, Xiaoyue Chen, Yuxiang Shu, Yan Zhang, Yanran Chen, Yilei Xu, Yixian Zhang, Zekai Wang, Zhendong Liu, Zihao Zhou, Zikai Shi, Hongzhu Wang, Yi Zhao, Bing Wei, Hu Qu, Lin Wu, Chenfei |
| author_facet | Li, Niantong Hu, Guangzheng Qiao, Weixu Ba, Ying Hong, Qichen Shen, Shijun Wang, Jinlin Zhou, Fan Kang, Jianye Shang, Xin He, Ziyi Wang, Wei Li, Dalin Li, Jiahao Zhang, Jie Gao, Kaiyuan Yan, Kun Jiang, Lihan Tang, Ningyuan Yin, Shengming Wu, Tianhe Xu, Xiao Chen, Xiaoyue Chen, Yuxiang Shu, Yan Zhang, Yanran Chen, Yilei Xu, Yixian Zhang, Zekai Wang, Zhendong Liu, Zihao Zhou, Zikai Shi, Hongzhu Wang, Yi Zhao, Bing Wei, Hu Qu, Lin Wu, Chenfei |
| contents | Text-to-Image generation has evolved from basic image synthesis into a frequently used core capability in professional creative workflows, where simple text-image alignment can no longer satisfy users' pressing demands for faithful real-world reconstruction and genuine creative expression. Existing benchmarks, however, remain anchored in these foundational criteria and do not yet capture the nuanced capabilities that matter in authentic artistic practice, making it difficult to reliably distinguish state-of-the-art T2I models. To address the gap, we introduce Qwen-Image-Bench, a creator-centric benchmark co-designed with professional artists and grounded in real-world creation scenarios. Qwen-Image-Bench enriches conventional evaluation with two application-driven dimensions: Real-world Fidelity and Creative Generation. Drawing on the staged reasoning inherent in professional artistic workflows, we organize these five pillars into a top-down hierarchical taxonomy that further decomposes into 23 second-level sub-capabilities and 56 third-level verifiable rubrics. To ensure broad coverage, we curate 1000 stratified prompts with each prompt jointly exercising more than four fine-grained facets across multiple pillars. We train a unified judge model Q-Judger based on Qwen3.6-27B, supervised by 80 professional annotators from global art academies under blind labeling and triple-review protocols, that scores every image across all 56 verifiable facets, producing fine-grained, rubric-grounded, and fully attributable diagnostics rather than a single opaque score. Empirically, Qwen-Image-Bench reliably distinguishes leading T2I models, achieving the greatest separation on the two application-driven dimensions of Real-world Fidelity and Creative Generation where existing benchmarks provide little insight, while also providing a trustworthy optimization signal for production-level T2I development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_28091 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation Li, Niantong Hu, Guangzheng Qiao, Weixu Ba, Ying Hong, Qichen Shen, Shijun Wang, Jinlin Zhou, Fan Kang, Jianye Shang, Xin He, Ziyi Wang, Wei Li, Dalin Li, Jiahao Zhang, Jie Gao, Kaiyuan Yan, Kun Jiang, Lihan Tang, Ningyuan Yin, Shengming Wu, Tianhe Xu, Xiao Chen, Xiaoyue Chen, Yuxiang Shu, Yan Zhang, Yanran Chen, Yilei Xu, Yixian Zhang, Zekai Wang, Zhendong Liu, Zihao Zhou, Zikai Shi, Hongzhu Wang, Yi Zhao, Bing Wei, Hu Qu, Lin Wu, Chenfei Computer Vision and Pattern Recognition Text-to-Image generation has evolved from basic image synthesis into a frequently used core capability in professional creative workflows, where simple text-image alignment can no longer satisfy users' pressing demands for faithful real-world reconstruction and genuine creative expression. Existing benchmarks, however, remain anchored in these foundational criteria and do not yet capture the nuanced capabilities that matter in authentic artistic practice, making it difficult to reliably distinguish state-of-the-art T2I models. To address the gap, we introduce Qwen-Image-Bench, a creator-centric benchmark co-designed with professional artists and grounded in real-world creation scenarios. Qwen-Image-Bench enriches conventional evaluation with two application-driven dimensions: Real-world Fidelity and Creative Generation. Drawing on the staged reasoning inherent in professional artistic workflows, we organize these five pillars into a top-down hierarchical taxonomy that further decomposes into 23 second-level sub-capabilities and 56 third-level verifiable rubrics. To ensure broad coverage, we curate 1000 stratified prompts with each prompt jointly exercising more than four fine-grained facets across multiple pillars. We train a unified judge model Q-Judger based on Qwen3.6-27B, supervised by 80 professional annotators from global art academies under blind labeling and triple-review protocols, that scores every image across all 56 verifiable facets, producing fine-grained, rubric-grounded, and fully attributable diagnostics rather than a single opaque score. Empirically, Qwen-Image-Bench reliably distinguishes leading T2I models, achieving the greatest separation on the two application-driven dimensions of Real-world Fidelity and Creative Generation where existing benchmarks provide little insight, while also providing a trustworthy optimization signal for production-level T2I development. |
| title | Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.28091 |