Cover Image

GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kamath, Amita, Chang, Kai-Wei, Krishna, Ranjay, Zettlemoyer, Luke, Hu, Yushi, Ghazvininejad, Marjan
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image
by: Hu, Yushi, et al.
Published: (2025)

Unified Text-Image Generation with Weakness-Targeted Post-Training
by: Chen, Jiahui, et al.
Published: (2026)

Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models
by: Yasunaga, Michihiro, et al.
Published: (2025)

The Hard Positive Truth about Vision-Language Compositionality
by: Kamath, Amita, et al.
Published: (2024)

Self-Improving VLM Judges Without Human Annotations
by: Lin, Inna Wanyin, et al.
Published: (2025)

Scale Can't Overcome Pragmatics: The Impact of Reporting Bias on Vision-Language Reasoning
by: Kamath, Amita, et al.
Published: (2026)

Eval3D: Interpretable and Fine-grained Evaluation for 3D Generation
by: Duggal, Shivam, et al.
Published: (2025)

Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models
by: Hu, Yushi, et al.
Published: (2024)

Visual Representations inside the Language Model
by: Liu, Benlin, et al.
Published: (2025)

JPEG-LM: LLMs as Image Generators with Canonical Codec Representations
by: Han, Xiaochuang, et al.
Published: (2024)

GenColorBench: A Color Evaluation Benchmark for Text-to-Image Generation Models
by: Butt, Muhammad Atif, et al.
Published: (2025)

Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation
by: Cho, Jaemin, et al.
Published: (2023)

Computational Tradeoffs in Image Synthesis: Diffusion, Masked-Token, and Next-Token Prediction
by: Kilian, Maciej, et al.
Published: (2024)

VideoGen-Eval: Agent-based System for Video Generation Evaluation
by: Yang, Yuhang, et al.
Published: (2025)

UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generation
by: Wang, Yibin, et al.
Published: (2025)

Exposing and Addressing Cross-Task Inconsistency in Unified Vision-Language Models
by: Maharana, Adyasha, et al.
Published: (2023)

m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks
by: Ma, Zixian, et al.
Published: (2024)

BizGenEval: A Systematic Benchmark for Commercial Visual Content Generation
by: Li, Yan, et al.
Published: (2026)

Matryoshka Query Transformer for Large Vision-Language Models
by: Hu, Wenbo, et al.
Published: (2024)

Negative Token Merging: Image-based Adversarial Feature Guidance
by: Singh, Jaskirat, et al.
Published: (2024)

SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation
by: Ma, Yingzi, et al.
Published: (2026)

Ablate-to-Validate: Are Vision-Language Models Really Using Continuous Thought Tokens?
by: Zhang, Tianyi, et al.
Published: (2026)

T2I-FineEval: Fine-Grained Compositional Metric for Text-to-Image Evaluation
by: Hosseini, Seyed Mohammad Hadi, et al.
Published: (2025)

DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models
by: Wang, Juntong, et al.
Published: (2026)

CompAlign: Improving Compositional Text-to-Image Generation with a Complex Benchmark and Fine-Grained Feedback
by: Wan, Yixin, et al.
Published: (2025)

DynamicEval: Rethinking Evaluation for Dynamic Text-to-Video Synthesis
by: Babu, Nithin C., et al.
Published: (2025)

Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models
by: Hu, Yushi, et al.
Published: (2023)

EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
by: Liu, Yaofang, et al.
Published: (2023)

OmniEval: A Benchmark for Evaluating Omni-modal Models with Visual, Auditory, and Textual Inputs
by: Zhang, Yiman, et al.
Published: (2025)

WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation
by: Zhang, Daoan, et al.
Published: (2025)

RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation
by: Wang, Yi Ru, et al.
Published: (2025)

LegalEval-Q: A New Benchmark for The Quality Evaluation of LLM-Generated Legal Text
by: yunhan, Li, et al.
Published: (2025)

Q-Eval-100K: Evaluating Visual Quality and Alignment Level for Text-to-Vision Content
by: Zhang, Zicheng, et al.
Published: (2025)

The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better
by: Geng, Scott, et al.
Published: (2024)

CharGen: High Accurate Character-Level Visual Text Generation Model with MultiModal Encoder
by: Ma, Lichen, et al.
Published: (2024)

Iterated Learning Improves Compositionality in Large Vision-Language Models
by: Zheng, Chenhao, et al.
Published: (2024)

The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects
by: Wan, Yixin, et al.
Published: (2024)

TabletopGen: Instance-Level Interactive 3D Tabletop Scene Generation from Text or Single Image
by: Wang, Ziqian, et al.
Published: (2025)

CAT: Content-Adaptive Image Tokenization
by: Shen, Junhong, et al.
Published: (2025)

SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesis
by: Tam, Hou In Ivan, et al.
Published: (2025)