Cover Image

MMMG: a Comprehensive and Reliable Evaluation Suite for Multitask Multimodal Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yao, Jihan, Hu, Yushi, Yi, Yujie, Han, Bin, Feng, Shangbin, Yang, Guang, Wen, Bingbing, Krishna, Ranjay, Wang, Lucy Lu, Tsvetkov, Yulia, Smith, Noah A., Zhu, Banghua
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Know Your Limits: A Survey of Abstention in Large Language Models
by: Wen, Bingbing, et al.
Published: (2024)

Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only
by: Yao, Jihan, et al.
Published: (2024)

Reading Between the Lines: Abstaining from VLM-Generated OCR Errors via Latent Representation Probes
by: Yao, Jihan, et al.
Published: (2025)

Data Swarms: Optimizable Generation of Synthetic Evaluation Data
by: Feng, Shangbin, et al.
Published: (2025)

ScienceMeter: Tracking Scientific Knowledge Updates in Language Models
by: Wang, Yike, et al.
Published: (2025)

The Single-Multi Evolution Loop for Self-Improving Model Collaboration Systems
by: Feng, Shangbin, et al.
Published: (2026)

Among Us: Measuring and Mitigating Malicious Contributions in Model Collaboration Systems
by: Yang, Ziyuan, et al.
Published: (2026)

GuessBench: Sensemaking Multimodal Creativity in the Wild
by: Zhu, Zifeng, et al.
Published: (2025)

FACTS&EVIDENCE: An Interactive Tool for Transparent Fine-Grained Factual Verification of Machine-Generated Text
by: Boonsanong, Varich, et al.
Published: (2025)

Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models
by: Hu, Yushi, et al.
Published: (2024)

SPARTA ALIGNMENT: Collectively Aligning Multiple Language Models through Combat
by: Jiang, Yuru, et al.
Published: (2025)

MentorCollab: Selective Large-to-Small Inference-Time Guidance for Efficient Reasoning
by: Wang, Haojin, et al.
Published: (2026)

KGQuiz: Evaluating the Generalization of Encoded Knowledge in Large Language Models
by: Bai, Yuyang, et al.
Published: (2023)

MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
by: Li, Shuyue Stella, et al.
Published: (2024)

Escaping the SpuriVerse: Can Large Vision-Language Models Generalize Beyond Seen Spurious Correlations?
by: Yang, Yiwei, et al.
Published: (2025)

DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection
by: Wan, Herun, et al.
Published: (2024)

What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection
by: Feng, Shangbin, et al.
Published: (2024)

Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
by: Feng, Shangbin, et al.
Published: (2024)

Don't Throw Away Your Pretrained Model
by: Feng, Shangbin, et al.
Published: (2025)

Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models
by: Feng, Shangbin, et al.
Published: (2023)

Small Reward Models via Backward Inference
by: Wang, Yike, et al.
Published: (2026)

BLINK: Multimodal Large Language Models Can See but Not Perceive
by: Fu, Xingyu, et al.
Published: (2024)

Can Language Models Solve Graph Problems in Natural Language?
by: Wang, Heng, et al.
Published: (2023)

Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation
by: Cho, Jaemin, et al.
Published: (2023)

Finding Flawed Fictions: Evaluating Complex Reasoning in Language Models via Plot Hole Detection
by: Ahuja, Kabir, et al.
Published: (2025)

ComPO: Community Preferences for Language Model Personalization
by: Kumar, Sachin, et al.
Published: (2024)

Resolving Knowledge Conflicts in Large Language Models
by: Wang, Yike, et al.
Published: (2023)

Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment
by: Zhang, Yizhuo, et al.
Published: (2025)

Tuning Language Models by Proxy
by: Liu, Alisa, et al.
Published: (2024)

Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration
by: Feng, Shangbin, et al.
Published: (2024)

Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models
by: Ding, Wenxuan, et al.
Published: (2023)

Eval3D: Interpretable and Fine-grained Evaluation for 3D Generation
by: Duggal, Shivam, et al.
Published: (2025)

BASS: Benchmarking Audio LMs for Musical Structure and Semantic Reasoning
by: Jang, Min, et al.
Published: (2026)

MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoning
by: Luo, Yuxuan, et al.
Published: (2025)

Can LLM Graph Reasoning Generalize beyond Pattern Memorization?
by: Zhang, Yizhuo, et al.
Published: (2024)

GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluation
by: Kamath, Amita, et al.
Published: (2025)

P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models
by: Liu, Yuhan, et al.
Published: (2023)

MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization
by: Ahia, Orevaoghene, et al.
Published: (2024)

Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks
by: Wang, Yichen, et al.
Published: (2024)

Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformers
by: Ahuja, Kabir, et al.
Published: (2024)