Image de couverture de livre

MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yue, Xiang, Zheng, Tianyu, Ni, Yuansheng, Wang, Yubo, Zhang, Kai, Tong, Shengbang, Sun, Yuxuan, Yu, Botao, Zhang, Ge, Sun, Huan, Su, Yu, Chen, Wenhu, Neubig, Graham
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Computer Vision and Pattern Recognition
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
par: Yue, Xiang, et autres
Publié: (2023)

Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
par: Zou, Kai, et autres
Publié: (2025)

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
par: Wang, Yubo, et autres
Publié: (2024)

JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation
par: Onohara, Shota, et autres
Publié: (2024)

CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark
par: Zhang, Ge, et autres
Publié: (2024)

JMMMU-Pro: Image-based Japanese Multi-discipline Multimodal Understanding Benchmark via Vibe Benchmark Construction
par: Miyai, Atsuyuki, et autres
Publié: (2025)

MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing
par: Zhang, Kai, et autres
Publié: (2023)

Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate
par: Wang, Yubo, et autres
Publié: (2025)

SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding
par: Cai, Songcheng, et autres
Publié: (2026)

MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale
par: Guo, Jarvis, et autres
Publié: (2024)

VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation
par: Ma, Wentao, et autres
Publié: (2025)

Harnessing Webpage UIs for Text-Rich Visual Understanding
par: Liu, Junpeng, et autres
Publié: (2024)

EcomMMMU: Strategic Utilization of Visuals for Robust Multimodal E-commerce Models
par: Ling, Xinyi, et autres
Publié: (2025)

Mass-Producing Failures of Multimodal Systems with Language Models
par: Tong, Shengbang, et autres
Publié: (2023)

Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformers
par: Ren, Weiming, et autres
Publié: (2025)

VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation
par: Ni, Yuansheng, et autres
Publié: (2025)

Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos
par: Hu, Kairui, et autres
Publié: (2025)

GenAI Arena: An Open Evaluation Platform for Generative Models
par: Jiang, Dongfu, et autres
Publié: (2024)

Demystifying Long Chain-of-Thought Reasoning in LLMs
par: Yeo, Edward, et autres
Publié: (2025)

II-Bench: An Image Implication Understanding Benchmark for Multimodal Large Language Models
par: Liu, Ziqiang, et autres
Publié: (2024)

MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation
par: Huang, Jinsheng, et autres
Publié: (2024)

ZINA: Multimodal Fine-grained Hallucination Detection and Editing
par: Wada, Yuiga, et autres
Publié: (2025)

LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset
par: Yu, Botao, et autres
Publié: (2024)

On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
par: Zhang, Charlie, et autres
Publié: (2025)

Asymmetric Idiosyncrasies in Multimodal Models
par: Tao, Muzi, et autres
Publié: (2026)

TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understanding
par: Ku, Max, et autres
Publié: (2025)

UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents
par: Ji, Yifan, et autres
Publié: (2026)

Go-Browse: Training Web Agents with Structured Exploration
par: Gandhi, Apurva, et autres
Publié: (2025)

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models
par: Tjuatja, Lindia, et autres
Publié: (2025)

OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI
par: Huang, Zhen, et autres
Publié: (2024)

Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
par: Chen, Xiaokang, et autres
Publié: (2025)

Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering
par: Wang, Yubo, et autres
Publié: (2023)

LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts
par: Xiao, Yijia, et autres
Publié: (2024)

ProLex: A Benchmark for Language Proficiency-oriented Lexical Substitution
par: Zhang, Xuanming, et autres
Publié: (2024)

Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains
par: Tang, Zitian, et autres
Publié: (2023)

Effective Strategies for Asynchronous Software Engineering Agents
par: Geng, Jiayi, et autres
Publié: (2026)

From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models
par: Fang, Irving, et autres
Publié: (2025)

MetaMorph: Multimodal Understanding and Generation via Instruction Tuning
par: Tong, Shengbang, et autres
Publié: (2024)

PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathology
par: Sun, Yuxuan, et autres
Publié: (2024)

Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Following
par: Xiong, Tianyi, et autres
Publié: (2025)