Imagen de Portada

MMLU-Reason: Benchmarking Multi-Task Multi-modal Language Understanding and Reasoning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Tie, Guiyao, Zhou, Xueyang, Gu, Tianhe, Zhang, Ruihang, Hu, Chaoran, Zhang, Sizhe, Sun, Mengqu, Zhang, Yan, Zhou, Pan, Sun, Lichao
Formato:	Preprint
Publicado:	2025
Materias:	Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs
por: Tie, Guiyao, et al.
Publicado: (2025)

A Survey of AI Scientists
por: Tie, Guiyao, et al.
Publicado: (2025)

BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization
por: Zhou, Xueyang, et al.
Publicado: (2025)

Exploring the Necessity of Reasoning in LLM-based Agent Scenarios
por: Zhou, Xueyang, et al.
Publicado: (2025)

BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoning
por: Tie, Guiyao, et al.
Publicado: (2026)

MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark
por: Zhao, Qihao, et al.
Publicado: (2024)

LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization
por: Zhou, Xueyang, et al.
Publicado: (2025)

EmbodiedClaw: Conversational Workflow Execution for Embodied AI Development
por: Zhou, Xueyang, et al.
Publicado: (2026)

IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding
por: KJ, Sankalp, et al.
Publicado: (2025)

CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoning
por: He, Zheqi, et al.
Publicado: (2024)

MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark
por: Xing, Junjie, et al.
Publicado: (2025)

SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator
por: Zhou, Xueyang, et al.
Publicado: (2025)

Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models
por: Jin, Yilun, et al.
Publicado: (2024)

Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
por: Song, Enxin, et al.
Publicado: (2025)

Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning
por: Zhao, Bingchen, et al.
Publicado: (2024)

CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs
por: Manh, Dung Nguyen, et al.
Publicado: (2024)

M$^3$-Med: A Benchmark for Multi-lingual, Multi-modal, and Multi-hop Reasoning in Medical Instructional Video Understanding
por: Liu, Shenxi, et al.
Publicado: (2025)

Mobile-MMLU: A Mobile Intelligence Language Understanding Benchmark
por: Bsharat, Sondos Mahmoud, et al.
Publicado: (2025)

MMLU-SR: A Benchmark for Stress-Testing Reasoning Capability of Large Language Models
por: Wang, Wentian, et al.
Publicado: (2024)

METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark
por: Yang, Xu, et al.
Publicado: (2025)

TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models
por: Yu, Fangxu, et al.
Publicado: (2026)

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
por: Wang, Yubo, et al.
Publicado: (2024)

Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism
por: Wang, Zhiwei, et al.
Publicado: (2024)

PointCoT: A Multi-modal Benchmark for Explicit 3D Geometric Reasoning
por: Zhang, Dongxu, et al.
Publicado: (2026)

MIRAGE: A Multi-modal Benchmark for Spatial Perception, Reasoning, and Intelligence
por: Liu, Chonghan, et al.
Publicado: (2025)

Hierarchical Attacks for Multi-Modal Multi-Agent Reasoning
por: Zhou, Hao, et al.
Publicado: (2026)

Kinship Data Benchmark for Multi-hop Reasoning
por: Sun, Tianda, et al.
Publicado: (2026)

Enabling Extensible Embodied Capabilities with Tools
por: Zhou, Xueyang, et al.
Publicado: (2026)

MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark
por: Sakshi, S, et al.
Publicado: (2024)

MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
por: Yue, Xiang, et al.
Publicado: (2023)

SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding
por: Sun, Ryan, et al.
Publicado: (2024)

Polymath: A Challenging Multi-modal Mathematical Reasoning Benchmark
por: Gupta, Himanshu, et al.
Publicado: (2024)

Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?
por: Fan, Chenrui, et al.
Publicado: (2025)

Prompt Injection Attack to Tool Selection in LLM Agents
por: Shi, Jiawen, et al.
Publicado: (2025)

MFE-ETP: A Comprehensive Evaluation Benchmark for Multi-modal Foundation Models on Embodied Task Planning
por: Zhang, Min, et al.
Publicado: (2024)

Integrating Text and Image Pre-training for Multi-modal Algorithmic Reasoning
por: Zhang, Zijian, et al.
Publicado: (2024)

Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models
por: Li, Yanwei, et al.
Publicado: (2024)

MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks
por: Niu, Yadong, et al.
Publicado: (2025)

Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation
por: Yue, Junrong, et al.
Publicado: (2025)

From Consistency to Complementarity: Aligned and Disentangled Multi-modal Learning for Time Series Understanding and Reasoning
por: Ni, Hang, et al.
Publicado: (2026)