Cover Image

VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Yueqi, Ou, Tianyue, Kong, Yibo, Li, Zecheng, Neubig, Graham, Yue, Xiang
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

What Is Missing in Multilingual Visual Reasoning and How to Fix It
by: Song, Yueqi, et al.
Published: (2024)

Grounding Multilingual Multimodal LLMs With Cultural Knowledge
by: Nyandwi, Jean de Dieu, et al.
Published: (2025)

Harnessing Webpage UIs for Text-Rich Visual Understanding
by: Liu, Junpeng, et al.
Published: (2024)

On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
by: Zhang, Charlie, et al.
Published: (2025)

An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance
by: Khanuja, Simran, et al.
Published: (2024)

Beyond Browsing: API-Based Web Agents
by: Song, Yueqi, et al.
Published: (2024)

VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?
by: Liu, Junpeng, et al.
Published: (2024)

Demystifying Long Chain-of-Thought Reasoning in LLMs
by: Yeo, Edward, et al.
Published: (2025)

Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
by: Yue, Xiang, et al.
Published: (2024)

MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale
by: Guo, Jarvis, et al.
Published: (2024)

Scaling Evaluation-time Compute with Reasoning Models as Evaluators
by: Kim, Seungone, et al.
Published: (2025)

JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation
by: Onohara, Shota, et al.
Published: (2024)

CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigation
by: Huq, Faria, et al.
Published: (2025)

Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities
by: Bertsch, Amanda, et al.
Published: (2025)

Go-Browse: Training Web Agents with Structured Exploration
by: Gandhi, Apurva, et al.
Published: (2025)

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models
by: Tjuatja, Lindia, et al.
Published: (2025)

TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoning
by: Liu, Daixian, et al.
Published: (2026)

Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
by: Huan, Maggie, et al.
Published: (2025)

Coding Agents with Multimodal Browsing are Generalist Problem Solvers
by: Soni, Aditya Bharat, et al.
Published: (2025)

VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
by: Koh, Jing Yu, et al.
Published: (2024)

Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate
by: Chern, Steffi, et al.
Published: (2024)

Analyzing Information Sharing and Coordination in Multi-Agent Planning
by: Ou, Tianyue, et al.
Published: (2025)

Effective Strategies for Asynchronous Software Engineering Agents
by: Geng, Jiayi, et al.
Published: (2026)

PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehunts
by: Li, Hengzhi, et al.
Published: (2025)

Modeling Distinct Human Interaction in Web Agents
by: Huq, Faria, et al.
Published: (2026)

ZINA: Multimodal Fine-grained Hallucination Detection and Editing
by: Wada, Yuiga, et al.
Published: (2025)

Towards Automatic Evaluation for Image Transcreation
by: Khanuja, Simran, et al.
Published: (2024)

Checklists Are Better Than Reward Models For Aligning Language Models
by: Viswanathan, Vijay, et al.
Published: (2025)

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model
by: Huo, Jiahao, et al.
Published: (2024)

Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning
by: Sutawika, Lintang, et al.
Published: (2026)

Mix-CPT: A Domain Adaptation Framework via Decoupling Knowledge Learning and Format Alignment
by: Jiang, Jinhao, et al.
Published: (2024)

GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning
by: Zhang, Jianghangfan, et al.
Published: (2025)

Midtraining Bridges Pretraining and Posttraining Distributions
by: Liu, Emmy, et al.
Published: (2025)

An Incomplete Loop: Instruction Inference, Instruction Following, and In-context Learning in Language Models
by: Liu, Emmy, et al.
Published: (2024)

Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach
by: Kaneko, Masahiro, et al.
Published: (2023)

Evaluating Language Models as Synthetic Data Generators
by: Kim, Seungone, et al.
Published: (2024)

From Evidence-Based Medicine to Knowledge Graph: Retrieval-Augmented Generation for Sports Rehabilitation and a Domain Benchmark
by: Zhang, Jinning, et al.
Published: (2026)

Multi-Turn Puzzles: Evaluating Interactive Reasoning and Strategic Dialogue in LLMs
by: Badola, Kartikeya, et al.
Published: (2025)

Language Modeling with Editable External Knowledge
by: Li, Belinda Z., et al.
Published: (2024)

Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing
by: Wang, Changyue, et al.
Published: (2025)