Cover Image

Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Hengyi, Shi, Haizhou, Tan, Shiwei, Qin, Weiyi, Wang, Wenyuan, Zhang, Tunyu, Nambi, Akshay, Ganu, Tanuja, Wang, Hao
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Causal Decoding for Hallucination-Resistant Multimodal Large Language Models
by: Tan, Shiwei, et al.
Published: (2026)

MMCTAgent: Multi-modal Critical Thinking Agent Framework for Complex Visual Reasoning
by: Kumar, Somnath, et al.
Published: (2024)

Do You See Me : A Multidimensional Benchmark for Evaluating Visual Perception in Multimodal LLMs
by: Kanade, Aditya, et al.
Published: (2025)

Needle In A Multimodal Haystack
by: Wang, Weiyun, et al.
Published: (2024)

Continual Learning of Large Language Models: A Comprehensive Survey
by: Shi, Haizhou, et al.
Published: (2024)

PromptWizard: Task-Aware Prompt Optimization Framework
by: Agarwal, Eshaan, et al.
Published: (2024)

Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts
by: Yu, Yifei, et al.
Published: (2025)

Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs
by: Kancheti, Sai Srinivas, et al.
Published: (2026)

Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contexts
by: Sharma, Aditya, et al.
Published: (2024)

Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
by: Kumar, Somnath, et al.
Published: (2023)

Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Models
by: Hengle, Amey, et al.
Published: (2024)

NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables
by: Wang, Lanrui, et al.
Published: (2025)

RadPhi-3: Small Language Models for Radiology
by: Ranjit, Mercy, et al.
Published: (2024)

Mind's Eye: A Benchmark of Visual Abstraction, Transformation and Composition for Multimodal LLMs
by: Sinha, Rohit, et al.
Published: (2026)

Reasoning on Multiple Needles In A Haystack
by: Wang, Yidong
Published: (2025)

U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack
by: Gao, Yunfan, et al.
Published: (2025)

Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark
by: Huybrechts, Goeric, et al.
Published: (2025)

Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models
by: Wang, Hengyi, et al.
Published: (2024)

Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmark
by: Wu, Tsung-Han, et al.
Published: (2024)

TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning
by: Zhang, Tunyu, et al.
Published: (2025)

Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
by: Kumar, Somnath, et al.
Published: (2024)

Needle in the Haystack for Memory Based Large Language Models
by: Nelson, Elliot, et al.
Published: (2024)

Faithful GRPO: Improving Visual Spatial Reasoning in Multimodal Language Models via Constrained Policy Optimization
by: Kancheti, Sai Srinivas, et al.
Published: (2026)

Variational Language Concepts for Interpreting Foundation Language Models
by: Wang, Hengyi, et al.
Published: (2024)

Shiksha Copilot: Teacher-AI Collaboration for Curating and Customizing Lesson Plans in Low-Resource Schools
by: Dennison, Deepak Varuvel, et al.
Published: (2025)

GOLD PANNING: Strategic Context Shuffling for Needle-in-Haystack Reasoning
by: Byerly, Adam, et al.
Published: (2025)

Designing Culturally Aligned AI Systems For Social Good in Non-Western Contexts
by: Dennison, Deepak Varuvel, et al.
Published: (2025)

CLEAR-HPV: Interpretable concept discovery for human-papillomavirus-associated morphology in whole-slide histology
by: Qin, Weiyi, et al.
Published: (2026)

Two Causally Related Needles in a Video Haystack
by: Li, Miaoyu, et al.
Published: (2025)

CODIS: Benchmarking Context-Dependent Visual Comprehension for Multimodal Large Language Models
by: Luo, Fuwen, et al.
Published: (2024)

DENIAHL: In-Context Features Influence LLM Needle-In-A-Haystack Abilities
by: Dai, Hui, et al.
Published: (2024)

TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning
by: Wu, Nemin, et al.
Published: (2024)

Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation
by: Li, Mufei, et al.
Published: (2025)

XL$^2$Bench: A Benchmark for Extremely Long Context Understanding with Long-range Dependencies
by: Ni, Xuanfan, et al.
Published: (2024)

Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Input
by: Li, Chenxu, et al.
Published: (2025)

A Survey on Benchmarks of Multimodal Large Language Models
by: Li, Jian, et al.
Published: (2024)

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
by: Xiong, Zheyang, et al.
Published: (2024)

NeedleChain: Measuring Intact Context Comprehension Capability of Large Language Models
by: Moon, Hyeonseok, et al.
Published: (2025)

Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models
by: Sun, Haoyuan, et al.
Published: (2025)

Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model
by: Zhang, Yuting, et al.
Published: (2025)