Cover Image

NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Baiqi, Lin, Zhiqiu, Peng, Wenxuan, Nyandwi, Jean de Dieu, Jiang, Daniel, Ma, Zixian, Khanuja, Simran, Krishna, Ranjay, Neubig, Graham, Ramanan, Deva
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Grounding Multilingual Multimodal LLMs With Cultural Knowledge
by: Nyandwi, Jean de Dieu, et al.
Published: (2025)

Evaluating Text-to-Visual Generation with Image-to-Text Generation
by: Lin, Zhiqiu, et al.
Published: (2024)

GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation
by: Li, Baiqi, et al.
Published: (2024)

What Is Missing in Multilingual Visual Reasoning and How to Fix It
by: Song, Yueqi, et al.
Published: (2024)

Towards Automatic Evaluation for Image Transcreation
by: Khanuja, Simran, et al.
Published: (2024)

Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
by: Yue, Xiang, et al.
Published: (2024)

An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance
by: Khanuja, Simran, et al.
Published: (2024)

CAIRe: Cultural Attribution of Images by Retrieval-Augmented Evaluation
by: Yayavaram, Arnav, et al.
Published: (2025)

Revisiting the Role of Language Priors in Vision-Language Models
by: Lin, Zhiqiu, et al.
Published: (2023)

TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs
by: Li, Baiqi, et al.
Published: (2026)

Language Models as Black-Box Optimizers for Vision-Language Models
by: Liu, Shihong, et al.
Published: (2023)

The Neglected Tails in Vision-Language Models
by: Parashar, Shubham, et al.
Published: (2024)

m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks
by: Ma, Zixian, et al.
Published: (2024)

MERLIN: A Testbed for Multilingual Multimodal Entity Recognition and Linking
by: Ramamoorthy, Sathyanarayanan, et al.
Published: (2025)

Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models
by: Lin, Zhiqiu, et al.
Published: (2023)

Predicting Long-horizon Futures by Conditioning on Geometry and Time
by: Khurana, Tarasha, et al.
Published: (2024)

PAI-Bench: A Comprehensive Benchmark For Physical AI
by: Zhou, Fengzhe, et al.
Published: (2025)

Revisiting Few-Shot Object Detection with Vision-Language Models
by: Madan, Anish, et al.
Published: (2023)

Rethinking Human Preference Evaluation of LLM Rationales
by: Li, Ziang, et al.
Published: (2025)

Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Features
by: Mitra, Chancharik, et al.
Published: (2024)

RefAV: Towards Planning-Centric Scenario Mining
by: Davidson, Cainan, et al.
Published: (2025)

InstructPart: Task-Oriented Part Segmentation with Instruction Reasoning
by: Wan, Zifu, et al.
Published: (2025)

Evaluating a VR System for Collecting Safety-Critical Vehicle-Pedestrian Interactions
by: Weng, Erica, et al.
Published: (2023)

Using Diffusion Priors for Video Amodal Segmentation
by: Chen, Kaihua, et al.
Published: (2024)

Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos
by: Chen, Kaihua, et al.
Published: (2025)

You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass
by: Yang, Yinuo, et al.
Published: (2026)

CMULAB: An Open-Source Framework for Training and Deployment of Natural Language Processing Models
by: Sheikh, Zaid, et al.
Published: (2024)

Ablate-to-Validate: Are Vision-Language Models Really Using Continuous Thought Tokens?
by: Zhang, Tianyi, et al.
Published: (2026)

Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations
by: Li, Linjie, et al.
Published: (2025)

The Hard Positive Truth about Vision-Language Compositionality
by: Kamath, Amita, et al.
Published: (2024)

SMORE: Simultaneous Map and Object REconstruction
by: Chodosh, Nathaniel, et al.
Published: (2024)

Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models
by: Robicheaux, Peter, et al.
Published: (2025)

VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models
by: He, Qijia, et al.
Published: (2026)

Go-Browse: Training Web Agents with Structured Exploration
by: Gandhi, Apurva, et al.
Published: (2025)

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models
by: Tjuatja, Lindia, et al.
Published: (2025)

Iterated Learning Improves Compositionality in Large Vision-Language Models
by: Zheng, Chenhao, et al.
Published: (2024)

Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate
by: Chern, Steffi, et al.
Published: (2024)

Shelf-Supervised Cross-Modal Pre-Training for 3D Object Detection
by: Khurana, Mehar, et al.
Published: (2024)

Self-Disclosure to AI: The Paradox of Trust and Vulnerability in Human-Machine Interactions
by: Jiang, Zoe Zhiqiu
Published: (2024)

Effective Strategies for Asynchronous Software Engineering Agents
by: Geng, Jiayi, et al.
Published: (2026)