Cover Image

Benchmarking Vision Language Models for Cultural Understanding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nayak, Shravan, Jain, Kanishk, Awal, Rabiul, Reddy, Siva, van Steenkiste, Sjoerd, Hendricks, Lisa Anne, Stańczak, Karolina, Agrawal, Aishwarya
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metrics
by: Nayak, Shravan, et al.
Published: (2025)

Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering
by: Awal, Rabiul, et al.
Published: (2023)

VisMin: Visual Minimal-Change Understanding
by: Awal, Rabiul, et al.
Published: (2024)

Value Drifts: Tracing Value Alignment During LLM Post-Training
by: Bhatia, Mehar, et al.
Published: (2025)

Discovering Failure Modes in Vision-Language Models using RL
by: Jain, Kanishk, et al.
Published: (2026)

Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understanding
by: Zhang, Le, et al.
Published: (2023)

UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction
by: Nayak, Shravan, et al.
Published: (2025)

How Does Code Pretraining Affect Language Model Task Performance?
by: Petty, Jackson, et al.
Published: (2024)

A Multilingual Perspective on Probing Gender Bias
by: Stańczak, Karolina
Published: (2024)

Social Bias Probing: Fairness Benchmarking for Language Models
by: Manerba, Marta Marchiori, et al.
Published: (2023)

REARANK: Reasoning Re-ranking Agent via Reinforcement Learning
by: Zhang, Le, et al.
Published: (2025)

The Impact of Depth on Compositional Generalization in Transformer Language Models
by: Petty, Jackson, et al.
Published: (2023)

Quantifying Gender Biases Towards Politicians on Reddit
by: Marjanovic, Sara, et al.
Published: (2021)

Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models
by: Qiu, Linlu, et al.
Published: (2025)

WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
by: Awal, Rabiul, et al.
Published: (2025)

A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models
by: Eisape, Tiwalayo, et al.
Published: (2023)

CTRL-O: Language-Controllable Object-Centric Visual Representation Learning
by: Didolkar, Aniket, et al.
Published: (2025)

VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
by: Yu, Haorui, et al.
Published: (2026)

Investigating Adversarial Trigger Transfer in Large Language Models
by: Meade, Nicholas, et al.
Published: (2024)

The Causal Influence of Grammatical Gender on Distributional Semantics
by: Stańczak, Karolina, et al.
Published: (2023)

SafeArena: Evaluating the Safety of Autonomous Web Agents
by: Tur, Ada Defne, et al.
Published: (2025)

Faithfulness Measurable Masked Language Models
by: Madsen, Andreas, et al.
Published: (2023)

Improving Automatic VQA Evaluation Using Large Language Models
by: Mañas, Oscar, et al.
Published: (2023)

A Latent-Variable Model for Intrinsic Probing
by: Stańczak, Karolina, et al.
Published: (2022)

Non-literal Understanding of Number Words by Language Models
by: Tsvilodub, Polina, et al.
Published: (2025)

Understanding the Influence of Synthetic Data for Text Embedders
by: Springer, Jacob Mitchell, et al.
Published: (2025)

An Examination of the Robustness of Reference-Free Image Captioning Evaluation Metrics
by: Ahmadi, Saba, et al.
Published: (2023)

Decompose and Compare Consistency: Measuring VLMs' Answer Reliability via Task-Decomposition Consistency Comparison
by: Yang, Qian, et al.
Published: (2024)

Bifocal Attention: Harmonizing Geometric and Spectral Positional Embeddings for Algorithmic Generalization
by: Awadhiya, Kanishk
Published: (2026)

AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
by: Lù, Xing Han, et al.
Published: (2025)

Language Models Largely Exhibit Human-like Constituent Ordering Preferences
by: Tur, Ada Defne, et al.
Published: (2025)

How to Get Your LLM to Generate Challenging Problems for Evaluation
by: Patel, Arkil, et al.
Published: (2025)

Not All Data Are Unlearned Equally
by: Krishnan, Aravind, et al.
Published: (2025)

Benchmarking Multimodal Models for Ukrainian Language Understanding Across Academic and Cultural Domains
by: Paniv, Yurii, et al.
Published: (2024)

VideoNorms: Benchmarking Cultural Awareness of Video Language Models
by: Varimalla, Nikhil Reddy, et al.
Published: (2025)

How Culturally Aware are Vision-Language Models?
by: Burda-Lassen, Olena, et al.
Published: (2024)

Scope Ambiguities in Large Language Models
by: Kamath, Gaurav, et al.
Published: (2024)

Are self-explanations from Large Language Models faithful?
by: Madsen, Andreas, et al.
Published: (2024)

BELL: Benchmarking the Explainability of Large Language Models
by: Ahmed, Syed Quiser, et al.
Published: (2025)

The Promise of RL for Autoregressive Image Editing
by: Ahmadi, Saba, et al.
Published: (2025)