Cover Image

Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Roy, Subhadeep, Bhatia, Gagan, Eger, Steffen
Format:	Preprint
Published:	2026
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning
by: Greisinger, Christian, et al.
Published: (2026)

Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition
by: Bhatia, Gagan, et al.
Published: (2024)

Uncovering Conceptual Blindspots in Generative Image Models Using Sparse Autoencoders
by: Bohacek, Matyas, et al.
Published: (2025)

3DFroMLLM: 3D Prototype Generation only from Pretrained Multimodal LLMs
by: Ahmed, Noor, et al.
Published: (2025)

ScImage: How Good Are Multimodal Large Language Models at Scientific Text-to-Image Generation?
by: Zhang, Leixin, et al.
Published: (2024)

MAJORScore: A Novel Metric for Evaluating Multimodal Relevance via Joint Representation
by: Du, Zhicheng, et al.
Published: (2025)

Multimodal Prototype-Enhanced Network for Few-Shot Action Recognition
by: Ni, Xinzhe, et al.
Published: (2022)

DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection
by: Li, Shawn, et al.
Published: (2024)

Multimodal Prototype Alignment for Semi-supervised Pathology Image Segmentation
by: Fu, Mingxi, et al.
Published: (2025)

Freeze and Reveal: Exposing Modality Bias in Vision-Language Models
by: Kavuri, Vivek Hruday, et al.
Published: (2025)

QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering
by: Jung, Woojun, et al.
Published: (2026)

PISA-Bench: The PISA Index as a Multilingual and Multimodal Metric for the Evaluation of Vision-Language Models
by: Haller, Patrick, et al.
Published: (2025)

Pedestrian Crossing Intention Prediction Using Multimodal Fusion Network
by: Li, Yuanzhe, et al.
Published: (2025)

ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts
by: Choi, Jinho, et al.
Published: (2025)

CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metrics
by: Nayak, Shravan, et al.
Published: (2025)

Physics-Based Benchmarking Metrics for Multimodal Synthetic Images
by: Gupta, Kishor Datta, et al.
Published: (2025)

StyleX: A Trainable Metric for X-ray Style Distances
by: Eckert, Dominik, et al.
Published: (2024)

Multimodal Political Bias Identification and Neutralization
by: Bernard, Cedric, et al.
Published: (2025)

Sum of Group Error Differences: A Critical Examination of Bias Evaluation in Biometric Verification and a Dual-Metric Measure
by: Elobaid, Alaa, et al.
Published: (2024)

Revealing Temporal Label Noise in Multimodal Hateful Video Classification
by: Yang, Shuonan, et al.
Published: (2025)

CROC: Evaluating and Training T2I Metrics with Pseudo- and Human-Labeled Contrastive Robustness Checks
by: Leiter, Christoph, et al.
Published: (2025)

SAGE: Spuriousness-Aware Guided Prompt Exploration for Mitigating Multimodal Bias
by: Ye, Wenqian, et al.
Published: (2025)

Process Integrated Computer Vision for Real-Time Failure Prediction in Steel Rolling Mill
by: Kurrey, Vaibhav, et al.
Published: (2025)

FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model
by: Lee, Yebin, et al.
Published: (2024)

SHOE: Semantic HOI Open-Vocabulary Evaluation Metric
by: Noack, Maja, et al.
Published: (2026)

Attribute Based Interpretable Evaluation Metrics for Generative Models
by: Kim, Dongkyun, et al.
Published: (2023)

Demographic Fairness in Multimodal LLMs: A Benchmark of Gender and Ethnicity Bias in Face Verification
by: Öztürk, Ünsal, et al.
Published: (2026)

Libra-MIL: Multimodal Prototypes Stereoscopic Infused with Task-specific Language Priors for Few-shot Whole Slide Image Classification
by: Zhuang, Zhenfeng, et al.
Published: (2025)

Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models
by: Yang, Songlin, et al.
Published: (2026)

Logic Unseen: Revealing the Logical Blindspots of Vision-Language Models
by: Zhou, Yuchen, et al.
Published: (2025)

BodyMetric: Evaluating the Realism of Human Bodies in Text-to-Image Generation
by: Andreou, Nefeli, et al.
Published: (2024)

SAMScore: A Content Structural Similarity Metric for Image Translation Evaluation
by: Li, Yunxiang, et al.
Published: (2023)

Comprehensive Equity Index (CEI): Definition and Application to Bias Evaluation in Biometrics
by: Solano, Imanol, et al.
Published: (2024)

MIBench: Evaluating LMMs on Multimodal Interaction
by: Miao, Yu, et al.
Published: (2026)

STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models
by: Kim, Pum Jun, et al.
Published: (2024)

Fairness Under Cover: Evaluating the Impact of Occlusions on Demographic Bias in Facial Recognition
by: Mamede, Rafael M., et al.
Published: (2024)

FAIntbench: A Holistic and Precise Benchmark for Bias Evaluation in Text-to-Image Models
by: Luo, Hanjun, et al.
Published: (2024)

Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
by: Chandu, Khyathi Raghavi, et al.
Published: (2024)

Polos: Multimodal Metric Learning from Human Feedback for Image Captioning
by: Wada, Yuiga, et al.
Published: (2024)

Negative Prototypes Guided Contrastive Learning for WSOD
by: Zhang, Yu, et al.
Published: (2024)