Copertina

MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chen, Dongping, Chen, Ruoxi, Zhang, Shilin, Liu, Yinuo, Wang, Yaochen, Zhou, Huichi, Zhang, Qihui, Wan, Yao, Zhou, Pan, Sun, Lichao
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Judge Anything: MLLM as a Judge Across Any Modality
di: Pu, Shu, et al.
Pubblicazione: (2025)

Are We on the Right Way to Assessing LLM-as-a-Judge?
di: Feng, Yuanning, et al.
Pubblicazione: (2025)

LLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected?
di: Zhang, Qihui, et al.
Pubblicazione: (2024)

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
di: Ye, Jiayi, et al.
Pubblicazione: (2024)

HonestLLM: Toward an Honest and Helpful Large Language Model
di: Gao, Chujie, et al.
Pubblicazione: (2024)

GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding
di: Chen, Dongping, et al.
Pubblicazione: (2024)

Evaluate-and-Purify: Fortifying Code Language Models Against Adversarial Attacks Using LLM-as-a-Judge
di: Mu, Wenhan, et al.
Pubblicazione: (2025)

MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge
di: Lee, Sua, et al.
Pubblicazione: (2026)

SEAL: Can Saturated Benchmarks Be Revived by LLM-as-a-Meta-Judge?
di: Chen, Jiamin, et al.
Pubblicazione: (2026)

Self-Cognition in Large Language Models: An Exploratory Study
di: Chen, Dongping, et al.
Pubblicazione: (2024)

Benchmarking LLM-as-a-Judge for Long-Form Output Evaluation
di: Chen, Junjie, et al.
Pubblicazione: (2026)

ProJudge: A Multi-Modal Multi-Discipline Benchmark and Instruction-Tuning Dataset for MLLM-based Process Judges
di: Ai, Jiaxin, et al.
Pubblicazione: (2025)

JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems
di: Bellibatlu, Rohith Reddy, et al.
Pubblicazione: (2026)

DataGen: Unified Synthetic Dataset Generation via Large Language Models
di: Huang, Yue, et al.
Pubblicazione: (2024)

MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces
di: Luera, Reuben A., et al.
Pubblicazione: (2025)

MLLM-as-a-Judge for Image Safety without Human Labeling
di: Wang, Zhenting, et al.
Pubblicazione: (2024)

FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge
di: Yang, Bo, et al.
Pubblicazione: (2026)

MR. Judge: Multimodal Reasoner as a Judge
di: Pi, Renjie, et al.
Pubblicazione: (2025)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
di: Jiang, Hongchao, et al.
Pubblicazione: (2025)

VideoJudge: Bootstrapping Enables Scalable Supervision of MLLM-as-a-Judge for Video Understanding
di: Waheed, Abdul, et al.
Pubblicazione: (2025)

Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?
di: Shen, Wenxuan, et al.
Pubblicazione: (2025)

Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge
di: Wu, Junjie, et al.
Pubblicazione: (2026)

UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning
di: Gu, Tiancheng, et al.
Pubblicazione: (2025)

Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
di: Zhou, Yilun, et al.
Pubblicazione: (2025)

Optimization-based Prompt Injection Attack to LLM-as-a-Judge
di: Shi, Jiawen, et al.
Pubblicazione: (2024)

EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models
di: Su, Jiamin, et al.
Pubblicazione: (2025)

Judge Model for Large-scale Multimodality Benchmarks
di: Shih, Min-Han, et al.
Pubblicazione: (2026)

TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
di: Wang, Yidong, et al.
Pubblicazione: (2025)

Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Following
di: Xiong, Tianyi, et al.
Pubblicazione: (2025)

BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
di: Tong, Terry, et al.
Pubblicazione: (2025)

RankJudge: A Multi-Turn LLM-as-a-Judge Synthetic Benchmark Generator
di: Tang, Zhenwei, et al.
Pubblicazione: (2026)

Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis
di: Liu, Runzhou, et al.
Pubblicazione: (2026)

HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment
di: Yang, Langqi, et al.
Pubblicazione: (2025)

MLLM-as-a-Judge Exhibits Model Preference Bias
di: Koyama, Shuitsu, et al.
Pubblicazione: (2026)

Fairness or Fluency? An Investigation into Language Bias of Pairwise LLM-as-a-Judge
di: Zhou, Xiaolin, et al.
Pubblicazione: (2026)

JudgeBench: A Benchmark for Evaluating LLM-based Judges
di: Tan, Sijun, et al.
Pubblicazione: (2024)

The Impact of Large Language Models in Academia: from Writing to Speaking
di: Geng, Mingmeng, et al.
Pubblicazione: (2024)

Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models
di: Ding, Meidan, et al.
Pubblicazione: (2025)

MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
di: Huang, Yue, et al.
Pubblicazione: (2023)

R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
di: Yuan, Tongxin, et al.
Pubblicazione: (2024)