Cover Image

RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pan, Tianjun, Lin, Xuan, Yang, Wenyan, He, Qianyu, Chen, Shisong, Qi, Licai, Xu, Wanqing, Feng, Hongwei, Xu, Bo, Xiao, Yanghua
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
by: He, Yun, et al.
Published: (2025)

Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks
by: Xu, Tianze, et al.
Published: (2026)

From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judges
by: Hong, Yihan, et al.
Published: (2026)

ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents
by: Sharma, Manasi, et al.
Published: (2025)

TN-Eval: Rubric and Evaluation Protocols for Measuring the Quality of Behavioral Therapy Notes
by: Shah, Raj Sanjay, et al.
Published: (2025)

Learning to Judge: LLMs Designing and Applying Evaluation Rubrics
by: Siro, Clemencia, et al.
Published: (2026)

SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment
by: Jiang, Sihang, et al.
Published: (2026)

SedarEval: Automated Evaluation using Self-Adaptive Rubrics
by: Fan, Zhiyuan, et al.
Published: (2025)

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges
by: Ding, Ruomeng, et al.
Published: (2026)

OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment
by: Liu, Tianci, et al.
Published: (2025)

AdaRubric: Task-Adaptive Rubrics for Reliable LLM Agent Evaluation and Reward Learning
by: Ding, Liang
Published: (2026)

INSEva: A Comprehensive Chinese Benchmark for Large Language Models in Insurance
by: Chen, Shisong, et al.
Published: (2025)

LP-Eval: Rubric and Dataset for Measuring the Quality of Legal Proposition Generation
by: Xu, Shanshan, et al.
Published: (2026)

Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubrics
by: Pathak, Aditya, et al.
Published: (2025)

Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation
by: Liu, Xue, et al.
Published: (2026)

Autorubric: Unifying Rubric-based LLM Evaluation
by: Rao, Delip, et al.
Published: (2026)

Am I More Pointwise or Pairwise? Revealing Position Bias in Rubric-Based LLM-as-a-Judge
by: Xu, Yuzheng, et al.
Published: (2026)

RubricRAG: Towards Interpretable and Reliable LLM Evaluation via Domain Knowledge Retrieval for Rubric Generation
by: Dhole, Kaustubh D., et al.
Published: (2026)

RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards
by: Li, Gaotang, et al.
Published: (2026)

RubricBench: Aligning Model-Generated Rubrics with Human Standards
by: Zhang, Qiyuan, et al.
Published: (2026)

Rubrics.
by: Callison, Daniel
Published: (2000)

EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generation
by: Guan, Xin, et al.
Published: (2026)

AutoRubric: Rubric-Based Generative Rewards for Faithful Multimodal Reasoning
by: Jia, Mengzhao, et al.
Published: (2025)

Step-wise Rubric Rewards for LLM Reasoning
by: Xie, Weichu, et al.
Published: (2026)

LASER: An LLM-based ASR Scoring and Evaluation Rubric
by: Parulekar, Amruta, et al.
Published: (2025)

Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks
by: Shen, William F., et al.
Published: (2026)

Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning
by: Bhattarai, Manish, et al.
Published: (2026)

Beyond the Rubric: Cultural Misalignment in LLM Benchmarks for Sexual and Reproductive Health
by: Dey, Sumon Kanti, et al.
Published: (2025)

Think-with-Rubrics: From External Evaluator to Internal Reasoning Guidance
by: Yu, Jiachen, et al.
Published: (2026)

Visual Preference Optimization with Rubric Rewards
by: Yu, Ya-Qi, et al.
Published: (2026)

Reinforcement Learning with Robust Rubric Rewards
by: Yu, Ya-Qi, et al.
Published: (2026)

Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR
by: Tyagi, Utkarsh, et al.
Published: (2026)

Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubric
by: Jia, Ruipeng, et al.
Published: (2026)

Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards
by: Yuan, Youliang, et al.
Published: (2025)

From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models
by: He, Qianyu, et al.
Published: (2024)

Reinforcement Learning with Rubric Anchors
by: Huang, Zenan, et al.
Published: (2025)

Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modeling
by: Xie, Lipeng, et al.
Published: (2025)

Using Rubrics to Assess Learning in Course-Integrated Library Instruction
by: Gariepy, Laura W., et al.
Published: (2016)

MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Models
by: Son, Guijin, et al.
Published: (2024)

Preference-Aware Rubric Learning for Personalized Evaluation
by: Qiu, Yilun, et al.
Published: (2026)