Imagen de Portada

CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Li, Haitao, Chen, Junjie, Ai, Qingyao, Chu, Zhumin, Zhou, Yujia, Dong, Qian, Liu, Yiqun
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Overview of the NTCIR-18 Automatic Evaluation of LLMs (AEOLLM) Task
por: Chen, Junjie, et al.
Publicado: (2025)

LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
por: Li, Haitao, et al.
Publicado: (2024)

PRE: A Peer Review Based Large Language Model Evaluator
por: Chu, Zhumin, et al.
Publicado: (2024)

ATACompressor: Adaptive Task-Aware Compression for Efficient Long-Context Processing in LLMs
por: Li, Xuancheng, et al.
Publicado: (2026)

Benchmarking LLM-as-a-Judge for Long-Form Output Evaluation
por: Chen, Junjie, et al.
Publicado: (2026)

Auto-PRE: An Automatic and Cost-Efficient Peer-Review Framework for Language Generation Evaluation
por: Chen, Junjie, et al.
Publicado: (2024)

LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models
por: Li, Haitao, et al.
Publicado: (2024)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
por: Ye, Ziyi, et al.
Publicado: (2024)

RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects
por: Tu, Yiteng, et al.
Publicado: (2025)

Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing
por: Wang, Changyue, et al.
Publicado: (2025)

Beyond Experience Retrieval: Learning to Generate Utility-Optimized Structured Experience for Frozen LLMs
por: Li, Xuancheng, et al.
Publicado: (2026)

Parametric Social Identity Injection and Diversification in Public Opinion Simulation
por: Wang, Hexi, et al.
Publicado: (2026)

Evaluation Ethics of LLMs in Legal Domain
por: Zhang, Ruizhe, et al.
Publicado: (2024)

BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models
por: Li, Haitao, et al.
Publicado: (2024)

Dynamic and Parametric Retrieval-Augmented Generation
por: Su, Weihang, et al.
Publicado: (2025)

DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment
por: Li, Haitao, et al.
Publicado: (2024)

Augmenting Multi-Agent Communication with State Delta Trajectory
por: Tang, Yichen, et al.
Publicado: (2025)

MulFeRL: Enhancing Reinforcement Learning with Verbal Feedback in a Multi-turn Loop
por: Li, Xuancheng, et al.
Publicado: (2026)

Mitigating Entity-Level Hallucination in Large Language Models
por: Su, Weihang, et al.
Publicado: (2024)

How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study
por: Zhu, Shuqi, et al.
Publicado: (2026)

Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models
por: Su, Weihang, et al.
Publicado: (2024)

LegalOne: A Family of Foundation Models for Reliable Legal Reasoning
por: Li, Haitao, et al.
Publicado: (2026)

Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models
por: Wang, Changyue, et al.
Publicado: (2025)

TEC: A Collection of Human Trial-and-error Trajectories for Problem Solving
por: Zhang, Xinkai, et al.
Publicado: (2026)

Decoupling Knowledge and Task Subspaces for Composable Parametric Retrieval Augmented Generation
por: Su, Weihang, et al.
Publicado: (2026)

CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generation
por: Li, Haitao, et al.
Publicado: (2025)

Improve Large Language Model Systems with User Logs
por: Wang, Changyue, et al.
Publicado: (2026)

Multi-Field Tool Retrieval
por: Tang, Yichen, et al.
Publicado: (2026)

ValueSim: Generating Backstories to Model Individual Value Systems
por: Du, Bangde, et al.
Publicado: (2025)

Option-ID Based Elimination For Multiple Choice Questions
por: Zhu, Zhenhao, et al.
Publicado: (2025)

LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversation
por: Li, Haitao, et al.
Publicado: (2025)

Enhancing LLM-Based Agents via Global Planning and Hierarchical Execution
por: Chen, Junjie, et al.
Publicado: (2025)

CyclicJudge: Mitigating Judge Bias Efficiently in LLM-based Evaluation
por: Zhu, Ziyi, et al.
Publicado: (2026)

Knowledge Editing through Chain-of-Thought
por: Wang, Changyue, et al.
Publicado: (2024)

Enhancing Judgment Document Generation via Agentic Legal Information Collection and Rubric-Guided Optimization
por: Su, Weihang, et al.
Publicado: (2026)

DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Models
por: Su, Weihang, et al.
Publicado: (2024)

Parametric Retrieval Augmented Generation
por: Su, Weihang, et al.
Publicado: (2025)

Disclosure and Mitigation of Gender Bias in LLMs
por: Dong, Xiangjue, et al.
Publicado: (2024)

Capability-aware Prompt Reformulation Learning for Text-to-Image Generation
por: Zhan, Jingtao, et al.
Publicado: (2024)

What Scales in Cross-Entropy Scaling Law?
por: Yan, Junxi, et al.
Publicado: (2025)