Image de couverture de livre

Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Leyao, He, Yanan, Chen, Peng, Yehudai, Asaf, Liu, Yixin, Ying, Rex, Shmueli-Scheuer, Michal, Cohan, Arman
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents
par: Yehudai, Asaf, et autres
Publié: (2026)

Survey on Evaluation of LLM-based Agents
par: Yehudai, Asaf, et autres
Publié: (2025)

CLEAR: Error Analysis via LLM-as-a-Judge Made Easy
par: Yehudai, Asaf, et autres
Publié: (2025)

A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks
par: Keren, Tomer, et autres
Publié: (2026)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
par: Liu, Yixin, et autres
Publié: (2025)

Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench
par: Perlitz, Yotam, et autres
Publié: (2024)

Growing Pains: Extensible and Efficient LLM Benchmarking Via Fixed Parameter Calibration
par: Habba, Eliya, et autres
Publié: (2026)

Mediocrity is the key for LLM as a Judge Anchor Selection
par: Don-Yehiya, Shachar, et autres
Publié: (2026)

Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models
par: Kour, George, et autres
Publié: (2025)

JuStRank: Benchmarking LLM Judges for System Ranking
par: Gera, Ariel, et autres
Publié: (2024)

Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
par: Bui, Ngoc, et autres
Publié: (2026)

General Agent Evaluation
par: Bandel, Elron, et autres
Publié: (2026)

ResearchGym: Evaluating Language Model Agents on Real-World AI Research
par: Garikaparthi, Aniketh, et autres
Publié: (2026)

Understanding Reference Policies in Direct Preference Optimization
par: Liu, Yixin, et autres
Publié: (2024)

When LLMs are Unfit Use FastFit: Fast and Effective Text Classification with Many Classes
par: Yehudai, Asaf, et autres
Publié: (2024)

The Mighty ToRR: A Benchmark for Table Reasoning and Robustness
par: Ashury-Tahan, Shir, et autres
Publié: (2025)

Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?
par: Zhang, Mingqiao, et autres
Publié: (2026)

Can We Trust LLM Detectors?
par: Sandhan, Jivnesh, et autres
Publié: (2026)

LLM-REVal: Can We Trust LLM Reviewers Yet?
par: Li, Rui, et autres
Publié: (2025)

Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
par: Liu, Yixin, et autres
Publié: (2026)

Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge
par: Ma, Chiyu, et autres
Publié: (2025)

Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge
par: Schroeder, Kayla, et autres
Publié: (2024)

Teaching Values to Machines: Simulating Human-Like Behavior in LLMs
par: Yehudai, Asaf, et autres
Publié: (2026)

Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
par: Xu, Zhijian, et autres
Publié: (2025)

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference
par: Gao, Mingqi, et autres
Publié: (2024)

ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Models
par: Ashury-Tahan, Shir, et autres
Publié: (2026)

Robustness as an Emergent Property of Task Performance
par: Ashury-Tahan, Shir, et autres
Publié: (2026)

Can We Trust AI to Govern AI? Benchmarking LLM Performance on Privacy and AI Governance Exams
par: Witherspoon, Zane, et autres
Publié: (2025)

SAGE: Benchmarking and Improving Retrieval for Deep Research Agents
par: Hu, Tiansheng, et autres
Publié: (2026)

References Improve LLM Alignment in Non-Verifiable Domains
par: Shi, Kejian, et autres
Publié: (2026)

SRTJ: Self-Evolving Rule-Driven Training-Free LLM Jailbreaking
par: Li, Jindong, et autres
Publié: (2026)

YaleNLP @ PerAnsSumm 2025: Multi-Perspective Integration via Mixture-of-Agents for Enhanced Healthcare QA Summarization
par: Jang, Dongsuk, et autres
Publié: (2025)

DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation
par: Habba, Eliya, et autres
Publié: (2025)

Observable Propagation: Uncovering Feature Vectors in Transformers
par: Dunefsky, Jacob, et autres
Publié: (2023)

One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs
par: Dunefsky, Jacob, et autres
Publié: (2025)

Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems
par: Jiao, Ruochen, et autres
Publié: (2024)

Guided Query Refinement: Multimodal Hybrid Retrieval with Test-Time Optimization
par: Uzan, Omri, et autres
Publié: (2025)

Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
par: Li, Alan, et autres
Publié: (2025)

Calibrating Long-form Generations from Large Language Models
par: Huang, Yukun, et autres
Publié: (2024)

COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences
par: Liu, Yixin, et autres
Publié: (2024)