Cover Image

On the Shelf Life of Fine-Tuned LLM-Judges: Future-Proofing, Backward-Compatibility, and Question Generalization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Singh, Janvijay, Xu, Austin, Zhou, Yilun, Zhou, Yefan, Hakkani-Tur, Dilek, Joty, Shafiq
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Variation in Verification: Understanding Verification Dynamics in Large Language Models
by: Zhou, Yefan, et al.
Published: (2025)

Do LLMs Encode Functional Importance of Reasoning Tokens?
by: Singh, Janvijay, et al.
Published: (2026)

Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
by: Zhou, Yilun, et al.
Published: (2025)

Neural Networks for Learnable and Scalable Influence Estimation of Instruction Fine-Tuning Data
by: Agarwal, Ishika, et al.
Published: (2025)

The Hidden Signal of Verifier Strictness: Controlling and Improving Step-Wise Verification via Selective Latent Steering
by: Zhou, Yefan, et al.
Published: (2026)

Question Generation for Assessing Early Literacy Reading Comprehension
by: Yang, Xiaocheng, et al.
Published: (2025)

J4R: Learning to Judge with Equivalent Initial State Group Relative Policy Optimization
by: Xu, Austin, et al.
Published: (2025)

Instruct, Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code Debugging
by: Kargupta, Priyanka, et al.
Published: (2024)

Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings
by: Xu, Austin, et al.
Published: (2025)

Direct Judgement Preference Optimization
by: Wang, Peifeng, et al.
Published: (2024)

Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue
by: Dongre, Vardhan, et al.
Published: (2026)

Confidence Estimation for LLM-Based Dialogue State Tracking
by: Sun, Yi-Jyun, et al.
Published: (2024)

From Fact to Judgment: Investigating the Impact of Task Framing on LLM Conviction in Dialogue Systems
by: Rabbani, Parisa, et al.
Published: (2025)

AURA: A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agents
by: Kim, Takyoung, et al.
Published: (2025)

Large Language Models as User-Agents for Evaluating Task-Oriented-Dialogue Systems
by: Kazi, Taaha, et al.
Published: (2024)

Sparking Scientific Creativity via LLM-Driven Interdisciplinary Inspiration
by: Kargupta, Priyanka, et al.
Published: (2026)

Self-Improving LLM Agents at Test-Time
by: Acikgoz, Emre Can, et al.
Published: (2025)

Simulating User Agents for Embodied Conversational-AI
by: Philipov, Daniel, et al.
Published: (2024)

Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling
by: Dey, Suvodip, et al.
Published: (2025)

A Rising Tide Lifts All Boats: MTQE Rewards for Idioms Improve General Translation Quality
by: Agarwal, Ishika, et al.
Published: (2026)

Goal Alignment in LLM-Based User Simulators for Conversational AI
by: Mehri, Shuhaib, et al.
Published: (2025)

Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains
by: Xu, Austin, et al.
Published: (2025)

Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
by: Bozdag, Nimet Beyza, et al.
Published: (2025)

Beyond Sample-Level Feedback: Using Reference-Level Feedback to Guide Data Synthesis
by: Mehri, Shuhaib, et al.
Published: (2025)

LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language
by: Ge, Yubin, et al.
Published: (2025)

ReasoningFlow: Semantic Structure of Complex Reasoning Traces
by: Lee, Jinu, et al.
Published: (2025)

DocCHA: Towards LLM-Augmented Interactive Online diagnosis System
by: Liu, Xinyi, et al.
Published: (2025)

SMILE: A Composite Lexical-Semantic Metric for Question-Answering Evaluation
by: Kendre, Shrikant, et al.
Published: (2025)

Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization
by: Li, Chuyuan, et al.
Published: (2025)

Language Specific Knowledge: Do Models Know Better in X than in English?
by: Agarwal, Ishika, et al.
Published: (2025)

A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions
by: Acikgoz, Emre Can, et al.
Published: (2025)

Dialog Flow Induction for Constrainable LLM-Based Chatbots
by: Agrawal, Stuti, et al.
Published: (2024)

User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction
by: Hao, Yuren, et al.
Published: (2026)

Unsupervised Human Preference Learning
by: Shashidhar, Sumuk, et al.
Published: (2024)

JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking
by: Niu, Tong, et al.
Published: (2024)

YourBench: Easy Custom Evaluation Sets for Everyone
by: Shashidhar, Sumuk, et al.
Published: (2025)

Plan Verification for LLM-Based Embodied Task Completion Agents
by: Hariharan, Ananth, et al.
Published: (2025)

From Context to Action: Analysis of the Impact of State Representation and Context on the Generalization of Multi-Turn Web Navigation Agents
by: Tiwary, Nalin, et al.
Published: (2024)

Premise-Augmented Reasoning Chains Improve Error Identification in Math reasoning with LLMs
by: Mukherjee, Sagnik, et al.
Published: (2025)

DialDefer: A Framework for Detecting and Mitigating LLM Dialogic Deference
by: Rabbani, Parisa, et al.
Published: (2026)