Cover Image

YourBench: Easy Custom Evaluation Sets for Everyone

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shashidhar, Sumuk, Fourrier, Clémentine, Lozovskia, Alina, Wolf, Thomas, Tur, Gokhan, Hakkani-Tür, Dilek
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence I.2.1
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Unsupervised Human Preference Learning
by: Shashidhar, Sumuk, et al.
Published: (2024)

Question Generation for Assessing Early Literacy Reading Comprehension
by: Yang, Xiaocheng, et al.
Published: (2025)

Simulating User Agents for Embodied Conversational-AI
by: Philipov, Daniel, et al.
Published: (2024)

Large Language Models as User-Agents for Evaluating Task-Oriented-Dialogue Systems
by: Kazi, Taaha, et al.
Published: (2024)

AURA: A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agents
by: Kim, Takyoung, et al.
Published: (2025)

Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
by: Bozdag, Nimet Beyza, et al.
Published: (2025)

Confidence Estimation for LLM-Based Dialogue State Tracking
by: Sun, Yi-Jyun, et al.
Published: (2024)

Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling
by: Dey, Suvodip, et al.
Published: (2025)

Do LLMs Encode Functional Importance of Reasoning Tokens?
by: Singh, Janvijay, et al.
Published: (2026)

Neural Networks for Learnable and Scalable Influence Estimation of Instruction Fine-Tuning Data
by: Agarwal, Ishika, et al.
Published: (2025)

Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue
by: Dongre, Vardhan, et al.
Published: (2026)

Self-Improving LLM Agents at Test-Time
by: Acikgoz, Emre Can, et al.
Published: (2025)

Towards Safer Chatbots: Automated Policy Compliance Evaluation of Custom GPTs
by: Rodriguez, David, et al.
Published: (2025)

Goal Alignment in LLM-Based User Simulators for Conversational AI
by: Mehri, Shuhaib, et al.
Published: (2025)

ATOD: An Evaluation Framework and Benchmark for Agentic Task-Oriented Dialogue Systems
by: Zhang, Yifei, et al.
Published: (2026)

Plan Verification for LLM-Based Embodied Task Completion Agents
by: Hariharan, Ananth, et al.
Published: (2025)

TD-EVAL: Revisiting Task-Oriented Dialogue Evaluation by Combining Turn-Level Precision with Dialogue-Level Comparisons
by: Acikgoz, Emre Can, et al.
Published: (2025)

User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction
by: Hao, Yuren, et al.
Published: (2026)

Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviors
by: Mehri, Shuhaib, et al.
Published: (2026)

Evaluating Large Language Models on Historical Health Crisis Knowledge in Resource-Limited Settings: A Hybrid Multi-Metric Study
by: Hasan, Mohammed Rakibul
Published: (2026)

ReSpAct: Harmonizing Reasoning, Speaking, and Acting Towards Building Large Language Model-Based Conversational AI Agents
by: Dongre, Vardhan, et al.
Published: (2024)

Spark: A System for Scientifically Creative Idea Generation
by: Sanyal, Aishik, et al.
Published: (2025)

ReIn: Conversational Error Recovery with Reasoning Inception
by: Kim, Takyoung, et al.
Published: (2026)

How to Evaluate Medical AI
by: Kopanichuk, Ilia, et al.
Published: (2025)

Large Language Models for Judicial Entity Extraction: A Comparative Study
by: Hussain, Atin Sakkeer, et al.
Published: (2024)

MIRAGE: A Benchmark for Multimodal Information-Seeking and Reasoning in Agricultural Expert-Guided Conversations
by: Dongre, Vardhan, et al.
Published: (2025)

A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions
by: Acikgoz, Emre Can, et al.
Published: (2025)

Sparking Scientific Creativity via LLM-Driven Interdisciplinary Inspiration
by: Kargupta, Priyanka, et al.
Published: (2026)

Combinatorial Creativity: A New Frontier in Generalization Abilities
by: Schapiro, Samuel, et al.
Published: (2025)

Reinforcement of Explainability of ChatGPT Prompts by Embedding Breast Cancer Self-Screening Rules into AI Responses
by: Khan, Yousef, et al.
Published: (2024)

ToolRL: Reward is All Tool Learning Needs
by: Qian, Cheng, et al.
Published: (2025)

SMART: Self-Aware Agent for Tool Overuse Mitigation
by: Qian, Cheng, et al.
Published: (2025)

Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model
by: Acikgoz, Emre Can, et al.
Published: (2025)

The Need for Guardrails with Large Language Models in Medical Safety-Critical Settings: An Artificial Intelligence Application in the Pharmacovigilance Ecosystem
by: Hakim, Joe B, et al.
Published: (2024)

AskSport: Web Application for Sports Question-Answering
by: Onofre, Enzo B, et al.
Published: (2025)

Comparing the Performance of LLMs in RAG-based Question-Answering: A Case Study in Computer Science Literature
by: Dayarathne, Ranul, et al.
Published: (2025)

ReTreVal: Reasoning Tree with Validation -- A Hybrid Framework for Enhanced LLM Multi-Step Reasoning
by: HS, Abhishek, et al.
Published: (2026)

Challenges and Opportunities of NLP for HR Applications: A Discussion Paper
by: Leidner, Jochen L., et al.
Published: (2024)

Comparative Analysis of AI Agent Architectures for Entity Relationship Classification
by: Berijanian, Maryam, et al.
Published: (2025)

Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date overview
by: Han, Lifeng, et al.
Published: (2016)