Copertina

A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Keren, Tomer, Calderon, Nitay, Yehudai, Asaf, Perlitz, Yotam, Shmueli-Scheuer, Michal, Reichert, Roi
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

CLEAR: Error Analysis via LLM-as-a-Judge Made Easy
di: Yehudai, Asaf, et al.
Pubblicazione: (2025)

Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents
di: Yehudai, Asaf, et al.
Pubblicazione: (2026)

General Agent Evaluation
di: Bandel, Elron, et al.
Pubblicazione: (2026)

JuStRank: Benchmarking LLM Judges for System Ranking
di: Gera, Ariel, et al.
Pubblicazione: (2024)

Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench
di: Perlitz, Yotam, et al.
Pubblicazione: (2024)

Growing Pains: Extensible and Efficient LLM Benchmarking Via Fixed Parameter Calibration
di: Habba, Eliya, et al.
Pubblicazione: (2026)

Survey on Evaluation of LLM-based Agents
di: Yehudai, Asaf, et al.
Pubblicazione: (2025)

On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs
di: Calderon, Nitay, et al.
Pubblicazione: (2024)

Efficient Benchmarking of Language Models
di: Perlitz, Yotam, et al.
Pubblicazione: (2023)

LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals
di: Toker, Gilat, et al.
Pubblicazione: (2026)

ELT-Bench-Verified: Benchmark Quality Issues Underestimate AI Agent Capabilities
di: Zanoli, Christopher, et al.
Pubblicazione: (2026)

The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs
di: Calderon, Nitay, et al.
Pubblicazione: (2025)

The Mighty ToRR: A Benchmark for Table Reasoning and Robustness
di: Ashury-Tahan, Shir, et al.
Pubblicazione: (2025)

Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI
di: Bandel, Elron, et al.
Pubblicazione: (2024)

Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models
di: Kour, George, et al.
Pubblicazione: (2025)

CUBE: A Standard for Unifying Agent Benchmarks
di: Lacoste, Alexandre, et al.
Pubblicazione: (2026)

When LLMs are Unfit Use FastFit: Fast and Effective Text Classification with Many Classes
di: Yehudai, Asaf, et al.
Pubblicazione: (2024)

Teaching Values to Machines: Simulating Human-Like Behavior in LLMs
di: Yehudai, Asaf, et al.
Pubblicazione: (2026)

The Colorful Future of LLMs: Evaluating and Improving LLMs as Emotional Supporters for Queer Youth
di: Lissak, Shir, et al.
Pubblicazione: (2024)

ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Models
di: Ashury-Tahan, Shir, et al.
Pubblicazione: (2026)

Robustness as an Emergent Property of Task Performance
di: Ashury-Tahan, Shir, et al.
Pubblicazione: (2026)

NL-Eye: Abductive NLI for Images
di: Ventura, Mor, et al.
Pubblicazione: (2024)

Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
di: Wang, Leyao, et al.
Pubblicazione: (2026)

WildIFEval: Instruction Following in the Wild
di: Lior, Gili, et al.
Pubblicazione: (2025)

DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation
di: Habba, Eliya, et al.
Pubblicazione: (2025)

How Safe is Your Safety Metric? Automatic Concatenation Tests for Metric Reliability
di: Fandina, Ora Nova, et al.
Pubblicazione: (2024)

Selective Self-Rehearsal: A Fine-Tuning Approach to Improve Generalization in Large Language Models
di: Gupta, Sonam, et al.
Pubblicazione: (2024)

RSRCC: A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of-N Ranking
di: Kazoom, Roie, et al.
Pubblicazione: (2026)

A Nurse is Blue and Elephant is Rugby: Cross Domain Alignment in Large Language Models Reveal Human-like Patterns
di: Yehudai, Asaf, et al.
Pubblicazione: (2024)

Interactive Explanations for Reinforcement-Learning Agents
di: Amitai, Yotam, et al.
Pubblicazione: (2025)

Can We Govern the Agent-to-Agent Economy?
di: Chaffer, Tomer Jordi
Pubblicazione: (2025)

ParaCodex: A Profiling-Guided Autonomous Coding Agent for Reliable Parallel Code Generation and Translation
di: Kaplan, Erel, et al.
Pubblicazione: (2026)

MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning
di: Wang, Xukai, et al.
Pubblicazione: (2025)

Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality
di: Calderon, Nitay, et al.
Pubblicazione: (2026)

Stay Tuned: An Empirical Study of the Impact of Hyperparameters on LLM Tuning in Real-World Applications
di: Halfon, Alon, et al.
Pubblicazione: (2024)

Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning
di: Gerstgrasser, Matthias, et al.
Pubblicazione: (2023)

Workflows vs Agents for Code Translation
di: Gray, Henry, et al.
Pubblicazione: (2025)

Repository Intelligence Graph: Deterministic Architectural Map for LLM Code Assistants
di: Cherny-Shahar, Tsvi, et al.
Pubblicazione: (2026)

All You Need is Sally-Anne: ToM in AI Strongly Supported After Surpassing Tests for 3-Year-Olds
di: Alon, Nitay, et al.
Pubblicazione: (2025)

TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Design
di: Zhu, Haonan, et al.
Pubblicazione: (2026)