Cover Image

BenchAgents: Multi-Agent Systems for Structured Benchmark Creation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Butt, Natasha, Chandrasekaran, Varun, Joshi, Neel, Nushi, Besmira, Balachandran, Vidhisha
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Attention Speaks Volumes: Localizing and Mitigating Bias in Language Models
by: Adiga, Rishabh, et al.
Published: (2024)

MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation
by: Joshi, Siddharth, et al.
Published: (2025)

Improving Instruction-Following in Language Models through Activation Steering
by: Stolfo, Alessandro, et al.
Published: (2024)

Unearthing Skill-Level Insights for Understanding Trade-Offs of Foundation Models
by: Moayeri, Mazda, et al.
Published: (2024)

Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness
by: Shayegani, Erfan, et al.
Published: (2025)

Eureka: Evaluating and Understanding Large Foundation Models
by: Balachandran, Vidhisha, et al.
Published: (2024)

Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead
by: Balachandran, Vidhisha, et al.
Published: (2025)

Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
by: Yuksekgonul, Mert, et al.
Published: (2023)

Diversity of Thought Improves Reasoning Abilities of LLMs
by: Naik, Ranjita, et al.
Published: (2023)

Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models
by: Bordt, Sebastian, et al.
Published: (2024)

Bench to the Future: A Pastcasting Benchmark for Forecasting Agents
by: FutureSearch, et al.
Published: (2025)

DataSciBench: An LLM Agent Benchmark for Data Science
by: Zhang, Dan, et al.
Published: (2025)

AgentBench: Evaluating LLMs as Agents
by: Liu, Xiao, et al.
Published: (2023)

Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning
by: Vilas, Martina G., et al.
Published: (2025)

VeRO: An Evaluation Harness for Agents to Optimize Agents
by: Ursekar, Varun, et al.
Published: (2026)

EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective
by: Wang, Yuyao, et al.
Published: (2026)

LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
by: Long, Xiang, et al.
Published: (2026)

AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
by: Lù, Xing Han, et al.
Published: (2025)

Recursive Multi-Agent Systems
by: Yang, Xiyuan, et al.
Published: (2026)

AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
by: Mazumder, Aritra, et al.
Published: (2026)

Geak: Introducing Triton Kernel AI Agent & Evaluation Benchmarks
by: Wang, Jianghui, et al.
Published: (2025)

Latent Collaboration in Multi-Agent Systems
by: Zou, Jiaru, et al.
Published: (2025)

ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
by: Chen, Ziru, et al.
Published: (2024)

MCP-AgentBench: Evaluating Real-World Language Agent Performance with MCP-Mediated Tools
by: Guo, Zikang, et al.
Published: (2025)

From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization
by: Ji, Haonian, et al.
Published: (2025)

IntellAgent: A Multi-Agent Framework for Evaluating Conversational AI Systems
by: Levi, Elad, et al.
Published: (2025)

AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
by: Andriushchenko, Maksym, et al.
Published: (2024)

RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code
by: Gautam, Dhruv, et al.
Published: (2025)

VoiceAgentBench: Are Voice Assistants ready for agentic tasks?
by: Jain, Dhruv, et al.
Published: (2025)

UserBench: An Interactive Gym Environment for User-Centric Agents
by: Qian, Cheng, et al.
Published: (2025)

LiveBench: A Challenging, Contamination-Limited LLM Benchmark
by: White, Colin, et al.
Published: (2024)

AI Knowledge Assist: An Automated Approach for the Creation of Knowledge Bases for Conversational AI Agents
by: Laskar, Md Tahmid Rahman, et al.
Published: (2025)

Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies
by: Tang, Zirui, et al.
Published: (2026)

Soft Tokens, Hard Truths
by: Butt, Natasha, et al.
Published: (2025)

ProcBench: Benchmark for Multi-Step Reasoning and Following Procedure
by: Fujisawa, Ippei, et al.
Published: (2024)

EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents
by: Qian, Cheng, et al.
Published: (2024)

AgentKernelArena: Generalization-Aware Benchmarking of GPU Kernel Optimization Agents
by: Younesian, Sharareh, et al.
Published: (2026)

MAS-ZERO: Designing Multi-Agent Systems with Zero Supervision
by: Ke, Zixuan, et al.
Published: (2025)

MASEval: Extending Multi-Agent Evaluation from Models to Systems
by: Emde, Cornelius, et al.
Published: (2026)

\$OneMillion-Bench: How Far are Language Agents from Human Experts?
by: Yang, Qianyu, et al.
Published: (2026)