Cover Image

Evaluation and Benchmarking of LLM Agents: A Survey

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mohammadi, Mahmoud, Li, Yipeng, Lo, Jane, Yip, Wendy
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Survey on Evaluation of LLM-based Agents
by: Yehudai, Asaf, et al.
Published: (2025)

TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Tasks
by: Weng, Muyan, et al.
Published: (2026)

AUTOCT: Automating Interpretable Clinical Trial Prediction with LLM Agents
by: Liu, Fengze, et al.
Published: (2025)

The Evaluation Game: Beyond Static LLM Benchmarking
by: Wang, Paul, et al.
Published: (2026)

FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction
by: Zeng, Zhiyuan, et al.
Published: (2025)

AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
by: Andriushchenko, Maksym, et al.
Published: (2024)

Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
by: Thaman, Kunvar
Published: (2026)

RxEval: A Prescription-Level Benchmark for Evaluating LLM Medication Recommendation
by: Chen, Shuhao, et al.
Published: (2026)

BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks
by: Anupam, Sagnik, et al.
Published: (2025)

MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness
by: Hathidara, Ashutosh, et al.
Published: (2026)

Evaluating Long-Context Reasoning in LLM-Based WebAgents
by: Chung, Andy, et al.
Published: (2025)

Evaluating LLM Agent Adherence to Hierarchical Safety Principles: A Lightweight Benchmark for Probing Foundational Controllability Components
by: Potham, Ram
Published: (2025)

DataSciBench: An LLM Agent Benchmark for Data Science
by: Zhang, Dan, et al.
Published: (2025)

OPT-BENCH: Evaluating LLM Agent on Large-Scale Search Spaces Optimization Problems
by: Li, Xiaozhe, et al.
Published: (2025)

SmartPlay: A Benchmark for LLMs as Intelligent Agents
by: Wu, Yue, et al.
Published: (2023)

The LLM Data Auditor: A Metric-oriented Survey on Quality and Trustworthiness in Evaluating Synthetic Data
by: Zhang, Kaituo, et al.
Published: (2026)

MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents
by: Rosen, Simon, et al.
Published: (2026)

From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning
by: Yoa, Seungdong, et al.
Published: (2026)

On the Importance of Task Complexity in Evaluating LLM-Based Multi-Agent Systems
by: Tang, Bohan, et al.
Published: (2025)

A Survey on Code Generation with LLM-based Agents
by: Dong, Yihong, et al.
Published: (2025)

MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents
by: Jiang, Yixing, et al.
Published: (2025)

BalanceBenchmark: A Survey for Multimodal Imbalance Learning
by: Xu, Shaoxuan, et al.
Published: (2025)

JudgeBench: A Benchmark for Evaluating LLM-based Judges
by: Tan, Sijun, et al.
Published: (2024)

Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments
by: Vishwakarma, Harsh, et al.
Published: (2025)

GenoTEX: An LLM Agent Benchmark for Automated Gene Expression Data Analysis
by: Liu, Haoyang, et al.
Published: (2024)

AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
by: Rawles, Christopher, et al.
Published: (2024)

RouterBench: A Benchmark for Multi-LLM Routing System
by: Hu, Qitian Jason, et al.
Published: (2024)

Geak: Introducing Triton Kernel AI Agent & Evaluation Benchmarks
by: Wang, Jianghui, et al.
Published: (2025)

Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking
by: Xu, Yang, et al.
Published: (2026)

LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
by: Long, Xiang, et al.
Published: (2026)

Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insights
by: Parashar, Shubham, et al.
Published: (2025)

Integrating Temporal and Structural Context in Graph Transformers for Relational Deep Learning
by: Lachi, Divyansha, et al.
Published: (2025)

WAREX: Web Agent Reliability Evaluation on Existing Benchmarks
by: Kara, Su, et al.
Published: (2025)

BERT-LSH: Reducing Absolute Compute For Attention
by: Li, Zezheng, et al.
Published: (2024)

Learning Multi-Agent Communication with Contrastive Learning
by: Lo, Yat Long, et al.
Published: (2023)

HEARTS: Benchmarking LLM Reasoning on Health Time Series
by: Li, Sirui, et al.
Published: (2026)

Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities
by: Anurin, Andrey, et al.
Published: (2024)

AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
by: Ma, Chang, et al.
Published: (2024)

From Time Series Analysis to Question Answering: A Survey in the LLM Era
by: Li, Wei, et al.
Published: (2025)

Optimal Singular Damage: Efficient LLM Inference in Low Storage Regimes
by: Alipour, Mohammadsajad, et al.
Published: (2025)