LiveAgentBench: Comprehensive Benchmarking of Agentic Systems Across 104 Real-World Challenges
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918367339741184 |
|---|---|
| author | Li, Hao Wang, Huan Gu, Jinjie Wang, Wenjie Zhuang, Chenyi Bian, Sikang |
| author_facet | Li, Hao Wang, Huan Gu, Jinjie Wang, Wenjie Zhuang, Chenyi Bian, Sikang |
| contents | As large language models grow more capable, general AI agents have become increasingly prevalent in practical applications. However, existing benchmarks face significant limitations, failing to represent real-world user tasks accurately. To address this gap, we present LiveAgentBench, a comprehensive benchmark with 104 scenarios that reflect real user requirements. It is constructed from publicly sourced questions on social media and real-world products. Central to our approach is the Social Perception-Driven Data Generation (SPDG) method, a novel process we developed to ensure each question's real-world relevance, task complexity, and result verifiability. We evaluate various models, frameworks, and commercial products using LiveAgentBench, revealing their practical performance and identifying areas for improvement. This release includes 374 tasks, with 125 for validation and 249 for testing. The SPDG process enables continuous updates with fresh queries from real-world interactions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_02586 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LiveAgentBench: Comprehensive Benchmarking of Agentic Systems Across 104 Real-World Challenges Li, Hao Wang, Huan Gu, Jinjie Wang, Wenjie Zhuang, Chenyi Bian, Sikang Artificial Intelligence As large language models grow more capable, general AI agents have become increasingly prevalent in practical applications. However, existing benchmarks face significant limitations, failing to represent real-world user tasks accurately. To address this gap, we present LiveAgentBench, a comprehensive benchmark with 104 scenarios that reflect real user requirements. It is constructed from publicly sourced questions on social media and real-world products. Central to our approach is the Social Perception-Driven Data Generation (SPDG) method, a novel process we developed to ensure each question's real-world relevance, task complexity, and result verifiability. We evaluate various models, frameworks, and commercial products using LiveAgentBench, revealing their practical performance and identifying areas for improvement. This release includes 374 tasks, with 125 for validation and 249 for testing. The SPDG process enables continuous updates with fresh queries from real-world interactions. |
| title | LiveAgentBench: Comprehensive Benchmarking of Agentic Systems Across 104 Real-World Challenges |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2603.02586 |