Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.21244 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915617619050496 |
|---|---|
| author | Xu, Pengyu Li, Shijia Sun, Ao Zhang, Feng Li, Yahan Wu, Bo Ma, Zhanyu Li, Jiguo Xu, Jun Gao, Jiuchong Hao, Jinghua He, Renqing Wang, Rui Liu, Yang Hu, Xiaobo Yang, Fan Zheng, Jia Yao, Guanghua |
| author_facet | Xu, Pengyu Li, Shijia Sun, Ao Zhang, Feng Li, Yahan Wu, Bo Ma, Zhanyu Li, Jiguo Xu, Jun Gao, Jiuchong Hao, Jinghua He, Renqing Wang, Rui Liu, Yang Hu, Xiaobo Yang, Fan Zheng, Jia Yao, Guanghua |
| contents | We propose OutboundEval, a comprehensive benchmark for evaluating large language models (LLMs) in expert-level intelligent outbound calling scenarios. Unlike existing methods that suffer from three key limitations - insufficient dataset diversity and category coverage, unrealistic user simulation, and inaccurate evaluation metrics - OutboundEval addresses these issues through a structured framework. First, we design a benchmark spanning six major business domains and 30 representative sub-scenarios, each with scenario-specific process decomposition, weighted scoring, and domain-adaptive metrics. Second, we develop a large-model-driven User Simulator that generates diverse, persona-rich virtual users with realistic behaviors, emotional variability, and communication styles, providing a controlled yet authentic testing environment. Third, we introduce a dynamic evaluation method that adapts to task variations, integrating automated and human-in-the-loop assessment to measure task execution accuracy, professional knowledge application, adaptability, and user experience quality. Experiments on 12 state-of-the-art LLMs reveal distinct trade-offs between expert-level task completion and interaction fluency, offering practical insights for building reliable, human-like outbound AI systems. OutboundEval establishes a practical, extensible, and domain-oriented standard for benchmarking LLMs in professional applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_21244 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VoiceAgentEval: A Dual-Dimensional Benchmark for Expert-Level Intelligent Voice-Agent Evaluation of Xbench's Professional-Aligned Series Xu, Pengyu Li, Shijia Sun, Ao Zhang, Feng Li, Yahan Wu, Bo Ma, Zhanyu Li, Jiguo Xu, Jun Gao, Jiuchong Hao, Jinghua He, Renqing Wang, Rui Liu, Yang Hu, Xiaobo Yang, Fan Zheng, Jia Yao, Guanghua Artificial Intelligence We propose OutboundEval, a comprehensive benchmark for evaluating large language models (LLMs) in expert-level intelligent outbound calling scenarios. Unlike existing methods that suffer from three key limitations - insufficient dataset diversity and category coverage, unrealistic user simulation, and inaccurate evaluation metrics - OutboundEval addresses these issues through a structured framework. First, we design a benchmark spanning six major business domains and 30 representative sub-scenarios, each with scenario-specific process decomposition, weighted scoring, and domain-adaptive metrics. Second, we develop a large-model-driven User Simulator that generates diverse, persona-rich virtual users with realistic behaviors, emotional variability, and communication styles, providing a controlled yet authentic testing environment. Third, we introduce a dynamic evaluation method that adapts to task variations, integrating automated and human-in-the-loop assessment to measure task execution accuracy, professional knowledge application, adaptability, and user experience quality. Experiments on 12 state-of-the-art LLMs reveal distinct trade-offs between expert-level task completion and interaction fluency, offering practical insights for building reliable, human-like outbound AI systems. OutboundEval establishes a practical, extensible, and domain-oriented standard for benchmarking LLMs in professional applications. |
| title | VoiceAgentEval: A Dual-Dimensional Benchmark for Expert-Level Intelligent Voice-Agent Evaluation of Xbench's Professional-Aligned Series |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.21244 |