Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2506.13651 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911007536840704 |
|---|---|
| author | Chen, Kaiyuan Ren, Yixin Liu, Yang Hu, Xiaobo Tian, Haotong Xie, Tianbao Liu, Fangfu Zhang, Haoye Liu, Hongzhang Gong, Yuan Sun, Chen Hou, Han Yang, Hui Pan, James Lou, Jianan Mao, Jiayi Liu, Jizheng Li, Jinpeng Liu, Kangyi Liu, Kenkun Wang, Rui Li, Run Niu, Tong Zhang, Wenlong Yan, Wenqi Wang, Xuanzheng Zhang, Yuchen Hung, Yi-Hsin Jiang, Yuan Liu, Zexuan Yin, Zihan Ma, Zijian Mo, Zhiwen |
| author_facet | Chen, Kaiyuan Ren, Yixin Liu, Yang Hu, Xiaobo Tian, Haotong Xie, Tianbao Liu, Fangfu Zhang, Haoye Liu, Hongzhang Gong, Yuan Sun, Chen Hou, Han Yang, Hui Pan, James Lou, Jianan Mao, Jiayi Liu, Jizheng Li, Jinpeng Liu, Kangyi Liu, Kenkun Wang, Rui Li, Run Niu, Tong Zhang, Wenlong Yan, Wenqi Wang, Xuanzheng Zhang, Yuchen Hung, Yi-Hsin Jiang, Yuan Liu, Zexuan Yin, Zihan Ma, Zijian Mo, Zhiwen |
| contents | We introduce xbench, a dynamic, profession-aligned evaluation suite designed to bridge the gap between AI agent capabilities and real-world productivity. While existing benchmarks often focus on isolated technical skills, they may not accurately reflect the economic value agents deliver in professional settings. To address this, xbench targets commercially significant domains with evaluation tasks defined by industry professionals. Our framework creates metrics that strongly correlate with productivity value, enables prediction of Technology-Market Fit (TMF), and facilitates tracking of product capabilities over time. As our initial implementations, we present two benchmarks: Recruitment and Marketing. For Recruitment, we collect 50 tasks from real-world headhunting business scenarios to evaluate agents' abilities in company mapping, information retrieval, and talent sourcing. For Marketing, we assess agents' ability to match influencers with advertiser needs, evaluating their performance across 50 advertiser requirements using a curated pool of 836 candidate influencers. We present initial evaluation results for leading contemporary agents, establishing a baseline for these professional domains. Our continuously updated evalsets and evaluations are available at https://xbench.org. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_13651 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations Chen, Kaiyuan Ren, Yixin Liu, Yang Hu, Xiaobo Tian, Haotong Xie, Tianbao Liu, Fangfu Zhang, Haoye Liu, Hongzhang Gong, Yuan Sun, Chen Hou, Han Yang, Hui Pan, James Lou, Jianan Mao, Jiayi Liu, Jizheng Li, Jinpeng Liu, Kangyi Liu, Kenkun Wang, Rui Li, Run Niu, Tong Zhang, Wenlong Yan, Wenqi Wang, Xuanzheng Zhang, Yuchen Hung, Yi-Hsin Jiang, Yuan Liu, Zexuan Yin, Zihan Ma, Zijian Mo, Zhiwen Machine Learning We introduce xbench, a dynamic, profession-aligned evaluation suite designed to bridge the gap between AI agent capabilities and real-world productivity. While existing benchmarks often focus on isolated technical skills, they may not accurately reflect the economic value agents deliver in professional settings. To address this, xbench targets commercially significant domains with evaluation tasks defined by industry professionals. Our framework creates metrics that strongly correlate with productivity value, enables prediction of Technology-Market Fit (TMF), and facilitates tracking of product capabilities over time. As our initial implementations, we present two benchmarks: Recruitment and Marketing. For Recruitment, we collect 50 tasks from real-world headhunting business scenarios to evaluate agents' abilities in company mapping, information retrieval, and talent sourcing. For Marketing, we assess agents' ability to match influencers with advertiser needs, evaluating their performance across 50 advertiser requirements using a curated pool of 836 candidate influencers. We present initial evaluation results for leading contemporary agents, establishing a baseline for these professional domains. Our continuously updated evalsets and evaluations are available at https://xbench.org. |
| title | xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.13651 |