Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.02503 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911722454908928 |
|---|---|
| author | Zhang, Qiaohong Ye, Weihao Chen, Jialong Luo, Yi Li, BoYuan Deng, Bowen Zheng, Zibin Lin, Jianhao Zheng, Wei-Shi Chen, Chuan |
| author_facet | Zhang, Qiaohong Ye, Weihao Chen, Jialong Luo, Yi Li, BoYuan Deng, Bowen Zheng, Zibin Lin, Jianhao Zheng, Wei-Shi Chen, Chuan |
| contents | Autonomous data analysis agents are increasingly expected to conduct exploratory analysis with limited human guidance about data. However, existing benchmarks typically evaluate such agents in prior-guided settings, providing selected data sources, explicit data schemas, or cleaned data, thereby understating the exploratory burden. To evaluate this realistic exploratory data analysis task, we introduce DataClawBench, a benchmark built from financial think-tank consulting scenarios where agents must independently explore unfamiliar, noisy, cross-domain data and produce verifiable conclusions. DataClawBench provides a unified real-world data environment with approximately 2.06 million records across enterprise, industry, and policy domains, with native data noise preserved. On top of this data environment, it defines 492 multi-step cross-domain tasks, each annotated with intermediate milestones that diagnose exploration and reasoning failures beyond outcome accuracy. A systematic evaluation of eight advanced LLMs under the OpenClaw agent reveals that exploratory data analysis breaks agent reliability: more exploration does not reliably translate into task-relevant progress or correct final answers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_02503 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DataClawBench: An Agent Benchmark for Exploratory Real-World Financial Data Analysis Zhang, Qiaohong Ye, Weihao Chen, Jialong Luo, Yi Li, BoYuan Deng, Bowen Zheng, Zibin Lin, Jianhao Zheng, Wei-Shi Chen, Chuan Artificial Intelligence Autonomous data analysis agents are increasingly expected to conduct exploratory analysis with limited human guidance about data. However, existing benchmarks typically evaluate such agents in prior-guided settings, providing selected data sources, explicit data schemas, or cleaned data, thereby understating the exploratory burden. To evaluate this realistic exploratory data analysis task, we introduce DataClawBench, a benchmark built from financial think-tank consulting scenarios where agents must independently explore unfamiliar, noisy, cross-domain data and produce verifiable conclusions. DataClawBench provides a unified real-world data environment with approximately 2.06 million records across enterprise, industry, and policy domains, with native data noise preserved. On top of this data environment, it defines 492 multi-step cross-domain tasks, each annotated with intermediate milestones that diagnose exploration and reasoning failures beyond outcome accuracy. A systematic evaluation of eight advanced LLMs under the OpenClaw agent reveals that exploratory data analysis breaks agent reliability: more exploration does not reliably translate into task-relevant progress or correct final answers. |
| title | DataClawBench: An Agent Benchmark for Exploratory Real-World Financial Data Analysis |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2605.02503 |