SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910274933489664 |
|---|---|
| author | Shen, Yujiong Yang, Yajie Xi, Zhiheng Hu, Binze Sha, Huayu Zhang, Jiazheng Peng, Qiyuan Shang, Junlin Huang, Jixuan Fan, Yutao Tong, Jingqi Dou, Shihan Zhang, Ming Bai, Lei Yin, Zhenfei Gui, Tao Ma, Xingjun Zhang, Qi Huang, Xuanjing Jiang, Yu-Gang |
| author_facet | Shen, Yujiong Yang, Yajie Xi, Zhiheng Hu, Binze Sha, Huayu Zhang, Jiazheng Peng, Qiyuan Shang, Junlin Huang, Jixuan Fan, Yutao Tong, Jingqi Dou, Shihan Zhang, Ming Bai, Lei Yin, Zhenfei Gui, Tao Ma, Xingjun Zhang, Qi Huang, Xuanjing Jiang, Yu-Gang |
| contents | Scientific reasoning inherently demands integrating sophisticated toolkits to navigate domain-specific knowledge. Yet, current benchmarks largely overlook agents' ability to orchestrate tools for such rigorous workflows. To bridge this gap, we introduce SciAgentGym, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines, supported by a robust execution infrastructure. Complementing this, we present SciAgentBench, a tiered evaluation suite designed to stress-test agentic capabilities from elementary actions to long-horizon workflows. Our evaluation identifies a critical bottleneck: state-of-the-art models still struggle with complex scientific tool-use, and their performance degrades substantially as interaction horizons extend. To address this, we propose SciForge, a data synthesis method that models the tool action space as a dependency graph to generate logic-aware training trajectories. By fine-tuning on these trajectories, our SciAgent-8B outperforms the significantly larger Qwen3-VL-235B-Instruct while exhibiting positive cross-domain transfer of scientific tool-use capabilities. These results underscore the promising potential of next-generation autonomous scientific agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_12984 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents Shen, Yujiong Yang, Yajie Xi, Zhiheng Hu, Binze Sha, Huayu Zhang, Jiazheng Peng, Qiyuan Shang, Junlin Huang, Jixuan Fan, Yutao Tong, Jingqi Dou, Shihan Zhang, Ming Bai, Lei Yin, Zhenfei Gui, Tao Ma, Xingjun Zhang, Qi Huang, Xuanjing Jiang, Yu-Gang Computation and Language Scientific reasoning inherently demands integrating sophisticated toolkits to navigate domain-specific knowledge. Yet, current benchmarks largely overlook agents' ability to orchestrate tools for such rigorous workflows. To bridge this gap, we introduce SciAgentGym, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines, supported by a robust execution infrastructure. Complementing this, we present SciAgentBench, a tiered evaluation suite designed to stress-test agentic capabilities from elementary actions to long-horizon workflows. Our evaluation identifies a critical bottleneck: state-of-the-art models still struggle with complex scientific tool-use, and their performance degrades substantially as interaction horizons extend. To address this, we propose SciForge, a data synthesis method that models the tool action space as a dependency graph to generate logic-aware training trajectories. By fine-tuning on these trajectories, our SciAgent-8B outperforms the significantly larger Qwen3-VL-235B-Instruct while exhibiting positive cross-domain transfer of scientific tool-use capabilities. These results underscore the promising potential of next-generation autonomous scientific agents. |
| title | SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2602.12984 |