Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.27977 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917453332742144 |
|---|---|
| author | Moussa, Hanane Nour Li, Yifei Li, Zhuoyang Yang, Yankai Tang, Cheng Zhang, Tianshu Ahmed, Nesreen K. Payani, Ali Chen, Ziru Sun, Huan |
| author_facet | Moussa, Hanane Nour Li, Yifei Li, Zhuoyang Yang, Yankai Tang, Cheng Zhang, Tianshu Ahmed, Nesreen K. Payani, Ali Chen, Ziru Sun, Huan |
| contents | Despite recent progress in language models and agents for scientific data-driven discovery, further advancing their capabilities is held back by the absence of verifiable environments representing real-world scientific tasks. To fill this gap, we introduce D3-Gym, the first automatically constructed dataset with verifiable environments for scientific Data-Driven Discovery. D3-Gym comprises (1) 565 tasks sourced from 239 real scientific repositories across four disciplines where (2) each task is equipped with a natural language instruction, an executable environment with pre-installed dependencies, input dataset and artifact previews, a reference code solution, and an automatically synthesized evaluation script. Rigorous evaluation of the quality of the verification signal in D3-Gym confirms that our evaluation scripts achieve 87.5% agreement with human-annotated gold standards and strong alignment in domain-specific evaluation logic, showing their scientific soundness. Further, training on trajectories sampled from D3-Gym yields consistent and substantial gains across Qwen3 models of varying sizes on ScienceAgentBench, boosting Qwen3-32B by 7.8 absolute points and substantially shrinking the gap with strong proprietary models. All D3-Gym artifacts (environments, creation workflow, trajectories, and models) can be found at https://github.com/OSU-NLP-Group/D3-Gym. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_27977 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery Moussa, Hanane Nour Li, Yifei Li, Zhuoyang Yang, Yankai Tang, Cheng Zhang, Tianshu Ahmed, Nesreen K. Payani, Ali Chen, Ziru Sun, Huan Artificial Intelligence Machine Learning Despite recent progress in language models and agents for scientific data-driven discovery, further advancing their capabilities is held back by the absence of verifiable environments representing real-world scientific tasks. To fill this gap, we introduce D3-Gym, the first automatically constructed dataset with verifiable environments for scientific Data-Driven Discovery. D3-Gym comprises (1) 565 tasks sourced from 239 real scientific repositories across four disciplines where (2) each task is equipped with a natural language instruction, an executable environment with pre-installed dependencies, input dataset and artifact previews, a reference code solution, and an automatically synthesized evaluation script. Rigorous evaluation of the quality of the verification signal in D3-Gym confirms that our evaluation scripts achieve 87.5% agreement with human-annotated gold standards and strong alignment in domain-specific evaluation logic, showing their scientific soundness. Further, training on trajectories sampled from D3-Gym yields consistent and substantial gains across Qwen3 models of varying sizes on ScienceAgentBench, boosting Qwen3-32B by 7.8 absolute points and substantially shrinking the gap with strong proprietary models. All D3-Gym artifacts (environments, creation workflow, trajectories, and models) can be found at https://github.com/OSU-NLP-Group/D3-Gym. |
| title | D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2604.27977 |