Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.16149 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915684331552768 |
|---|---|
| author | Chen, Hao Hu, Zhexin Chai, Jiajun Yang, Haocheng He, Hang Wang, Xiaohan Lin, Wei Wang, Luhang Yin, Guojun zhao, Zhuofeng |
| author_facet | Chen, Hao Hu, Zhexin Chai, Jiajun Yang, Haocheng He, Hang Wang, Xiaohan Lin, Wei Wang, Luhang Yin, Guojun zhao, Zhuofeng |
| contents | Training LLMs to invoke tools and leverage retrieved information necessitates high-quality, diverse data. However, existing pipelines for synthetic data generation often rely on tens of thousands of real API calls to enhance generalization, incurring prohibitive costs while lacking multi-hop reasoning and self-reflection. To address these limitations, we introduce ToolForge, an automated synthesis framework that achieves strong real-world tool-calling performance by constructing only a small number of virtual tools, eliminating the need for real API calls. ToolForge leverages a (question, golden context, answer) triple to synthesize large-scale tool-learning data specifically designed for multi-hop search scenarios, further enriching the generated data through multi-hop reasoning and self-reflection mechanisms. To ensure data fidelity, we employ a Multi-Layer Validation Framework that integrates both rule-based and model-based assessments. Empirical results show that a model with only 8B parameters, when trained on our synthesized data, outperforms GPT-4o on multiple benchmarks. Our code and dataset are publicly available at https://github.com/Buycar-arb/ToolForge . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16149 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ToolForge: A Data Synthesis Pipeline for Multi-Hop Search without Real-World APIs Chen, Hao Hu, Zhexin Chai, Jiajun Yang, Haocheng He, Hang Wang, Xiaohan Lin, Wei Wang, Luhang Yin, Guojun zhao, Zhuofeng Artificial Intelligence I.2.7; I.2.8; H.3.3 Training LLMs to invoke tools and leverage retrieved information necessitates high-quality, diverse data. However, existing pipelines for synthetic data generation often rely on tens of thousands of real API calls to enhance generalization, incurring prohibitive costs while lacking multi-hop reasoning and self-reflection. To address these limitations, we introduce ToolForge, an automated synthesis framework that achieves strong real-world tool-calling performance by constructing only a small number of virtual tools, eliminating the need for real API calls. ToolForge leverages a (question, golden context, answer) triple to synthesize large-scale tool-learning data specifically designed for multi-hop search scenarios, further enriching the generated data through multi-hop reasoning and self-reflection mechanisms. To ensure data fidelity, we employ a Multi-Layer Validation Framework that integrates both rule-based and model-based assessments. Empirical results show that a model with only 8B parameters, when trained on our synthesized data, outperforms GPT-4o on multiple benchmarks. Our code and dataset are publicly available at https://github.com/Buycar-arb/ToolForge . |
| title | ToolForge: A Data Synthesis Pipeline for Multi-Hop Search without Real-World APIs |
| topic | Artificial Intelligence I.2.7; I.2.8; H.3.3 |
| url | https://arxiv.org/abs/2512.16149 |