APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916302327644160 |
|---|---|
| author | Liu, Zuxin Hoang, Thai Zhang, Jianguo Zhu, Ming Lan, Tian Kokane, Shirley Tan, Juntao Yao, Weiran Liu, Zhiwei Feng, Yihao Murthy, Rithesh Yang, Liangwei Savarese, Silvio Niebles, Juan Carlos Wang, Huan Heinecke, Shelby Xiong, Caiming |
| author_facet | Liu, Zuxin Hoang, Thai Zhang, Jianguo Zhu, Ming Lan, Tian Kokane, Shirley Tan, Juntao Yao, Weiran Liu, Zhiwei Feng, Yihao Murthy, Rithesh Yang, Liangwei Savarese, Silvio Niebles, Juan Carlos Wang, Huan Heinecke, Shelby Xiong, Caiming |
| contents | The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets. This paper presents APIGen, an automated data generation pipeline designed to synthesize verifiable high-quality datasets for function-calling applications. We leverage APIGen and collect 3,673 executable APIs across 21 different categories to generate diverse function-calling datasets in a scalable and structured manner. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness. We demonstrate that models trained with our curated datasets, even with only 7B parameters, can achieve state-of-the-art performance on the Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models. Moreover, our 1B model achieves exceptional performance, surpassing GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000 high-quality entries, aiming to advance the field of function-calling agent domains. The dataset is available on Huggingface: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the project homepage: https://apigen-pipeline.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18518 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets Liu, Zuxin Hoang, Thai Zhang, Jianguo Zhu, Ming Lan, Tian Kokane, Shirley Tan, Juntao Yao, Weiran Liu, Zhiwei Feng, Yihao Murthy, Rithesh Yang, Liangwei Savarese, Silvio Niebles, Juan Carlos Wang, Huan Heinecke, Shelby Xiong, Caiming Computation and Language Artificial Intelligence Machine Learning Software Engineering The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets. This paper presents APIGen, an automated data generation pipeline designed to synthesize verifiable high-quality datasets for function-calling applications. We leverage APIGen and collect 3,673 executable APIs across 21 different categories to generate diverse function-calling datasets in a scalable and structured manner. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness. We demonstrate that models trained with our curated datasets, even with only 7B parameters, can achieve state-of-the-art performance on the Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models. Moreover, our 1B model achieves exceptional performance, surpassing GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000 high-quality entries, aiming to advance the field of function-calling agent domains. The dataset is available on Huggingface: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the project homepage: https://apigen-pipeline.github.io/ |
| title | APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets |
| topic | Computation and Language Artificial Intelligence Machine Learning Software Engineering |
| url | https://arxiv.org/abs/2406.18518 |