Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipeline
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916535569743872 |
|---|---|
| author | Zeng, Guancheng Ding, Wentao Xu, Beining Zhang, Chi Han, Wenqiang Li, Gang Mo, Jingjing Qiu, Pengxu Tao, Xinran Tao, Wang Hu, Haowen |
| author_facet | Zeng, Guancheng Ding, Wentao Xu, Beining Zhang, Chi Han, Wenqiang Li, Gang Mo, Jingjing Qiu, Pengxu Tao, Xinran Tao, Wang Hu, Haowen |
| contents | Enterprises possess a vast array of API assets scattered across various functions, forming the backbone of existing business processes. By leveraging these APIs as functional tools, enterprises can design diverse, scenario-specific agent applications, driven by on-premise function-calling models as the core engine. However, generic models often fail to meet enterprise requirements in terms of computational efficiency, output accuracy, and stability, necessitating scenario-specific adaptation. In this paper, we propose a training pipeline for function-calling capabilities tailored to real-world business scenarios. This pipeline includes the synthesis and augmentation of scenario-specific function-calling data, model fine-tuning, and performance evaluation and analysis. Using this pipeline, we generated 1,260 fully AI-generated samples and 1,035 augmented manually-labeled samples in digital HR agent scenario. The Qwen2.5-Coder-7B-Instruct model was employed as the base model and fine-tuned using the LoRA method on four GPUs with 24GB VRAM. Our fine-tuned model demonstrated outstanding performance in evaluations and practical applications, surpassing GPT-4 and GPT-4o in accuracy on the test set. These results validate the reliability of the proposed pipeline for training scenario-specific function-calling models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15660 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipeline Zeng, Guancheng Ding, Wentao Xu, Beining Zhang, Chi Han, Wenqiang Li, Gang Mo, Jingjing Qiu, Pengxu Tao, Xinran Tao, Wang Hu, Haowen Artificial Intelligence Computation and Language Software Engineering Enterprises possess a vast array of API assets scattered across various functions, forming the backbone of existing business processes. By leveraging these APIs as functional tools, enterprises can design diverse, scenario-specific agent applications, driven by on-premise function-calling models as the core engine. However, generic models often fail to meet enterprise requirements in terms of computational efficiency, output accuracy, and stability, necessitating scenario-specific adaptation. In this paper, we propose a training pipeline for function-calling capabilities tailored to real-world business scenarios. This pipeline includes the synthesis and augmentation of scenario-specific function-calling data, model fine-tuning, and performance evaluation and analysis. Using this pipeline, we generated 1,260 fully AI-generated samples and 1,035 augmented manually-labeled samples in digital HR agent scenario. The Qwen2.5-Coder-7B-Instruct model was employed as the base model and fine-tuned using the LoRA method on four GPUs with 24GB VRAM. Our fine-tuned model demonstrated outstanding performance in evaluations and practical applications, surpassing GPT-4 and GPT-4o in accuracy on the test set. These results validate the reliability of the proposed pipeline for training scenario-specific function-calling models. |
| title | Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipeline |
| topic | Artificial Intelligence Computation and Language Software Engineering |
| url | https://arxiv.org/abs/2412.15660 |