API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910453125349376 |
|---|---|
| author | Basu, Kinjal Abdelaziz, Ibrahim Chaudhury, Subhajit Dan, Soham Crouse, Maxwell Munawar, Asim Kumaravel, Sadhana Muthusamy, Vinod Kapanipathi, Pavan Lastras, Luis A. |
| author_facet | Basu, Kinjal Abdelaziz, Ibrahim Chaudhury, Subhajit Dan, Soham Crouse, Maxwell Munawar, Asim Kumaravel, Sadhana Muthusamy, Vinod Kapanipathi, Pavan Lastras, Luis A. |
| contents | There is a growing need for Large Language Models (LLMs) to effectively use tools and external Application Programming Interfaces (APIs) to plan and complete tasks. As such, there is tremendous interest in methods that can acquire sufficient quantities of train and test data that involve calls to tools / APIs. Two lines of research have emerged as the predominant strategies for addressing this challenge. The first has focused on synthetic data generation techniques, while the second has involved curating task-adjacent datasets which can be transformed into API / Tool-based tasks. In this paper, we focus on the task of identifying, curating, and transforming existing datasets and, in turn, introduce API-BLEND, a large corpora for training and systematic testing of tool-augmented LLMs. The datasets mimic real-world scenarios involving API-tasks such as API / tool detection, slot filling, and sequencing of the detected APIs. We demonstrate the utility of the API-BLEND dataset for both training and benchmarking purposes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_15491 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs Basu, Kinjal Abdelaziz, Ibrahim Chaudhury, Subhajit Dan, Soham Crouse, Maxwell Munawar, Asim Kumaravel, Sadhana Muthusamy, Vinod Kapanipathi, Pavan Lastras, Luis A. Computation and Language Artificial Intelligence There is a growing need for Large Language Models (LLMs) to effectively use tools and external Application Programming Interfaces (APIs) to plan and complete tasks. As such, there is tremendous interest in methods that can acquire sufficient quantities of train and test data that involve calls to tools / APIs. Two lines of research have emerged as the predominant strategies for addressing this challenge. The first has focused on synthetic data generation techniques, while the second has involved curating task-adjacent datasets which can be transformed into API / Tool-based tasks. In this paper, we focus on the task of identifying, curating, and transforming existing datasets and, in turn, introduce API-BLEND, a large corpora for training and systematic testing of tool-augmented LLMs. The datasets mimic real-world scenarios involving API-tasks such as API / tool detection, slot filling, and sequencing of the detected APIs. We demonstrate the utility of the API-BLEND dataset for both training and benchmarking purposes. |
| title | API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.15491 |