AutoFeedback: An LLM-based Framework for Efficient and Accurate API Request Generation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929533886660608 |
|---|---|
| author | Liu, Huanxi Liao, Jiaqi Feng, Dawei Xu, Kele Wang, Huaimin |
| author_facet | Liu, Huanxi Liao, Jiaqi Feng, Dawei Xu, Kele Wang, Huaimin |
| contents | Large Language Models (LLMs) leverage external tools primarily through generating the API request to enhance task completion efficiency. The accuracy of API request generation significantly determines the capability of LLMs to accomplish tasks.
Due to the inherent hallucinations within the LLM, it is difficult to efficiently and accurately generate the correct API request.
Current research uses prompt-based feedback to facilitate the LLM-based API request generation. However, existing methods lack factual information and are insufficiently detailed.
To address these issues, we propose AutoFeedback, an LLM-based framework for efficient and accurate API request generation, with a Static Scanning Component (SSC) and a Dynamic Analysis Component (DAC). SSC incorporates errors detected in the API requests as pseudo-facts into the feedback, enriching the factual information. DAC retrieves information from API documentation, enhancing the level of detail in feedback.
Based on this two components, Autofeedback implementes two feedback loops during the process of generating API requests by the LLM.
Extensive experiments demonstrate that it significantly improves accuracy of API request generation and reduces the interaction cost. AutoFeedback achieves an accuracy of 100.00\% on a real-world API dataset and reduces the cost of interaction with GPT-3.5 Turbo by 23.44\%, and GPT-4 Turbo by 11.85\%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06943 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AutoFeedback: An LLM-based Framework for Efficient and Accurate API Request Generation Liu, Huanxi Liao, Jiaqi Feng, Dawei Xu, Kele Wang, Huaimin Software Engineering Artificial Intelligence Large Language Models (LLMs) leverage external tools primarily through generating the API request to enhance task completion efficiency. The accuracy of API request generation significantly determines the capability of LLMs to accomplish tasks. Due to the inherent hallucinations within the LLM, it is difficult to efficiently and accurately generate the correct API request. Current research uses prompt-based feedback to facilitate the LLM-based API request generation. However, existing methods lack factual information and are insufficiently detailed. To address these issues, we propose AutoFeedback, an LLM-based framework for efficient and accurate API request generation, with a Static Scanning Component (SSC) and a Dynamic Analysis Component (DAC). SSC incorporates errors detected in the API requests as pseudo-facts into the feedback, enriching the factual information. DAC retrieves information from API documentation, enhancing the level of detail in feedback. Based on this two components, Autofeedback implementes two feedback loops during the process of generating API requests by the LLM. Extensive experiments demonstrate that it significantly improves accuracy of API request generation and reduces the interaction cost. AutoFeedback achieves an accuracy of 100.00\% on a real-world API dataset and reduces the cost of interaction with GPT-3.5 Turbo by 23.44\%, and GPT-4 Turbo by 11.85\%. |
| title | AutoFeedback: An LLM-based Framework for Efficient and Accurate API Request Generation |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2410.06943 |