Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2412.05366 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912439073767424 |
|---|---|
| author | Wang, Yunkun Zhang, Yue Qin, Zhen Zhi, Chen Li, Binhua Huang, Fei Li, Yongbin Deng, Shuiguang |
| author_facet | Wang, Yunkun Zhang, Yue Qin, Zhen Zhi, Chen Li, Binhua Huang, Fei Li, Yongbin Deng, Shuiguang |
| contents | Large language models face intrinsic limitations in coding with APIs that are unseen in their training corpora. As libraries continuously evolve, it becomes impractical to exhaustively retrain LLMs with new API knowledge. This limitation hampers LLMs from solving programming problems which require newly introduced or privately maintained libraries. Inspired by exploratory programming paradigm in human behavior, we propose ExploraCoder, a training-free framework that empowers LLMs to invoke multiple unseen APIs in code solution by (1) planning a complex problem into several API invocation subtasks, and (2) experimenting with correct API usage at intermediate steps through a novel chain-of-API-exploration. We conduct evaluation on program synthesizing tasks involving complex API interactions. Experimental results demonstrate that ExploraCoder significantly improves performance for models lacking prior API knowledge, achieving absolute increases of up to 11.99% over retrieval-based approaches and 17.28% over pretraining-based methods in pass@10. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_05366 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ExploraCoder: Advancing code generation for multiple unseen APIs via planning and chained exploration Wang, Yunkun Zhang, Yue Qin, Zhen Zhi, Chen Li, Binhua Huang, Fei Li, Yongbin Deng, Shuiguang Software Engineering Large language models face intrinsic limitations in coding with APIs that are unseen in their training corpora. As libraries continuously evolve, it becomes impractical to exhaustively retrain LLMs with new API knowledge. This limitation hampers LLMs from solving programming problems which require newly introduced or privately maintained libraries. Inspired by exploratory programming paradigm in human behavior, we propose ExploraCoder, a training-free framework that empowers LLMs to invoke multiple unseen APIs in code solution by (1) planning a complex problem into several API invocation subtasks, and (2) experimenting with correct API usage at intermediate steps through a novel chain-of-API-exploration. We conduct evaluation on program synthesizing tasks involving complex API interactions. Experimental results demonstrate that ExploraCoder significantly improves performance for models lacking prior API knowledge, achieving absolute increases of up to 11.99% over retrieval-based approaches and 17.28% over pretraining-based methods in pass@10. |
| title | ExploraCoder: Advancing code generation for multiple unseen APIs via planning and chained exploration |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2412.05366 |