SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908874863280128 |
|---|---|
| author | Chen, Shiqi Gai, Jingze Zhou, Ruochen Zhang, Jinghan Zhu, Tongyao Li, Junlong Wang, Kangrui Wang, Zihan Chen, Zhengyu Kaleb, Klara Miao, Ning Gao, Siyang Lu, Cong Li, Manling He, Junxian Teh, Yee Whye |
| author_facet | Chen, Shiqi Gai, Jingze Zhou, Ruochen Zhang, Jinghan Zhu, Tongyao Li, Junlong Wang, Kangrui Wang, Zihan Chen, Zhengyu Kaleb, Klara Miao, Ning Gao, Siyang Lu, Cong Li, Manling He, Junxian Teh, Yee Whye |
| contents | Real-world tool-using agents operate over long-horizon workflows with recurring structure and diverse demands, where effective behavior requires not only invoking atomic tools but also abstracting, and reusing higher-level tool compositions. However, existing benchmarks mainly measure instance-level success under static tool sets, offering limited insight into agents' ability to acquire such reusable skills. We address this gap by introducing SkillCraft, a benchmark explicitly stress-test agent ability to form and reuse higher-level tool compositions, where we call Skills. SkillCraft features realistic, highly compositional tool-use scenarios with difficulty scaled along both quantitative and structural dimensions, designed to elicit skill abstraction and cross-task reuse. We further propose a lightweight evaluation protocol that enables agents to auto-compose atomic tools into executable Skills, cache and reuse them inside and across tasks, thereby improving efficiency while accumulating a persistent library of reusable skills. Evaluating state-of-the-art agents on SkillCraft, we observe substantial efficiency gains, with token usage reduced by up to 80% by skill saving and reuse. Moreover, success rate strongly correlates with tool composition ability at test time, underscoring compositional skill acquisition as a core capability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_00718 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SkillCraft: Can LLM Agents Learn to Use Tools Skillfully? Chen, Shiqi Gai, Jingze Zhou, Ruochen Zhang, Jinghan Zhu, Tongyao Li, Junlong Wang, Kangrui Wang, Zihan Chen, Zhengyu Kaleb, Klara Miao, Ning Gao, Siyang Lu, Cong Li, Manling He, Junxian Teh, Yee Whye Computation and Language Software Engineering Real-world tool-using agents operate over long-horizon workflows with recurring structure and diverse demands, where effective behavior requires not only invoking atomic tools but also abstracting, and reusing higher-level tool compositions. However, existing benchmarks mainly measure instance-level success under static tool sets, offering limited insight into agents' ability to acquire such reusable skills. We address this gap by introducing SkillCraft, a benchmark explicitly stress-test agent ability to form and reuse higher-level tool compositions, where we call Skills. SkillCraft features realistic, highly compositional tool-use scenarios with difficulty scaled along both quantitative and structural dimensions, designed to elicit skill abstraction and cross-task reuse. We further propose a lightweight evaluation protocol that enables agents to auto-compose atomic tools into executable Skills, cache and reuse them inside and across tasks, thereby improving efficiency while accumulating a persistent library of reusable skills. Evaluating state-of-the-art agents on SkillCraft, we observe substantial efficiency gains, with token usage reduced by up to 80% by skill saving and reuse. Moreover, success rate strongly correlates with tool composition ability at test time, underscoring compositional skill acquisition as a core capability. |
| title | SkillCraft: Can LLM Agents Learn to Use Tools Skillfully? |
| topic | Computation and Language Software Engineering |
| url | https://arxiv.org/abs/2603.00718 |