AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909651594903552 |
|---|---|
| author | Qiu, Jiahao Juan, Xinzhe Wang, Yimin Yang, Ling Qi, Xuan Zhang, Tongcheng Guo, Jiacheng Lu, Yifu Yao, Zixin Wang, Hongru Liu, Shilong Jiang, Xun Leqi, Liu Wang, Mengdi |
| author_facet | Qiu, Jiahao Juan, Xinzhe Wang, Yimin Yang, Ling Qi, Xuan Zhang, Tongcheng Guo, Jiacheng Lu, Yifu Yao, Zixin Wang, Hongru Liu, Shilong Jiang, Xun Leqi, Liu Wang, Mengdi |
| contents | While knowledge distillation has become a mature field for compressing large language models (LLMs) into smaller ones by aligning their outputs or internal representations, the distillation of LLM-based agents, which involve planning, memory, and tool use, remains relatively underexplored. Existing agent distillation methods typically replay full teacher trajectories or imitate step-by-step teacher tool usage, but they often struggle to train student agents to dynamically plan and act in novel environments. We propose AgentDistill, a novel, training-free agent distillation framework that enables efficient and scalable knowledge transfer via direct reuse of Model-Context-Protocols (MCPs), which are structured and reusable task-solving modules autonomously generated by teacher agents. The reuse of these distilled MCPs enables student agents to generalize their capabilities across domains and solve new problems with minimal supervision or human intervention. Experiments on biomedical and mathematical benchmarks demonstrate that our distilled student agents, built on small language models, can achieve performance comparable to advanced systems using large LLMs such as OctoTools (GPT-4o), highlighting the effectiveness of our framework in building scalable and cost-efficient intelligent agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_14728 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes Qiu, Jiahao Juan, Xinzhe Wang, Yimin Yang, Ling Qi, Xuan Zhang, Tongcheng Guo, Jiacheng Lu, Yifu Yao, Zixin Wang, Hongru Liu, Shilong Jiang, Xun Leqi, Liu Wang, Mengdi Artificial Intelligence While knowledge distillation has become a mature field for compressing large language models (LLMs) into smaller ones by aligning their outputs or internal representations, the distillation of LLM-based agents, which involve planning, memory, and tool use, remains relatively underexplored. Existing agent distillation methods typically replay full teacher trajectories or imitate step-by-step teacher tool usage, but they often struggle to train student agents to dynamically plan and act in novel environments. We propose AgentDistill, a novel, training-free agent distillation framework that enables efficient and scalable knowledge transfer via direct reuse of Model-Context-Protocols (MCPs), which are structured and reusable task-solving modules autonomously generated by teacher agents. The reuse of these distilled MCPs enables student agents to generalize their capabilities across domains and solve new problems with minimal supervision or human intervention. Experiments on biomedical and mathematical benchmarks demonstrate that our distilled student agents, built on small language models, can achieve performance comparable to advanced systems using large LLMs such as OctoTools (GPT-4o), highlighting the effectiveness of our framework in building scalable and cost-efficient intelligent agents. |
| title | AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.14728 |