Skill Weaving: Efficient LLM Improvement via Modular Skillpacks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914587474919424 |
|---|---|
| author | Li, Zhuo Du, Guodong Shi, Zesheng Guo, Weiyang Yao, Weijun Zhou, Yuan Zhang, Jiabo Li, Jing |
| author_facet | Li, Zhuo Du, Guodong Shi, Zesheng Guo, Weiyang Yao, Weijun Zhou, Yuan Zhang, Jiabo Li, Jing |
| contents | Large language models increasingly require specialization across diverse domains, yet existing approaches struggle to balance multi-domain capacities with strict memory and inference constraints. In this work, we introduce SkillWeave, a modular improvement framework that enables LLMs to specialize under fixed memory budgets. SkillWeave partitions full capabilities of a general-purpose model into skillpacks -- lightweight, domain-specific delta modules -- that reorganize and refine the model's internal knowledge. For efficient deployment, SkillWeave integrates SkillZip to compress skillpacks into compact and inference-ready format, enabling strong multi-domain performance with low-latency execution. On multi-task and agentic benchmarks, a 9B SkillWeave model outperforms several baselines and even surpasses a 32B monolithic LLM, while achieving up to 4x speedup. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_22205 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Skill Weaving: Efficient LLM Improvement via Modular Skillpacks Li, Zhuo Du, Guodong Shi, Zesheng Guo, Weiyang Yao, Weijun Zhou, Yuan Zhang, Jiabo Li, Jing Artificial Intelligence Machine Learning Large language models increasingly require specialization across diverse domains, yet existing approaches struggle to balance multi-domain capacities with strict memory and inference constraints. In this work, we introduce SkillWeave, a modular improvement framework that enables LLMs to specialize under fixed memory budgets. SkillWeave partitions full capabilities of a general-purpose model into skillpacks -- lightweight, domain-specific delta modules -- that reorganize and refine the model's internal knowledge. For efficient deployment, SkillWeave integrates SkillZip to compress skillpacks into compact and inference-ready format, enabling strong multi-domain performance with low-latency execution. On multi-task and agentic benchmarks, a 9B SkillWeave model outperforms several baselines and even surpasses a 32B monolithic LLM, while achieving up to 4x speedup. |
| title | Skill Weaving: Efficient LLM Improvement via Modular Skillpacks |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2605.22205 |