Image de couverture de livre

DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Chengyu, Yan, Junbing, Yue, Yuanhao, Huang, Jun
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Thinking with DistilQwen: A Tale of Four Distilled Reasoning and Reward Model Series
par: Cai, Wenrui, et autres
Publié: (2025)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
par: Wang, Chengyu, et autres
Publié: (2025)

AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
par: Lyu, Yuanjie, et autres
Publié: (2026)

Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning
par: Yue, Yuanhao, et autres
Publié: (2024)

OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
par: Yue, Yuanhao, et autres
Publié: (2026)

Do Large Language Models Understand Logic or Just Mimick Context?
par: Yan, Junbing, et autres
Publié: (2024)

From Correction to Mastery: Reinforced Distillation of Large Language Model Agents
par: Lyu, Yuanjie, et autres
Publié: (2025)

Building a Family of Data Augmentation Models for Low-cost LLM Fine-tuning on the Cloud
par: Yue, Yuanhao, et autres
Publié: (2024)

Qwen2.5 Technical Report
par: Qwen, et autres
Publié: (2024)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
par: Yan, Junbing, et autres
Publié: (2024)

Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations
par: Cai, Wenrui, et autres
Publié: (2025)

Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment
par: Cai, Wenrui, et autres
Publié: (2025)

Qwen2.5-VL Technical Report
par: Bai, Shuai, et autres
Publié: (2025)

Qwen2.5-Coder Technical Report
par: Hui, Binyuan, et autres
Publié: (2024)

Qwen2.5-1M Technical Report
par: Yang, An, et autres
Publié: (2025)

Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources
par: Wang, Weizhi, et autres
Publié: (2025)

Qwen2.5-Omni Technical Report
par: Xu, Jin, et autres
Publié: (2025)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
par: Tang, Shengkun, et autres
Publié: (2026)

Comprehensive and Efficient Distillation for Lightweight Sentiment Analysis Models
par: Xie, Guangyu, et autres
Publié: (2025)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
par: Gu, Yuxian, et autres
Publié: (2024)

Large Language Models Explore by Latent Distilling
par: Zeng, Yuanhao, et autres
Publié: (2026)

Evolving Knowledge Distillation for Lightweight Neural Machine Translation
par: Zhang, Xuewen, et autres
Publié: (2026)

1.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training
par: Zhao, Han, et autres
Publié: (2025)

HARNESS: Lightweight Distilled Arabic Speech Foundation Models
par: sukhadia, Vrunda N., et autres
Publié: (2025)

On-Policy Context Distillation for Language Models
par: Ye, Tianzhu, et autres
Publié: (2026)

D2LLM: Decomposed and Distilled Large Language Models for Semantic Search
par: Liao, Zihan, et autres
Publié: (2024)

Distilling Token-Trained Models into Byte-Level Models
par: Bao, Zishuo, et autres
Publié: (2026)

Knowledge Distillation with Training Wheels
par: Liu, Guanlin, et autres
Publié: (2025)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
par: Zhao, Siyan, et autres
Publié: (2026)

Augmenting Human-Annotated Training Data with Large Language Model Generation and Distillation in Open-Response Assessment
par: Borchers, Conrad, et autres
Publié: (2025)

ICH-Qwen: A Large Language Model Towards Chinese Intangible Cultural Heritage
par: Ye, Wenhao, et autres
Publié: (2025)

QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management
par: Shen, Weizhou, et autres
Publié: (2025)

Training Language Models to Generate Text with Citations via Fine-grained Rewards
par: Huang, Chengyu, et autres
Publié: (2024)

RM-Distiller: Exploiting Generative LLM for Reward Model Distillation
par: Zhou, Hongli, et autres
Publié: (2026)

Fine-Tuning Qwen 2.5 3B for Realistic Movie Dialogue Generation
par: Gupta, Kartik
Publié: (2025)

Knowledge Distillation of Black-Box Large Language Models
par: Chen, Hongzhan, et autres
Publié: (2024)

Leveraging Large Language Models for Enhanced NLP Task Performance through Knowledge Distillation and Optimized Training Strategies
par: Huang, Yining, et autres
Publié: (2024)

Quantification of Large Language Model Distillation
par: Lee, Sunbowen, et autres
Publié: (2025)

Structured Agent Distillation for Large Language Model
par: Liu, Jun, et autres
Publié: (2025)

Collaborative Distillation Strategies for Parameter-Efficient Language Model Deployment
par: Meng, Xiandong, et autres
Publié: (2025)