Cover Image

Thinking with DistilQwen: A Tale of Four Distilled Reasoning and Reward Model Series

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cai, Wenrui, Wang, Chengyu, Yan, Junbing, Huang, Jun, Fang, Xiangzhong
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models
by: Wang, Chengyu, et al.
Published: (2025)

Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations
by: Cai, Wenrui, et al.
Published: (2025)

Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment
by: Cai, Wenrui, et al.
Published: (2025)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

Do Large Language Models Understand Logic or Just Mimick Context?
by: Yan, Junbing, et al.
Published: (2024)

From Correction to Mastery: Reinforced Distillation of Large Language Model Agents
by: Lyu, Yuanjie, et al.
Published: (2025)

Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners
by: Paliotta, Daniele, et al.
Published: (2025)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
by: Tang, Shengkun, et al.
Published: (2026)

Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?
by: Zhang, Yudi, et al.
Published: (2025)

Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
by: Chen, Wei-Rui, et al.
Published: (2025)

AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
by: Lyu, Yuanjie, et al.
Published: (2026)

Enhancing Reasoning Capabilities in SLMs with Reward Guided Dataset Distillation
by: Padarha, Shreyansh
Published: (2025)

A Short Survey on Small Reasoning Models: Training, Inference, Applications and Research Directions
by: Wang, Chengyu, et al.
Published: (2025)

Distilling Mathematical Reasoning Capabilities into Small Language Models
by: Zhu, Xunyu, et al.
Published: (2024)

Detecting Distillation Data from Reasoning Models
by: Zhang, Hengxiang, et al.
Published: (2025)

LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
by: Li, Dongheng, et al.
Published: (2024)

Mixed Distillation Helps Smaller Language Model Better Reasoning
by: Li, Chenglin, et al.
Published: (2023)

Skill-Conditioned Gated Self-Distillation for LLM Reasoning
by: Huang, Jiazhen, et al.
Published: (2026)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
by: Zhu, Xunyu, et al.
Published: (2024)

Hide to See: Reasoning-prefix Masking for Visual-anchored Thinking in VLM Distillation
by: Yu, Seonghoon, et al.
Published: (2026)

Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphs
by: Xiao, Yilin, et al.
Published: (2025)

Controlling Thinking Speed in Reasoning Models
by: Lin, Zhengkai, et al.
Published: (2025)

Is It Thinking or Cheating? Detecting Implicit Reward Hacking by Measuring Reasoning Effort
by: Wang, Xinpeng, et al.
Published: (2025)

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
by: Wang, Wei, et al.
Published: (2024)

Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillation
by: Zhu, Xunyu, et al.
Published: (2024)

Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
by: Wang, Bing, et al.
Published: (2026)

Qwen2 Technical Report
by: Yang, An, et al.
Published: (2024)

Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation
by: Phan, Phuc, et al.
Published: (2024)

ThinkTuning: Instilling Cognitive Reflections without Distillation
by: RRV, Aswin, et al.
Published: (2025)

Style over Substance: Distilled Language Models Reason Via Stylistic Replication
by: Lippmann, Philip, et al.
Published: (2025)

Process Reward Models That Think
by: Khalifa, Muhammad, et al.
Published: (2025)

EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs
by: Lin, Liang, et al.
Published: (2026)

Exploring Reasoning Reward Model for Agents
by: Fan, Kaixuan, et al.
Published: (2026)

Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation
by: Zhou, Yuhang, et al.
Published: (2024)

MixReasoning: Switching Modes to Think
by: Lu, Haiquan, et al.
Published: (2025)

MiniLLM: On-Policy Distillation of Large Language Models
by: Gu, Yuxian, et al.
Published: (2023)

Chunk-Distilled Language Modeling
by: Li, Yanhong, et al.
Published: (2024)

Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs
by: Hoang, Cong Duy Vu, et al.
Published: (2025)

SOD: Step-wise On-policy Distillation for Small Language Model Agents
by: Zhong, Qiyong, et al.
Published: (2026)

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
by: Yang, Chuanpeng, et al.
Published: (2024)