Cover Image

Merge-of-Thought Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shen, Zhanming, Qin, Zeyu, Huang, Zenan, Chen, Hao, Hu, Jiaqi, Zhuang, Yihong, Lu, Guoshan, Chen, Gang, Zhao, Junbo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Training-Trajectory-Aware Token Selection
by: Shen, Zhanming, et al.
Published: (2026)

Reinforcement Learning with Rubric Anchors
by: Huang, Zenan, et al.
Published: (2025)

Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority
by: Shen, Zhanming, et al.
Published: (2026)

CYCLE-INSTRUCT: Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency
by: Shen, Zhanming, et al.
Published: (2025)

ABench-Physics: Benchmarking Physical Reasoning in LLMs via High-Difficulty and Dynamic Physics Problems
by: Zhang, Yiming, et al.
Published: (2025)

LLaDA2.0: Scaling Up Diffusion Language Models to 100B
by: Bie, Tiwei, et al.
Published: (2025)

FedSDR: Federated Self-Distillation with Rectification
by: Ren, Ziheng, et al.
Published: (2026)

Channel Merging: Preserving Specialization for Merged Experts
by: Zhang, Mingyang, et al.
Published: (2024)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
by: Lu, Zhenyi, et al.
Published: (2024)

Self-Distilled Agentic Reinforcement Learning
by: Lu, Zhengxi, et al.
Published: (2026)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Energy-based Automated Model Evaluation
by: Peng, Ru, et al.
Published: (2024)

Merge to Mix: Mixing Datasets via Model Merging
by: Tao, Zhixu Silvia, et al.
Published: (2025)

MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
by: Liu, Qin, et al.
Published: (2025)

Generating Chain-of-Thoughts with a Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought
by: Zhang, Zhen-Yu, et al.
Published: (2024)

Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information
by: Shen, Guobin, et al.
Published: (2026)

Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective
by: Kajitsuka, Tokio, et al.
Published: (2026)

Multilingual Safety Alignment via Self-Distillation
by: Qin, Ruiyang, et al.
Published: (2026)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
by: Chen, Jennifer, et al.
Published: (2025)

Understanding Reasoning in Chain-of-Thought from the Hopfieldian View
by: Hu, Lijie, et al.
Published: (2024)

Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models
by: Ye, Jiacheng, et al.
Published: (2024)

Model Merging for Knowledge Editing
by: Fu, Zichuan, et al.
Published: (2025)

LLM Pruning and Distillation in Practice: The Minitron Approach
by: Sreenivas, Sharath Turuvekere, et al.
Published: (2024)

Toward Adaptive Reasoning in Large Language Models with Thought Rollback
by: Chen, Sijia, et al.
Published: (2024)

STAR: Spectral Truncation and Rescale for Model Merging
by: Lee, Yu-Ang, et al.
Published: (2025)

Towards the Law of Capacity Gap in Distilling Language Models
by: Zhang, Chen, et al.
Published: (2023)

AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
by: Hu, Yuezhou, et al.
Published: (2025)

Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging
by: Wang, Zhixiang, et al.
Published: (2025)

Brain-Inspired Two-Stage Approach: Enhancing Mathematical Reasoning by Imitating Human Thought Processes
by: Chen, Yezeng, et al.
Published: (2024)

PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation
by: Fan, Tao, et al.
Published: (2025)

Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
by: Hu, Xinyang, et al.
Published: (2024)

Safety Reasoning with Guidelines
by: Wang, Haoyu, et al.
Published: (2025)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

Arcee's MergeKit: A Toolkit for Merging Large Language Models
by: Goddard, Charles, et al.
Published: (2024)

Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models
by: Chen, Sijia, et al.
Published: (2024)

Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion
by: Tan, Zhen, et al.
Published: (2026)

Leveraging Large Language Models with Chain-of-Thought and Prompt Engineering for Traffic Crash Severity Analysis and Inference
by: Zhen, Hao, et al.
Published: (2024)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
by: Shenaj, Donald, et al.
Published: (2025)