Cover Image

Mixed Distillation Helps Smaller Language Model Better Reasoning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Chenglin, Chen, Qianglong, Li, Liangyue, Wang, Caiyu, Li, Yicheng, Chen, Zulong, Zhang, Yin
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search
by: Li, Chenglin, et al.
Published: (2024)

Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration
by: Lv, Hang, et al.
Published: (2026)

Think Thrice Before You Act: Progressive Thought Refinement in Large Language Models
by: Du, Chengyu, et al.
Published: (2024)

VideoCogQA: A Controllable Benchmark for Evaluating Cognitive Abilities in Video-Language Models
by: Li, Chenglin, et al.
Published: (2024)

Enhancing Generalization in Chain of Thought Reasoning for Smaller Models
by: Yin, Maxwell J., et al.
Published: (2025)

Iterative Zoom-In: Temporal Interval Exploration for Long Video Understanding
by: Li, Chenglin, et al.
Published: (2025)

Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?
by: Zhang, Yudi, et al.
Published: (2025)

TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models
by: Chu, Zheng, et al.
Published: (2023)

CausalEval: Towards Better Causal Reasoning in Language Models
by: Yu, Longxuan, et al.
Published: (2024)

Timo: Towards Better Temporal Reasoning for Language Models
by: Su, Zhaochen, et al.
Published: (2024)

Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
by: Bansal, Hritik, et al.
Published: (2024)

Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought
by: Lee, Jooyoung, et al.
Published: (2024)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
by: Tian, Yijun, et al.
Published: (2024)

Distilling Mathematical Reasoning Capabilities into Small Language Models
by: Zhu, Xunyu, et al.
Published: (2024)

Learning to Self-Verify Makes Language Models Better Reasoners
by: Chen, Yuxin, et al.
Published: (2026)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
by: Zhu, Xunyu, et al.
Published: (2024)

LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?
by: Wang, Jingyuan, et al.
Published: (2025)

Abstraction-of-Thought Makes Language Models Better Reasoners
by: Hong, Ruixin, et al.
Published: (2024)

Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillation
by: Zhu, Xunyu, et al.
Published: (2024)

Detecting Distillation Data from Reasoning Models
by: Zhang, Hengxiang, et al.
Published: (2025)

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models
by: Chen, Yanjun, et al.
Published: (2024)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

SR-FoT: A Syllogistic-Reasoning Framework of Thought for Large Language Models Tackling Knowledge-based Reasoning Tasks
by: Wan, Wentao, et al.
Published: (2025)

VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning
by: Li, Chenglin, et al.
Published: (2026)

Mix of Experts Language Model for Named Entity Recognition
by: Chen, Xinwei, et al.
Published: (2024)

Active Confusion Expression in Large Language Models: Leveraging World Models toward Better Social Reasoning
by: Du, Jialu, et al.
Published: (2025)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

Chunk-Distilled Language Modeling
by: Li, Yanhong, et al.
Published: (2024)

Channel-Wise Mixed-Precision Quantization for Large Language Models
by: Chen, Zihan, et al.
Published: (2024)

Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphs
by: Xiao, Yilin, et al.
Published: (2025)

GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving
by: Zhang, Jiaxin, et al.
Published: (2024)

Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging
by: Lyu, Mengxian, et al.
Published: (2026)

When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models
by: Zheng, Yingming, et al.
Published: (2025)

Do Thinking Tokens Help or Trap? Towards More Efficient Large Reasoning Model
by: Ding, Bowen, et al.
Published: (2025)

Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate System
by: Wang, Haotian, et al.
Published: (2023)

Language Model Distillation: A Temporal Difference Imitation Learning Perspective
by: Yu, Zishun, et al.
Published: (2025)

MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate
by: Wang, Jianze, et al.
Published: (2026)

Integrating Arithmetic Learning Improves Mathematical Reasoning in Smaller Models
by: Gangwar, Neeraj, et al.
Published: (2025)

A Survey on Enhancing Causal Reasoning Ability of Large Language Models
by: Li, Xin, et al.
Published: (2025)

TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance
by: Wang, Haorui, et al.
Published: (2024)