Cover Image

$\textit{SKIntern}$: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liao, Huanxuan, He, Shizhu, Hao, Yupu, Li, Xiang, Zhang, Yuanzhe, Zhao, Jun, Liu, Kang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Neural-Symbolic Collaborative Distillation: Advancing Small Language Models for Complex Reasoning Tasks
by: Liao, Huanxuan, et al.
Published: (2024)

Awakening Augmented Generation: Learning to Awaken Internal Knowledge of Large Language Models for Question Answering
by: Liao, Huanxuan, et al.
Published: (2024)

DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning
by: Liao, Huanxuan, et al.
Published: (2025)

Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
by: Tan, Yuqiao, et al.
Published: (2025)

From Instance Training to Instruction Learning: Task Adapters Generation from Instructions
by: Liao, Huanxuan, et al.
Published: (2024)

CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performance
by: Hao, Yupu, et al.
Published: (2024)

ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoning
by: Liao, Huanxuan, et al.
Published: (2026)

Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention
by: Liao, Huanxuan, et al.
Published: (2025)

Shuttle Between the Instructions and the Parameters of Large Language Models
by: Sun, Wangtao, et al.
Published: (2025)

Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity
by: Hao, Yupu, et al.
Published: (2025)

ExpNote: Black-box Large Language Models are Better Task Solvers with Experience Notebook
by: Sun, Wangtao, et al.
Published: (2023)

Neural Incompatibility: The Unbridgeable Gap of Cross-Scale Parametric Knowledge Transfer in Large Language Models
by: Tan, Yuqiao, et al.
Published: (2025)

Mastering Symbolic Operations: Augmenting Language Models with Compiled Neural Networks
by: Weng, Yixuan, et al.
Published: (2023)

Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints
by: Song, Ran, et al.
Published: (2024)

From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
by: Deng, Yuntian, et al.
Published: (2024)

Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoning
by: Zhao, Shangziqi, et al.
Published: (2025)

Towards Efficient CoT Distillation: Self-Guided Rationale Selector for Better Performance with Fewer Rationales
by: Yan, Jianzhi, et al.
Published: (2025)

SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning
by: Liao, Huanxuan, et al.
Published: (2025)

ControlLM: Crafting Diverse Personalities for Language Models
by: Weng, Yixuan, et al.
Published: (2024)

Efficient Long CoT Reasoning in Small Language Models
by: Wang, Zhaoyang, et al.
Published: (2025)

SC-Taxo: Hierarchical Taxonomy Generation under Semantic Consistency Constraints using Large Language Models
by: Cai, Shiqiang, et al.
Published: (2026)

MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation
by: Cui, Jin, et al.
Published: (2026)

Investigating Mysteries of CoT-Augmented Distillation
by: Wadhwa, Somin, et al.
Published: (2024)

AS-ES Learning: Towards Efficient CoT Learning in Small Models
by: Xi, Nuwa, et al.
Published: (2024)

The Zero-Step Thinking: An Empirical Study of Mode Selection as Harder Early Exit in Reasoning Models
by: Tan, Yuqiao, et al.
Published: (2025)

Find Parent then Label Children: A Two-stage Taxonomy Completion Method with Pre-trained Language Model
by: Xia, Fei, et al.
Published: (2024)

S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models
by: Lei, Fangyu, et al.
Published: (2023)

Through the Valley: Path to Effective Long CoT Training for Small Language Models
by: Luo, Renjie, et al.
Published: (2025)

Distilling Mathematical Reasoning Capabilities into Small Language Models
by: Zhu, Xunyu, et al.
Published: (2024)

Generating Effective CoT Traces for Mitigating Causal Hallucination
by: Zhao, Yiheng, et al.
Published: (2026)

Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning
by: Yan, Shaotian, et al.
Published: (2026)

Efficient Data Learning for Open Information Extraction with Pre-trained Language Models
by: Fan, Zhiyuan, et al.
Published: (2023)

CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning
by: Gan, Zeyu, et al.
Published: (2025)

CoT is Not the Chain of Truth: An Empirical Internal Analysis of Reasoning LLMs for Fake News Generation
by: Tong, Zhao, et al.
Published: (2026)

CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning
by: Feng, Kehua, et al.
Published: (2025)

MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models
by: Luo, Tongxu, et al.
Published: (2024)

A Survey on Symbolic Knowledge Distillation of Large Language Models
by: Acharya, Kamal, et al.
Published: (2024)

ItD: Large Language Models Can Teach Themselves Induction through Deduction
by: Sun, Wangtao, et al.
Published: (2024)

Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning
by: Jin, Senjie, et al.
Published: (2025)

Does Knowledge Localization Hold True? Surprising Differences Between Entity and Relation Perspectives in Language Models
by: Wei, Yifan, et al.
Published: (2024)