Cover Image

Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Junlin, Tang, Yihong, Nie, Tong, Li, Guilong, Yang, Binyu, Du, Jinxiao, Sun, Lijun, Ma, Wei
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LLMSynthor: Macro-Aligned Micro-Records Synthesis with Large Language Models
by: Tang, Yihong, et al.
Published: (2025)

Estimating Real Demand Using a Flipped Queueing Model: A Case of Shared Micro-Mobility Services
by: Yang, Binyu, et al.
Published: (2025)

MobEvolve: An Agentic Self-Evolving Heuristic System for Interpretable Human Mobility Generation
by: He, Junlin, et al.
Published: (2026)

Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization
by: He, Junlin, et al.
Published: (2024)

Joint Estimation and Prediction of City-wide Delivery Demand: A Large Language Model Empowered Graph-based Learning Approach
by: Nie, Tong, et al.
Published: (2024)

Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learning
by: He, Junlin, et al.
Published: (2024)

Bridge: Retrieval-Augmented Spatiotemporal Modeling for Urban Delivery Demand
by: Tang, Yihong, et al.
Published: (2026)

ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving
by: Nie, Tong, et al.
Published: (2026)

DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models
by: Jiang, Yuxuan, et al.
Published: (2025)

Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
by: Chen, Wei-Rui, et al.
Published: (2025)

Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization
by: He, Junlin, et al.
Published: (2024)

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
by: Liu, Yihong, et al.
Published: (2023)

Activity-aware Human Mobility Prediction with Hierarchical Graph Attention Recurrent Network
by: Tang, Yihong, et al.
Published: (2022)

Crosslingual On-Policy Self-Distillation for Multilingual Reasoning
by: Liu, Yihong, et al.
Published: (2026)

Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs
by: Sun, Jia Ao, et al.
Published: (2025)

CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents
by: Tang, Yihong, et al.
Published: (2026)

Self-Enhanced Reasoning Training: Activating Latent Reasoning in Small Models for Enhanced Reasoning Distillation
by: Zhang, Yong, et al.
Published: (2025)

Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners
by: Xu, Xin, et al.
Published: (2025)

ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
by: Yuan, Zhihang, et al.
Published: (2023)

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
by: Özeren, Enes, et al.
Published: (2025)

E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving
by: Tang, Yihong, et al.
Published: (2025)

Balanced Actor Initialization: Stable RLHF Training of Distillation-Based Reasoning Models
by: Zheng, Chen, et al.
Published: (2025)

The Valley of Code Reasoning: Scaling Knowledge Distillation of Large Language Models
by: He, Muyu, et al.
Published: (2025)

MAR: Efficient Large Language Models via Module-aware Architecture Refinement
by: Cai, Junhong, et al.
Published: (2026)

MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching
by: Yue, Liang, et al.
Published: (2025)

Activation-Space Anchored Access Control for Multi-Class Permission Reasoning in Large Language Models
by: Zhang, Zhaopeng, et al.
Published: (2026)

Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
by: Xing, Wang, et al.
Published: (2026)

A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
by: Qu, Xiaoye, et al.
Published: (2025)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
by: Zhu, Xunyu, et al.
Published: (2024)

ORPP: Self-Optimizing Role-playing Prompts to Enhance Language Model Capabilities
by: Duan, Yifan, et al.
Published: (2025)

AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Control
by: Li, Ruosen, et al.
Published: (2025)

Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation
by: Qi, Chengwen, et al.
Published: (2025)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
by: Zhao, Siyan, et al.
Published: (2026)

REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once
by: Pan, Zhuoshi, et al.
Published: (2025)

Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners
by: Liu, Yihong, et al.
Published: (2026)

Quantification of Large Language Model Distillation
by: Lee, Sunbowen, et al.
Published: (2025)

HADES: Hardware Accelerated Decoding for Efficient Speculation in Large Language Models
by: Yang, Ze, et al.
Published: (2024)

Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression
by: Huang, Jiameng, et al.
Published: (2025)

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
by: Lin, Ji, et al.
Published: (2023)

Automated Benchmark Auditing for AI Agents and Large Language Models
by: Wang, Junlin, et al.
Published: (2026)