Imagen de Portada

Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	He, Junlin, Tang, Yihong, Nie, Tong, Li, Guilong, Yang, Binyu, Du, Jinxiao, Sun, Lijun, Ma, Wei
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

LLMSynthor: Macro-Aligned Micro-Records Synthesis with Large Language Models
por: Tang, Yihong, et al.
Publicado: (2025)

Estimating Real Demand Using a Flipped Queueing Model: A Case of Shared Micro-Mobility Services
por: Yang, Binyu, et al.
Publicado: (2025)

MobEvolve: An Agentic Self-Evolving Heuristic System for Interpretable Human Mobility Generation
por: He, Junlin, et al.
Publicado: (2026)

Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization
por: He, Junlin, et al.
Publicado: (2024)

Joint Estimation and Prediction of City-wide Delivery Demand: A Large Language Model Empowered Graph-based Learning Approach
por: Nie, Tong, et al.
Publicado: (2024)

Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learning
por: He, Junlin, et al.
Publicado: (2024)

Bridge: Retrieval-Augmented Spatiotemporal Modeling for Urban Delivery Demand
por: Tang, Yihong, et al.
Publicado: (2026)

ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving
por: Nie, Tong, et al.
Publicado: (2026)

DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models
por: Jiang, Yuxuan, et al.
Publicado: (2025)

Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
por: Chen, Wei-Rui, et al.
Publicado: (2025)

Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization
por: He, Junlin, et al.
Publicado: (2024)

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
por: Liu, Yihong, et al.
Publicado: (2023)

Activity-aware Human Mobility Prediction with Hierarchical Graph Attention Recurrent Network
por: Tang, Yihong, et al.
Publicado: (2022)

Crosslingual On-Policy Self-Distillation for Multilingual Reasoning
por: Liu, Yihong, et al.
Publicado: (2026)

Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs
por: Sun, Jia Ao, et al.
Publicado: (2025)

CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents
por: Tang, Yihong, et al.
Publicado: (2026)

Self-Enhanced Reasoning Training: Activating Latent Reasoning in Small Models for Enhanced Reasoning Distillation
por: Zhang, Yong, et al.
Publicado: (2025)

Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners
por: Xu, Xin, et al.
Publicado: (2025)

ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
por: Yuan, Zhihang, et al.
Publicado: (2023)

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
por: Özeren, Enes, et al.
Publicado: (2025)

Balanced Actor Initialization: Stable RLHF Training of Distillation-Based Reasoning Models
por: Zheng, Chen, et al.
Publicado: (2025)

The Valley of Code Reasoning: Scaling Knowledge Distillation of Large Language Models
por: He, Muyu, et al.
Publicado: (2025)

E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving
por: Tang, Yihong, et al.
Publicado: (2025)

MAR: Efficient Large Language Models via Module-aware Architecture Refinement
por: Cai, Junhong, et al.
Publicado: (2026)

MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching
por: Yue, Liang, et al.
Publicado: (2025)

Activation-Space Anchored Access Control for Multi-Class Permission Reasoning in Large Language Models
por: Zhang, Zhaopeng, et al.
Publicado: (2026)

Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
por: Xing, Wang, et al.
Publicado: (2026)

A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
por: Qu, Xiaoye, et al.
Publicado: (2025)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
por: Zhu, Xunyu, et al.
Publicado: (2024)

ORPP: Self-Optimizing Role-playing Prompts to Enhance Language Model Capabilities
por: Duan, Yifan, et al.
Publicado: (2025)

AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Control
por: Li, Ruosen, et al.
Publicado: (2025)

Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation
por: Qi, Chengwen, et al.
Publicado: (2025)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
por: Zhao, Siyan, et al.
Publicado: (2026)

REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once
por: Pan, Zhuoshi, et al.
Publicado: (2025)

Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners
por: Liu, Yihong, et al.
Publicado: (2026)

Quantification of Large Language Model Distillation
por: Lee, Sunbowen, et al.
Publicado: (2025)

HADES: Hardware Accelerated Decoding for Efficient Speculation in Large Language Models
por: Yang, Ze, et al.
Publicado: (2024)

Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression
por: Huang, Jiameng, et al.
Publicado: (2025)

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
por: Lin, Ji, et al.
Publicado: (2023)

Automated Benchmark Auditing for AI Agents and Large Language Models
por: Wang, Junlin, et al.
Publicado: (2026)