Cover Image

From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pei, Zehua, Zhen, Hui-Ling, Yu, Xianzhi, Pan, Sinno Jialin, Yuan, Mingxuan, Yu, Bei
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning
by: Pei, Zehua, et al.
Published: (2026)

Verilog-Evolve: Feedback-Driven and Skill-Evolving Verilog Generation
by: Pei, Zehua, et al.
Published: (2026)

Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
by: Pei, Zehua, et al.
Published: (2025)

MemDLM: Memory-Enhanced DLM Training
by: Pei, Zehua, et al.
Published: (2026)

BetterV: Controlled Verilog Generation with Discriminative Guidance
by: Pei, Zehua, et al.
Published: (2024)

DiLA: Enhancing LLM Tool Learning with Differential Logic Layer
by: Zhang, Yu, et al.
Published: (2024)

Behavioral Fingerprinting of Large Language Models
by: Pei, Zehua, et al.
Published: (2025)

SCOPE: Prompt Evolution for Enhancing Agent Effectiveness
by: Pei, Zehua, et al.
Published: (2025)

Beyond Speedup -- Utilizing KV Cache for Sampling and Reasoning
by: Xing, Zeyu, et al.
Published: (2026)

PreMoE: Proactive Inference for Efficient Mixture-of-Experts
by: Pei, Zehua, et al.
Published: (2025)

KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
by: Li, Xing, et al.
Published: (2025)

What Matters For Safety Alignment?
by: Li, Xing, et al.
Published: (2026)

SwiftMem: Fast Agentic Memory via Query-aware Indexing
by: Tian, Anxin, et al.
Published: (2026)

SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention
by: Yankun, Hong, et al.
Published: (2025)

MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generation
by: Jiang, Weisen, et al.
Published: (2025)

Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
by: Yu, Fengming, et al.
Published: (2025)

MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
by: Jiang, Weisen, et al.
Published: (2026)

MOSS: Efficient and Accurate FP8 LLM Training with Microscaling and Automatic Scaling
by: Zhang, Yu, et al.
Published: (2025)

BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization
by: Li, Ji-Fu, et al.
Published: (2026)

Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
by: Sun, Shengyin, et al.
Published: (2025)

TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scaling
by: Lin, Weizhe, et al.
Published: (2025)

Revisiting Judge Decoding from First Principles via Training-Free Distributional Divergence
by: Sun, Shengyin, et al.
Published: (2026)

Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats
by: Zhao, Pengxiang, et al.
Published: (2026)

Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats
by: Zhang, Manyi, et al.
Published: (2026)

E$^3$-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
by: Yuan, Tao, et al.
Published: (2025)

SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Prompts
by: Pei, Aihua, et al.
Published: (2024)

Fast Graph Generation via Spectral Diffusion
by: Luo, Tianze, et al.
Published: (2022)

KGPA: Robustness Evaluation for Large Language Models via Cross-Domain Knowledge Graphs
by: Pei, Aihua, et al.
Published: (2024)

CLAUSE: Agentic Neuro-Symbolic Knowledge Graph Reasoning via Dynamic Learnable Context Engineering
by: Zhao, Yang, et al.
Published: (2025)

Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
by: Long, Quanyu, et al.
Published: (2024)

PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery
by: He, Bowei, et al.
Published: (2025)

PermLLM: Learnable Channel Permutation for N:M Sparse Large Language Models
by: Zou, Lancheng, et al.
Published: (2025)

Knowledge Fusion via Bidirectional Information Aggregation
by: Zhai, Songlin, et al.
Published: (2025)

IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
by: Li, Yixiao, et al.
Published: (2025)

PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
by: Sun, Shengyin, et al.
Published: (2026)

The Graph's Apprentice: Teaching an LLM Low Level Knowledge for Circuit Quality Estimation
by: Moravej, Reza, et al.
Published: (2024)

A Simple Linear Patch Revives Layer-Pruned Large Language Models
by: Chen, Xinrui, et al.
Published: (2025)

LeanK: Learnable K Cache Channel Pruning for Efficient Decoding
by: Zhang, Yike, et al.
Published: (2025)

Accelerating Large Language Model Reasoning via Speculative Search
by: Wang, Zhihai, et al.
Published: (2025)

Multilingual Jailbreak Challenges in Large Language Models
by: Deng, Yue, et al.
Published: (2023)