Cover Image

Demystifying When Pruning Works via Representation Hierarchies

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Shwai, Sun, Guoheng, Zhang, Haichao, Fu, Yun, Li, Ang
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

What Matters in Transformers? Not All Attention is Needed
by: He, Shwai, et al.
Published: (2024)

Rethinking Pruning for Vision-Language Models: Strategies for Effective Sparsity and Performance Restoration
by: He, Shwai, et al.
Published: (2024)

Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Experts
by: He, Shwai, et al.
Published: (2025)

SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning
by: He, Yexiao, et al.
Published: (2024)

Dense Video Understanding with Gated Residual Tokenization
by: Zhang, Haichao, et al.
Published: (2025)

Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs
by: Fu, Yao, et al.
Published: (2025)

Disentangling the Roles of Representation and Selection in Data Pruning
by: Du, Yupei, et al.
Published: (2025)

Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
by: Li, Ming, et al.
Published: (2024)

Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe
by: Wu, Xixi, et al.
Published: (2026)

ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model
by: Zhang, Haichao, et al.
Published: (2026)

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
by: Lee, Jaeseong, et al.
Published: (2024)

Demystifying Long Chain-of-Thought Reasoning in LLMs
by: Yeo, Edward, et al.
Published: (2025)

Demystifying Verbatim Memorization in Large Language Models
by: Huang, Jing, et al.
Published: (2024)

Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers
by: He, Shwai, et al.
Published: (2024)

Demystifying the Slash Pattern in Attention: The Role of RoPE
by: Cheng, Yuan, et al.
Published: (2026)

SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning
by: Wang, Yiting, et al.
Published: (2025)

VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation
by: Wang, Yiting, et al.
Published: (2025)

MoEless: Efficient MoE LLM Serving via Serverless Computing
by: Yu, Hanfei, et al.
Published: (2026)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
by: Cao, Mingyu, et al.
Published: (2024)

Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Models
by: Luo, Feng, et al.
Published: (2026)

SAP: Syntactic Attention Pruning for Transformer-based Language Models
by: Lee, Tzu-Yun, et al.
Published: (2025)

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention
by: Tian, Yuandong, et al.
Published: (2023)

Towards counterfactual fairness through auxiliary variables
by: Tian, Bowei, et al.
Published: (2024)

Language Models as Hierarchy Encoders
by: He, Yuan, et al.
Published: (2024)

Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
by: Li, Zhaoyi, et al.
Published: (2026)

Demystifying Language Model Forgetting with Low-rank Example Associations
by: Jin, Xisen, et al.
Published: (2024)

Demystifying Chains, Trees, and Graphs of Thoughts
by: Besta, Maciej, et al.
Published: (2024)

Reformatted Alignment
by: Fan, Run-Ze, et al.
Published: (2024)

Olica: Efficient Structured Pruning of Large Language Models without Retraining
by: He, Jiujun, et al.
Published: (2025)

When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
by: Xu, Zhen, et al.
Published: (2025)

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing
by: Le, Qi, et al.
Published: (2025)

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations
by: Petrov, Aleksandar, et al.
Published: (2023)

VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs
by: Goel, Raghavv, et al.
Published: (2025)

Dissecting Language Models: Machine Unlearning via Selective Pruning
by: Pochinkov, Nicholas, et al.
Published: (2024)

Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends
by: Yao, Chaorui, et al.
Published: (2025)

When Reward Hacking Rebounds: Understanding and Mitigating It with Representation-Level Signals
by: Wu, Rui, et al.
Published: (2026)

Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes
by: Kolawole, Steven, et al.
Published: (2024)

MLP-KAN: Unifying Deep Representation and Function Learning
by: He, Yunhong, et al.
Published: (2024)

DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration
by: Gu, Tianteng, et al.
Published: (2025)

Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processing
by: Liu, Yudong, et al.
Published: (2025)