Image de couverture de livre

Data-Free Pruning of Self-Attention Layers in LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Saikumar, Dhananjay, Varghese, Blesson
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Signal Collapse in One-Shot Pruning: When Sparse Models Fail to Distinguish Neural Representations
par: Saikumar, Dhananjay, et autres
Publié: (2025)

DRIVE: Dual Gradient-Based Rapid Iterative Pruning
par: Saikumar, Dhananjay, et autres
Publié: (2024)

NeuroFlux: Memory-Efficient CNN Training Using Adaptive Local Learning
par: Saikumar, Dhananjay, et autres
Publié: (2024)

Mosaic: Composite Projection Pruning for Resource-efficient LLMs
par: Eccles, Bailey J., et autres
Publié: (2025)

Rapid Deployment of DNNs for Edge Computing via Structured Pruning at Initialization
par: Eccles, Bailey J., et autres
Publié: (2024)

DNNShifter: An Efficient DNN Pruning System for Edge Computing
par: Eccles, Bailey J., et autres
Publié: (2023)

Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs
par: Yun, Vincent-Daniel, et autres
Publié: (2026)

When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs
par: Wang, Keyu, et autres
Publié: (2025)

Post-Pruning Accuracy Recovery via Data-Free Knowledge Distillation
par: Tripurwar, Chinmay, et autres
Publié: (2025)

SwiftPrune: Hessian-Free Weight Pruning for Large Language Models
par: Kang, Yuhan, et autres
Publié: (2025)

Layer Collapse Can be Induced by Unstructured Pruning
par: Liao, Zhu, et autres
Publié: (2024)

PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
par: Yu, Tongzhou, et autres
Publié: (2025)

A Little Human Data Goes A Long Way
par: Ashok, Dhananjay, et autres
Publié: (2024)

Dispatch-Aware Ragged Attention for Pruned Vision Transformers
par: Abdellatif, Seifeldin, et autres
Publié: (2026)

Safe Pruning LoRA: Robust Distance-Guided Pruning for Safety Alignment in Adaptation of LLMs
par: Ao, Shuang, et autres
Publié: (2025)

CLOVER: Cross-Layer Orthogonal Vectors Pruning and Fine-Tuning
par: Meng, Fanxu, et autres
Publié: (2024)

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models
par: Shrestha, Safal, et autres
Publié: (2026)

PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs
par: Zimmer, Max, et autres
Publié: (2023)

Multi-Layer Attention-Based Explainability via Transformers for Tabular Data
par: Gavito, Andrea Treviño, et autres
Publié: (2023)

The Structural Scalpel: Automated Contiguous Layer Pruning for Large Language Models
par: Lu, Yao, et autres
Publié: (2025)

TopoPrune: Robust Data Pruning via Unified Latent Space Topology
par: Roy, Arjun, et autres
Publié: (2026)

Mixture of Layers with Hybrid Attention
par: Ternovtsii, Ivan, et autres
Publié: (2026)

Hierarchical Attention-based Graph Neural Network with Relevance-driven Pruning
par: Kum, Seungwoo
Publié: (2026)

Severing Spurious Correlations with Data Pruning
par: Mulchandani, Varun, et autres
Publié: (2025)

EMP: Enhance Memory in Data Pruning
par: Xiao, Jinying, et autres
Publié: (2024)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
par: Qin, Jiayu, et autres
Publié: (2025)

Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference
par: Taniguchi, Rei, et autres
Publié: (2026)

A Free Lunch in LLM Compression: Revisiting Retraining after Pruning
par: Wagner, Moritz, et autres
Publié: (2025)

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing
par: Le, Qi, et autres
Publié: (2025)

IDAP++: Advancing Divergence-Based Pruning via Filter-Level and Layer-Level Optimization
par: Samarin, Aleksei, et autres
Publié: (2025)

A Generic Layer Pruning Method for Signal Modulation Recognition Deep Learning Models
par: Lu, Yao, et autres
Publié: (2024)

UniPruning: Unifying Local Metric and Global Feedback for Scalable Sparse LLMs
par: Ding, Yizhuo, et autres
Publié: (2025)

Maximum Redundancy Pruning: A Principle-Driven Layerwise Sparsity Allocation for LLMs
par: Gao, Chang, et autres
Publié: (2025)

Geometric Median (GM) Matching for Robust Data Pruning
par: Acharya, Anish, et autres
Publié: (2024)

Dynamic Vocabulary Pruning in Early-Exit LLMs
par: Vincenti, Jort, et autres
Publié: (2024)

FAIR-Pruner: A Flexible Framework for Automatic Layer-Wise Pruning via Tolerance of Difference
par: Lin, Chenqing, et autres
Publié: (2025)

DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning
par: Han, Yizhou, et autres
Publié: (2026)

When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models
par: Sanyal, Sunny, et autres
Publié: (2024)

IntraSlice: Towards High-Performance Structural Pruning with Block-Intra PCA for LLMs
par: Li, Meng, et autres
Publié: (2026)

Gradient Boosting within a Single Attention Layer
par: Sargolzaei, Saleh
Publié: (2026)