:: Library Catalog

Cover Image

Saved in:

Bibliographic Details
Main Authors:	Guo, Yuxi, Sheridan, Paul
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	https://arxiv.org/abs/2602.04491
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Efficient LLMs with AMP: Attention Heads and MLP Pruning
by: Mugnaini, Leandro Giusti, et al.
Published: (2025)

Rényi Attention Entropy for Patch Pruning
by: Aizawa, Hiroaki, et al.
Published: (2026)

Convergence of Alternating Gradient Descent for Matrix Factorization
by: Ward, Rachel, et al.
Published: (2023)

Attention Head Entropy of LLMs Predicts Answer Correctness
by: Ostmeier, Sophie, et al.
Published: (2026)

Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
by: Choi, Minsik, et al.
Published: (2025)

Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
by: Yukhimchuk, Alexander, et al.
Published: (2026)

Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformers
by: Gabetni, Firas, et al.
Published: (2025)

Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix
by: Liang, Yingyu, et al.
Published: (2024)

Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfe
by: Roux, Christophe, et al.
Published: (2025)

A Greedy Hierarchical Approach to Whole-Network Filter-Pruning in CNNs
by: Purohit, Kiran, et al.
Published: (2024)

GradientStabilizer:Fix the Norm, Not the Gradient
by: Huang, Tianjin, et al.
Published: (2025)

Interleaved Head Attention
by: Duvvuri, Sai Surya, et al.
Published: (2026)

Pruning for Generalization: A Transfer-Oriented Spatiotemporal Graph Framework
by: Jing, Zihao, et al.
Published: (2026)

ETAGE: Enhanced Test Time Adaptation with Integrated Entropy and Gradient Norms for Robust Model Performance
by: Shamsi, Afshar, et al.
Published: (2024)

Multi-Head Low-Rank Attention
by: Liu, Songtao, et al.
Published: (2026)

Norm$\times$Direction: Restoring the Missing Query Norm in Vision Linear Attention
by: Meng, Weikang, et al.
Published: (2025)

SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
by: Wang, Hanrui, et al.
Published: (2020)

The Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization
by: Kravatskiy, Alexey, et al.
Published: (2025)

Gradient Dynamics of Attention: How Cross-Entropy Sculpts Bayesian Manifolds
by: Agarwal, Naman, et al.
Published: (2025)

On the Role of Attention Masks and LayerNorm in Transformers
by: Wu, Xinyi, et al.
Published: (2024)

Greedy Low-Rank Gradient Compression for Distributed Learning with Convergence Guarantees
by: Chen, Chuyan, et al.
Published: (2025)

Unlearning Backdoor Attacks through Gradient-Based Model Pruning
by: Dunnett, Kealan, et al.
Published: (2024)

Post-Norm can Resharpen Attention
by: Zsámboki, Pál, et al.
Published: (2025)

ASAP: Attention Sink Anchored Pruning
by: Lee, Jaehyuk, et al.
Published: (2026)

Accelerating Matrix Diagonalization through Decision Transformers with Epsilon-Greedy Optimization
by: Bhatta, Kshitij, et al.
Published: (2024)

DRIVE: Dual Gradient-Based Rapid Iterative Pruning
by: Saikumar, Dhananjay, et al.
Published: (2024)

Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns
by: Zhao, Yang, et al.
Published: (2024)

Adaptive Head Budgeting for Efficient Multi-Head Attention
by: Faye, Bilal, et al.
Published: (2026)

SHRP: Specialized Head Routing and Pruning for Efficient Encoder Compression
by: Su, Zeli, et al.
Published: (2025)

A Capacity-Based Rationale for Multi-Head Attention
by: Adler, Micah
Published: (2025)

Efficient Over-parameterized Matrix Sensing from Noisy Measurements via Alternating Preconditioned Gradient Descent
by: Liu, Zhiyu, et al.
Published: (2025)

Geometry and Dynamics of LayerNorm
by: Riechers, Paul M.
Published: (2024)

Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques
by: Dao, Manh Cuong, et al.
Published: (2025)

Relative Error Bound Analysis for Nuclear Norm Regularized Matrix Completion
by: Zhang, Lijun, et al.
Published: (2015)

A Granger-Causal Perspective on Gradient Descent with Application to Pruning
by: Shah, Aditya, et al.
Published: (2024)

Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks
by: Xue, Lulu, et al.
Published: (2024)

On the Upper Bounds for the Matrix Spectral Norm
by: Naumov, Alexey, et al.
Published: (2025)

Small Gradient Norm Regret for Online Convex Optimization
by: Gao, Wenzhi, et al.
Published: (2026)

Pruning for Sparse Diffusion Models based on Gradient Flow
by: Wan, Ben, et al.
Published: (2025)

High-Layer Attention Pruning with Rescaling
by: Liu, Songtao, et al.
Published: (2025)