Cover Image

OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Stephen, Papyan, Vardan
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparsest Models Elude Pruning: An Exposé of Pruning's Current Capabilities
by: Zhang, Stephen, et al.
Published: (2024)

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
by: Zhang, Stephen, et al.
Published: (2025)

Transformer Block Coupling and its Correlation with Generalization in LLMs
by: Aubry, Murdock, et al.
Published: (2024)

MUXQ: Mixed-to-Uniform Precision MatriX Quantization via Low-Rank Outlier Decomposition
by: Lee, Seoungsub, et al.
Published: (2026)

SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs
by: Mozaffari, Mohammad, et al.
Published: (2024)

Laplacian Heads Improve Transformers by Smoothing Token Representations
by: Zhang, Yuchong, et al.
Published: (2026)

Residual Alignment: Uncovering the Mechanisms of Residual Networks
by: Li, Jianing, et al.
Published: (2024)

On the Importance of Gaussianizing Representations
by: Eftekhari, Daniel, et al.
Published: (2025)

Long-Term Outlier Prediction Through Outlier Score Modeling
by: Aoki, Yuma, et al.
Published: (2026)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
by: Yang, Cheng, et al.
Published: (2024)

Low-Rank Tensor Decompositions for the Theory of Neural Networks
by: Borsoi, Ricardo, et al.
Published: (2025)

Hierarchical Sparse Plus Low Rank Compression of LLM
by: Kumar, Pawan, et al.
Published: (2025)

Identifying Sparsely Active Circuits Through Local Loss Landscape Decomposition
by: Chrisman, Brianna, et al.
Published: (2025)

Optimal Policy Sparsification and Low Rank Decomposition for Deep Reinforcement Learning
by: Goddla, Vikram
Published: (2024)

CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention
by: Zhou, Zhongzhu, et al.
Published: (2026)

Sparse Decomposition of Graph Neural Networks
by: Hu, Yaochen, et al.
Published: (2024)

GeoRA: Geometry-Aware Low-Rank Adaptation for RLVR
by: Zhang, Jiaying, et al.
Published: (2026)

Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
by: Wei, Boyi, et al.
Published: (2024)

Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decomposition
by: Wu, Xinghao, et al.
Published: (2024)

Pushing Boundaries: Mixup's Influence on Neural Collapse
by: Fisher, Quinn, et al.
Published: (2024)

Compressing Large Language Models using Low Rank and Low Precision Decomposition
by: Saha, Rajarshi, et al.
Published: (2024)

CALR: Corrective Adaptive Low-Rank Decomposition for Efficient Large Language Model Layer Compression
by: Kautsar, Muchammad Daniyal, et al.
Published: (2025)

Sparse High Rank Adapters
by: Bhardwaj, Kartikeya, et al.
Published: (2024)

SAFE: Finding Sparse and Flat Minima to Improve Pruning
by: Lee, Dongyeop, et al.
Published: (2025)

Hessian Aware Low-Rank Perturbation for Order-Robust Continual Learning
by: Li, Jiaqi, et al.
Published: (2023)

SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
by: Tang, Anke, et al.
Published: (2024)

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
by: Zhang, Longteng, et al.
Published: (2026)

Robust Statistical Scaling of Outlier Scores: Improving the Quality of Outlier Probabilities for Outliers (Extended Version)
by: Röchner, Philipp, et al.
Published: (2024)

Fast and Low-Cost Genomic Foundation Models via Outlier Removal
by: Luo, Haozheng, et al.
Published: (2025)

Low-Rank Quantization-Aware Training for LLMs
by: Bondarenko, Yelysei, et al.
Published: (2024)

Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning
by: Choi, Moonseok, et al.
Published: (2023)

SparseSwaps: Tractable LLM Pruning Mask Refinement at Scale
by: Zimmer, Max, et al.
Published: (2025)

The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models
by: Xiao, Yang, et al.
Published: (2025)

TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models
by: Mu, Lin, et al.
Published: (2026)

An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning
by: Li, Cen-Jhih, et al.
Published: (2025)

LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis
by: Zhang, Qingyue, et al.
Published: (2025)

Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training
by: Emadi, Seyed Morteza
Published: (2026)

Cross-Modal Bayesian Low-Rank Adaptation for Uncertainty-Aware Multimodal Learning
by: Naderi, Habibeh, et al.
Published: (2026)

ThanoRA: Task Heterogeneity-Aware Multi-Task Low-Rank Adaptation
by: Liang, Jian, et al.
Published: (2025)

The Primacy of Magnitude in Low-Rank Adaptation
by: Zhang, Zicheng, et al.
Published: (2025)