Cover Image

Hierarchical Sparse Plus Low Rank Compression of LLM

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kumar, Pawan, Gupta, Aditi
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs
by: Mozaffari, Mohammad, et al.
Published: (2024)

DeltaLLM: Compress LLMs with Low-Rank Deltas between Shared Weights
by: Mikaelyan, Liana, et al.
Published: (2025)

Low-Rank Compression of Language Models via Differentiable Rank Selection
by: Sundrani, Sidhant, et al.
Published: (2025)

Low-Rank Adapters Meet Neural Architecture Search for LLM Compression
by: Muñoz, J. Pablo, et al.
Published: (2025)

Palu: Compressing KV-Cache with Low-Rank Projection
by: Chang, Chi-Chih, et al.
Published: (2024)

Verified Neural Compressed Sensing
by: Bunel, Rudy, et al.
Published: (2024)

OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition
by: Zhang, Stephen, et al.
Published: (2024)

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
by: Shi, Jiang-Xin, et al.
Published: (2025)

MGAA: Multi-Granular Adaptive Allocation fof Low-Rank Compression of LLMs
by: Li, Guangyan, et al.
Published: (2025)

Lillama: Large Language Models Compression via Low-Rank Feature Distillation
by: Sy, Yaya, et al.
Published: (2024)

MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
by: Modoranu, Ionut-Vlad, et al.
Published: (2026)

Sparse High Rank Adapters
by: Bhardwaj, Kartikeya, et al.
Published: (2024)

Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
by: Cong, Peizhuang, et al.
Published: (2025)

Compressing Large Language Models using Low Rank and Low Precision Decomposition
by: Saha, Rajarshi, et al.
Published: (2024)

CALR: Corrective Adaptive Low-Rank Decomposition for Efficient Large Language Model Layer Compression
by: Kautsar, Muchammad Daniyal, et al.
Published: (2025)

Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation
by: Yaras, Can, et al.
Published: (2024)

UltraSketchLLM: Saliency-Driven Sketching for Ultra-Low Bit LLM Compression
by: Zou, Sunan, et al.
Published: (2025)

Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression
by: Zhang, Xi, et al.
Published: (2025)

ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration
by: Yan, Xianglong, et al.
Published: (2025)

Lossless Model Compression via Joint Low-Rank Factorization Optimization
by: Zhang, Boyang, et al.
Published: (2024)

Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
by: Saxena, Utkarsh, et al.
Published: (2024)

Low-Rank Compression of Pretrained Models via Randomized Subspace Iteration
by: Pourkamali-Anaraki, Farhad
Published: (2026)

SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
by: Tang, Anke, et al.
Published: (2024)

Breaking the Blocks: Continuous Low-Rank Decomposed Scaling for Unified LLM Quantization and Adaptation
by: Tang, Pingzhi, et al.
Published: (2026)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
by: Zhu, Yuxuan, et al.
Published: (2025)

SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling
by: Ji, Xiaodong, et al.
Published: (2025)

Memorization Sinks: Isolating Memorization during LLM Training
by: Ghosal, Gaurav R., et al.
Published: (2025)

GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection
by: Su, DiJia, et al.
Published: (2025)

Lotus: Efficient LLM Training by Randomized Low-Rank Gradient Projection with Adaptive Subspace Switching
by: Miao, Tianhao, et al.
Published: (2026)

LANCE: Low Rank Activation Compression for Efficient On-Device Continual Learning
by: Apolinario, Marco Paul E., et al.
Published: (2025)

D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
by: Li, Junlin, et al.
Published: (2026)

Large Language Model Compression with Global Rank and Sparsity Optimization
by: Zhou, Changhai, et al.
Published: (2025)

Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
by: Gong, Wenbo, et al.
Published: (2025)

Towards Symmetric Low-Rank Adapters
by: Panoutsos, Tales, et al.
Published: (2025)

The Primacy of Magnitude in Low-Rank Adaptation
by: Zhang, Zicheng, et al.
Published: (2025)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
by: Yang, Cheng, et al.
Published: (2024)

Mixture-of-Subspaces in Low-Rank Adaptation
by: Wu, Taiqiang, et al.
Published: (2024)

Global Low-Rank, Local Full-Rank: The Holographic Encoding of Learned Algorithms
by: Xu, Yongzhong
Published: (2026)

LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
by: Zhang, Rongzhi, et al.
Published: (2024)

Understanding Transformers for Time Series: Rank Structure, Flow-of-ranks, and Compressibility
by: Yu, Annan, et al.
Published: (2025)