Cover Image

Pivoting Factorization: A Compact Meta Low-Rank Representation of Sparsity for Efficient Inference in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Jialin, Zhang, Yingtao, Cannistraci, Carlo Vittorio
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparse Spectral Training and Inference on Euclidean and Hyperbolic Neural Networks
by: Zhao, Jialin, et al.
Published: (2024)

Brain network science modelling of sparse neural networks enables Transformers and LLMs to perform as fully connected
by: Zhang, Yingtao, et al.
Published: (2025)

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
by: Zhang, Longteng, et al.
Published: (2026)

Efficient Low Rank Attention for Long-Context Inference in Large Language Models
by: Li, Tenghui, et al.
Published: (2025)

R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference
by: Zhang, Zhenyu, et al.
Published: (2025)

ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
by: Li, Jiaxi, et al.
Published: (2026)

Large Language Model Compression with Global Rank and Sparsity Optimization
by: Zhou, Changhai, et al.
Published: (2025)

SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference
by: Ma, Hao, et al.
Published: (2026)

GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference
by: Zeng, Chao, et al.
Published: (2024)

Activity Sparsity Complements Weight Sparsity for Efficient RNN Inference
by: Mukherji, Rishav, et al.
Published: (2023)

ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
by: Zhao, Youpeng, et al.
Published: (2024)

Novel Pivoted Cholesky Decompositions for Efficient Gaussian Process Inference
by: de Roos, Filip, et al.
Published: (2025)

Meta-Learning at Scale for Large Language Models via Low-Rank Amortized Bayesian Meta-Learning
by: Zhang, Liyi, et al.
Published: (2025)

Dynamic Low-Rank Sparse Adaptation for Large Language Models
by: Huang, Weizhong, et al.
Published: (2025)

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

Large Language Models As Evolution Strategies
by: Lange, Robert Tjarko, et al.
Published: (2024)

Learn To be Efficient: Build Structured Sparsity in Large Language Models
by: Zheng, Haizhong, et al.
Published: (2024)

Decentralized Low-Rank Fine-Tuning of Large Language Models
by: Ghiasvand, Sajjad, et al.
Published: (2025)

MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators
by: Huang, Beichen, et al.
Published: (2025)

FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
by: Shao, Zishan, et al.
Published: (2025)

Compact Lifted Relaxations for Low-Rank Optimization
by: Cory-Wright, Ryan, et al.
Published: (2026)

BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation
by: Xu, Peng, et al.
Published: (2024)

Activation Sparsity Opportunities for Compressing General Large Language Models
by: Dhar, Nobel, et al.
Published: (2024)

RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
by: Zhang, Yilang, et al.
Published: (2025)

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
by: Tang, Jiaming, et al.
Published: (2024)

Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations
by: Zhao, Yize, et al.
Published: (2024)

Efficient Reward Identification In Max Entropy Reinforcement Learning with Sparsity and Rank Priors
by: Shehab, Mohamad Louai, et al.
Published: (2025)

Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
by: An, Tai, et al.
Published: (2025)

QR-LoRA: QR-Based Low-Rank Adaptation for Efficient Fine-Tuning of Large Language Models
by: Liang, Jessica, et al.
Published: (2025)

CALR: Corrective Adaptive Low-Rank Decomposition for Efficient Large Language Model Layer Compression
by: Kautsar, Muchammad Daniyal, et al.
Published: (2025)

EmbedLLM: Learning Compact Representations of Large Language Models
by: Zhuang, Richard, et al.
Published: (2024)

Universal Properties of Activation Sparsity in Modern Large Language Models
by: Szatkowski, Filip, et al.
Published: (2025)

E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity
by: Li, Yun, et al.
Published: (2023)

CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models
by: Lee, Donghyun, et al.
Published: (2024)

Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
by: Samplawski, Colin, et al.
Published: (2025)

Provable Meta-Learning with Low-Rank Adaptations
by: Block, Jacob L., et al.
Published: (2024)

Low-Rank Adaptation Redux for Large Models
by: Li, Bingcong, et al.
Published: (2026)

Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models
by: Asante, Daniel Agyei, et al.
Published: (2026)

LoLCATs: On Low-Rank Linearizing of Large Language Models
by: Zhang, Michael, et al.
Published: (2024)

Semi-supervised Symmetric Non-negative Matrix Factorization with Low-Rank Tensor Representation
by: Jia, Yuheng, et al.
Published: (2024)