Imagen de Portada

QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Yutong, Wang, Haiyu, Zhang, Sai Qian
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

WSVD: Weighted Low-Rank Approximation for Fast and Efficient Execution of Low-Precision Vision-Language Models
por: Wang, Haiyu, et al.
Publicado: (2026)

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models
por: Wang, Haiyu, et al.
Publicado: (2026)

SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression
por: Mozaffari, Mohammad, et al.
Publicado: (2024)

Loki: Low-rank Keys for Efficient Sparse Attention
por: Singhania, Prajwal, et al.
Publicado: (2024)

MLorc: Momentum Low-rank Compression for Memory Efficient Large Language Model Adaptation
por: Shen, Wei, et al.
Publicado: (2025)

Tensor Low-rank Approximation of Finite-horizon Value Functions
por: Rozada, Sergio, et al.
Publicado: (2024)

Efficient Differentiable Approximation of Generalized Low-rank Regularization
por: Li, Naiqi, et al.
Publicado: (2025)

Unified Low-rank Compression Framework for Click-through Rate Prediction
por: Yu, Hao, et al.
Publicado: (2024)

CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts
por: Yin, Xiangyang, et al.
Publicado: (2026)

AutoQRA: Joint Optimization of Mixed-Precision Quantization and Low-rank Adapters for Efficient LLM Fine-Tuning
por: Zhou, Changhai, et al.
Publicado: (2026)

Weighted Low-rank Approximation via Stochastic Gradient Descent on Manifolds
por: Xu, Conglong, et al.
Publicado: (2025)

Bayesian Low-rank Adaptation for Large Language Models
por: Yang, Adam X., et al.
Publicado: (2023)

Compressing Large Language Models using Low Rank and Low Precision Decomposition
por: Saha, Rajarshi, et al.
Publicado: (2024)

Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation
por: Song, Zhao, et al.
Publicado: (2023)

UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs
por: Chiang, Hung-Yueh, et al.
Publicado: (2025)

Low-Rank Key Value Attention
por: O'Neill, James, et al.
Publicado: (2026)

Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
por: Huang, Jia-Hong, et al.
Publicado: (2024)

Efficient Algorithms for Regularized Nonnegative Scale-invariant Low-rank Approximation Models
por: Cohen, Jeremy E., et al.
Publicado: (2024)

LOST: Low-rank and Sparse Pre-training for Large Language Models
por: Li, Jiaxi, et al.
Publicado: (2025)

Low-Rank Tensor Approximation of Weights in Large Language Models via Cosine Lanczos Bidiagonalization
por: Ichi, A. El, et al.
Publicado: (2026)

Key and Value Weights Are Probably All You Need: On the Necessity of the Query, Key, Value weight Triplet in Self-Attention Transformers
por: Karbevski, Marko, et al.
Publicado: (2025)

Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
por: Gao, Wei, et al.
Publicado: (2025)

Collage: Light-Weight Low-Precision Strategy for LLM Training
por: Yu, Tao, et al.
Publicado: (2024)

Towards Reversible Model Merging For Low-rank Weights
por: Alipour, Mohammadsajad, et al.
Publicado: (2025)

Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures
por: Chen, Yiming, et al.
Publicado: (2024)

GNMR: Runtime Stability Control for Low-Precision Large Language Model Training
por: Kong, Boao, et al.
Publicado: (2026)

Reweighted Solutions for Weighted Low Rank Approximation
por: Woodruff, David P., et al.
Publicado: (2024)

Low-rank Momentum Factorization for Memory Efficient Training
por: Mahdavinia, Pouria, et al.
Publicado: (2025)

LRD-MPC: Efficient MPC Inference through Low-rank Decomposition
por: Tang, Tingting, et al.
Publicado: (2026)

Extreme Model Compression with Structured Sparsity at Low Precision
por: Liu, Dan, et al.
Publicado: (2025)

VALUE: Value-Aware Large Language Model for Query Rewriting via Weighted Trie in Sponsored Search
por: Zhang, Xiao, et al.
Publicado: (2025)

Low-Rank Matrix Approximation for Neural Network Compression
por: Cherukuri, Kalyan, et al.
Publicado: (2025)

Efficient Low-rank Identification via Accelerated Iteratively Reweighted Nuclear Norm Minimization
por: Wang, Hao, et al.
Publicado: (2024)

Dynamic Low-rank Approximation of Full-Matrix Preconditioner for Training Generalized Linear Models
por: Matveeva, Tatyana, et al.
Publicado: (2025)

Efficient Compression of Overparameterized Deep Models through Low-Dimensional Learning Dynamics
por: Kwon, Soo Min, et al.
Publicado: (2023)

Clustering-Based Low-Rank Matrix Approximation for Medical Image Compression
por: Hamlomo, Sisipho, et al.
Publicado: (2025)

Demystifying Language Model Forgetting with Low-rank Example Associations
por: Jin, Xisen, et al.
Publicado: (2024)

Towards Better Generalization: Weight Decay Induces Low-rank Bias for Neural Networks
por: Chen, Ke, et al.
Publicado: (2024)

Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities
por: Hao, Zhiwei, et al.
Publicado: (2025)

Neural Weight Compression for Language Models
por: Ryu, Jegwang, et al.
Publicado: (2025)