Cover Image

Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Qinsi, Ke, Jinghan, Tomizuka, Masayoshi, Chen, Yiran, Keutzer, Kurt, Xu, Chenfeng
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Angles Don't Lie: Unlocking Training-Efficient RL Through the Model's Own Signals
by: Wang, Qinsi, et al.
Published: (2025)

IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression
by: Abbasi, Ali, et al.
Published: (2026)

DipSVD: Dual-importance Protected SVD for Efficient LLM Compression
by: Ding, Xuan, et al.
Published: (2025)

FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
by: Shao, Zishan, et al.
Published: (2025)

AA-SVD : Anchored and Adaptive SVD for Large Language Model Compression
by: Sinha, Atul Kumar, et al.
Published: (2026)

SVD-NO: Learning PDE Solution Operators with SVD Integral Kernels
by: Koren, Noam, et al.
Published: (2025)

Zero Sum SVD: Balancing Loss Sensitivity for Low Rank LLM Compression
by: Abbasi, Ali, et al.
Published: (2026)

Beyond Uniform SVD:Dual-Level Optimization across Columns and Modules for LLM Compression
by: Xv, Lin, et al.
Published: (2025)

SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models
by: Hong, Chengjie, et al.
Published: (2026)

SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression
by: Wang, Xin, et al.
Published: (2024)

Low-Rank Prehab: Preparing Neural Networks for SVD Compression
by: Qin, Haoran, et al.
Published: (2025)

SVD Contextual Sparsity Predictors for Fast LLM Inference
by: Serbin, Georgii, et al.
Published: (2026)

Residual Policy Gradient: A Reward View of KL-regularized Objective
by: Wang, Pengcheng, et al.
Published: (2025)

KQ-SVD: Compressing the KV Cache with Provable Guarantees on Attention Fidelity
by: Lesens, Damien, et al.
Published: (2025)

Enhancing Delta Compression in LLMs via SVD-based Quantization Error Minimization
by: Xiong, Boya, et al.
Published: (2025)

PCA, SVD, and Centering of Data
by: Kim, Donggun, et al.
Published: (2023)

Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compression
by: Zhu, Hengyi, et al.
Published: (2026)

$k$-SVD with Gradient Descent
by: Jedra, Yassir, et al.
Published: (2025)

ARA: Adaptive Rank Allocation for Efficient Large Language Model SVD Compression
by: Xv, Lin, et al.
Published: (2025)

Concatenated Matrix SVD: Compression Bounds, Incremental Approximation, and Error-Constrained Clustering
by: Shamrai, Maksym
Published: (2026)

tenSVD algorithm for compression
by: Gallo, Michele
Published: (2025)

FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
by: Wu, Wenhao, et al.
Published: (2026)

A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervision
by: Peng, Chensheng, et al.
Published: (2024)

StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation
by: Kodaira, Akio, et al.
Published: (2023)

LASER: Low-Rank Activation SVD for Efficient Recursion
by: Çakar, Ege, et al.
Published: (2026)

Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integration
by: Baharav, Tavor Z., et al.
Published: (2025)

PixelGaussian: Generalizable 3D Gaussian Reconstruction from Arbitrary Views
by: Fei, Xin, et al.
Published: (2024)

Driv3R: Learning Dense 4D Reconstruction for Autonomous Driving
by: Fei, Xin, et al.
Published: (2024)

Efficient GPU implementation of randomized SVD and its applications
by: Struski, Łukasz, et al.
Published: (2021)

SVD-AE: Simple Autoencoders for Collaborative Filtering
by: Hong, Seoyoung, et al.
Published: (2024)

Post-processing for Fair Regression via Explainable SVD
by: Zuo, Zhiqun, et al.
Published: (2025)

Improved Immiscible Diffusion: Accelerate Diffusion Training by Reducing Its Miscibility
by: Li, Yiheng, et al.
Published: (2025)

Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment
by: Li, Yiheng, et al.
Published: (2024)

Looking Backward: Streaming Video-to-Video Translation with Feature Banks
by: Liang, Feng, et al.
Published: (2024)

Bisimulation metric for Model Predictive Control
by: Shimizu, Yutaka, et al.
Published: (2024)

Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models
by: Chekalina, Viktoriia, et al.
Published: (2025)

Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective
by: Yao, Xinhao, et al.
Published: (2024)

Operator SVD with Neural Networks via Nested Low-Rank Approximation
by: Ryu, J. Jon, et al.
Published: (2024)

SAES-SVD: Self-Adaptive Suppression of Accumulated and Local Errors for SVD-based LLM Compression
by: Hu, Xing, et al.
Published: (2026)

SOLAR: SVD-Optimized Lifelong Attention for Recommendation
by: Zhang, Chenghao, et al.
Published: (2026)