Cover Image

MAP: Revisiting Weight Decomposition for Low-Rank Adaptation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Si, Chongjie, Shi, Zhiyi, Wang, Yadao, Yang, Xiaokang, Rahardja, Susanto, Shen, Wei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Task-Specific Directions: Definition, Exploration, and Utilization in Parameter Efficient Fine-Tuning
by: Si, Chongjie, et al.
Published: (2024)

NAN: A Training-Free Solution to Coefficient Estimation in Model Merging
by: Si, Chongjie, et al.
Published: (2025)

Weight Spectra Induced Efficient Model Adaptation
by: Si, Chongjie, et al.
Published: (2025)

See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition
by: Si, Chongjie, et al.
Published: (2024)

Unveiling the Mystery of Weight in Large Foundation Models: Gaussian Distribution Never Fades
by: Si, Chongjie, et al.
Published: (2025)

Revisiting Sparsity Constraint Under High-Rank Property in Partial Multi-Label Learning
by: Si, Chongjie, et al.
Published: (2025)

EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition
by: Nasiri, Hamid, et al.
Published: (2025)

Generalized Tensor-based Parameter-Efficient Fine-Tuning via Lie Group Transformations
by: Si, Chongjie, et al.
Published: (2025)

LoTR: Low Tensor Rank Weight Adaptation
by: Bershatsky, Daniel, et al.
Published: (2024)

SBoRA: Low-Rank Adaptation with Regional Weight Updates
by: Po, Lai-Man, et al.
Published: (2024)

FlexLoRA: Entropy-Guided Flexible Low-Rank Adaptation
by: Liu, Muqing, et al.
Published: (2026)

Dual Decomposition of Weights and Singular Value Low Rank Adaptation
by: Han, Jialong, et al.
Published: (2025)

BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation
by: Qin, Peijia, et al.
Published: (2024)

Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition
by: Cho, Yoonjun, et al.
Published: (2025)

Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
by: He, Zhengfu, et al.
Published: (2025)

Variational Low-Rank Adaptation Using IVON
by: Cong, Bai, et al.
Published: (2024)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
by: Wang, Shaowen, et al.
Published: (2024)

BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models
by: Wang, Yibin, et al.
Published: (2024)

Contextually Guided Transformers via Low-Rank Adaptation
by: Zhmoginov, Andrey, et al.
Published: (2025)

A Bayesian Interpretation of Adaptive Low-Rank Adaptation
by: Chen, Haolin, et al.
Published: (2024)

Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
by: Li, Shiwei, et al.
Published: (2025)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
by: Yang, Yaming, et al.
Published: (2024)

Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications
by: Li, Yang, et al.
Published: (2024)

OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
by: Feng, Jinyuan, et al.
Published: (2025)

FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing
by: Smith, James Seale, et al.
Published: (2025)

CSULoRA: Closest Safe Update Low-Rank Adaptation
by: Breneur, Oleksandr Marchenko, et al.
Published: (2026)

The Expressive Power of Low-Rank Adaptation
by: Zeng, Yuchen, et al.
Published: (2023)

Appeal: Allow Mislabeled Samples the Chance to be Rectified in Partial Label Learning
by: Si, Chongjie, et al.
Published: (2023)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
by: Tang, Chuanyu, et al.
Published: (2024)

FLoE: Fisher-Based Layer Selection for Efficient Sparse Adaptation of Low-Rank Experts
by: Wang, Xinyi, et al.
Published: (2025)

CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models
by: Xiao, Xiaojun, et al.
Published: (2024)

Batched Low-Rank Adaptation of Foundation Models
by: Wen, Yeming, et al.
Published: (2023)

Low Rank Adaptation for Adversarial Perturbation
by: Liu, Han, et al.
Published: (2026)

Why Can Accurate Models Be Learned from Inaccurate Annotations?
by: Si, Chongjie, et al.
Published: (2025)

SwitchLoRA: Switched Low-Rank Adaptation Can Learn Full-Rank Information
by: Zhou, Kaiye, et al.
Published: (2024)

MSS-PAE: Saving Autoencoder-based Outlier Detection from Unexpected Reconstruction
by: Tan, Xu, et al.
Published: (2023)

Weakly-supervised anomaly detection for multimodal data distributions
by: Tan, Xu, et al.
Published: (2024)

Optimizing Fine-Tuning through Advanced Initialization Strategies for Low-Rank Adaptation
by: Xue, Yongfu
Published: (2025)

Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
by: Sengupta, Ayan, et al.
Published: (2024)

LoRMA: Low-Rank Multiplicative Adaptation for LLMs
by: Bihany, Harsh, et al.
Published: (2025)