Cover Image

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shi, Jiang-Xin, Wei, Wen-Da, Qi, Jin-Fei, Chen, Xuanyu, Wei, Tong, Li, Yu-Feng
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Penrose Tiled Low-Rank Compression and Section-Wise Q&A Fine-Tuning: A General Framework for Domain-Specific Large Language Model Adaptation
by: Kuo, Chuan-Wei, et al.
Published: (2025)

Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning
by: Han, Xiao, et al.
Published: (2025)

TsqLoRA: Towards Sensitivity and Quality Low-Rank Adaptation for Efficient Fine-Tuning
by: Chen, Yu, et al.
Published: (2025)

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
by: Zhang, Longteng, et al.
Published: (2026)

LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization
by: Wang, Xujia, et al.
Published: (2025)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
by: Zhang, Chengqian, et al.
Published: (2026)

Joint Localization and Activation Editing for Low-Resource Fine-Tuning
by: Lai, Wen, et al.
Published: (2025)

Improving Anomalous Sound Detection via Low-Rank Adaptation Fine-Tuning of Pre-Trained Audio Models
by: Zheng, Xinhu, et al.
Published: (2024)

Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
by: Zhang, Wenhao, et al.
Published: (2026)

MSPLoRA: A Multi-Scale Pyramid Low-Rank Adaptation for Efficient Model Fine-Tuning
by: Zhao, Jiancheng, et al.
Published: (2025)

LANCE: Low Rank Activation Compression for Efficient On-Device Continual Learning
by: Apolinario, Marco Paul E., et al.
Published: (2025)

ElaLoRA: Elastic & Learnable Low-Rank Adaptation for Efficient Model Fine-Tuning
by: Chang, Huandong, et al.
Published: (2025)

LoR2C : Low-Rank Residual Connection Adaptation for Parameter-Efficient Fine-Tuning
by: Zhao, Jiancheng, et al.
Published: (2025)

LARGO: Low-Rank Regulated Gradient Projection for Robust Parameter Efficient Fine-Tuning
by: Zhang, Haotian, et al.
Published: (2025)

FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference
by: Lu, Yu-Chen, et al.
Published: (2025)

FRoD: Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence
by: Wan, Guoan, et al.
Published: (2025)

Memory-Efficient Vision Transformers: An Activation-Aware Mixed-Rank Compression Strategy
by: Azizi, Seyedarmin, et al.
Published: (2024)

APAR: Modeling Irregular Target Functions in Tabular Regression via Arithmetic-Aware Pre-Training and Adaptive-Regularized Fine-Tuning
by: Wu, Hong-Wei, et al.
Published: (2024)

GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Jung, Yeonjoon, et al.
Published: (2025)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
by: Sun, Mengyang, et al.
Published: (2025)

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression
by: Huang, Xinhao, et al.
Published: (2026)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
by: Liu, Zeyu, et al.
Published: (2024)

Palu: Compressing KV-Cache with Low-Rank Projection
by: Chang, Chi-Chih, et al.
Published: (2024)

RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
by: Liu, Jun, et al.
Published: (2025)

RankAdaptor: Hierarchical Rank Allocation for Efficient Fine-Tuning Pruned LLMs via Performance Model
by: Zhou, Changhai, et al.
Published: (2024)

SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping
by: Lu, Yu-Chen, et al.
Published: (2025)

PAFT: Prompt-Agnostic Fine-Tuning
by: Wei, Chenxing, et al.
Published: (2025)

Localized LoRA: A Structured Low-Rank Approximation for Efficient Fine-Tuning
by: Barazandeh, Babak, et al.
Published: (2025)

Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation
by: Yang, Yuchen, et al.
Published: (2024)

Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
by: Cong, Peizhuang, et al.
Published: (2025)

On-Policy Supervised Fine-Tuning for Efficient Reasoning
by: Zhao, Anhao, et al.
Published: (2026)

FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
by: Shao, Zishan, et al.
Published: (2025)

Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
by: Raje, Arian, et al.
Published: (2025)

Low-Rank Compression of Language Models via Differentiable Rank Selection
by: Sundrani, Sidhant, et al.
Published: (2025)

EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation
by: Liu, Shih-Yang, et al.
Published: (2024)

ClusterUCB: Efficient Gradient-Based Data Selection for Targeted Fine-Tuning of LLMs
by: Wang, Zige, et al.
Published: (2025)

H2Tune: Federated Foundation Model Fine-Tuning with Hybrid Heterogeneity
by: Guo, Wei, et al.
Published: (2025)

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training
by: Xi, Haocheng, et al.
Published: (2024)

Probe-Free Low-Rank Activation Intervention
by: Jiang, Chonghe, et al.
Published: (2025)

Flexora: Flexible Low Rank Adaptation for Large Language Models
by: Wei, Chenxing, et al.
Published: (2024)