Cover Image

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lu, Xudong, Zhou, Aojun, Xu, Yuhui, Zhang, Renrui, Gao, Peng, Li, Hongsheng
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
by: Zhang, Renrui, et al.
Published: (2023)

BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation
by: Xu, Peng, et al.
Published: (2024)

TerDiT: Ternary Diffusion Models with Transformers
by: Lu, Xudong, et al.
Published: (2024)

Long Exposure: Accelerating Parameter-Efficient Fine-Tuning for LLMs under Shadowy Sparsity
by: Wang, Tuowei, et al.
Published: (2025)

Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models
by: Weyssow, Martin, et al.
Published: (2023)

DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model
by: Gao, Chao, et al.
Published: (2024)

Less but Better: Parameter-Efficient Fine-Tuning of Large Language Models for Personality Detection
by: Shen, Lingzhi, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning via Circular Convolution
by: Chen, Aochuan, et al.
Published: (2024)

L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models
by: Jeon, Hyesung, et al.
Published: (2024)

SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
by: Lin, Zheng, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning for Foundation Models
by: Zhang, Dan, et al.
Published: (2025)

Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models
by: Liao, Xiaoxuan, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of State Space Models
by: Galim, Kevin, et al.
Published: (2024)

Parameter-Efficient Tuning Large Language Models for Graph Representation Learning
by: Zhu, Qi, et al.
Published: (2024)

Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models
by: Ghiasvand, Sajjad, et al.
Published: (2024)

LoRA-SP: Streamlined Partial Parameter Adaptation for Resource-Efficient Fine-Tuning of Large Language Models
by: Wu, Yichao, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies
by: Wang, Luping, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning with Discrete Fourier Transform
by: Gao, Ziqi, et al.
Published: (2024)

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
by: Chen, Hao, et al.
Published: (2026)

SVFit: Parameter-Efficient Fine-Tuning of Large Pre-Trained Models Using Singular Values
by: Sun, Chengwei, et al.
Published: (2024)

BIPEFT: Budget-Guided Iterative Search for Parameter Efficient Fine-Tuning of Large Pretrained Language Models
by: Chang, Aofei, et al.
Published: (2024)

MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
by: Zhang, Renrui, et al.
Published: (2024)

Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation
by: Ma, Xinyu, et al.
Published: (2024)

SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
by: Liu, Yongkang, et al.
Published: (2026)

Graph-Based Spectral Decomposition for Parameter Coordination in Language Model Fine-Tuning
by: Zhang, Hanlu, et al.
Published: (2025)

Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning
by: Lin, Zekai, et al.
Published: (2026)

SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models
by: Liu, Dongyang, et al.
Published: (2024)

Crafting Efficient Fine-Tuning Strategies for Large Language Models
by: Oliver, Michael, et al.
Published: (2024)

Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
by: Wang, Fei, et al.
Published: (2024)

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
by: Jiang, Ting, et al.
Published: (2024)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
by: Yang, Yifan, et al.
Published: (2024)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
by: Liu, Zeyu, et al.
Published: (2024)

Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models
by: Doering, Nigel, et al.
Published: (2024)

TPP-LLM: Modeling Temporal Point Processes by Efficiently Fine-Tuning Large Language Models
by: Liu, Zefang, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain
by: Gema, Aryo Pradipta, et al.
Published: (2023)

Dissecting Fine-Tuning Unlearning in Large Language Models
by: Hong, Yihuai, et al.
Published: (2024)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Zhang, Haojie
Published: (2025)

Energy-Efficient Split Learning for Fine-Tuning Large Language Models in Edge Networks
by: Li, Zuguang, et al.
Published: (2024)

AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning
by: Zhou, Han, et al.
Published: (2023)