Cover Image

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gu, Naibin, Fu, Peng, Liu, Xiyu, Shen, Bowen, Lin, Zheng, Wang, Weiping
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models
by: Gu, Naibin, et al.
Published: (2025)

CBP-Tuning: Efficient Local Customization for Black-box Large Language Models
by: Zhao, Jiaxuan, et al.
Published: (2025)

Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing
by: Liu, Xiyu, et al.
Published: (2025)

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
by: Feng, Yuchen, et al.
Published: (2025)

Relation Also Knows: Rethinking the Recall and Editing of Factual Associations in Auto-Regressive Transformer Language Models
by: Liu, Xiyu, et al.
Published: (2024)

PEFT-U: Parameter-Efficient Fine-Tuning for User Personalization
by: Clarke, Christopher, et al.
Published: (2024)

ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning
by: Li, Xianming, et al.
Published: (2026)

PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark
by: Belanec, Robert, et al.
Published: (2025)

MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba
by: Yoshimura, Masakazu, et al.
Published: (2024)

TS-PEFT: Unveiling Token-Level Redundancy in Parameter-Efficient Fine-Tuning
by: Ma, Dabiao, et al.
Published: (2025)

BeamLoRA: Beam-Constraint Low-Rank Adaptation
by: Gu, Naibin, et al.
Published: (2025)

PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models
by: Belanec, Robert, et al.
Published: (2025)

AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning
by: Zhou, Han, et al.
Published: (2023)

Advantageous Parameter Expansion Training Makes Better Large Language Models
by: Gu, Naibin, et al.
Published: (2025)

Beyond the Covariance Trap: Unlocking Generalization in Same-Subject Knowledge Editing for Large Language Models
by: Liu, Xiyu, et al.
Published: (2026)

PEFT A2Z: Parameter-Efficient Fine-Tuning Survey for Large Language and Vision Models
by: Prottasha, Nusrat Jahan, et al.
Published: (2025)

X-PEFT: eXtremely Parameter-Efficient Fine-Tuning for Extreme Multi-Profile Scenarios
by: Kwak, Namju, et al.
Published: (2024)

Pruning Large Language Models to Intra-module Low-rank Architecture with Transitional Activations
by: Shen, Bowen, et al.
Published: (2024)

Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Models
by: Peng, Zhiyuan, et al.
Published: (2024)

Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning
by: Wang, Shaobo, et al.
Published: (2025)

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
by: Huang, Yangyi, et al.
Published: (2026)

Parameter-Efficient Fine-Tuning via Circular Convolution
by: Chen, Aochuan, et al.
Published: (2024)

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
by: Chen, Hao, et al.
Published: (2026)

OBCache: Optimal Brain KV Cache Pruning for Efficient Long-Context LLM Inference
by: Gu, Yuzhe, et al.
Published: (2025)

Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?
by: Wang, Yatong, et al.
Published: (2026)

High-Rank Structured Modulation for Parameter-Efficient Fine-Tuning
by: Liu, Yongkang, et al.
Published: (2026)

ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization
by: Yadav, Prateek, et al.
Published: (2023)

On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
by: Lab, Mind, et al.
Published: (2026)

TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimization
by: Yao, Dingyu, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning With Adapters
by: Chen, Keyu, et al.
Published: (2024)

Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation
by: Su, Tong, et al.
Published: (2024)

Weights-Rotated Preference Optimization for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)

FISH-Tuning: Enhancing PEFT Methods with Fisher Information
by: Xue, Kang, et al.
Published: (2025)

PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
by: Yu, Tongzhou, et al.
Published: (2025)

Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation
by: Ma, Xinyu, et al.
Published: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
by: Gu, Naibin, et al.
Published: (2025)

DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models
by: Deng, Wenlong, et al.
Published: (2024)

Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models
by: Wang, Jifeng, et al.
Published: (2024)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
by: Zhao, Bowen, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning with Discrete Fourier Transform
by: Gao, Ziqi, et al.
Published: (2024)