Cover Image

Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yao, Kai, Gao, Penglei, Li, Lichun, Zhao, Yuan, Wang, Xiaofeng, Wang, Wei, Zhu, Jianke
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models
by: Yao, Kai, et al.
Published: (2025)

GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning
by: Tian, Kaiyuan, et al.
Published: (2026)

ScaleOT: Privacy-utility-scalable Offsite-tuning with Dynamic LayerReplace and Selective Rank Compression
by: Yao, Kai, et al.
Published: (2024)

Less Could Be Better: Parameter-efficient Fine-tuning Advances Medical Vision Foundation Models
by: Lian, Chenyu, et al.
Published: (2024)

OPa-Ma: Text Guided Mamba for 360-degree Image Out-painting
by: Gao, Penglei, et al.
Published: (2024)

Less but Better: Parameter-Efficient Fine-Tuning of Large Language Models for Personality Detection
by: Shen, Lingzhi, et al.
Published: (2025)

Full Parameter Fine-tuning for Large Language Models with Limited Resources
by: Lv, Kai, et al.
Published: (2023)

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
by: Liu, Yuxi, et al.
Published: (2025)

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
by: Liu, Yong, et al.
Published: (2024)

LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views
by: Roh, Yuji, et al.
Published: (2024)

HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
by: Zhao, Huaqin, et al.
Published: (2024)

Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts
by: Zhang, Xue, et al.
Published: (2025)

The Harder The Better: Maintaining Supervised Fine-tuning Generalization with Less but Harder Data
by: Shang, Zhaoyang, et al.
Published: (2025)

Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction
by: Liu, Zhexiong, et al.
Published: (2025)

Explain Less, Understand More: Jargon Detection via Personalized Parameter-Efficient Fine-tuning
by: Wu, Bohao, et al.
Published: (2025)

EMLoC: Emulator-based Memory-efficient Fine-tuning with LoRA Correction
by: Lin, Hsi-Che, et al.
Published: (2025)

Forgetting: A New Mechanism Towards Better Large Language Model Fine-tuning
by: Taheri, Ali, et al.
Published: (2025)

Stage-wise Fine-tuning for Graph-to-Text Generation
by: Wang, Qingyun, et al.
Published: (2021)

JudgeLM: Fine-tuned Large Language Models are Scalable Judges
by: Zhu, Lianghui, et al.
Published: (2023)

MeLo: Low-rank Adaptation is Better than Fine-tuning for Medical Image Diagnosis
by: Zhu, Yitao, et al.
Published: (2023)

Structure-based RNA Design by Step-wise Optimization of Latent Diffusion Model
by: Si, Qi, et al.
Published: (2026)

Parameter-efficient Fine-tuning in Hyperspherical Space for Open-vocabulary Semantic Segmentation
by: Peng, Zelin, et al.
Published: (2024)

EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models
by: Wu, Jialin, et al.
Published: (2025)

Layer-mediated tuning of spin and valley physics in stacked tetragonal altermagnetic bilayers
by: Tian, Jianke, et al.
Published: (2026)

Harmony in Divergence: Towards Fast, Accurate, and Memory-efficient Zeroth-order LLM Fine-tuning
by: Tan, Qitao, et al.
Published: (2025)

A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models
by: Zhao, Pengxiang, et al.
Published: (2024)

RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuning
by: Wang, Haoyu, et al.
Published: (2024)

Exploiting Layer Normalization Fine-tuning in Visual Transformer Foundation Models for Classification
by: Tan, Zhaorui, et al.
Published: (2025)

Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation
by: Huang, Kai, et al.
Published: (2023)

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
by: Ben-Zaken, Elad, et al.
Published: (2021)

Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models
by: Liu, Hongbin, et al.
Published: (2024)

External Prompt Features Enhanced Parameter-efficient Fine-tuning for Salient Object Detection
by: Liang, Wen, et al.
Published: (2024)

Less Forgetting for Better Generalization: Exploring Continual-learning Fine-tuning Methods for Speech Self-supervised Representations
by: Zaiem, Salah, et al.
Published: (2024)

Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning
by: Tan, Zhaoxuan, et al.
Published: (2024)

Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models
by: Agarwal, Aradhye, et al.
Published: (2024)

Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Models
by: Peng, Zhiyuan, et al.
Published: (2024)

Iterative Importance Fine-tuning of Diffusion Models
by: Denker, Alexander, et al.
Published: (2025)

A Layer-wise Analysis of Supervised Fine-Tuning
by: Zhao, Qinghua, et al.
Published: (2026)

Fine-tuning of Large Language Models for Domain-Specific Cybersecurity Knowledge
by: Huang, Yuan
Published: (2025)

LayerKV: Optimizing Large Language Model Serving with Layer-wise KV Cache Management
by: Xiong, Yi, et al.
Published: (2024)