Cover Image

From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Runwal, Bharat, Pedapati, Tejaswini, Chen, Pin-Yu
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Intermediate Representations are Strong AI-Generated Image Detectors
by: Huang, Zhenhan, et al.
Published: (2026)

Graph is all you need? Lightweight data-agnostic neural architecture search without training
by: Huang, Zhenhan, et al.
Published: (2024)

Differentiable Prompt Learning for Vision Language Models
by: Huang, Zhenhan, et al.
Published: (2024)

Sparse Gradient Compression for Fine-Tuning Large Language Models
by: Yang, David H., et al.
Published: (2025)

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
by: Huang, Yangyi, et al.
Published: (2026)

Uncovering the Hidden Cost of Model Compression
by: Misra, Diganta, et al.
Published: (2023)

PRISM: Demystifying Retention and Interaction in Mid-Training
by: Runwal, Bharat, et al.
Published: (2026)

ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval
by: Yang, David H., et al.
Published: (2026)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
by: Zhu, Yuxuan, et al.
Published: (2025)

STAR: Spectral Truncation and Rescale for Model Merging
by: Lee, Yu-Ang, et al.
Published: (2025)

PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
by: Yu, Tongzhou, et al.
Published: (2025)

DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised $h$-transform
by: Denker, Alexander, et al.
Published: (2024)

Large Language Model Confidence Estimation via Black-Box Access
by: Pedapati, Tejaswini, et al.
Published: (2024)

Harli: SLO-Aware Co-location of LLM Inference and PEFT-based Finetuning on Model-as-a-Service Platforms
by: Xu, Ao, et al.
Published: (2025)

AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning
by: Zhou, Han, et al.
Published: (2023)

Large Language Models can be Strong Self-Detoxifiers
by: Ko, Ching-Yun, et al.
Published: (2024)

ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization
by: Yadav, Prateek, et al.
Published: (2023)

MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba
by: Yoshimura, Masakazu, et al.
Published: (2024)

CoFrNets: Interpretable Neural Architecture Inspired by Continued Fractions
by: Puri, Isha, et al.
Published: (2025)

On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
by: Lab, Mind, et al.
Published: (2026)

Shape it Up! Restoring LLM Safety during Finetuning
by: Peng, ShengYun, et al.
Published: (2025)

Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models
by: Peng, ShengYun, et al.
Published: (2024)

NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models
by: Dhurandhar, Amit, et al.
Published: (2024)

X-PEFT: eXtremely Parameter-Efficient Fine-Tuning for Extreme Multi-Profile Scenarios
by: Kwak, Namju, et al.
Published: (2024)

Assessing the Portability of Parameter Matrices Trained by Parameter-Efficient Finetuning Methods
by: Sabry, Mohammed, et al.
Published: (2024)

Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models
by: Tang, Yiwen, et al.
Published: (2023)

Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition
by: Mai, Zheda, et al.
Published: (2024)

Modular Prompt Learning Improves Vision-Language Models
by: Huang, Zhenhan, et al.
Published: (2025)

Monkey Jump : MoE-Style PEFT for Efficient Multi-Task Learning
by: Prottasha, Nusrat Jahan, et al.
Published: (2026)

Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation
by: Lashkarashvili, Nineli, et al.
Published: (2024)

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
by: Liu, Weiyang, et al.
Published: (2023)

SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
by: Jia, Jinghan, et al.
Published: (2024)

Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Models
by: Peng, Zhiyuan, et al.
Published: (2024)

Improving Transformers using Faithful Positional Encoding
by: Idé, Tsuyoshi, et al.
Published: (2024)

$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning
by: Wang, Runqian, et al.
Published: (2024)

Eigenpruning: an Interpretability-Inspired PEFT Method
by: Vergara-Browne, Tomás, et al.
Published: (2024)

Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything Model
by: Zhong, Zihan, et al.
Published: (2024)

Quantum-PEFT: Ultra parameter-efficient fine-tuning
by: Koike-Akino, Toshiaki, et al.
Published: (2025)

Beyond Parameter Finetuning: Test-Time Representation Refinement for Node Classification
by: Zhang, Jiaxin, et al.
Published: (2026)

Calibrating and Rotating: A Unified Framework for Weight Conditioning in PEFT
by: Chang, Da, et al.
Published: (2025)