Cover Image

SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chekalina, Viktoriia, Rudenko, Anna, Mezentsev, Gleb, Mikhalev, Alexander, Panchenko, Alexander, Oseledets, Ivan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Exploring the Hidden Capacity of LLMs for One-Step Text Generation
by: Mezentsev, Gleb, et al.
Published: (2025)

Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality
by: Chekalina, Viktoriia, et al.
Published: (2024)

LoTR: Low Tensor Rank Weight Adaptation
by: Bershatsky, Daniel, et al.
Published: (2024)

When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs
by: Seleznyov, Mikhail, et al.
Published: (2025)

GradPruner: Gradient-Guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMs
by: Huang, Wei, et al.
Published: (2026)

Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction
by: Liu, Zhexiong, et al.
Published: (2025)

MultiParaDetox: Extending Text Detoxification with Parallel Data to New Languages
by: Dementieva, Daryna, et al.
Published: (2024)

<think> So let's replace this phrase with insult... </think> Lessons learned from generation of toxic texts with LLMs
by: Pletenev, Sergey, et al.
Published: (2025)

S3: A Simple Strong Sample-effective Multimodal Dialog System
by: Rykov, Elisei, et al.
Published: (2024)

Evolutionary Search for Automated Design of Uncertainty Quantification Methods
by: Seleznyov, Mikhail, et al.
Published: (2026)

Efficient Model Editing with Task-Localized Sparse Fine-tuning
by: Iurada, Leonardo, et al.
Published: (2025)

MTUncertainty: Assessing the Need for Post-editing of Machine Translation Outputs by Fine-tuning OpenAI LLMs
by: Gladkoff, Serge, et al.
Published: (2023)

Fine-tuning and Utilization Methods of Domain-specific LLMs
by: Jeong, Cheonsu
Published: (2024)

On the Spatial Structure of Mixture-of-Experts in Transformers
by: Bershatsky, Daniel, et al.
Published: (2025)

Increasing the Robustness of the Fine-tuned Multilingual Machine-Generated Text Detectors
by: Macko, Dominik, et al.
Published: (2025)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations
by: Solano, Jesus, et al.
Published: (2023)

Scaling Sparse Fine-Tuning to Large Language Models
by: Ansell, Alan, et al.
Published: (2024)

Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning
by: Huang, Shih-Cheng, et al.
Published: (2022)

Token-level Data Selection for Safe LLM Fine-tuning
by: Li, Yanping, et al.
Published: (2026)

Efficient Model Development through Fine-tuning Transfer
by: Lin, Pin-Jie, et al.
Published: (2025)

SmurfCat at SemEval-2024 Task 6: Leveraging Synthetic Data for Hallucination Detection
by: Rykov, Elisei, et al.
Published: (2024)

Inference-Time Selective Debiasing to Enhance Fairness in Text Classification Models
by: Kuzmin, Gleb, et al.
Published: (2024)

Layer-Wise Evolution of Representations in Fine-Tuned Transformers: Insights from Sparse AutoEncoders
by: Nadipalli, Suneel
Published: (2025)

APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
by: Marín, Javier
Published: (2025)

Aloe: A Family of Fine-tuned Open Healthcare LLMs
by: Gururajan, Ashwin Kumar, et al.
Published: (2024)

When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications
by: Liu, Qidong, et al.
Published: (2023)

Small Updates, Big Doubts: Does Parameter-Efficient Fine-tuning Enhance Hallucination Detection ?
by: Hu, Xu, et al.
Published: (2026)

Parameter Efficient Fine-tuning via Explained Variance Adaptation
by: Paischer, Fabian, et al.
Published: (2024)

Efficient End-to-end Language Model Fine-tuning on Graphs
by: Xue, Rui, et al.
Published: (2023)

Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning
by: Dymkiewicz, Kajetan, et al.
Published: (2025)

PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging
by: He, Jinlong, et al.
Published: (2024)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
by: Lin, Haowei, et al.
Published: (2024)

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning
by: Zou, Heming, et al.
Published: (2025)

Joint Localization and Activation Editing for Low-Resource Fine-Tuning
by: Lai, Wen, et al.
Published: (2025)

GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning
by: Yang, Ningyuan, et al.
Published: (2026)

The Chronicles of RiDiC: Generating Datasets with Controlled Popularity Distribution for Long-form Factuality Evaluation
by: Braslavski, Pavel, et al.
Published: (2026)

Stage-wise Fine-tuning for Graph-to-Text Generation
by: Wang, Qingyun, et al.
Published: (2021)

Semi-supervised Fine-tuning for Large Language Models
by: Luo, Junyu, et al.
Published: (2024)

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
by: Xu, Jing, et al.
Published: (2024)