Cover Image

GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yao, Kai, Song, Zhenghan, Wu, Kaixin, Zhong, Mingjie, Cheng, Danzhao, Tan, Zhaorui, Ji, Yixin, Gao, Penglei
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models
by: Yao, Kai, et al.
Published: (2025)

GAST: A Generic AST Representation for Language-Independent Source Code Analysis
by: Jason Leiton-Jimenez
Published: (2023)

Sparse Gradient Compression for Fine-Tuning Large Language Models
by: Yang, David H., et al.
Published: (2025)

Prompt-aligned Gradient for Prompt Tuning
by: Zhu, Beier, et al.
Published: (2022)

Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models
by: Yao, Kai, et al.
Published: (2024)

Detoxification of Large Language Models through Output-layer Fusion with a Calibration Model
by: Tian, Yuanhe, et al.
Published: (2025)

RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selection
by: Yang, Yixin, et al.
Published: (2025)

Natural Language based Specification and Verification
by: Li, Zhaorui, et al.
Published: (2026)

Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencoder
by: Yang, Xianjun, et al.
Published: (2025)

OPa-Ma: Text Guided Mamba for 360-degree Image Out-painting
by: Gao, Penglei, et al.
Published: (2024)

Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection
by: Zhao, Yang, et al.
Published: (2025)

HLoRA: Efficient Federated Learning System for LLM Heterogeneous Fine-Tuning
by: Liu, Qianli, et al.
Published: (2025)

Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models
by: Liu, Xinxin, et al.
Published: (2024)

ReCIT: Reconstructing Full Private Data from Gradient in Parameter-Efficient Fine-Tuning of Large Language Models
by: Xie, Jin, et al.
Published: (2025)

ScaleOT: Privacy-utility-scalable Offsite-tuning with Dynamic LayerReplace and Selective Rank Compression
by: Yao, Kai, et al.
Published: (2024)

Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
by: Das, Rocktim Jyoti, et al.
Published: (2023)

Entropy-Gated Selective Policy Optimization:Token-Level Gradient Allocation for Hybrid Training of Large Language Models
by: Hu, Yuelin, et al.
Published: (2026)

Scaling Sparse Fine-Tuning to Large Language Models
by: Ansell, Alan, et al.
Published: (2024)

Instruction Mining: Instruction Data Selection for Tuning Large Language Models
by: Cao, Yihan, et al.
Published: (2023)

Neuron-Aware Data Selection In Instruction Tuning For Large Language Models
by: Chen, Xin, et al.
Published: (2026)

Gradient-based Parameter Selection for Efficient Fine-Tuning
by: Zhang, Zhi, et al.
Published: (2023)

Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning
by: Wang, Shaobo, et al.
Published: (2025)

CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuning
by: Zhang, Ping, et al.
Published: (2025)

RecRanker: Instruction Tuning Large Language Model as Ranker for Top-k Recommendation
by: Luo, Sichun, et al.
Published: (2023)

Achieving Sparse Activation in Small Language Models
by: Song, Jifeng, et al.
Published: (2024)

Large Language Models (LLMs) for Materials Design
by: Lei Zhang, et al.
Published: (2025)

Beyond Policy Optimization: A Data Curation Flywheel for Sparse-Reward Long-Horizon Planning
by: Wang, Yutong, et al.
Published: (2025)

Unveiling the Vulnerability of Private Fine-Tuning in Split-Based Frameworks for Large Language Models: A Bidirectionally Enhanced Attack
by: Chen, Guanzhong, et al.
Published: (2024)

Non-parabolic Spatial Hybrid Framed Curves and Their Applications in the Spatial Hybrid Number Space
by: Yao, Kaixin
Published: (2026)

Alignment Tuning for Large Language Models: A Data-Centric Lens on Alignment Data Pipelines
by: Song, Hwanjun
Published: (2026)

SCMix: Stochastic Compound Mixing for Open Compound Domain Adaptation in Semantic Segmentation
by: Yao, Kai, et al.
Published: (2024)

An Efficient Gradient-Aware Error-Bounded Lossy Compressor for Federated Learning
by: Ye, Zhijing, et al.
Published: (2025)

Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning
by: Yuan, Zhihang, et al.
Published: (2026)

Evaluating the Effectiveness of Large Language Models in Representing and Understanding Movement Trajectories
by: Ji, Yuhan, et al.
Published: (2024)

Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
by: Li, Haoling, et al.
Published: (2024)

IterSelectTune: An Iterative Training Framework for Efficient Instruction-Tuning Data Selection
by: Song, Jielin, et al.
Published: (2024)

Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearning
by: Wan, Yixin, et al.
Published: (2025)

Large-Scale Data Selection for Instruction Tuning
by: Ivison, Hamish, et al.
Published: (2025)

GRIP: In-Parameter Graph Reasoning through Fine-Tuning Large Language Models
by: Feng, Jiarui, et al.
Published: (2025)

Disentangling Tabular Data Towards Better One-Class Anomaly Detection
by: Ye, Jianan, et al.
Published: (2024)