Cover Image

Learn More, Forget Less: A Gradient-Aware Data Selection Approach for LLM

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Yibai, Wang, Shihang, Liu, Zeming, Song, Zheming, Wang, Junzhe, Liu, Jingjing, Liu, Qingjie, Wang, Yunhong
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Less is More: Improving LLM Alignment via Preference Data Selection
by: Deng, Xun, et al.
Published: (2025)

ContextQFormer: A New Context Modeling Method for Multi-Turn Multi-Modal Conversations
by: Lei, Yiming, et al.
Published: (2025)

GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Art
by: Lei, Yiming, et al.
Published: (2025)

LLM Unlearning via Loss Adjustment with Only Forget Data
by: Wang, Yaxuan, et al.
Published: (2024)

LoRA Learns Less and Forgets Less
by: Biderman, Dan, et al.
Published: (2024)

LIMR: Less is More for RL Scaling
by: Li, Xuefeng, et al.
Published: (2025)

Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs
by: Zhang, Yifei, et al.
Published: (2024)

SeriesBench: A Benchmark for Narrative-Driven Drama Series Understanding
by: Zhang, Chenkai, et al.
Published: (2025)

Less is More: Understanding Word-level Textual Adversarial Attack via n-gram Frequency Descend
by: Lu, Ning, et al.
Published: (2023)

Less is More for Improving Automatic Evaluation of Factual Consistency
by: Wang, Tong, et al.
Published: (2024)

When More is Less: Understanding Chain-of-Thought Length in LLMs
by: Wu, Yuyang, et al.
Published: (2025)

ToolSpectrum : Towards Personalized Tool Utilization for Large Language Models
by: Cheng, Zihao, et al.
Published: (2025)

Less is More: Selective Reflection for Compatible and Efficient Knowledge Distillation in Large Language Models
by: Liu, Lingyuan, et al.
Published: (2025)

EntroCut: Entropy-Guided Adaptive Truncation for Efficient Chain-of-Thought Reasoning in Small-scale Large Reasoning Models
by: Yan, Hongxi, et al.
Published: (2026)

SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
by: Liu, Liangxin, et al.
Published: (2024)

GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning
by: Yang, Ningyuan, et al.
Published: (2026)

SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
by: Huang, Tianjin, et al.
Published: (2025)

FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning
by: Feng, Yujie, et al.
Published: (2026)

2M-NER: Contrastive Learning for Multilingual and Multimodal NER with Language and Modal Fusion
by: Wang, Dongsheng, et al.
Published: (2024)

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
by: Liu, Yuxing, et al.
Published: (2026)

LIMO: Less is More for Reasoning
by: Ye, Yixin, et al.
Published: (2025)

Explain Less, Understand More: Jargon Detection via Personalized Parameter-Efficient Fine-tuning
by: Wu, Bohao, et al.
Published: (2025)

Soundwave: Less is More for Speech-Text Alignment in LLMs
by: Zhang, Yuhao, et al.
Published: (2025)

Offline Learning and Forgetting for Reasoning with Large Language Models
by: Ni, Tianwei, et al.
Published: (2025)

Real Time Detection and Quantitative Analysis of Spurious Forgetting in Continual Learning
by: Wang, Weiwei
Published: (2025)

Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning
by: Yuan, Zhihang, et al.
Published: (2026)

Mitigating Shortcut Reasoning in Language Models: A Gradient-Aware Training Approach
by: Cao, Hongyu, et al.
Published: (2026)

Supernova: Achieving More with Less in Transformer Architectures
by: Tanase, Andrei-Valentin, et al.
Published: (2025)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
by: Zhang, Zhixin, et al.
Published: (2025)

Less Noise, More Voice: Reinforcement Learning for Reasoning via Instruction Purification
by: Guo, Yiju, et al.
Published: (2026)

BAGEN: Are LLM Agents Budget-Aware?
by: Lin, Yuxiang, et al.
Published: (2026)

MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
by: Chen, Zhixun, et al.
Published: (2025)

Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior
by: Li, Yulin, et al.
Published: (2025)

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
by: Lai, Song, et al.
Published: (2025)

C$^2$GSPG: Confidence-calibrated Group Sequence Policy Gradient towards Self-aware Reasoning
by: Liu, Haotian, et al.
Published: (2025)

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
by: Shen, Junzhe, et al.
Published: (2026)

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
by: Liu, Yong, et al.
Published: (2024)

Policy Split: Incentivizing Dual-Mode Exploration in LLM Reinforcement with Dual-Mode Entropy Regularization
by: Yao, Jiashu, et al.
Published: (2026)

Less is More: Resource-Efficient Low-Rank Adaptation
by: Tian, Chunlin, et al.
Published: (2025)

Graceful Forgetting in Generative Language Models
by: Jiang, Chunyang, et al.
Published: (2025)