Image de couverture de livre

RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Yuduo, Shi, Xiaofeng, Kou, Qian, Yu, Longbin, Zhou, Hua
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy
par: Shi, Xiaofeng, et autres
Publié: (2025)

SPAR: Scholar Paper Retrieval with LLM-based Agents for Enhanced Academic Search
par: Shi, Xiaofeng, et autres
Publié: (2025)

Forget the Data and Fine-Tuning! Just Fold the Network to Compress
par: Wang, Dong, et autres
Publié: (2025)

SciSage: A Multi-Agent Framework for High-Quality Scientific Survey Generation
par: Shi, Xiaofeng, et autres
Publié: (2025)

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting
par: Diao, Muxi, et autres
Publié: (2026)

Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
par: Schweighofer, Kajetan, et autres
Publié: (2026)

Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
par: Sanyal, Sunny, et autres
Publié: (2025)

RL Fine-Tuning Heals OOD Forgetting in SFT
par: Jin, Hangzhan, et autres
Publié: (2025)

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
par: Yu, Ziming, et autres
Publié: (2024)

SLIM-RAFT: A Novel Fine-Tuning Approach to Improve Cross-Linguistic Performance for Mercosur Common Nomenclature
par: Di Oliveira, Vinícius, et autres
Publié: (2024)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
par: Chen, Yupeng, et autres
Publié: (2024)

Can LLMs Alleviate Catastrophic Forgetting in Graph Continual Learning? A Systematic Study
par: Cheng, Ziyang, et autres
Publié: (2025)

Delayed Bottlenecking: Alleviating Forgetting in Pre-trained Graph Neural Networks
par: Zhao, Zhe, et autres
Publié: (2024)

Enhancing Reinforcement Learning Fine-Tuning with an Online Refiner
par: Ma, Hao, et autres
Publié: (2026)

Not All Forgetting Is Equal: Architecture-Dependent Retention Dynamics in Fine-Tuned Image Classifiers
par: Daga, Miit, et autres
Publié: (2026)

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
par: Lai, Song, et autres
Publié: (2025)

FLAME: Towards Federated Fine-Tuning Large Language Models Through Adaptive SMoE
par: Le, Khiem, et autres
Publié: (2025)

Defense without Forgetting: Continual Adversarial Defense with Anisotropic & Isotropic Pseudo Replay
par: Zhou, Yuhang, et autres
Publié: (2024)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
par: Fawi, Muhammad
Publié: (2024)

Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler
par: Hu, Zixuan, et autres
Publié: (2025)

Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning
par: Zhang, Jingyuan, et autres
Publié: (2024)

Distilling Linearized Behavior into Non-Linear Fine-Tuning for Effective Task Arithmetic
par: Sommariva, Thomas, et autres
Publié: (2026)

Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning
par: Jiang, Gangwei, et autres
Publié: (2025)

Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data
par: Ling, Zhenqing, et autres
Publié: (2025)

Policy Gradient with Adaptive Entropy Annealing for Continual Fine-Tuning
par: Zhang, Yaqian, et autres
Publié: (2026)

Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning
par: Liu, Siyuan, et autres
Publié: (2026)

Balancing Fidelity and Plasticity: Aligning Mixed-Precision Fine-Tuning with Linguistic Hierarchies
par: Zhou, Changhai, et autres
Publié: (2025)

ASFT: Aligned Supervised Fine-Tuning through Absolute Likelihood
par: Wang, Ruoyu, et autres
Publié: (2024)

FaLW: A Forgetting-aware Loss Reweighting for Long-tailed Unlearning
par: Yu, Liheng, et autres
Publié: (2026)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
par: Fernando, Heshan, et autres
Publié: (2024)

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
par: Shi, Jiang-Xin, et autres
Publié: (2025)

Silent Inconsistency in Data-Parallel Full Fine-Tuning: Diagnosing Worker-Level Optimization Misalignment
par: Li, Hong, et autres
Publié: (2026)

Gradual Fine-Tuning with Graph Routing for Multi-Source Unsupervised Domain Adaptation
par: Ma, Yao, et autres
Publié: (2024)

Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection
par: Lai, Zhixin, et autres
Publié: (2024)

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
par: Ren, Weijieying, et autres
Publié: (2024)

Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning
par: Ye, Kai, et autres
Publié: (2025)

MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning
par: Lu, Yiyang, et autres
Publié: (2026)

TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models
par: Shing, Makoto, et autres
Publié: (2025)

SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning
par: Chen, Jinpeng, et autres
Publié: (2025)

Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Design
par: Su, Xingyu, et autres
Publié: (2025)