Imagen de Portada

Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Sanyal, Sunny, Prairie, Hayden, Das, Rudrajit, Kavis, Ali, Sanghavi, Sujay
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models
por: Sanyal, Sunny, et al.
Publicado: (2024)

Understanding Self-Supervised Learning via Gaussian Mixture Models
por: Bansal, Parikshit, et al.
Publicado: (2024)

Context-Free Synthetic Data Mitigates Forgetting
por: Bansal, Parikshit, et al.
Publicado: (2025)

HiSpec: Hierarchical Speculative Decoding for LLMs
por: Kumar, Avinash, et al.
Publicado: (2025)

Understanding the Training Speedup from Sampling with Approximate Losses
por: Das, Rudrajit, et al.
Publicado: (2024)

Geometric Median (GM) Matching for Robust Data Pruning
por: Acharya, Anish, et al.
Publicado: (2024)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
por: Chen, Yupeng, et al.
Publicado: (2024)

Towards Quantifying the Preconditioning Effect of Adam
por: Das, Rudrajit, et al.
Publicado: (2024)

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
por: Lai, Song, et al.
Publicado: (2025)

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting
por: Diao, Muxi, et al.
Publicado: (2026)

SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors
por: Lingam, Vijay, et al.
Publicado: (2024)

Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization
por: Jiang, Ruichen, et al.
Publicado: (2024)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
por: Fawi, Muhammad
Publicado: (2024)

RL Fine-Tuning Heals OOD Forgetting in SFT
por: Jin, Hangzhan, et al.
Publicado: (2025)

Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
por: Schweighofer, Kajetan, et al.
Publicado: (2026)

Forget the Data and Fine-Tuning! Just Fold the Network to Compress
por: Wang, Dong, et al.
Publicado: (2025)

RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
por: Li, Yuduo, et al.
Publicado: (2026)

Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning
por: Jiang, Gangwei, et al.
Publicado: (2025)

In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
por: Collins, Liam, et al.
Publicado: (2024)

Entropy Aware Reward Guidance for Diffusion Language Model Alignment
por: Tejaswi, Atula, et al.
Publicado: (2026)

Not All Forgetting Is Equal: Architecture-Dependent Retention Dynamics in Fine-Tuned Image Classifiers
por: Daga, Miit, et al.
Publicado: (2026)

Sculpting Latent Spaces With MMD: Disentanglement With Programmable Priors
por: Fruytier, Quentin, et al.
Publicado: (2025)

Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity
por: Hou, Charlie, et al.
Publicado: (2023)

Positive Unlabeled Contrastive Learning
por: Acharya, Anish, et al.
Publicado: (2022)

Thompson Sampling via Fine-Tuning of LLMs
por: Menet, Nicolas, et al.
Publicado: (2025)

Sequencing to Mitigate Catastrophic Forgetting in Continual Learning
por: Moussa, Hesham G., et al.
Publicado: (2025)

MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation
por: Ekbote, Chanakya, et al.
Publicado: (2025)

Enabling Approximate Joint Sampling in Diffusion LMs
por: Bansal, Parikshit, et al.
Publicado: (2025)

FedRef: Bayesian Fine-Tuning using a Reference Model to Mitigate Catastrophic Forgetting for Heterogeneous Federated Learning
por: Yoon, Taehwan, et al.
Publicado: (2025)

Evaluating Negative Sampling Approaches for Neural Topic Models
por: Adhya, Suman, et al.
Publicado: (2025)

Catastrophic Forgetting Mitigation Through Plateau Phase Activity Profiling
por: Mashiach, Idan, et al.
Publicado: (2025)

RIFT: Repurposing Negative Samples via Reward-Informed Fine-Tuning
por: Liu, Zehua, et al.
Publicado: (2026)

CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay
por: Zhang, Jianshu, et al.
Publicado: (2024)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
por: Zhang, Zhixin, et al.
Publicado: (2025)

Unintended Misalignment from Agentic Fine-Tuning: Risks and Mitigation
por: Hahm, Dongyoon, et al.
Publicado: (2025)

Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
por: Ran, Yide, et al.
Publicado: (2025)

Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation
por: Liu, Jinmei, et al.
Publicado: (2026)

Modular Jets for Supervised Pipelines: Diagnosing Mirage vs Identifiability
por: Sanyal, Suman
Publicado: (2025)

Perception Learning: A Formal Separation of Sensory Representation Learning from Decision Learning
por: Sanyal, Suman
Publicado: (2025)

SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning
por: Chen, Jinpeng, et al.
Publicado: (2025)