Cover Image

Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Schweighofer, Kajetan, Hayes, Conor F., Dailey, Roberto, Miikkulainen, Risto, Qiu, Xin
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning
by: Qiu, Xin, et al.
Published: (2025)

Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost
by: Xu, Yinggan, et al.
Published: (2026)

Efficient Pre-Training of LLMs through Truncated SVD Layers
by: Kamali, Kaivan, et al.
Published: (2026)

The Blessing of Dimensionality in LLM Fine-tuning: A Variance-Curvature Perspective
by: Liang, Qiyao, et al.
Published: (2026)

EVOTER: Evolution of Transparent Explainable Rule-sets
by: Shahrzad, Hormoz, et al.
Published: (2022)

On Information-Theoretic Measures of Predictive Uncertainty
by: Schweighofer, Kajetan, et al.
Published: (2024)

Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation
by: Ielanskyi, Mykyta, et al.
Published: (2025)

Improving Uncertainty Estimation through Semantically Diverse Language Generation
by: Aichberger, Lukas, et al.
Published: (2024)

The Disparate Benefits of Deep Ensembles
by: Schweighofer, Kajetan, et al.
Published: (2024)

Solving a Million-Step LLM Task with Zero Errors
by: Meyerson, Elliot, et al.
Published: (2025)

The Odyssey of the Fittest: Can Agents Survive and Still Be Good?
by: Waldner, Dylan, et al.
Published: (2025)

Fine-Tuning Language Models to Know What They Know
by: Park, Sangjun, et al.
Published: (2026)

Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space
by: Qiu, Xin, et al.
Published: (2024)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
by: Chen, Yupeng, et al.
Published: (2024)

Optimizing the Design of an Artificial Pancreas to Improve Diabetes Management
by: Khanna, Ashok, et al.
Published: (2024)

Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
by: Sanyal, Sunny, et al.
Published: (2025)

RL Fine-Tuning Heals OOD Forgetting in SFT
by: Jin, Hangzhan, et al.
Published: (2025)

Optimize Wider, Not Deeper: Consensus Aggregation for Policy Optimization
by: Su, Zelal, et al.
Published: (2026)

Forget the Data and Fine-Tuning! Just Fold the Network to Compress
by: Wang, Dong, et al.
Published: (2025)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

NeuroBack: Improving CDCL SAT Solving using Graph Neural Networks
by: Wang, Wenxi, et al.
Published: (2021)

RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
by: Li, Yuduo, et al.
Published: (2026)

Not All Forgetting Is Equal: Architecture-Dependent Retention Dynamics in Fine-Tuned Image Classifiers
by: Daga, Miit, et al.
Published: (2026)

Overcoming Growth-Induced Forgetting in Task-Agnostic Continual Learning
by: Zhao, Yuqing, et al.
Published: (2024)

Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System
by: Chen, Chen, et al.
Published: (2024)

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting
by: Diao, Muxi, et al.
Published: (2026)

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
by: Lai, Song, et al.
Published: (2025)

Alignment Dynamics in LLM Fine-Tuning
by: Huang, Yuhan, et al.
Published: (2026)

Effective Regularization Through Loss-Function Metalearning
by: Gonzalez, Santiago, et al.
Published: (2020)

Overcoming Catastrophic Forgetting in Tabular Data Classification: A Pseudorehearsal-based approach
by: García-Santaclara, Pablo, et al.
Published: (2024)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
by: Fernando, Heshan, et al.
Published: (2024)

ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning
by: Sun, Zhishen, et al.
Published: (2026)

Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure
by: Aichberger, Lukas, et al.
Published: (2024)

DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy
by: Xu, Kaixuan, et al.
Published: (2025)

Exploring Gradient Subspaces: Addressing and Overcoming LoRA's Limitations in Federated Fine-Tuning of Large Language Models
by: Mahla, Navyansh, et al.
Published: (2024)

Internalizing Curriculum Judgment for LLM Reinforcement Fine-Tuning
by: Zheng, Han, et al.
Published: (2026)

Goal-Conditioned Supervised Learning for LLM Fine-Tuning
by: Li, Shijun, et al.
Published: (2026)

LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection
by: Zeng, Xinyue, et al.
Published: (2025)

SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning
by: Hossain, Saad, et al.
Published: (2025)

Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning
by: Liu, Siyuan, et al.
Published: (2026)