Cover Image

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Yuxing, Wang, Jianyu, Zhang, Tong
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
by: Yu, Dingzhi, et al.
Published: (2026)

Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
by: Glentis, Athanasios, et al.
Published: (2025)

Unbiased Gradient Low-Rank Projection
by: Pan, Rui, et al.
Published: (2025)

Constructing Industrial-Scale Optimization Modeling Benchmark
by: Li, Zhong, et al.
Published: (2026)

The Road Less Scheduled
by: Defazio, Aaron, et al.
Published: (2024)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
by: Fernando, Heshan, et al.
Published: (2024)

From Large Language Models and Optimization to Decision Optimization CoPilot: A Research Manifesto
by: Wasserkrug, Segev, et al.
Published: (2024)

ARO: A New Lens On Matrix Optimization For Large Models
by: Gong, Wenbo, et al.
Published: (2026)

Contextual Distributionally Robust Optimization with Causal and Continuous Structure: An Interpretable and Tractable Approach
by: Zhang, Fenglin, et al.
Published: (2026)

On the Condition Number Dependency in Bilevel Optimization
by: Chen, Lesi, et al.
Published: (2025)

Budget-aware Auto Optimizer Configurator
by: Liu, Kang, et al.
Published: (2026)

A Minimalist Bayesian Framework for Stochastic Optimization
by: Wang, Kaizheng
Published: (2025)

Client-Centric Federated Adaptive Optimization
by: Sun, Jianhui, et al.
Published: (2025)

DiaBlo: Diagonal Blocks Are Sufficient For Finetuning
by: Gurses, Selcuk, et al.
Published: (2025)

OptiRepair: Closed-Loop Diagnosis and Repair of Supply Chain Optimization Models with LLM Agents
by: Ao, Ruicheng, et al.
Published: (2026)

Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks
by: Kiyani, Elham, et al.
Published: (2025)

From Soliloquy to Agora: Memory-Enhanced LLM Agents with Decentralized Debate for Optimization Modeling
by: Lin, Jianghao, et al.
Published: (2026)

The Newton-Muon Optimizer
by: Du, Zhehang, et al.
Published: (2026)

Riemannian Bilevel Optimization
by: Dutta, Sanchayan, et al.
Published: (2024)

MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters
by: Sharifnassab, Arsalan, et al.
Published: (2024)

Optimizer-Induced Mode Connectivity: From AdamW to Muon
by: Zhang, Fangzhao, et al.
Published: (2026)

LLM Serving Optimization with Variable Prefill and Decode Lengths
by: Wang, Meixuan, et al.
Published: (2025)

Zeroth-Order Optimization Finds Flat Minima
by: Zhang, Liang, et al.
Published: (2025)

Stochastic Optimization of Inventory at Large-scale Supply Chains
by: Jin, Zhaoyang Larry, et al.
Published: (2025)

MiMuon: Mixed Muon Optimizer with Improved Generalization for Large Models
by: Huang, Feihu, et al.
Published: (2026)

HoP: Homeomorphic Polar Learning for Hard Constrained Optimization
by: Deng, Ke, et al.
Published: (2025)

Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization
by: Du, Zhehang, et al.
Published: (2026)

Isotropic Curvature Model for Understanding Deep Learning Optimization: Is Gradient Orthogonalization Optimal?
by: Su, Weijie
Published: (2025)

Unsupervised Training of Diffusion Models for Feasible Solution Generation in Neural Combinatorial Optimization
by: Hong, Seong-Hyun, et al.
Published: (2024)

DT-PBO: an Interpretable Tree-based Surrogate Model for Preferential Bayesian Optimization
by: Leenders, Nick, et al.
Published: (2025)

EquivaMap: Leveraging LLMs for Automatic Equivalence Checking of Optimization Formulations
by: Zhai, Haotian, et al.
Published: (2025)

Jacobian Descent for Multi-Objective Optimization
by: Quinton, Pierre, et al.
Published: (2024)

Optimization and Generalization Guarantees for Weight Normalization
by: Cisneros-Velarde, Pedro, et al.
Published: (2024)

Neural Solver Selection for Combinatorial Optimization
by: Gao, Chengrui, et al.
Published: (2024)

Dynamic Memory Based Adaptive Optimization
by: Szegedy, Balázs, et al.
Published: (2024)

On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis
by: Chen, Lesi, et al.
Published: (2023)

Benchmarking PtO and PnO Methods in the Predictive Combinatorial Optimization Regime
by: Geng, Haoyu, et al.
Published: (2023)

Neural Combinatorial Optimization for Stochastic Flexible Job Shop Scheduling Problems
by: Smit, Igor G., et al.
Published: (2024)

Recursive Entropic Risk Optimization in Discounted MDPs: Sample Complexity Bounds with a Generative Model
by: Mortensen, Oliver, et al.
Published: (2025)

Hierarchical Mixture-of-Experts with Two-Stage Optimization
by: Molodtsov, Gleb, et al.
Published: (2026)