Cover Image

Why Zeroth-Order Adaptation May Forget Less: A Randomized Shaping Theory

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shu, Yao, Mu, Jian, Dai, Zhongxiang
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Refining Adaptive Zeroth-Order Optimization at Ease
by: Shu, Yao, et al.
Published: (2025)

Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization
by: Shu, Yao, et al.
Published: (2026)

RL's Razor: Why Online Reinforcement Learning Forgets Less
by: Shenfeld, Idan, et al.
Published: (2025)

Zeroth-Order Optimization is Secretly Single-Step Policy Optimization
by: Qiu, Junbin, et al.
Published: (2025)

Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens
by: Qiu, Junbin, et al.
Published: (2026)

ZOTTA: Test-Time Adaptation with Gradient-Free Zeroth-Order Optimization
by: Zhang, Ronghao, et al.
Published: (2026)

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
by: Yu, Ziming, et al.
Published: (2024)

Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimization
by: Li, Zhe, et al.
Published: (2025)

Meta-Prompt Optimization for LLM-Based Sequential Decision Making
by: Kong, Mingze, et al.
Published: (2025)

More Than Memory Savings: Zeroth-Order Optimization Mitigates Forgetting in Continual Learning
by: Yu, Wanhao, et al.
Published: (2025)

A Randomized Zeroth-Order Hierarchical Framework for Heterogeneous Federated Learning
by: Qiu, Yuyang, et al.
Published: (2025)

Elucidating Subspace Perturbation in Zeroth-Order Optimization: Theory and Practice at Scale
by: Park, Sihwan, et al.
Published: (2025)

Why Less is More (Sometimes): A Theory of Data Curation
by: Dohmatob, Elvis, et al.
Published: (2025)

Forget Less, Generalize More: Unifying Temporal and Structural Adaptation for Dynamic Graphs
by: Chang, Qian, et al.
Published: (2026)

Minimisation of Polyak-Łojasewicz Functions Using Random Zeroth-Order Oracles
by: Farzin, Amir Ali, et al.
Published: (2024)

Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer
by: Zhao, Yanjun, et al.
Published: (2024)

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
by: Liu, Yong, et al.
Published: (2024)

Words & Weights: Streamlining Multi-Turn Interactions via Co-Adaptation
by: Wei, Chenxing, et al.
Published: (2026)

LoRA Learns Less and Forgets Less
by: Biderman, Dan, et al.
Published: (2024)

On Adaptivity in Zeroth-Order Optimization
by: Dbouk, Hassan, et al.
Published: (2026)

Robustifying and Boosting Training-Free Neural Architecture Search
by: He, Zhenfeng, et al.
Published: (2024)

Minimisation of Submodular Functions Using Gaussian Zeroth-Order Random Oracles
by: Farzin, Amir Ali, et al.
Published: (2025)

On the Convergence of Zeroth-Order Federated Tuning for Large Language Models
by: Ling, Zhenqing, et al.
Published: (2024)

Private Zeroth-Order Optimization with Public Data
by: Gong, Xuchen, et al.
Published: (2025)

The Multi-Query Paradox in Zeroth-Order Optimization
by: Lin, Wei, et al.
Published: (2025)

Learning Dynamics of Zeroth-Order Optimization: A Kernel Perspective
by: Li, Zhe, et al.
Published: (2026)

Minimisation of Quasar-Convex Functions Using Random Zeroth-Order Oracles
by: Farzin, Amir Ali, et al.
Published: (2025)

LLM Zeroth-Order Fine-Tuning is an Inference Workload
by: Li, Zelin, et al.
Published: (2026)

Alternating Diffusion for Proximal Sampling with Zeroth Order Queries
by: Takagi, Hirohane, et al.
Published: (2026)

Improving the Straight-Through Estimator with Zeroth-Order Information
by: Yang, Ningfeng, et al.
Published: (2025)

Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer
by: Feng, Jian, et al.
Published: (2026)

AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
by: Chen, Ziye, et al.
Published: (2026)

Zeroth-Order Optimization at the Edge of Stability
by: Song, Minhak, et al.
Published: (2026)

Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
by: Liu, Sijia, et al.
Published: (2026)

Efficient Federated RLHF via Zeroth-Order Policy Optimization
by: Wang, Deyi, et al.
Published: (2026)

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
by: Zhang, Kairun, et al.
Published: (2025)

Z0-Inf: Zeroth Order Approximation for Data Influence
by: Kokhlikyan, Narine, et al.
Published: (2025)

Stochastic Dimension-Free Zeroth-Order Estimator for High-Dimensional and High-Order PINNs
by: Liang, Zhangyong, et al.
Published: (2026)

Mitigating Forgetting in Low Rank Adaptation
by: Sliwa, Joanna, et al.
Published: (2025)

Zeroth-Order primal-dual Alternating Projection Gradient Algorithms for Nonconvex Minimax Problems with Coupled linear Constraints
by: Zhang, Huiling, et al.
Published: (2024)