Cover Image

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Shuo, Chen, Ziyu, Tang, Ming
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
by: Liu, Yong, et al.
Published: (2024)

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
by: Zhou, Jiajun, et al.
Published: (2025)

MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models
by: Zhang, Zhen, et al.
Published: (2025)

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
by: Cai, Zhijie, et al.
Published: (2026)

Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer
by: Feng, Jian, et al.
Published: (2026)

AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
by: Chen, Ziye, et al.
Published: (2026)

OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning
by: Long, Jikai, et al.
Published: (2025)

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
by: Yu, Ziming, et al.
Published: (2024)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
by: Guo, Wentao, et al.
Published: (2024)

Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
by: Ran, Yide, et al.
Published: (2025)

Refining Adaptive Zeroth-Order Optimization at Ease
by: Shu, Yao, et al.
Published: (2025)

Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning
by: Seung, Hyunseok, et al.
Published: (2025)

AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning
by: Yang, Yifan, et al.
Published: (2024)

FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed
by: Dang, Sizhe, et al.
Published: (2025)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
by: Gautam, Tanmay, et al.
Published: (2024)

RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
by: Liu, Jun, et al.
Published: (2025)

ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activation
by: Meng, Chuiyang, et al.
Published: (2026)

Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oracles
by: Tang, Zhiwei, et al.
Published: (2023)

ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithm
by: Xie, Lunchen, et al.
Published: (2025)

ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory
by: Wang, Liangyu, et al.
Published: (2025)

Subspace-based Approximate Hessian Method for Zeroth-Order Optimization
by: Kim, Dongyoon, et al.
Published: (2025)

Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuning
by: Luo, Yu, et al.
Published: (2026)

Zeroth-Order Optimization Finds Flat Minima
by: Zhang, Liang, et al.
Published: (2025)

Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection
by: Lai, Zhixin, et al.
Published: (2024)

Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA
by: Yang, Nuocheng, et al.
Published: (2026)

HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
by: Zhao, Huaqin, et al.
Published: (2024)

TeZO: Empowering the Low-Rankness on the Temporal Dimension in the Zeroth-Order Optimization for Fine-tuning LLMs
by: Sun, Yan, et al.
Published: (2025)

ElasticZO: A Memory-Efficient On-Device Learning with Combined Zeroth- and First-Order Optimization
by: Sugiura, Keisuke, et al.
Published: (2025)

A Historical Trajectory Assisted Optimization Method for Zeroth-Order Federated Learning
by: Wu, Chenlin, et al.
Published: (2024)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
by: Zhang, Yihua, et al.
Published: (2024)

MallowsPO: Fine-Tune Your LLM with Preference Dispersions
by: Chen, Haoxian, et al.
Published: (2024)

Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization
by: Liang, Chen, et al.
Published: (2026)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
by: Chen, Yupeng, et al.
Published: (2024)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
by: Liu, Yilun, et al.
Published: (2024)

On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
by: Ye, Rongguang, et al.
Published: (2025)

Alignment Dynamics in LLM Fine-Tuning
by: Huang, Yuhan, et al.
Published: (2026)

Prior-Informed Zeroth-Order Optimization with Adaptive Direction Alignment for Memory-Efficient LLM Fine-Tuning
by: Jin, Feihu, et al.
Published: (2026)

DistZO2: High-Throughput and Memory-Efficient Zeroth-Order Fine-tuning LLMs with Distributed Parallel Computing
by: Wang, Liangyu, et al.
Published: (2025)

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
by: Ma, Shaocong, et al.
Published: (2025)

When Foresight Pruning Meets Zeroth-Order Optimization: Efficient Federated Learning for Low-Memory Devices
by: Zhang, Pengyu, et al.
Published: (2024)