Cover Image

OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Long, Jikai, Hu, Zijian, Yu, Xiaodong, Xie, Jianwen, Xu, Zhaozhuo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
by: Ran, Yide, et al.
Published: (2025)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
by: Guo, Wentao, et al.
Published: (2024)

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
by: Yu, Ziming, et al.
Published: (2024)

Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer
by: Feng, Jian, et al.
Published: (2026)

FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed
by: Dang, Sizhe, et al.
Published: (2025)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
by: Wang, Shuo, et al.
Published: (2026)

Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves
by: Deng, Yihe, et al.
Published: (2023)

Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models
by: Prasad, Archiki, et al.
Published: (2023)

MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models
by: Zhang, Zhen, et al.
Published: (2025)

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
by: Zhou, Jiajun, et al.
Published: (2025)

Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning
by: Liu, Ziwen, et al.
Published: (2026)

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
by: Liu, Yong, et al.
Published: (2024)

AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
by: Chen, Ziye, et al.
Published: (2026)

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
by: Cai, Zhijie, et al.
Published: (2026)

AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning
by: Yang, Yifan, et al.
Published: (2024)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
by: Gautam, Tanmay, et al.
Published: (2024)

Refining Adaptive Zeroth-Order Optimization at Ease
by: Shu, Yao, et al.
Published: (2025)

To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentration
by: Yang, Zeyu, et al.
Published: (2025)

OAT: Ordered Action Tokenization
by: Liu, Chaoqi, et al.
Published: (2026)

Subspace-based Approximate Hessian Method for Zeroth-Order Optimization
by: Kim, Dongyoon, et al.
Published: (2025)

TATRA: Training-Free Instance-Adaptive Prompting Through Rephrasing and Aggregation
by: Dziuba, Bartosz, et al.
Published: (2026)

Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations
by: Mi, Zhendong, et al.
Published: (2025)

Zeroth-Order Optimization Finds Flat Minima
by: Zhang, Liang, et al.
Published: (2025)

Warming Up for Zeroth-Order Federated Pre-Training with Low Resource Clients
by: Legate, Gwen, et al.
Published: (2025)

HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
by: Zhao, Huaqin, et al.
Published: (2024)

DART: An AIGT Detector using AMR of Rephrased Text
by: Park, Hyeonchu, et al.
Published: (2024)

Task-Aware Parameter-Efficient Fine-Tuning of Large Pre-Trained Models at the Edge
by: Hu, Senkang, et al.
Published: (2025)

A Historical Trajectory Assisted Optimization Method for Zeroth-Order Federated Learning
by: Wu, Chenlin, et al.
Published: (2024)

RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models
by: Najafi, Saeed, et al.
Published: (2024)

Perturbation-efficient Zeroth-order Optimization for Hardware-friendly On-device Training
by: Tan, Qitao, et al.
Published: (2025)

RPO:Reinforcement Fine-Tuning with Partial Reasoning Optimization
by: Yi, Hongzhu, et al.
Published: (2026)

LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning
by: Chen, Junyu, et al.
Published: (2025)

Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oracles
by: Tang, Zhiwei, et al.
Published: (2023)

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
by: Ma, Shaocong, et al.
Published: (2025)

MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning
by: Lu, Yiyang, et al.
Published: (2026)

Token-wise Influential Training Data Retrieval for Large Language Models
by: Lin, Huawei, et al.
Published: (2024)

Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning
by: Seung, Hyunseok, et al.
Published: (2025)

Selection of LLM Fine-Tuning Data based on Orthogonal Rules
by: Li, Xiaomin, et al.
Published: (2024)

Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning
by: Liu, Siyuan, et al.
Published: (2026)

Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training
by: Cunegatti, Elia, et al.
Published: (2024)