Imagen de Portada

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhou, Jiajun, Yang, Yifan, Zhen, Kai, Liu, Ziyue, Zhao, Yequan, Banijamali, Ershad, Mouchtaris, Athanasios, Wong, Ngai, Zhang, Zheng
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning
por: Yang, Yifan, et al.
Publicado: (2024)

MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models
por: Zhang, Zhen, et al.
Publicado: (2025)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
por: Yang, Yifan, et al.
Publicado: (2024)

ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory
por: Wang, Liangyu, et al.
Publicado: (2025)

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
por: Liu, Yong, et al.
Publicado: (2024)

ZO-ASR: Zeroth-Order Fine-Tuning of Speech Foundation Models without Back-Propagation
por: Peng, Yuezhang, et al.
Publicado: (2025)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
por: Wang, Shuo, et al.
Publicado: (2026)

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
por: Zhang, Kairun, et al.
Publicado: (2025)

AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
por: Chen, Ziye, et al.
Publicado: (2026)

TeZO: Empowering the Low-Rankness on the Temporal Dimension in the Zeroth-Order Optimization for Fine-tuning LLMs
por: Sun, Yan, et al.
Publicado: (2025)

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
por: Cai, Zhijie, et al.
Publicado: (2026)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
por: Gautam, Tanmay, et al.
Publicado: (2024)

Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models
por: Solgi, Ryan, et al.
Publicado: (2025)

FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed
por: Dang, Sizhe, et al.
Publicado: (2025)

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
por: Yu, Ziming, et al.
Publicado: (2024)

DistZO2: High-Throughput and Memory-Efficient Zeroth-Order Fine-tuning LLMs with Distributed Parallel Computing
por: Wang, Liangyu, et al.
Publicado: (2025)

ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithm
por: Xie, Lunchen, et al.
Publicado: (2025)

AIS: Adaptive Importance Sampling for Quantized RL
por: Zhou, Jiajun, et al.
Publicado: (2026)

SharpZO: Hybrid Sharpness-Aware Vision Language Model Prompt Tuning via Forward-Only Passes
por: Yang, Yifan, et al.
Publicado: (2025)

ElasticZO: A Memory-Efficient On-Device Learning with Combined Zeroth- and First-Order Optimization
por: Sugiura, Keisuke, et al.
Publicado: (2025)

Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
por: Wang, Fei, et al.
Publicado: (2024)

LLM Zeroth-Order Fine-Tuning is an Inference Workload
por: Li, Zelin, et al.
Publicado: (2026)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
por: Guo, Wentao, et al.
Publicado: (2024)

On the Convergence of Zeroth-Order Federated Tuning for Large Language Models
por: Ling, Zhenqing, et al.
Publicado: (2024)

MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
por: Su, Yupeng, et al.
Publicado: (2026)

On-Device Fine-Tuning via Backprop-Free Zeroth-Order Optimization
por: Katti, Prabodh, et al.
Publicado: (2025)

Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning
por: Seung, Hyunseok, et al.
Publicado: (2025)

Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer
por: Zhao, Yanjun, et al.
Publicado: (2024)

Fine-tuning Quantized Neural Networks with Zeroth-order Optimization
por: Shang, Sifeng, et al.
Publicado: (2025)

On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
por: Ye, Rongguang, et al.
Publicado: (2025)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
por: Zhang, Yihua, et al.
Publicado: (2024)

Model Evolution Under Zeroth-Order Optimization: A Neural Tangent Kernel Perspective
por: Zhang, Chen, et al.
Publicado: (2026)

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
por: Li, Zeman, et al.
Publicado: (2024)

MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines
por: Gao, Lei, et al.
Publicado: (2024)

PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
por: Xiao, He, et al.
Publicado: (2025)

Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
por: Ran, Yide, et al.
Publicado: (2025)

OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning
por: Long, Jikai, et al.
Publicado: (2025)

Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer
por: Feng, Jian, et al.
Publicado: (2026)

Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization
por: Shu, Yao, et al.
Publicado: (2026)

QuIP: 2-Bit Quantization of Large Language Models With Guarantees
por: Chee, Jerry, et al.
Publicado: (2023)