Imagen de Portada

MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Zhen, Yang, Yifan, Zhen, Kai, Susanj, Nathan, Mouchtaris, Athanasios, Kunzmann, Siegfried, Zhang, Zheng
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
por: Zhou, Jiajun, et al.
Publicado: (2025)

AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning
por: Yang, Yifan, et al.
Publicado: (2024)

Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models
por: Solgi, Ryan, et al.
Publicado: (2025)

SharpZO: Hybrid Sharpness-Aware Vision Language Model Prompt Tuning via Forward-Only Passes
por: Yang, Yifan, et al.
Publicado: (2025)

ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory
por: Wang, Liangyu, et al.
Publicado: (2025)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
por: Wang, Shuo, et al.
Publicado: (2026)

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
por: Liu, Yong, et al.
Publicado: (2024)

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
por: Zhang, Kairun, et al.
Publicado: (2025)

ZO-ASR: Zeroth-Order Fine-Tuning of Speech Foundation Models without Back-Propagation
por: Peng, Yuezhang, et al.
Publicado: (2025)

AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
por: Chen, Ziye, et al.
Publicado: (2026)

Wanda++: Pruning Large Language Models via Regional Gradients
por: Yang, Yifan, et al.
Publicado: (2025)

TeZO: Empowering the Low-Rankness on the Temporal Dimension in the Zeroth-Order Optimization for Fine-tuning LLMs
por: Sun, Yan, et al.
Publicado: (2025)

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
por: Cai, Zhijie, et al.
Publicado: (2026)

ElasticZO: A Memory-Efficient On-Device Learning with Combined Zeroth- and First-Order Optimization
por: Sugiura, Keisuke, et al.
Publicado: (2025)

Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
por: Wang, Fei, et al.
Publicado: (2024)

FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed
por: Dang, Sizhe, et al.
Publicado: (2025)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
por: Zhang, Yihua, et al.
Publicado: (2024)

DistZO2: High-Throughput and Memory-Efficient Zeroth-Order Fine-tuning LLMs with Distributed Parallel Computing
por: Wang, Liangyu, et al.
Publicado: (2025)

ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithm
por: Xie, Lunchen, et al.
Publicado: (2025)

On-Device Fine-Tuning via Backprop-Free Zeroth-Order Optimization
por: Katti, Prabodh, et al.
Publicado: (2025)

Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning
por: Seung, Hyunseok, et al.
Publicado: (2025)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
por: Guo, Wentao, et al.
Publicado: (2024)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
por: Gautam, Tanmay, et al.
Publicado: (2024)

Quantization-Aware and Tensor-Compressed Training of Transformers for Natural Language Understanding
por: Yang, Zi, et al.
Publicado: (2023)

Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks
por: Hu, Jiacheng, et al.
Publicado: (2024)

LLM Zeroth-Order Fine-Tuning is an Inference Workload
por: Li, Zelin, et al.
Publicado: (2026)

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
por: Yu, Ziming, et al.
Publicado: (2024)

On the Convergence of Zeroth-Order Federated Tuning for Large Language Models
por: Ling, Zhenqing, et al.
Publicado: (2024)

Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer
por: Zhao, Yanjun, et al.
Publicado: (2024)

Boosting Large Language Models with Mask Fine-Tuning
por: Zhang, Mingyuan, et al.
Publicado: (2025)

OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning
por: Long, Jikai, et al.
Publicado: (2025)

Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer
por: Feng, Jian, et al.
Publicado: (2026)

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
por: Li, Zeman, et al.
Publicado: (2024)

KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning
por: Mi, Zhendong, et al.
Publicado: (2025)

Prior-Informed Zeroth-Order Optimization with Adaptive Direction Alignment for Memory-Efficient LLM Fine-Tuning
por: Jin, Feihu, et al.
Publicado: (2026)

Branch, or Layer? Zeroth-Order Optimization for Continual Learning of Vision-Language Models
por: Liu, Ziwei, et al.
Publicado: (2025)

Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models
por: Ghiasvand, Sajjad, et al.
Publicado: (2024)

Certified Multi-Fidelity Zeroth-Order Optimization
por: de Montbrun, Étienne, et al.
Publicado: (2023)

The Multi-Query Paradox in Zeroth-Order Optimization
por: Lin, Wei, et al.
Publicado: (2025)

Zeroth-Order Optimization at the Edge of Stability
por: Song, Minhak, et al.
Publicado: (2026)