Image de couverture de livre

ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Meng, Chuiyang, Tang, Ming, Wong, Vincent W. S.
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

FLoRG: Federated Fine-tuning with Low-rank Gram Matrices and Procrustes Alignment
par: Meng, Chuiyang, et autres
Publié: (2026)

ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning
par: Meng, Chuiyang, et autres
Publié: (2026)

Activated LoRA: Fine-tuned LLMs for Intrinsics
par: Greenewald, Kristjan, et autres
Publié: (2025)

HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
par: Zhao, Huaqin, et autres
Publié: (2024)

Harmony in Divergence: Towards Fast, Accurate, and Memory-efficient Zeroth-order LLM Fine-tuning
par: Tan, Qitao, et autres
Publié: (2025)

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
par: Yu, Ziming, et autres
Publié: (2024)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
par: Guo, Wentao, et autres
Publié: (2024)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
par: Wang, Shuo, et autres
Publié: (2026)

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
par: Cai, Zhijie, et autres
Publié: (2026)

SDFLoRA: Selective Decoupled Federated LoRA for Privacy-preserving Fine-tuning with Heterogeneous Clients
par: Shen, Zhikang, et autres
Publié: (2026)

MoBA: Mixture of Block Attention for Long-Context LLMs
par: Lu, Enzhe, et autres
Publié: (2025)

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
par: Zhou, Jiajun, et autres
Publié: (2025)

Learning Heterogeneous Performance-Fairness Trade-offs in Federated Learning
par: Ye, Rongguang, et autres
Publié: (2025)

Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
par: Ran, Yide, et autres
Publié: (2025)

Aggregating Low Rank Adapters in Federated Fine-tuning
par: Trautmann, Evelyn, et autres
Publié: (2025)

FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion
par: Fan, Tao, et autres
Publié: (2026)

New Insights on Unfolding and Fine-tuning Quantum Federated Learning
par: Nanayakkara, Shanika Iroshi, et autres
Publié: (2025)

Efficient Knowledge Graph Unlearning with Zeroth-order Information
par: Xiao, Yang, et autres
Publié: (2025)

HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization
par: Su, Yang, et autres
Publié: (2024)

Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMs
par: Chen, Jack, et autres
Publié: (2025)

RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization
par: Huang, Xijie, et autres
Publié: (2024)

PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
par: Yu, Tongzhou, et autres
Publié: (2025)

LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilities
par: Schmied, Thomas, et autres
Publié: (2025)

Perturbation-efficient Zeroth-order Optimization for Hardware-friendly On-device Training
par: Tan, Qitao, et autres
Publié: (2025)

FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition
par: Yan, Yuxuan, et autres
Publié: (2024)

Warming Up for Zeroth-Order Federated Pre-Training with Low Resource Clients
par: Legate, Gwen, et autres
Publié: (2025)

A Historical Trajectory Assisted Optimization Method for Zeroth-Order Federated Learning
par: Wu, Chenlin, et autres
Publié: (2024)

On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
par: Ye, Rongguang, et autres
Publié: (2025)

HeteroBA: A Structure-Manipulating Backdoor Attack on Heterogeneous Graphs
par: Gao, Honglin, et autres
Publié: (2025)

Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost
par: Xu, Yinggan, et autres
Publié: (2026)

Federated Fine-Tuning of LLMs: Framework Comparison and Research Directions
par: Yan, Na, et autres
Publié: (2025)

Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer
par: Feng, Jian, et autres
Publié: (2026)

OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning
par: Long, Jikai, et autres
Publié: (2025)

H2Tune: Federated Foundation Model Fine-Tuning with Hybrid Heterogeneity
par: Guo, Wei, et autres
Publié: (2025)

EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
par: Guo, Song, et autres
Publié: (2024)

Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning
par: Jan, Essa, et autres
Publié: (2024)

Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning
par: Tong, Anh, et autres
Publié: (2025)

MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models
par: Zhang, Zhen, et autres
Publié: (2025)

Deploying Models to Non-participating Clients in Federated Learning without Fine-tuning: A Hypernetwork-based Approach
par: Zhou, Yuhao, et autres
Publié: (2025)

AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
par: Song, Yurun, et autres
Publié: (2025)