Imagen de Portada

AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yang, Yifan, Zhen, Kai, Banijamal, Ershad, Mouchtaris, Athanasios, Zhang, Zheng
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
por: Zhou, Jiajun, et al.
Publicado: (2025)

MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models
por: Zhang, Zhen, et al.
Publicado: (2025)

Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models
por: Solgi, Ryan, et al.
Publicado: (2025)

Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
por: Wang, Fei, et al.
Publicado: (2024)

Prior-Informed Zeroth-Order Optimization with Adaptive Direction Alignment for Memory-Efficient LLM Fine-Tuning
por: Jin, Feihu, et al.
Publicado: (2026)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
por: Zhang, Yihua, et al.
Publicado: (2024)

Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models
por: Ghiasvand, Sajjad, et al.
Publicado: (2024)

AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
por: Cemri, Mert, et al.
Publicado: (2026)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
por: Yang, Yifan, et al.
Publicado: (2024)

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
por: Li, Zeman, et al.
Publicado: (2024)

On the Convergence of Zeroth-Order Federated Tuning for Large Language Models
por: Ling, Zhenqing, et al.
Publicado: (2024)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
por: Gautam, Tanmay, et al.
Publicado: (2024)

Hi-ZFO: Hierarchical Zeroth- and First-Order LLM Fine-Tuning via Importance-Guided Tensor Selection
por: Jin, Feihu, et al.
Publicado: (2026)

AdaFRUGAL: Adaptive Memory-Efficient Training with Dynamic Control
por: Bui, Quang-Hung, et al.
Publicado: (2025)

On-Device Fine-Tuning via Backprop-Free Zeroth-Order Optimization
por: Katti, Prabodh, et al.
Publicado: (2025)

AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Experts
por: Liu, Zefang, et al.
Publicado: (2024)

Reversing Large Language Models for Efficient Training and Fine-Tuning
por: Gal, Eshed, et al.
Publicado: (2025)

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
por: Liu, Yong, et al.
Publicado: (2024)

Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns
por: Zhao, Yang, et al.
Publicado: (2024)

Wanda++: Pruning Large Language Models via Regional Gradients
por: Yang, Yifan, et al.
Publicado: (2025)

Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training
por: Cunegatti, Elia, et al.
Publicado: (2024)

Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuning
por: Jie, Shibo, et al.
Publicado: (2024)

TLoRA+: A Low-Rank Parameter-Efficient Fine-Tuning Method for Large Language Models
por: Cao, Yarui, et al.
Publicado: (2026)

NeuroAda: Activating Each Neuron's Potential for Parameter-Efficient Fine-Tuning
por: Zhang, Zhi, et al.
Publicado: (2025)

Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models
por: Doering, Nigel, et al.
Publicado: (2024)

AdaSVD: Adaptive Singular Value Decomposition for Large Language Models
por: Li, Zhiteng, et al.
Publicado: (2025)

AdaComp: Extractive Context Compression with Adaptive Predictor for Retrieval-Augmented Large Language Models
por: Zhang, Qianchi, et al.
Publicado: (2024)

Private Fine-tuning of Large Language Models with Zeroth-order Optimization
por: Tang, Xinyu, et al.
Publicado: (2024)

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
por: Zhang, Kairun, et al.
Publicado: (2025)

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
por: Pan, Rui, et al.
Publicado: (2024)

Continual Pre-Training is (not) What You Need in Domain Adaption
por: Chen, Pin-Er, et al.
Publicado: (2025)

AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
por: Lu, Kuan-Wei, et al.
Publicado: (2025)

AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning
por: Refael, Yehonathan, et al.
Publicado: (2024)

AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents
por: Yan, Shannan, et al.
Publicado: (2026)

Memory-Efficient Fine-Tuning of Transformers via Token Selection
por: Simoulin, Antoine, et al.
Publicado: (2025)

Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation
por: Leong, Hui Yi, et al.
Publicado: (2024)

Visualising Policy-Reward Interplay to Inform Zeroth-Order Preference Optimisation of Large Language Models
por: Galatolo, Alessio, et al.
Publicado: (2025)

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning
por: Liu, Z, et al.
Publicado: (2024)

Parameter-Efficient Fine-Tuning of Large Language Models using Semantic Knowledge Tuning
por: Prottasha, Nusrat Jahan, et al.
Publicado: (2024)

ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory
por: Wang, Liangyu, et al.
Publicado: (2025)