Cover Image

AutoMixQ: Self-Adjusting Quantization for High Performance Memory-Efficient Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Changhai, Zhang, Shiyang, Zhou, Yuhua, Liu, Zekai, Weng, Shichao
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Balancing Fidelity and Plasticity: Aligning Mixed-Precision Fine-Tuning with Linguistic Hierarchies
by: Zhou, Changhai, et al.
Published: (2025)

AutoQRA: Joint Optimization of Mixed-Precision Quantization and Low-rank Adapters for Efficient LLM Fine-Tuning
by: Zhou, Changhai, et al.
Published: (2026)

Large Language Model Compression with Global Rank and Sparsity Optimization
by: Zhou, Changhai, et al.
Published: (2025)

RankAdaptor: Hierarchical Rank Allocation for Efficient Fine-Tuning Pruned LLMs via Performance Model
by: Zhou, Changhai, et al.
Published: (2024)

AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning
by: Zhou, Han, et al.
Published: (2023)

CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization
by: Deng, Yanxia, et al.
Published: (2025)

Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
by: Zhang, Zhengxin, et al.
Published: (2024)

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
by: Zhou, Jiajun, et al.
Published: (2025)

On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
by: Ye, Rongguang, et al.
Published: (2025)

Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis
by: Huang, Hong, et al.
Published: (2025)

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
by: Shi, Jiang-Xin, et al.
Published: (2025)

MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Models
by: Liu, Wenyuan, et al.
Published: (2025)

QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals
by: Zhang, Nan, et al.
Published: (2026)

RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
by: Wei, Quan, et al.
Published: (2025)

MELINOE: Fine-Tuning Enables Memory-Efficient Inference for Mixture-of-Experts Models
by: Raje, Arian, et al.
Published: (2026)

Memory-Efficient Fine-Tuning for Quantized Diffusion Model
by: Ryu, Hyogon, et al.
Published: (2024)

A Bayesian Hybrid Parameter-Efficient Fine-Tuning Method for Large Language Models
by: Chai, Yidong, et al.
Published: (2025)

RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks
by: Liu, Ningyuan, et al.
Published: (2025)

Adjusting Pretrained Backbones for Performativity
by: Demirel, Berker, et al.
Published: (2024)

FedKRSO: Communication and Memory Efficient Federated Fine-Tuning of Large Language Models
by: Yang, Guohao, et al.
Published: (2026)

Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation
by: Yang, Yuchen, et al.
Published: (2024)

An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning
by: Li, Cen-Jhih, et al.
Published: (2025)

AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
by: Song, Yurun, et al.
Published: (2025)

Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models
by: Arteaga, Gabriel Y., et al.
Published: (2024)

Q-realign: Piggybacking Realignment on Quantization for Safe and Efficient LLM Deployment
by: Tan, Qitao, et al.
Published: (2026)

mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs
by: Ye, Zhengmao, et al.
Published: (2023)

FRoD: Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence
by: Wan, Guoan, et al.
Published: (2025)

DONOD: Efficient and Generalizable Instruction Fine-Tuning for LLMs via Model-Intrinsic Dataset Pruning
by: Hu, Jucheng, et al.
Published: (2025)

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
by: Pan, Rui, et al.
Published: (2024)

Study of Training Dynamics for Memory-Constrained Fine-Tuning
by: Quélennec, Aël, et al.
Published: (2025)

Fine-Tuning without Performance Degradation
by: Wang, Han, et al.
Published: (2025)

GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning
by: Zhou, Sifan, et al.
Published: (2025)

Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning
by: Zhao, Maosen, et al.
Published: (2025)

LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning
by: Chen, Junyu, et al.
Published: (2025)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
by: Hong, Joey, et al.
Published: (2024)

DeltaDQ: Ultra-High Delta Compression for Fine-Tuned LLMs via Group-wise Dropout and Separate Quantization
by: Jiang, Yanfeng, et al.
Published: (2024)

Parameter-Efficient Continual Fine-Tuning: A Survey
by: Coleman, Eric Nuertey, et al.
Published: (2025)

Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies
by: Weng, Benjue
Published: (2024)

Diffusion Fine-Tuning via Reparameterized Policy Gradient of the Soft Q-Function
by: Kang, Hyeongyu, et al.
Published: (2025)

Self-Mined Hardness for Safety Fine-Tuning
by: Gupta, Prakhar, et al.
Published: (2026)