Image de couverture de livre

RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wei, Quan, Yau, Chung-Yiu, Wai, Hoi-To, Zhao, Yang Katie, Kang, Dongyeop, Park, Youngsuk, Hong, Mingyi
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

A Stochastic Approximation Approach for Efficient Decentralized Optimization on Random Networks
par: Yau, Chung-Yiu, et autres
Publié: (2024)

EMC$^2$: Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence
par: Yau, Chung-Yiu, et autres
Publié: (2024)

Decentralized Stochastic Optimization over Unreliable Networks via Two-timescales Updates
par: Liu, Haoming, et autres
Publié: (2025)

A Two-timescale Primal-dual Algorithm for Decentralized Optimization with Compression
par: Liu, Haoming, et autres
Publié: (2025)

EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
par: Yau, Chung-Yiu, et autres
Publié: (2026)

Revisiting the Adam-SGD Gap in LLM Pre-Training: The Role of Large Effective Learning Rates
par: Glentis, Athanasios, et autres
Publié: (2026)

OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models
par: Lee, Changhun, et autres
Publié: (2023)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
par: Gautam, Tanmay, et autres
Publié: (2024)

QWHA: Quantization-Aware Walsh-Hadamard Adaptation for Parameter-Efficient Fine-Tuning on Large Language Models
par: Jeon, Hyesung, et autres
Publié: (2025)

MuonBP: Faster Muon via Block-Periodic Orthogonalization
par: Khaled, Ahmed, et autres
Publié: (2025)

L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models
par: Jeon, Hyesung, et autres
Publié: (2024)

RoSA: Enhancing Parameter-Efficient Fine-Tuning via RoPE-aware Selective Adaptation in Large Language Models
par: Pan, Dayan, et autres
Publié: (2025)

EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models
par: He, Yefei, et autres
Publié: (2023)

QEFT: Quantization for Efficient Fine-Tuning of LLMs
par: Lee, Changhun, et autres
Publié: (2024)

Preserving Diversity in Supervised Fine-Tuning of Large Language Models
par: Li, Ziniu, et autres
Publié: (2024)

JacQuant: STE-Free Quantization-Aware Training via Learned Jacobian Surrogates
par: Yi, Kai, et autres
Publié: (2026)

Accurate and Efficient Fine-Tuning of Quantized Large Language Models Through Optimal Balance
par: Shen, Ao, et autres
Publié: (2024)

On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
par: Ye, Rongguang, et autres
Publié: (2025)

TPP-LLM: Modeling Temporal Point Processes by Efficiently Fine-Tuning Large Language Models
par: Liu, Zefang, et autres
Publié: (2024)

SFT-GO: Supervised Fine-Tuning with Group Optimization for Large Language Models
par: Kim, Gyuhak, et autres
Publié: (2025)

QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning
par: Ananta, Moses, et autres
Publié: (2025)

Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
par: Zhang, Zhengxin, et autres
Publié: (2024)

The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Models
par: Riemer, Matthew, et autres
Publié: (2025)

Spatial Sigma-Delta Modulation for Coarsely Quantized Massive MIMO Downlink: Flexible Designs by Convex Optimization
par: Keung, Wai-Yiu, et autres
Publié: (2023)

RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
par: Liu, Jun, et autres
Publié: (2025)

RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation
par: Nikdan, Mahdi, et autres
Publié: (2024)

Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
par: Li, Jiaxiang, et autres
Publié: (2024)

Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis
par: Huang, Hong, et autres
Publié: (2025)

3D-Aware Vision-Language Models Fine-Tuning with Geometric Distillation
par: Lee, Seonho, et autres
Publié: (2025)

Diversity in Large Language Models under Supervised Fine-Tuning
par: Klypa, Roman, et autres
Publié: (2026)

Learning Explainable Dense Reward Shapes via Bayesian Optimization
par: Koo, Ryan, et autres
Publié: (2025)

Chain-of-Instructions: Compositional Instruction Tuning on Large Language Models
par: Hayati, Shirley Anugrah, et autres
Publié: (2024)

Safety-Aware Fine-Tuning of Large Language Models
par: Choi, Hyeong Kyu, et autres
Publié: (2024)

Unveiling the Generalization Power of Fine-Tuned Large Language Models
par: Yang, Haoran, et autres
Publié: (2024)

Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback
par: Ahn, Daechul, et autres
Publié: (2024)

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
par: Zhang, Longteng, et autres
Publié: (2026)

Memory-Efficient Fine-Tuning for Quantized Diffusion Model
par: Ryu, Hyogon, et autres
Publié: (2024)

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
par: Zhou, Jiajun, et autres
Publié: (2025)

Aligning Large Language Model Agents with Rational and Moral Preferences: A Supervised Fine-Tuning Approach
par: Lu, Wei, et autres
Publié: (2025)

Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models
par: Li, Yuan, et autres
Publié: (2025)