Imagen de Portada

RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Huang, Xijie, Liu, Zechun, Liu, Shih-Yang, Cheng, Kwang-Ting
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

LLM-FP4: 4-Bit Floating-Point Quantized Transformers
por: Liu, Shih-yang, et al.
Publicado: (2023)

Efficient and Robust Quantization-aware Training via Adaptive Coreset Selection
por: Huang, Xijie, et al.
Publicado: (2023)

EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation
por: Liu, Shih-Yang, et al.
Publicado: (2024)

Activated LoRA: Fine-tuned LLMs for Intrinsics
por: Greenewald, Kristjan, et al.
Publicado: (2025)

LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models
por: Ran, Delong, et al.
Publicado: (2025)

Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning
por: Huang, Xijie, et al.
Publicado: (2023)

Outlier-weighed Layerwise Sampling for LLM Fine-tuning
por: Li, Pengxiang, et al.
Publicado: (2024)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
por: Chen, Yukang, et al.
Publicado: (2023)

LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report
por: Zhao, Justin, et al.
Publicado: (2024)

Uncertainty quantification in fine-tuned LLMs using LoRA ensembles
por: Balabanov, Oleksandr, et al.
Publicado: (2024)

Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning
por: Huang, Shih-Cheng, et al.
Publicado: (2022)

Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization
por: Wang, Dongwei, et al.
Publicado: (2024)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
por: Deng, Guanzhi, et al.
Publicado: (2026)

OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantization
por: Gadhikar, Advait, et al.
Publicado: (2025)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
por: Li, Dengchun, et al.
Publicado: (2024)

HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
por: Tian, Chunlin, et al.
Publicado: (2024)

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
por: Lee, Yu-Ang, et al.
Publicado: (2026)

RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
por: Su, Zunhai, et al.
Publicado: (2025)

FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition
por: Yan, Yuxuan, et al.
Publicado: (2024)

AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
por: Song, Yurun, et al.
Publicado: (2025)

Joint stereo 3D object detection and implicit surface reconstruction
por: Li, Shichao, et al.
Publicado: (2021)

LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuning
por: Zhang, Longteng, et al.
Publicado: (2023)

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
por: Xiao, Yicheng, et al.
Publicado: (2025)

LoRA-Squeeze: Simple and Effective Post-Tuning and In-Tuning Compression of LoRA Modules
por: Vulić, Ivan, et al.
Publicado: (2026)

MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning
por: Yang, Shu, et al.
Publicado: (2024)

Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision
por: Huang, Xijie, et al.
Publicado: (2023)

HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization
por: Su, Yang, et al.
Publicado: (2024)

HypeLoRA: Hyper-Network-Generated LoRA Adapters for Calibrated Language Model Fine-Tuning
por: Trojan, Bartosz, et al.
Publicado: (2026)

ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation
por: Song, Yurun, et al.
Publicado: (2024)

DoRA: Weight-Decomposed Low-Rank Adaptation
por: Liu, Shih-Yang, et al.
Publicado: (2024)

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
por: Qin, Haotong, et al.
Publicado: (2024)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
por: Azimi, Rambod, et al.
Publicado: (2024)

GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning
por: Zhou, Sifan, et al.
Publicado: (2025)

multiMentalRoBERTa: A Fine-tuned Multiclass Classifier for Mental Health Disorder
por: Islam, K M Sajjadul, et al.
Publicado: (2025)

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
por: Cheng, Wenhua, et al.
Publicado: (2023)

EvalYaks: Instruction Tuning Datasets and LoRA Fine-tuned Models for Automated Scoring of CEFR B2 Speaking Assessment Transcripts
por: Scaria, Nicy, et al.
Publicado: (2024)

Align, Don't Divide: Revisiting the LoRA Architecture in Multi-Task Learning
por: Liu, Jinda, et al.
Publicado: (2025)

Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs
por: Paglieri, Davide, et al.
Publicado: (2024)

FinLoRA: Benchmarking LoRA Methods for Fine-Tuning LLMs on Financial Datasets
por: Wang, Dannong, et al.
Publicado: (2025)

LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs
por: Arabpour, Reza, et al.
Publicado: (2025)