Cover Image

SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Khaki, Samir, Li, Xiuyu, Guo, Junxian, Zhu, Ligeng, Xu, Chenfeng, Plataniotis, Konstantinos N., Yazdanbakhsh, Amir, Keutzer, Kurt, Han, Song, Liu, Zhijian
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inference
by: Khaki, Samir, et al.
Published: (2025)

Sparse Refinement for Efficient High-Resolution Semantic Segmentation
by: Liu, Zhijian, et al.
Published: (2024)

The Need for Speed: Pruning Transformers with One Recipe
by: Khaki, Samir, et al.
Published: (2024)

Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
by: Xi, Haocheng, et al.
Published: (2025)

Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation
by: Yang, Shuo, et al.
Published: (2025)

ProbMCL: Simple Probabilistic Contrastive Learning for Multi-label Visual Classification
by: Sajedi, Ahmad, et al.
Published: (2024)

SqueezeLLM: Dense-and-Sparse Quantization
by: Kim, Sehoon, et al.
Published: (2023)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression
by: Mozaffari, Mohammad, et al.
Published: (2024)

Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA
by: Yang, Nuocheng, et al.
Published: (2026)

LoRAFusion: Efficient LoRA Fine-Tuning for LLMs
by: Zhu, Zhanda, et al.
Published: (2025)

LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
by: Xi, Haocheng, et al.
Published: (2026)

Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment
by: Li, Yiheng, et al.
Published: (2024)

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training
by: Xi, Haocheng, et al.
Published: (2024)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
by: Sheng, Ying, et al.
Published: (2023)

Improved Immiscible Diffusion: Accelerate Diffusion Training by Reducing Its Miscibility
by: Li, Yiheng, et al.
Published: (2025)

ATOM: Attention Mixer for Efficient Dataset Distillation
by: Khaki, Samir, et al.
Published: (2024)

Data-to-Model Distillation: Data-Efficient Learning Framework
by: Sajedi, Ahmad, et al.
Published: (2024)

DataDAM: Efficient Dataset Distillation with Attention Matching
by: Sajedi, Ahmad, et al.
Published: (2023)

MaLoRA: Gated Modality LoRA for Key-Space Alignment in Multimodal LLM Fine-Tuning
by: Zheng, Xinhan, et al.
Published: (2025)

HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
by: Tian, Chunlin, et al.
Published: (2024)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
by: Chen, Yukang, et al.
Published: (2023)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Zhang, Haojie
Published: (2025)

LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning
by: Huang, Yining, et al.
Published: (2025)

LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement
by: Bian, Jieming, et al.
Published: (2024)

CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models
by: Chen, Guanduo, et al.
Published: (2025)

Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives
by: Wang, Qinsi, et al.
Published: (2025)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)

Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
by: Wu, Chengyue, et al.
Published: (2025)

FinLoRA: Benchmarking LoRA Methods for Fine-Tuning LLMs on Financial Datasets
by: Wang, Dannong, et al.
Published: (2025)

Beyond Moore's Law: Harnessing the Redshift of Generative AI with Effective Hardware-Software Co-Design
by: Yazdanbakhsh, Amir
Published: (2025)

VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
by: Li, Yang, et al.
Published: (2024)

Rethinking the Rank Threshold for LoRA Fine-Tuning
by: Park, Juneyoung
Published: (2026)

CDM-QTA: Quantized Training Acceleration for Efficient LoRA Fine-Tuning of Diffusion Model
by: Lu, Jinming, et al.
Published: (2025)

LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
by: Liu, Yezi, et al.
Published: (2025)

Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA
by: Wang, Fei, et al.
Published: (2025)

L1RA: Dynamic Rank Assignment in LoRA Fine-Tuning
by: Singh, Raul, et al.
Published: (2025)

FGMP: Fine-Grained Mixed-Precision Weight and Activation Quantization for Hardware-Accelerated LLM Inference
by: Hooper, Coleman, et al.
Published: (2025)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
by: Kunwar, Pradip, et al.
Published: (2025)

FedTreeLoRA: Reconciling Statistical and Functional Heterogeneity in Federated LoRA Fine-Tuning
by: Bian, Jieming, et al.
Published: (2026)