Copertina

LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhou, Zikai, Zhang, Qizheng, Kumbong, Hermann, Olukotun, Kunle
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Hardware Architecture Computation and Language Performance
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
di: Zhang, Hang, et al.
Pubblicazione: (2025)

BitDecoding: Unlocking Tensor Cores for Long-Context LLMs with Low-Bit KV Cache
di: Du, Dayou, et al.
Pubblicazione: (2025)

LoRA-Edge: Tensor-Train-Assisted LoRA for Practical CNN Fine-Tuning on Edge Devices
di: Kwak, Hyunseok, et al.
Pubblicazione: (2025)

SSM-RDU: A Reconfigurable Dataflow Unit for Long-Sequence State-Space Models
di: Ko, Sho, et al.
Pubblicazione: (2025)

Concorde: Fast and Accurate CPU Performance Modeling with Compositional Analytical-ML Fusion
di: Nasr-Esfahany, Arash, et al.
Pubblicazione: (2025)

WaveTune: Wave-aware Bilinear Modeling for Efficient GPU Kernel Auto-tuning
di: Zhang, Kaixuan, et al.
Pubblicazione: (2026)

SONIQ: System-Optimized Noise-Injected Ultra-Low-Precision Quantization with Full-Precision Parity
di: Zhou, Cyrus, et al.
Pubblicazione: (2023)

FuseFlow: A Fusion-Centric Compilation Framework for Sparse Deep Learning on Streaming Dataflow
di: Lacouture, Rubens, et al.
Pubblicazione: (2025)

Implementing and Optimizing the Scaled Dot-Product Attention on Streaming Dataflow
di: Sohn, Gina, et al.
Pubblicazione: (2024)

Non-Monotonic Latency in Apple MPS Decoding: KV Cache Interactions and Execution Regimes
di: Hendria, Willy Fitra
Pubblicazione: (2026)

A$^3$PIM: An Automated, Analytic and Accurate Processing-in-Memory Offloader
di: Jiang, Qingcai, et al.
Pubblicazione: (2024)

Streaming Tensor Programs: A Streaming Abstraction for Dynamic Parallelism
di: Sohn, Gina, et al.
Pubblicazione: (2025)

Latency Based Tiling
di: Cashman, Jack
Pubblicazione: (2025)

HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture
di: Wu, Taiqiang, et al.
Pubblicazione: (2025)

Examem: Low-Overhead Memory Instrumentation for Intelligent Memory Systems
di: Poduval, Ashwin, et al.
Pubblicazione: (2024)

A2Q+: Improving Accumulator-Aware Weight Quantization
di: Colbert, Ian, et al.
Pubblicazione: (2024)

CDM-QTA: Quantized Training Acceleration for Efficient LoRA Fine-Tuning of Diffusion Model
di: Lu, Jinming, et al.
Pubblicazione: (2025)

Sieve: Dynamic Expert-Aware PIM Acceleration for Evolving Mixture-of-Experts Models
di: Kim, Jungwoo, et al.
Pubblicazione: (2026)

L1RA: Dynamic Rank Assignment in LoRA Fine-Tuning
di: Singh, Raul, et al.
Pubblicazione: (2025)

Automatic Generation of Fast and Accurate Performance Models for Deep Neural Network Accelerators
di: Lübeck, Konstantin, et al.
Pubblicazione: (2024)

GeneTEK: Low-power, high-performance and scalable FPGA architecture for exact unit-cost edit distance
di: Espinosa, Elena, et al.
Pubblicazione: (2025)

HiKonv: Maximizing the Throughput of Quantized Convolution With Novel Bit-wise Management and Computation
di: Chen, Yao, et al.
Pubblicazione: (2022)

USEFUSE: Uniform Stride for Enhanced Performance in Fused Layer Architecture of Deep Neural Networks
di: Ibrahim, Muhammad Sohail, et al.
Pubblicazione: (2024)

PoTAcc: A Pipeline for End-to-End Acceleration of Power-of-Two Quantized DNNs
di: Saha, Rappy, et al.
Pubblicazione: (2026)

Toward A Formalized Approach for Spike Sorting Algorithms and Hardware Evaluation
di: Zhang, Tim, et al.
Pubblicazione: (2022)

Search Your Block Floating Point Scales!
di: Gupta, Tanmaey, et al.
Pubblicazione: (2026)

GCL-Sampler: Discovering Kernel Similarity for Sampled GPU Simulation via Graph Contrastive Learning
di: Wang, Jiaqi, et al.
Pubblicazione: (2026)

Understanding the Performance Horizon of the Latest ML Workloads with NonGEMM Workloads
di: Karami, Rachid, et al.
Pubblicazione: (2024)

Prefill vs. Decode Bottlenecks: SRAM-Frequency Tradeoffs and the Memory-Bandwidth Ceiling
di: Atmer, Hannah, et al.
Pubblicazione: (2025)

Graph neural networks with configuration cross-attention for tensor compilers
di: Khizbullin, Dmitrii, et al.
Pubblicazione: (2024)

ETM2: Empowering Traditional Memory Bandwidth Regulation using ETM
di: Zuepke, Alexander, et al.
Pubblicazione: (2026)

CXL-Interference: Analysis and Characterization in Modern Computer Systems
di: Mao, Shunyu, et al.
Pubblicazione: (2024)

Cleaning up the Mess: Re-Evaluating the Real-System Modeling Accuracy of Ramulator 2.0
di: Bostanci, F. Nisa, et al.
Pubblicazione: (2025)

SparAMX: Accelerating Compressed LLMs Token Generation on AMX-powered CPUs
di: AbouElhamayed, Ahmed F., et al.
Pubblicazione: (2025)

RooflineBench: A Benchmarking Framework for On-Device LLMs via Roofline Analysis
di: Bi, Zhen, et al.
Pubblicazione: (2026)

DEER: Deep Runahead for Instruction Prefetching on Modern Mobile Workloads
di: Vahdatniya, Parmida, et al.
Pubblicazione: (2025)

LightningSimV2: Faster and Scalable Simulation for High-Level Synthesis via Graph Compilation and Optimization
di: Sarkar, Rishov, et al.
Pubblicazione: (2024)

DFModel: Design Space Optimization of Large-Scale Systems Exploiting Dataflow Mappings
di: Ko, Sho, et al.
Pubblicazione: (2024)

FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs
di: Xie, Xilong, et al.
Pubblicazione: (2025)

Linear Layouts: Robust Code Generation of Efficient Tensor Computation Using $\mathbb{F}_2$
di: Zhou, Keren, et al.
Pubblicazione: (2025)