Cover Image

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Hang, Shi, Jiuchen, Wang, Yixiao, Chen, Quan, Shan, Yizhou, Guo, Minyi
Format:	Preprint
Published:	2025
Subjects:	Hardware Architecture Artificial Intelligence Machine Learning Performance
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits
by: Zhou, Zikai, et al.
Published: (2025)

LoRA-Edge: Tensor-Train-Assisted LoRA for Practical CNN Fine-Tuning on Edge Devices
by: Kwak, Hyunseok, et al.
Published: (2025)

Enhancing Instruction Prefetching via Cache and TLB Management
by: Jamet, Alexandre Valentin, et al.
Published: (2026)

Adaptive Cache Pollution Control for Large Language Model Inference Workloads Using Temporal CNN-Based Prediction and Priority-Aware Replacement
by: Liu, Songze, et al.
Published: (2025)

Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
by: Fang, Yunhua, et al.
Published: (2025)

The Bicameral Cache: a split cache for vector architectures
by: Rebolledo, Susana, et al.
Published: (2024)

CDM-QTA: Quantized Training Acceleration for Efficient LoRA Fine-Tuning of Diffusion Model
by: Lu, Jinming, et al.
Published: (2025)

A Review on Proprietary Accelerators for Large Language Models
by: Park, Sihyeong, et al.
Published: (2025)

Non-Monotonic Latency in Apple MPS Decoding: KV Cache Interactions and Execution Regimes
by: Hendria, Willy Fitra
Published: (2026)

ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
by: Lei, Jianlong, et al.
Published: (2026)

BitDecoding: Unlocking Tensor Cores for Long-Context LLMs with Low-Bit KV Cache
by: Du, Dayou, et al.
Published: (2025)

SAHM: State-Aware Heterogeneous Multicore for Single-Thread Performance
by: Wadle, Shayne, et al.
Published: (2025)

Towards CPU Performance Prediction: New Challenge Benchmark Dataset and Novel Approach
by: Liu, Xiaoman
Published: (2024)

Data-Driven Power Modeling and Monitoring via Hardware Performance Counter Tracking
by: Mazzola, Sergio, et al.
Published: (2025)

PipeWeave: Synergizing Analytical and Learning Models for Unified GPU Performance Prediction
by: Zhang, Kaixuan, et al.
Published: (2026)

Single 32-bit Sub-Channel DDR5 DIMMs: Architecture, Performance Bounds, and Standardisation
by: Ke, Chih-Hua
Published: (2026)

ACALSim: A Scalable Parallel Simulation Framework for High-Performance System Design Space Exploration
by: Lin, Wei-Fen, et al.
Published: (2026)

WaveTune: Wave-aware Bilinear Modeling for Efficient GPU Kernel Auto-tuning
by: Zhang, Kaixuan, et al.
Published: (2026)

Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
by: Li, Jiaxi, et al.
Published: (2025)

ONNXim: A Fast, Cycle-level Multi-core NPU Simulator
by: Ham, Hyungkyu, et al.
Published: (2024)

3RSeT: Read Disturbance Rate Reduction in STT-MRAM Caches by Selective Tag Comparison
by: Cheshmikhani, Elham, et al.
Published: (2025)

HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture
by: Wu, Taiqiang, et al.
Published: (2025)

JSPIM: A Skew-Aware PIM Accelerator for High-Performance Databases Join and Select Operations
by: Tajdari, Sabiha, et al.
Published: (2025)

Heterogeneous Memory Benchmarking Toolkit
by: Ghaemi, Golsana, et al.
Published: (2025)

Simulation-Driven Evaluation of Chiplet-Based Architectures Using VisualSim
by: Ali, Wajid, et al.
Published: (2025)

ETM2: Empowering Traditional Memory Bandwidth Regulation using ETM
by: Zuepke, Alexander, et al.
Published: (2026)

Recurrent CircuitSAT Sampling for Sequential Circuits
by: Ardakani, Arash, et al.
Published: (2025)

Introducing the Arm-membench Throughput Benchmark
by: Burth, Cyrill, et al.
Published: (2025)

Enhancing software-hardware co-design for HEP by low-overhead profiling of single- and multi-threaded programs on diverse architectures with Adaptyst
by: Graczyk, Maksymilian, et al.
Published: (2025)

SPEC CPU2026: Characterization, Representativeness, and Cross-Suite Comparison
by: Li, Ruihao, et al.
Published: (2026)

Regular-Dead on Arrival: Characterizing and Protecting Against Dead-Entry TLB Misses in GPU Microarchitectures
by: Anik, Shafayat Mowla, et al.
Published: (2026)

Makinote: An FPGA-Based HW/SW Platform for Pre-Silicon Emulation of RISC-V Designs
by: Perdomo, Elias, et al.
Published: (2024)

AI Load Dynamics--A Power Electronics Perspective
by: Li, Yuzhuo, et al.
Published: (2025)

LightningSimV2: Faster and Scalable Simulation for High-Level Synthesis via Graph Compilation and Optimization
by: Sarkar, Rishov, et al.
Published: (2024)

Range, Not Precision: Block-Floating-Point Half-Precision FFT and SAR Imaging on Apple Silicon
by: Bergach, Mohamed Amine
Published: (2026)

CXL-Interference: Analysis and Characterization in Modern Computer Systems
by: Mao, Shunyu, et al.
Published: (2024)

OPTIMA: Design-Space Exploration of Discharge-Based In-SRAM Computing: Quantifying Energy-Accuracy Trade-Offs
by: Seyedfaraji, Saeed, et al.
Published: (2024)

An Analytical Cost Model for Fast Evaluation of Multiple Compute-Engine CNN Accelerators
by: Qararyah, Fareed, et al.
Published: (2025)

Characterizing and Optimizing Realistic Workloads on a Commercial Compute-in-SRAM Device
by: Zhang, Niansong, et al.
Published: (2025)

Accelerating Transistor-Level Simulation of Integrated Circuits via Equivalence of RC Long-Chain Structures
by: Tang, Ruibai, et al.
Published: (2025)