InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917391967977472 |
|---|---|
| author | Chen, Hongyu Ruan, Letian Xu, Zilin Li, Yuchen Chen, Xinyu Leng, Jingwen He, Bingsheng Guo, Minyi Sun, Shixuan |
| author_facet | Chen, Hongyu Ruan, Letian Xu, Zilin Li, Yuchen Chen, Xinyu Leng, Jingwen He, Bingsheng Guo, Minyi Sun, Shixuan |
| contents | LoRA enables efficient customization of LLMs and is widely used in multi-tenant and multi-task serving. However, emerging model architectures such as MoE significantly increase LoRA memory cost, making existing coupled LoRA serving designs poorly scalable and prone to tail-latency inflation. We present InfiniLoRA, a disaggregated LoRA serving system that decouples LoRA execution from base-model inference. InfiniLoRA introduces a shared LoRA Server with parallelism-aware execution, SLO-driven provisioning, and critical-path optimizations, including GPU-initiated communication and hardware-specialized LoRA kernels. Experiments show that InfiniLoRA can achieve an average $3.05\times$ increase in serviceable request rate under strict latency SLOs, and improve the percentage of LoRA adapters satisfying the SLO requirement by 54.0\%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_07173 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models Chen, Hongyu Ruan, Letian Xu, Zilin Li, Yuchen Chen, Xinyu Leng, Jingwen He, Bingsheng Guo, Minyi Sun, Shixuan Distributed, Parallel, and Cluster Computing LoRA enables efficient customization of LLMs and is widely used in multi-tenant and multi-task serving. However, emerging model architectures such as MoE significantly increase LoRA memory cost, making existing coupled LoRA serving designs poorly scalable and prone to tail-latency inflation. We present InfiniLoRA, a disaggregated LoRA serving system that decouples LoRA execution from base-model inference. InfiniLoRA introduces a shared LoRA Server with parallelism-aware execution, SLO-driven provisioning, and critical-path optimizations, including GPU-initiated communication and hardware-specialized LoRA kernels. Experiments show that InfiniLoRA can achieve an average $3.05\times$ increase in serviceable request rate under strict latency SLOs, and improve the percentage of LoRA adapters satisfying the SLO requirement by 54.0\%. |
| title | InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2604.07173 |