Imagen de Portada

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	He, Yiyuan, Xu, Minxian, Wu, Jingfeng, Hu, Jianmin, Ma, Chong, Shen, Min, Chen, Le, Xu, Chengzhong, Qu, Lin, Ye, Kejiang
Formato:	Preprint
Publicado:	2025
Materias:	Distributed, Parallel, and Cluster Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
por: Yiyuan He, et al.
Publicado: (2026)

UELLM: A Unified and Efficient Approach for LLM Inference Serving
por: He, Yiyuan, et al.
Publicado: (2024)

Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
por: Wu, Jingfeng, et al.
Publicado: (2025)

Cloud Native System for LLM Inference Serving
por: Xu, Minxian, et al.
Publicado: (2025)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
por: Hu, Jianmin, et al.
Publicado: (2025)

DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
por: Liao, Junhan, et al.
Publicado: (2025)

CloudNativeSim: a toolkit for modeling and simulation of cloud-native applications
por: Wu, Jingfeng, et al.
Publicado: (2024)

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
por: Zheng, Wanyi, et al.
Publicado: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
por: Hu, Cunchen, et al.
Publicado: (2024)

TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
por: Yoon, Dongha, et al.
Publicado: (2025)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
por: Ruan, Chaoyi, et al.
Publicado: (2025)

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
por: Lin, Yanying, et al.
Publicado: (2025)

Collaborative Resource Management and Workloads Scheduling in Cloud-Assisted Mobile Edge Computing across Timescales
por: Tang, Lujie, et al.
Publicado: (2024)

LSRAM: A Lightweight Autoscaling and SLO Resource Allocation Framework for Microservices Based on Gradient Descent
por: Hu, Kan, et al.
Publicado: (2024)

DRPC: Distributed Reinforcement Learning Approach for Scalable Resource Provisioning in Container-based Clusters
por: Bai, Haoyu, et al.
Publicado: (2024)

CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
por: Nian, Sean, et al.
Publicado: (2026)

PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
por: Woo, Sunghyeon, et al.
Publicado: (2026)

Efficient Multi-round LLM Inference over Disaggregated Serving
por: He, Wenhao, et al.
Publicado: (2026)

ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache
por: Wang, Shao, et al.
Publicado: (2026)

Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
por: Wang, Chao, et al.
Publicado: (2025)

Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking
por: Mo, Zizhao, et al.
Publicado: (2026)

LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure
por: Cho, Jaehong, et al.
Publicado: (2026)

KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
por: Liu, Zedong, et al.
Publicado: (2026)

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
por: Zhu, Jianian, et al.
Publicado: (2025)

TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
por: Bian, Zhuohang, et al.
Publicado: (2026)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
por: li, Fei, et al.
Publicado: (2026)

DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
por: Yuan, Ying, et al.
Publicado: (2026)

Auto-scaling Approaches for Microservice Applications: A Survey and Taxonomy
por: Xu, Minxian, et al.
Publicado: (2025)

StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving
por: Kumar, Satyam, et al.
Publicado: (2026)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
por: Zhong, Yinmin, et al.
Publicado: (2024)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
por: Kim, Kihyun, et al.
Publicado: (2025)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
por: Su, Zhaoyuan, et al.
Publicado: (2025)

SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving
por: Guo, Yipin, et al.
Publicado: (2026)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
por: Mo, Zizhao, et al.
Publicado: (2025)

TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity
por: Lai, Ruiqi, et al.
Publicado: (2025)

vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models
por: Yin, Peiqi, et al.
Publicado: (2026)

C-Koordinator: Interference-aware Management for Large-scale and Co-located Microservice Clusters
por: Song, Shengye, et al.
Publicado: (2025)

An Interference-aware Approach for Co-located Container Orchestration with Novel Metric
por: Li, Xiang, et al.
Publicado: (2024)

MSARS: A Meta-Learning and Reinforcement Learning Framework for SLO Resource Allocation and Adaptive Scaling for Microservices
por: Hu, Kan, et al.
Publicado: (2024)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
por: Liu, Dong, et al.
Publicado: (2025)