Imagen de Portada

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zheng, Wanyi, Xu, Minxian, Song, Shengye, Ye, Kejiang
Formato:	Preprint
Publicado:	2025
Materias:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

UELLM: A Unified and Efficient Approach for LLM Inference Serving
por: He, Yiyuan, et al.
Publicado: (2024)

DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
por: Liao, Junhan, et al.
Publicado: (2025)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
por: Hu, Jianmin, et al.
Publicado: (2025)

Cloud Native System for LLM Inference Serving
por: Xu, Minxian, et al.
Publicado: (2025)

Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
por: Wu, Jingfeng, et al.
Publicado: (2025)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
por: He, Yiyuan, et al.
Publicado: (2025)

TD3-Sched: Learning to Orchestrate Container-based Cloud-Edge Resources via Distributed Reinforcement Learning
por: Song, Shengye, et al.
Publicado: (2025)

DeServe: Towards Affordable Offline LLM Inference via Decentralization
por: Wu, Linyu, et al.
Publicado: (2025)

SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
por: Xie, Jincheng, et al.
Publicado: (2026)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
por: Liu, Dong, et al.
Publicado: (2025)

LLM Inference Serving: Survey of Recent Advances and Opportunities
por: Li, Baolin, et al.
Publicado: (2024)

KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving
por: Cheng, Rongxin, et al.
Publicado: (2024)

EPD-Serve: A Flexible Multimodal EPD Disaggregation Inference Serving System On Ascend
por: Bai, Fan, et al.
Publicado: (2026)

VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
por: Kamahori, Keisuke, et al.
Publicado: (2026)

StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving
por: Kumar, Satyam, et al.
Publicado: (2026)

FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
por: Ye, Zihao, et al.
Publicado: (2025)

KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
por: Yuan, Yichao, et al.
Publicado: (2026)

D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
por: Wang, Haodong, et al.
Publicado: (2025)

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
por: Lin, Yanying, et al.
Publicado: (2025)

Niyama : Breaking the Silos of LLM Inference Serving
por: Goel, Kanishk, et al.
Publicado: (2025)

On Evaluating Performance of LLM Inference Serving Systems
por: Agrawal, Amey, et al.
Publicado: (2025)

LAPS: A Length-Aware-Prefill LLM Serving System
por: She, Jianshu, et al.
Publicado: (2026)

C-Koordinator: Interference-aware Management for Large-scale and Co-located Microservice Clusters
por: Song, Shengye, et al.
Publicado: (2025)

LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale
por: Cho, Jaehong, et al.
Publicado: (2024)

LSRAM: A Lightweight Autoscaling and SLO Resource Allocation Framework for Microservices Based on Gradient Descent
por: Hu, Kan, et al.
Publicado: (2024)

GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving
por: Jayakody, Shakya, et al.
Publicado: (2026)

SkyServe: Serving AI Models across Regions and Clouds with Spot Instances
por: Mao, Ziming, et al.
Publicado: (2024)

FairBatching: Fairness-Aware Batch Formation for LLM Inference
por: Lyu, Hongtao, et al.
Publicado: (2025)

Scepsy: Serving Agentic Workflows Using Aggregate LLM Pipelines
por: Wagenländer, Marcel, et al.
Publicado: (2026)

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
por: Hankendi, Can, et al.
Publicado: (2026)

ProMoE: Fast MoE-based LLM Serving using Proactive Caching
por: Song, Xiaoniu, et al.
Publicado: (2024)

Taming the Titans: A Survey of Efficient LLM Inference Serving
por: Zhen, Ranran, et al.
Publicado: (2025)

ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
por: Qiu, Haoran, et al.
Publicado: (2025)

TCM-Serve: Modality-aware Scheduling for Multimodal Large Language Model Inference
por: Papaioannou, Konstantinos, et al.
Publicado: (2026)

Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
por: Da, Wei, et al.
Publicado: (2025)

ENOVA: Autoscaling towards Cost-effective and Stable Serverless LLM Serving
por: Huang, Tao, et al.
Publicado: (2024)

Efficient Serving of LLM Applications with Probabilistic Demand Modeling
por: Liu, Yifei, et al.
Publicado: (2025)

Regulating Branch Parallelism in LLM Serving
por: Gandhi, Swapnil, et al.
Publicado: (2026)

CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
por: Suo, Jiashun, et al.
Publicado: (2025)

Efficient Multi-round LLM Inference over Disaggregated Serving
por: He, Wenhao, et al.
Publicado: (2026)