Image de couverture de livre

Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Shi, Xiaoxiang, Cai, Colin, Du, Junjia, Jia, Zhihao
Format:	Preprint
Publié:	2025
Sujets:	Distributed, Parallel, and Cluster Computing Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
par: Woo, Sunghyeon, et autres
Publié: (2026)

RAPID-Serve: Resource-efficient and Accelerated P/D Intra-GPU Disaggregation
par: Masood, Amna, et autres
Publié: (2026)

Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
par: Wang, Chao, et autres
Publié: (2025)

SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
par: Zhang, Hengrui, et autres
Publié: (2025)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
par: Zhong, Yinmin, et autres
Publié: (2024)

SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference
par: Li, Luchang, et autres
Publié: (2026)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
par: Chen, Xing, et autres
Publié: (2025)

Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
par: Liu, Yunzhao, et autres
Publié: (2025)

EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
par: Du, Jiangsu, et autres
Publié: (2025)

Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation
par: Liang, Yunkai, et autres
Publié: (2025)

WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
par: Lou, Chiheng, et autres
Publié: (2025)

ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
par: Qiao, Yifan, et autres
Publié: (2024)

Revati: Transparent GPU-Free Time-Warp Emulation for LLM Serving
par: Agrawal, Amey, et autres
Publié: (2026)

Autellix: An Efficient Serving Engine for LLM Agents as General Programs
par: Luo, Michael, et autres
Publié: (2025)

From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
par: Lee, Gunjun, et autres
Publié: (2025)

BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architectures
par: Hu, Xiannan, et autres
Publié: (2025)

MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
par: Zhu, Ruidong, et autres
Publié: (2025)

P/D-Serve: Serving Disaggregated Large Language Model at Scale
par: Jin, Yibo, et autres
Publié: (2024)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
par: Ruan, Chaoyi, et autres
Publié: (2025)

Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels
par: Song, Mingcong, et autres
Publié: (2024)

AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
par: Li, Zikun, et autres
Publié: (2025)

Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
par: Chen, Shaoyuan, et autres
Publié: (2024)

SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving
par: Guo, Yipin, et autres
Publié: (2026)

Enabling Disaggregated Multi-Stage MLLM Inference via GPU-Internal Scheduling and Resource Sharing
par: Zhao, Lingxiao, et autres
Publié: (2025)

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
par: Oliaro, Gabriele, et autres
Publié: (2024)

SIMPLE: Disaggregating Sampling from GPU Inference into a Decision Plane for Faster Distributed LLM Serving
par: Zhao, Bohan, et autres
Publié: (2025)

How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
par: Wu, Hanjiang, et autres
Publié: (2026)

semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storage
par: Hong, Ke, et autres
Publié: (2025)

ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache
par: Wang, Shao, et autres
Publié: (2026)

SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
par: Miao, Xupeng, et autres
Publié: (2023)

KVDirect: Distributed Disaggregated LLM Inference
par: Chen, Shiyang, et autres
Publié: (2024)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
par: Griggs, Tyler, et autres
Publié: (2024)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
par: Hu, Cunchen, et autres
Publié: (2024)

Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving
par: Ding, Jianru, et autres
Publié: (2026)

DUET: Disaggregated Hybrid Mamba-Transformer LLMs with Prefill and Decode-Specific Packages
par: Kanani, Alish, et autres
Publié: (2026)

SLOs-Serve: Optimized Serving of Multi-SLO LLMs
par: Chen, Siyuan, et autres
Publié: (2025)

CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters
par: Huang, Shaoyuan, et autres
Publié: (2026)

Efficient Multi-round LLM Inference over Disaggregated Serving
par: He, Wenhao, et autres
Publié: (2026)

HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
par: Zhang, Zeyu, et autres
Publié: (2025)

LAPS: A Length-Aware-Prefill LLM Serving System
par: She, Jianshu, et autres
Publié: (2026)