Copertina

FlowPrefill: Decoupling Preemption from Prefill Scheduling Granularity to Mitigate Head-of-Line Blocking in LLM Serving

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Hsieh, Chia-chi, Zong, Zan, Chen, Xinyang, Li, Jianjiang, Zhai, Jidong, Wen, Lijie
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
di: Woo, Sunghyeon, et al.
Pubblicazione: (2026)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
di: Zhong, Yinmin, et al.
Pubblicazione: (2024)

PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications
di: Du, Kuntai, et al.
Pubblicazione: (2025)

Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
di: Wang, Chao, et al.
Pubblicazione: (2025)

From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
di: Lee, Gunjun, et al.
Pubblicazione: (2025)

LAPS: A Length-Aware-Prefill LLM Serving System
di: She, Jianshu, et al.
Pubblicazione: (2026)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
di: Chen, Xing, et al.
Pubblicazione: (2025)

Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
di: Shi, Xiaoxiang, et al.
Pubblicazione: (2025)

ContiguousKV: Accelerating LLM Prefill with Granularity-Aligned KV Cache Management
di: Zou, Jing, et al.
Pubblicazione: (2026)

RServe: Overlapping Encoding and Prefill for Efficient LMM Inference
di: Guo, Tianyu, et al.
Pubblicazione: (2025)

Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter
di: Qin, Ruoyu, et al.
Pubblicazione: (2026)

Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
di: Liu, Yunzhao, et al.
Pubblicazione: (2025)

A Flexible Programmable Pipeline Parallelism Framework for Efficient DNN Training
di: Jiang, Lijuan, et al.
Pubblicazione: (2025)

Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels
di: Song, Mingcong, et al.
Pubblicazione: (2024)

DUET: Disaggregated Hybrid Mamba-Transformer LLMs with Prefill and Decode-Specific Packages
di: Kanani, Alish, et al.
Pubblicazione: (2026)

FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
di: He, Jiaao, et al.
Pubblicazione: (2024)

GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management
di: Duan, Jiaang, et al.
Pubblicazione: (2025)

Large-Scale LLM Inference with Heterogeneous Workloads: Prefill-Decode Contention and Asymptotically Optimal Control
di: Lin, Ruihan, et al.
Pubblicazione: (2026)

SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
di: Zhang, Hengrui, et al.
Pubblicazione: (2025)

Preemption Aware Task Scheduling for Priority and Deadline Constrained DNN Inference Task Offloading in Homogeneous Mobile-Edge Networks
di: Cotter, Jamie, et al.
Pubblicazione: (2025)

Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption
di: Yildiz, Mert, et al.
Pubblicazione: (2026)

Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference
di: Zhu, Yue, et al.
Pubblicazione: (2025)

SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference
di: Li, Luchang, et al.
Pubblicazione: (2026)

A Granularity Characterization of Task Scheduling Effectiveness
di: Anvari, Sana Taghipour, et al.
Pubblicazione: (2026)

Hummingbird: SLO-Oriented GPU Preemption at Microsecond-scale
di: Hu, Tiancheng, et al.
Pubblicazione: (2026)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
di: Zhang, Chen, et al.
Pubblicazione: (2025)

Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
di: Da, Wei, et al.
Pubblicazione: (2025)

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
di: Zhu, Jianian, et al.
Pubblicazione: (2025)

Cortex: Workflow-Aware Resource Pooling and Scheduling for Agentic Serving
di: Pagonas, Nikos, et al.
Pubblicazione: (2025)

Past-Future Scheduler for LLM Serving under SLA Guarantees
di: Gong, Ruihao, et al.
Pubblicazione: (2025)

PROSERVE: Unified Multi-Priority Request Scheduling for LLM Serving
di: Huang, Weizhe, et al.
Pubblicazione: (2025)

Improved Methods of Task Assignment and Resource Allocation with Preemption in Edge Computing Systems
di: Rublein, Caroline, et al.
Pubblicazione: (2024)

Multi-stage Flow Scheduling for LLM Serving
di: Sun, Yijun, et al.
Pubblicazione: (2026)

ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
di: Xiang, Yuxing, et al.
Pubblicazione: (2025)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
di: Zheng, Size, et al.
Pubblicazione: (2026)

A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
di: Zhang, Yue, et al.
Pubblicazione: (2025)

Slice-Level Scheduling for High Throughput and Load Balanced LLM Serving
di: Cheng, Ke, et al.
Pubblicazione: (2024)

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
di: He, Xuan, et al.
Pubblicazione: (2025)

HydraInfer: Hybrid Disaggregated Scheduling for Multimodal Large Language Model Serving
di: Dong, Xianzhe, et al.
Pubblicazione: (2025)

SneakPeek: Data-Aware Model Selection and Scheduling for Inference Serving on the Edge
di: Wolfrath, Joel, et al.
Pubblicazione: (2025)