Imagen de Portada

SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Li, Luchang, Li, Dongfang, Gong, Bozhao, Zhang, Yu
Formato:	Preprint
Publicado:	2026
Materias:	Distributed, Parallel, and Cluster Computing Information Theory Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
por: Zhang, Hengrui, et al.
Publicado: (2025)

Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
por: Shi, Xiaoxiang, et al.
Publicado: (2025)

PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
por: Woo, Sunghyeon, et al.
Publicado: (2026)

Robust DNN Partitioning and Resource Allocation Under Uncertain Inference Time
por: Nan, Zhaojun, et al.
Publicado: (2025)

Taming Request Imbalance: SLO-Aware Scheduling for Disaggregated LLM Inference
por: Wang, Qipeng
Publicado: (2026)

Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
por: Wang, Chao, et al.
Publicado: (2025)

Harli: SLO-Aware Co-location of LLM Inference and PEFT-based Finetuning on Model-as-a-Service Platforms
por: Xu, Ao, et al.
Publicado: (2025)

SuperInfer: SLO-Aware Rotary Scheduling and Memory Management for LLM Inference on Superchips
por: Yu, Jiahuan, et al.
Publicado: (2026)

SERFLOW: A Cross-Service Cost Optimization Framework for SLO-Aware Dynamic ML Inference
por: Zhang, Zongshun, et al.
Publicado: (2025)

KVDirect: Distributed Disaggregated LLM Inference
por: Chen, Shiyang, et al.
Publicado: (2024)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
por: Chen, Xing, et al.
Publicado: (2025)

Over-the-Air Multi-Sensor Inference with Neural Networks Using Memristor-Based Analog Computing
por: Tegin, Busra, et al.
Publicado: (2025)

HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
por: Zhang, Zeyu, et al.
Publicado: (2025)

AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
por: Li, Zikun, et al.
Publicado: (2025)

Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation
por: Liang, Yunkai, et al.
Publicado: (2025)

Rethinking Resource Management in Edge Learning: A Joint Pre-training and Fine-tuning Design Paradigm
por: Lyu, Zhonghao, et al.
Publicado: (2024)

Autothrottle: A Practical Bi-Level Approach to Resource Management for SLO-Targeted Microservices
por: Wang, Zibo, et al.
Publicado: (2022)

SLO-Aware Scheduling for Large Language Model Inferences
por: Huang, Jinqi, et al.
Publicado: (2025)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
por: Zhong, Yinmin, et al.
Publicado: (2024)

STAR: Decode-Phase Rescheduling for LLM Inference
por: Wang, Zhibin, et al.
Publicado: (2025)

Double-Edge-Assisted Computation Offloading and Resource Allocation for Space-Air-Marine Integrated Networks
por: Wang, Zhen, et al.
Publicado: (2025)

Coded Computing for Resilient Distributed Computing: A Learning-Theoretic Framework
por: Moradi, Parsa, et al.
Publicado: (2024)

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
por: Oliaro, Gabriele, et al.
Publicado: (2024)

Enabling Disaggregated Multi-Stage MLLM Inference via GPU-Internal Scheduling and Resource Sharing
por: Zhao, Lingxiao, et al.
Publicado: (2025)

ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference
por: Oh, Hyungjun, et al.
Publicado: (2024)

Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
por: Chen, Shaoyuan, et al.
Publicado: (2024)

CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters
por: Huang, Shaoyuan, et al.
Publicado: (2026)

Out-of-Air Computation: Enabling Structured Extraction from Wireless Superposition
por: Azimi-Abarghouyi, Seyed Mohammad
Publicado: (2026)

Hierarchical Federated Learning for Networked AI: From Communication Saving to Architecture-Aware Design
por: Azimi-Abarghouyi, Seyed Mohammad, et al.
Publicado: (2026)

Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendation
por: Luo, Liang, et al.
Publicado: (2024)

NeRCC: Nested-Regression Coded Computing for Resilient Distributed Prediction Serving Systems
por: Moradi, Parsa, et al.
Publicado: (2024)

Heterogeneity-Aware Resource Allocation and Topology Design for Hierarchical Federated Edge Learning
por: Gao, Zhidong, et al.
Publicado: (2024)

SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines
por: Cheng, Ke, et al.
Publicado: (2024)

LatencyPrism: Online Non-intrusive Latency Sculpting for SLO-Guaranteed LLM Inference
por: Du, Yin, et al.
Publicado: (2026)

SLICE: SLO-Driven Scheduling for LLM Inference on Edge Computing Devices
por: Chow, Will
Publicado: (2025)

Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
por: Liu, Yunzhao, et al.
Publicado: (2025)

Joint Offloading and Resource Allocation for Hybrid Cloud and Edge Computing in SAGINs: A Decision Assisted Hybrid Action Space Deep Reinforcement Learning Approach
por: Huang, Chong, et al.
Publicado: (2024)

Personalized Federated Learning for Generative AI-Assisted Semantic Communications
por: Peng, Yubo, et al.
Publicado: (2024)

Decentralized Federated Learning Over Imperfect Communication Channels
por: Li, Weicai, et al.
Publicado: (2024)

PolyServe: Efficient Multi-SLO Serving at Scale
por: Zhu, Kan, et al.
Publicado: (2025)