Cover Image

DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liao, Junhan, Xu, Minxian, Zheng, Wanyi, Wang, Yan, Ye, Kejiang, Buyya, Rajkumar, Xu, Chengzhong
Format:	Preprint
Published:	2025
Subjects:	Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Cloud Native System for LLM Inference Serving
by: Xu, Minxian, et al.
Published: (2025)

UELLM: A Unified and Efficient Approach for LLM Inference Serving
by: He, Yiyuan, et al.
Published: (2024)

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
by: Zheng, Wanyi, et al.
Published: (2025)

Auto-scaling Approaches for Microservice Applications: A Survey and Taxonomy
by: Xu, Minxian, et al.
Published: (2025)

DRPC: Distributed Reinforcement Learning Approach for Scalable Resource Provisioning in Container-based Clusters
by: Bai, Haoyu, et al.
Published: (2024)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
by: Hu, Jianmin, et al.
Published: (2025)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
by: He, Yiyuan, et al.
Published: (2025)

Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
by: Wu, Jingfeng, et al.
Published: (2025)

Collaborative Resource Management and Workloads Scheduling in Cloud-Assisted Mobile Edge Computing across Timescales
by: Tang, Lujie, et al.
Published: (2024)

LSRAM: A Lightweight Autoscaling and SLO Resource Allocation Framework for Microservices Based on Gradient Descent
by: Hu, Kan, et al.
Published: (2024)

Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
by: Wang, Chao, et al.
Published: (2025)

CloudNativeSim: a toolkit for modeling and simulation of cloud-native applications
by: Wu, Jingfeng, et al.
Published: (2024)

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
by: Lin, Yanying, et al.
Published: (2025)

BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architectures
by: Hu, Xiannan, et al.
Published: (2025)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
by: Zhong, Yinmin, et al.
Published: (2024)

Multi-Layer Scheduling for MoE-Based LLM Reasoning
by: Sun, Yifan, et al.
Published: (2026)

ORACL: Optimized Reasoning for Autoscaling via Chain of Thought with LLMs for Microservices
by: Bai, Haoyu, et al.
Published: (2026)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
by: Du, Boxiao, et al.
Published: (2026)

Efficient Multi-round LLM Inference over Disaggregated Serving
by: He, Wenhao, et al.
Published: (2026)

An Interference-aware Approach for Co-located Container Orchestration with Novel Metric
by: Li, Xiang, et al.
Published: (2024)

MSARS: A Meta-Learning and Reinforcement Learning Framework for SLO Resource Allocation and Adaptive Scaling for Microservices
by: Hu, Kan, et al.
Published: (2024)

SealOS+: A Sealos-based Approach for Adaptive Resource Optimization Under Dynamic Workloads for Securities Trading System
by: Jia, Haojie, et al.
Published: (2025)

C-Koordinator: Interference-aware Management for Large-scale and Co-located Microservice Clusters
by: Song, Shengye, et al.
Published: (2025)

TempoScale: A Cloud Workloads Prediction Approach Integrating Short-Term and Long-Term Information
by: Wen, Linfeng, et al.
Published: (2024)

StatuScale: Status-aware and Elastic Scaling Strategy for Microservice Applications
by: Wen, Linfeng, et al.
Published: (2024)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
by: Mo, Zizhao, et al.
Published: (2025)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
by: Ruan, Chaoyi, et al.
Published: (2025)

TD3-Sched: Learning to Orchestrate Container-based Cloud-Edge Resources via Distributed Reinforcement Learning
by: Song, Shengye, et al.
Published: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
by: Hu, Cunchen, et al.
Published: (2024)

Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking
by: Mo, Zizhao, et al.
Published: (2026)

Aging-aware CPU Core Management for Embodied Carbon Amortization in Cloud LLM Inference
by: Hewage, Tharindu B., et al.
Published: (2025)

Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecture
by: Wu, Yu, et al.
Published: (2025)

Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
by: Qin, Ruoyu, et al.
Published: (2024)

A Joint Time and Energy-Efficient Federated Learning-based Computation Offloading Method for Mobile Edge Computing
by: Mukherjee, Anwesha, et al.
Published: (2024)

EnFed: An Energy-aware Federated Learning in Resource Constrained Environments for Human Activity Recognition
by: Mukherjee, Anwesha, et al.
Published: (2024)

Generative Federated Learning for Smart Prediction and Recommendation Applications
by: Mukherjee, Anwesha, et al.
Published: (2025)

TrustMesh: A Blockchain-Enabled Trusted Distributed Computing Framework for Open Heterogeneous IoT Environments
by: Rangwala, Murtaza, et al.
Published: (2024)

TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference
by: Zhang, Hongbin, et al.
Published: (2025)

Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads
by: Hu, Cunchen, et al.
Published: (2024)

Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda
by: Xu, Minxian, et al.
Published: (2026)