Image de couverture de livre

Serving Compound Inference Systems on Datacenter GPUs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Devata, Sriram, Singh, Rahul, Adve, Sarita
Format:	Preprint
Publié:	2026
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
par: Chen, Xing, et autres
Publié: (2025)

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
par: Jiang, Youhe, et autres
Publié: (2025)

ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RL
par: Gao, Wei, et autres
Publié: (2026)

Cloud Native System for LLM Inference Serving
par: Xu, Minxian, et autres
Publié: (2025)

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
par: He, Xuan, et autres
Publié: (2025)

EcoServe: Designing Carbon-Aware AI Inference Systems
par: Li, Yueying, et autres
Publié: (2025)

Datacenter Energy Optimized Power Profiles
par: Narayanaswamy, Sreedhar, et autres
Publié: (2025)

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization
par: Jiang, Youhe, et autres
Publié: (2026)

Loki: A System for Serving ML Inference Pipelines with Hardware and Accuracy Scaling
par: Ahmad, Sohaib, et autres
Publié: (2024)

A Tale of Two Scales: Reconciling Horizontal and Vertical Scaling for Inference Serving Systems
par: Razavi, Kamran, et autres
Publié: (2024)

DARIS: An Oversubscribed Spatio-Temporal Scheduler for Real-Time DNN Inference on GPUs
par: Babaei, Amir Fakhim, et autres
Publié: (2025)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
par: Du, Boxiao, et autres
Publié: (2026)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
par: Li, Suyi, et autres
Publié: (2024)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
par: Hu, Jianmin, et autres
Publié: (2025)

PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks
par: Zhan, Huiyou, et autres
Publié: (2025)

APEX: Asynchronous Parallel CPU-GPU Execution for Online LLM Inference on Constrained GPUs
par: Fan, Jiakun, et autres
Publié: (2025)

TridentServe: A Stage-level Serving System for Diffusion Pipelines
par: Xia, Yifei, et autres
Publié: (2025)

Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems
par: Wilkins, Grant, et autres
Publié: (2024)

Efficient Multi-round LLM Inference over Disaggregated Serving
par: He, Wenhao, et autres
Publié: (2026)

UELLM: A Unified and Efficient Approach for LLM Inference Serving
par: He, Yiyuan, et autres
Publié: (2024)

OCTOPINF: Workload-Aware Inference Serving for Edge Video Analytics
par: Nguyen, Thanh-Tung, et autres
Publié: (2025)

CrossPipe: Towards Optimal Pipeline Schedules for Cross-Datacenter Training
par: Chen, Tiancheng, et autres
Publié: (2025)

Tetris: Efficient Intra-Datacenter Calls Packing for Large Conferencing Services
par: Gandhi, Rohan, et autres
Publié: (2025)

Capsule: Efficient Player Isolation for Datacenters
par: Du, Zhouheng, et autres
Publié: (2025)

CALVO: Improve Serving Efficiency for LLM Inferences with Intense Network Demands
par: Wang, Weiye, et autres
Publié: (2026)

Sponge: Inference Serving with Dynamic SLOs Using In-Place Vertical Scaling
par: Razavi, Kamran, et autres
Publié: (2024)

The Ghost in the Datacenter: Link Flapping, Topology Knowledge Failures, and the FITO Category Mistake
par: Borrill, Paul
Publié: (2026)

PowerTrip: Exploiting Federated Heterogeneous Datacenter Power for Distributed ML Training
par: Mehboob, Talha, et autres
Publié: (2025)

SneakPeek: Data-Aware Model Selection and Scheduling for Inference Serving on the Edge
par: Wolfrath, Joel, et autres
Publié: (2025)

OpenDC-STEAM: Realistic Modeling and Systematic Exploration of Composable Techniques for Sustainable Datacenters
par: Niewenhuis, Dante, et autres
Publié: (2026)

Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter
par: Qin, Ruoyu, et autres
Publié: (2026)

DCGen 1.1 Technical Report: Generating Datacenter Configurations (including IT, Power, Cooling)
par: Gnibga, Wedan Emmanuel, et autres
Publié: (2026)

OpenDT: Exploring Datacenter Performance and Sustainability with a Self-Calibrating Digital Twin
par: Nicolae, Radu, et autres
Publié: (2026)

EPD-Serve: A Flexible Multimodal EPD Disaggregation Inference Serving System On Ascend
par: Bai, Fan, et autres
Publié: (2026)

An Adaptive Distributed Stencil Abstraction for GPUs
par: Bhosale, Aditya, et autres
Publié: (2025)

Accelerating Maximal Biclique Enumeration on GPUs
par: Hsieh, Chou-Ying, et autres
Publié: (2024)

Parallelizing Maximal Clique Enumeration on GPUs
par: Almasri, Mohammad, et autres
Publié: (2022)

Optimizing sDTW for AMD GPUs
par: Latta-Lin, Daniel, et autres
Publié: (2024)

DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
par: Liao, Junhan, et autres
Publié: (2025)

AgentServe: Algorithm-System Co-Design for Efficient Agentic AI Serving on a Consumer-Grade GPU
par: Zhang, Yuning, et autres
Publié: (2026)