Copertina

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Jiang, Youhe, Fu, Fangcheng, Yao, Xiaozhe, He, Guoliang, Miao, Xupeng, Klimovic, Ana, Cui, Bin, Yuan, Binhang, Yoneki, Eiko
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

Efficient Multi-round LLM Inference over Disaggregated Serving
di: He, Wenhao, et al.
Pubblicazione: (2026)

HexiScale: Facilitating Large Language Model Training over Heterogeneous Hardware
di: Yan, Ran, et al.
Pubblicazione: (2024)

Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
di: He, Guoliang, et al.
Pubblicazione: (2025)

HexGen: Generative Inference of Large Language Model over Heterogeneous Environment
di: Jiang, Youhe, et al.
Pubblicazione: (2023)

Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

Cascadia: An Efficient Cascade Serving System for Large Language Models
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
di: Yao, Xiaozhe, et al.
Pubblicazione: (2023)

TridentServe: A Stage-level Serving System for Diffusion Pipelines
di: Xia, Yifei, et al.
Pubblicazione: (2025)

HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environment
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
di: Peng, You, et al.
Pubblicazione: (2026)

Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs
di: Mei, Yixuan, et al.
Pubblicazione: (2026)

AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
di: Yan, Ran, et al.
Pubblicazione: (2025)

HexiSeq: Accommodating Long Context Training of LLMs over Heterogeneous Hardware
di: Liang, Yan, et al.
Pubblicazione: (2026)

Hetu v2: A General and Scalable Deep Learning System with Hierarchical and Heterogeneous Single Program Multiple Data Annotations
di: Li, Haoyang, et al.
Pubblicazione: (2025)

Improving Automatic Parallel Training via Balanced Memory Workload Optimization
di: Wang, Yujie, et al.
Pubblicazione: (2023)

Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow
di: Mei, Yixuan, et al.
Pubblicazione: (2024)

LMDeploy Accelerates Mixed-Precision LLM Inference with TurboMind
di: Zhang, Li, et al.
Pubblicazione: (2025)

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
di: He, Xuan, et al.
Pubblicazione: (2025)

Parallax: Efficient LLM Inference Service over Decentralized Environment
di: Tong, Chris, et al.
Pubblicazione: (2025)

LobRA: Multi-tenant Fine-tuning over Heterogeneous Data
di: Lin, Sheng, et al.
Pubblicazione: (2025)

DéjàVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving
di: Strati, Foteini, et al.
Pubblicazione: (2024)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
di: Nie, Xiaonan, et al.
Pubblicazione: (2024)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
di: Du, Boxiao, et al.
Pubblicazione: (2026)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
di: Li, Suyi, et al.
Pubblicazione: (2024)

Serving Compound Inference Systems on Datacenter GPUs
di: Devata, Sriram, et al.
Pubblicazione: (2026)

GreenLLM: Disaggregating Large Language Model Serving on Heterogeneous GPUs for Lower Carbon Emissions
di: Shi, Tianyao, et al.
Pubblicazione: (2024)

Spindle: Efficient Distributed Training of Multi-Task Large Models via Wavefront Scheduling
di: Wang, Yujie, et al.
Pubblicazione: (2024)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
di: Zhang, Chen, et al.
Pubblicazione: (2025)

Atlas: Hierarchical Partitioning for Quantum Circuit Simulation on GPUs (Extended Version)
di: Xu, Mingkuan, et al.
Pubblicazione: (2024)

EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
di: Du, Jiangsu, et al.
Pubblicazione: (2025)

ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RL
di: Gao, Wei, et al.
Pubblicazione: (2026)

Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
di: Strati, Foteini, et al.
Pubblicazione: (2025)

Revisiting the Time Cost Model of AllReduce
di: Xiong, Dian, et al.
Pubblicazione: (2024)

CALVO: Improve Serving Efficiency for LLM Inferences with Intense Network Demands
di: Wang, Weiye, et al.
Pubblicazione: (2026)

FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
di: Yan, Ran, et al.
Pubblicazione: (2025)

ByteScale: Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000 GPUs
di: Ge, Hao, et al.
Pubblicazione: (2025)

HetCCL: Accelerating LLM Training with Heterogeneous GPUs
di: Kim, Heehoon, et al.
Pubblicazione: (2026)