Copertina

Scaling Up Throughput-oriented LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Phung, Thanh Son, Thain, Douglas
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Efficiently Executing High-throughput Lightweight LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management
di: Phung, Thanh Son, et al.
Pubblicazione: (2025)

Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters
di: Guo, Runsheng Benson, et al.
Pubblicazione: (2025)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
di: Zhang, WenZheng, et al.
Pubblicazione: (2024)

High-Throughput LLM inference on Heterogeneous Clusters
di: Xiong, Yi, et al.
Pubblicazione: (2025)

Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
di: Liu, Yunzhao, et al.
Pubblicazione: (2025)

Frenzy: A Memory-Aware Serverless LLM Training System for Heterogeneous GPU Clusters
di: Chang, Zihan, et al.
Pubblicazione: (2024)

Optimal Resource Efficiency with Fairness in Heterogeneous GPU Clusters
di: Mo, Zizhao, et al.
Pubblicazione: (2024)

Predictable LLM Serving on GPU Clusters
di: Darzi, Erfan, et al.
Pubblicazione: (2025)

HARP: Orchestrating Automated Parallel Training on Heterogeneous GPU Clusters
di: Liang, Antian, et al.
Pubblicazione: (2025)

Cephalo: Harnessing Heterogeneous GPU Clusters for Training Transformer Models
di: Guo, Runsheng Benson, et al.
Pubblicazione: (2024)

AQUA: Network-Accelerated Memory Offloading for LLMs in Scale-Up GPU Domains
di: Kumar, Abhishek Vijaya, et al.
Pubblicazione: (2024)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
di: Mo, Zizhao, et al.
Pubblicazione: (2025)

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
di: Zhang, Shiwei, et al.
Pubblicazione: (2024)

ParvaGPU: Efficient Spatial GPU Sharing for Large-Scale DNN Inference in Cloud Environments
di: Lee, Munkyu, et al.
Pubblicazione: (2024)

Towards Fast Setup and High Throughput of GPU Serverless Computing
di: Zhao, Han, et al.
Pubblicazione: (2024)

Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
di: Hidayetoglu, Mert, et al.
Pubblicazione: (2025)

The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
di: Chen, Huamin, et al.
Pubblicazione: (2026)

FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
di: He, Jiaao, et al.
Pubblicazione: (2024)

PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers
di: Zhang, Hongbin, et al.
Pubblicazione: (2026)

Federated Inference for Heterogeneous LLM Communication and Collaboration
di: Chen, Zihan, et al.
Pubblicazione: (2026)

TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference
di: Zhang, Hongbin, et al.
Pubblicazione: (2025)

Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching
di: Pang, Bowen, et al.
Pubblicazione: (2025)

PRISM: Dynamic Primitive-Based Forecasting for Large-Scale GPU Cluster Workloads
di: Wu, Xin, et al.
Pubblicazione: (2026)

PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism
di: Kong, Z. Jonny, et al.
Pubblicazione: (2025)

CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control
di: Chen, Qiaoling, et al.
Pubblicazione: (2026)

Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
di: Argerich, Mauricio Fadel, et al.
Pubblicazione: (2026)

KIS-S: A GPU-Aware Kubernetes Inference Simulator with RL-Based Auto-Scaling
di: Zhang, Guilin, et al.
Pubblicazione: (2025)

APEX: Asynchronous Parallel CPU-GPU Execution for Online LLM Inference on Constrained GPUs
di: Fan, Jiakun, et al.
Pubblicazione: (2025)

DAK: Direct-Access-Enabled GPU Memory Offloading with Optimal Efficiency for LLM Inference
di: Lin, Shouxu, et al.
Pubblicazione: (2026)

Toward Sustainability-Aware LLM Inference on Edge Clusters
di: Rajashekar, Kolichala, et al.
Pubblicazione: (2025)

GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management
di: Duan, Jiaang, et al.
Pubblicazione: (2025)

Bandwidth-Aware LLM Inference on Heterogeneous Many-Core Supercomputers
di: Lu, Yao, et al.
Pubblicazione: (2026)

H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
di: Tang, Ding, et al.
Pubblicazione: (2025)

SiPipe: Bridging the CPU-GPU Utilization Gap for Efficient Pipeline-Parallel LLM Inference
di: He, Yongchao, et al.
Pubblicazione: (2025)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
di: Zhang, Chen, et al.
Pubblicazione: (2025)

Beyond Microservices: Testing Web-Scale RCA Methods on GPU-Driven LLM Workloads
di: Scheinert, Dominik, et al.
Pubblicazione: (2026)

FCPO: Federated Continual Policy Optimization for Real-Time High-Throughput Edge Video Analytics
di: Liebe, Lucas, et al.
Pubblicazione: (2025)

SIMPLE: Disaggregating Sampling from GPU Inference into a Decision Plane for Faster Distributed LLM Serving
di: Zhao, Bohan, et al.
Pubblicazione: (2025)

FaaSTube: Optimizing GPU-oriented Data Transfer for Serverless Computing
di: Wu, Hao, et al.
Pubblicazione: (2024)

SYMPHONY: Improving Memory Management for LLM Inference Workloads
di: Agarwal, Saurabh, et al.
Pubblicazione: (2024)