Image de couverture de livre

Efficiently Executing High-throughput Lightweight LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Phung, Thanh Son, Thain, Douglas
Format:	Preprint
Publié:	2025
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Scaling Up Throughput-oriented LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management
par: Phung, Thanh Son, et autres
Publié: (2025)

Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
par: Liu, Yunzhao, et autres
Publié: (2025)

Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters
par: Guo, Runsheng Benson, et autres
Publié: (2025)

APEX: Asynchronous Parallel CPU-GPU Execution for Online LLM Inference on Constrained GPUs
par: Fan, Jiakun, et autres
Publié: (2025)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
par: Zhang, WenZheng, et autres
Publié: (2024)

Frenzy: A Memory-Aware Serverless LLM Training System for Heterogeneous GPU Clusters
par: Chang, Zihan, et autres
Publié: (2024)

The Energy Cost of Execution-Idle in GPU Clusters
par: Lei, Yiran, et autres
Publié: (2026)

Optimal Resource Efficiency with Fairness in Heterogeneous GPU Clusters
par: Mo, Zizhao, et autres
Publié: (2024)

Predictable LLM Serving on GPU Clusters
par: Darzi, Erfan, et autres
Publié: (2025)

AGILE: Lightweight and Efficient Asynchronous GPU-SSD Integration
par: Yang, Zhuoping, et autres
Publié: (2025)

HARP: Orchestrating Automated Parallel Training on Heterogeneous GPU Clusters
par: Liang, Antian, et autres
Publié: (2025)

Cephalo: Harnessing Heterogeneous GPU Clusters for Training Transformer Models
par: Guo, Runsheng Benson, et autres
Publié: (2024)

PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism
par: Kong, Z. Jonny, et autres
Publié: (2025)

Augur: Pre-Execution Energy Prediction for Workflow Tasks in Heterogeneous Clusters
par: West, Kathleen, et autres
Publié: (2026)

SiPipe: Bridging the CPU-GPU Utilization Gap for Efficient Pipeline-Parallel LLM Inference
par: He, Yongchao, et autres
Publié: (2025)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
par: Mo, Zizhao, et autres
Publié: (2025)

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
par: Zhang, Shiwei, et autres
Publié: (2024)

Seesaw: High-throughput LLM Inference via Model Re-sharding
par: Su, Qidong, et autres
Publié: (2025)

ParvaGPU: Efficient Spatial GPU Sharing for Large-Scale DNN Inference in Cloud Environments
par: Lee, Munkyu, et autres
Publié: (2024)

The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
par: Chen, Huamin, et autres
Publié: (2026)

LLM-CoOpt: A Co-Design and Optimization Framework for Efficient LLM Inference on Heterogeneous Platforms
par: Kong, Jie, et autres
Publié: (2026)

PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers
par: Zhang, Hongbin, et autres
Publié: (2026)

Federated Inference for Heterogeneous LLM Communication and Collaboration
par: Chen, Zihan, et autres
Publié: (2026)

Zenix: Efficient Execution of Bulky Serverless Applications
par: Guo, Zhiyuan, et autres
Publié: (2022)

VDCores: Resource Decoupled Programming and Execution for Asynchronous GPU
par: He, Zijian, et autres
Publié: (2026)

Efficiently Reproducing Distributed Workflows in Notebook-based Systems
par: Azaz, Talha, et autres
Publié: (2026)

HAS-GPU: Efficient Hybrid Auto-scaling with Fine-grained GPU Allocation for SLO-aware Serverless Inferences
par: Gu, Jianfeng, et autres
Publié: (2025)

HybridGen: Efficient LLM Generative Inference via CPU-GPU Hybrid Computing
par: Lin, Mao, et autres
Publié: (2026)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
par: Du, Boxiao, et autres
Publié: (2026)

Fantasy: Efficient Large-scale Vector Search on GPU Clusters with GPUDirect Async
par: Liu, Yi, et autres
Publié: (2025)

Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
par: Argerich, Mauricio Fadel, et autres
Publié: (2026)

HarMoEny: Efficient Multi-GPU Inference of MoE Models
par: Doucet, Zachary, et autres
Publié: (2025)

Torpor: GPU-Enabled Serverless Computing for Low-Latency, Resource-Efficient Inference
par: Yu, Minchen, et autres
Publié: (2023)

DAK: Direct-Access-Enabled GPU Memory Offloading with Optimal Efficiency for LLM Inference
par: Lin, Shouxu, et autres
Publié: (2026)

Toward Sustainability-Aware LLM Inference on Edge Clusters
par: Rajashekar, Kolichala, et autres
Publié: (2025)

GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management
par: Duan, Jiaang, et autres
Publié: (2025)

Bandwidth-Aware LLM Inference on Heterogeneous Many-Core Supercomputers
par: Lu, Yao, et autres
Publié: (2026)

Cost-Efficient Multimodal LLM Inference via Cross-Tier GPU Heterogeneity
par: Yu, Donglin
Publié: (2026)

An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters
par: Zhang, Mingjun, et autres
Publié: (2025)

High-Throughput LLM inference on Heterogeneous Clusters
par: Xiong, Yi, et autres
Publié: (2025)