Copertina

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Griggs, Tyler, Liu, Xiaoxuan, Yu, Jiaxiang, Kim, Doyoung, Chiang, Wei-Lin, Cheung, Alvin, Stoica, Ion
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Distributed, Parallel, and Cluster Computing Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

SkyServe: Serving AI Models across Regions and Clouds with Spot Instances
di: Mao, Ziming, et al.
Pubblicazione: (2024)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
di: Zhang, Chen, et al.
Pubblicazione: (2025)

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
di: Duan, Jiangfei, et al.
Pubblicazione: (2024)

Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
di: Yu, Shan, et al.
Pubblicazione: (2025)

ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
di: Qiao, Yifan, et al.
Pubblicazione: (2024)

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
di: Qianli, Liu, et al.
Pubblicazione: (2025)

PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism
di: Kong, Z. Jonny, et al.
Pubblicazione: (2025)

Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony
di: Wang, Shaoyu, et al.
Pubblicazione: (2025)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
di: Mo, Zizhao, et al.
Pubblicazione: (2025)

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications
di: Du, Kuntai, et al.
Pubblicazione: (2025)

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

Towards Efficient and Practical GPU Multitasking in the Era of LLM
di: Xing, Jiarong, et al.
Pubblicazione: (2025)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
di: Cao, Shiyi, et al.
Pubblicazione: (2024)

AgentServe: Algorithm-System Co-Design for Efficient Agentic AI Serving on a Consumer-Grade GPU
di: Zhang, Yuning, et al.
Pubblicazione: (2026)

Predictable LLM Serving on GPU Clusters
di: Darzi, Erfan, et al.
Pubblicazione: (2025)

DeepServe: Serverless Large Language Model Serving at Scale
di: Hu, Junhao, et al.
Pubblicazione: (2025)

Cascadia: An Efficient Cascade Serving System for Large Language Models
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

Boosting LLM Serving through Spatial-Temporal GPU Resource Sharing
di: Lin, Zejia, et al.
Pubblicazione: (2025)

GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads
di: Ye, Fanjiang, et al.
Pubblicazione: (2026)

Delta Fair Sharing: Performance Isolation for Multi-Tenant Storage Systems
di: Griggs, Tyler, et al.
Pubblicazione: (2026)

Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
di: Liu, Xueshen, et al.
Pubblicazione: (2026)

MoLink: Distributed and Efficient Serving Framework for Large Models
di: Jin, Lewei, et al.
Pubblicazione: (2025)

EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
di: Du, Jiangsu, et al.
Pubblicazione: (2025)

RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMs
di: Wu, Yongji, et al.
Pubblicazione: (2025)

Harpagon: Minimizing DNN Serving Cost via Efficient Dispatching, Scheduling and Splitting
di: Zhao, Zhixin, et al.
Pubblicazione: (2024)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
di: Zhang, WenZheng, et al.
Pubblicazione: (2024)

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
di: Zhang, Shiwei, et al.
Pubblicazione: (2024)

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
di: He, Xuan, et al.
Pubblicazione: (2025)

ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
di: Xiang, Yuxing, et al.
Pubblicazione: (2025)

FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
di: He, Jiaao, et al.
Pubblicazione: (2024)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
di: Kim, Kihyun, et al.
Pubblicazione: (2025)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
di: Zhong, Yinmin, et al.
Pubblicazione: (2024)

TetriServe: Efficient DiT Serving for Heterogeneous Image Generation
di: Lu, Runyu, et al.
Pubblicazione: (2025)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
di: Peng, You, et al.
Pubblicazione: (2026)

BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models
di: Hu, Bodun, et al.
Pubblicazione: (2024)

HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs
di: Wu, Yongji, et al.
Pubblicazione: (2025)

Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
di: Liu, Yunzhao, et al.
Pubblicazione: (2025)

ParvaGPU: Efficient Spatial GPU Sharing for Large-Scale DNN Inference in Cloud Environments
di: Lee, Munkyu, et al.
Pubblicazione: (2024)