Copertina

Serving Chain-structured Jobs with Large Memory Footprints with Application to Large Foundation Model Serving

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Sun, Tingyang, He, Ting, Hou, I-Hong
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Distributed, Parallel, and Cluster Computing Performance
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architectures
di: Hu, Xiannan, et al.
Pubblicazione: (2025)

BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems
di: Wang, Yuxin, et al.
Pubblicazione: (2024)

CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
di: Suo, Jiashun, et al.
Pubblicazione: (2025)

GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving
di: Jayakody, Shakya, et al.
Pubblicazione: (2026)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
di: Hu, Junhao, et al.
Pubblicazione: (2024)

Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
di: Li, Jiaxi, et al.
Pubblicazione: (2025)

"Two-Stagification": Job Dispatching in Large-Scale Clusters via a Two-Stage Architecture
di: Yildiz, Mert, et al.
Pubblicazione: (2025)

FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
di: He, Jiaao, et al.
Pubblicazione: (2024)

Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
di: Miao, Xupeng, et al.
Pubblicazione: (2023)

DeepServe: Serverless Large Language Model Serving at Scale
di: Hu, Junhao, et al.
Pubblicazione: (2025)

Taming the Memory Footprint Crisis: System Design for Production Diffusion LLM Serving
di: Fan, Jiakun, et al.
Pubblicazione: (2025)

Asymptotically Optimal Scheduling of Multiple Parallelizable Job Classes
di: Berg, Benjamin, et al.
Pubblicazione: (2024)

An Empirical Characterization of Outages and Incidents in Public Services for Large Language Models
di: Chu, Xiaoyu, et al.
Pubblicazione: (2025)

ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
di: Fan, Ruibo, et al.
Pubblicazione: (2026)

LLload: Simplifying Real-Time Job Monitoring for HPC Users
di: Byun, Chansup, et al.
Pubblicazione: (2024)

HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
di: Zhao, Xuanlei, et al.
Pubblicazione: (2024)

Synthesizing Proxy Applications for MPI Programs
di: Luo, Jiyu, et al.
Pubblicazione: (2023)

ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
di: Xiang, Yuxing, et al.
Pubblicazione: (2025)

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
di: Qianli, Liu, et al.
Pubblicazione: (2025)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
di: Zhong, Yinmin, et al.
Pubblicazione: (2024)

CGSim: A Simulation Framework for Large Scale Distributed Computing Environment
di: Vatsavai, Sairam Sri, et al.
Pubblicazione: (2025)

Scaling Large-scale GNN Training to Thousands of Processors on CPU-based Supercomputers
di: Zhuang, Chen, et al.
Pubblicazione: (2024)

FalconFS: Distributed File System for Large-Scale Deep Learning Pipeline
di: Xu, Jingwei, et al.
Pubblicazione: (2025)

Denoising Application Performance Models with Noise-Resilient Priors
di: de Morais, Gustavo, et al.
Pubblicazione: (2025)

Prompt-Aware Scheduling for Low-Latency LLM Serving
di: Tao, Yiheng, et al.
Pubblicazione: (2025)

Architecture Specific Generation of Large Scale Lattice Boltzmann Methods for Sparse Complex Geometries
di: Suffa, Philipp, et al.
Pubblicazione: (2024)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
di: Hu, Cunchen, et al.
Pubblicazione: (2024)

Can Tensor Cores Benefit Memory-Bound Kernels? (No!)
di: Zhang, Lingqi, et al.
Pubblicazione: (2025)

Understanding Power Consumption Metric on Heterogeneous Memory Systems
di: Proaño, Andrès Rubio, et al.
Pubblicazione: (2024)

Parallel I/O Characterization and Optimization on Large-Scale HPC Systems: A 360-Degree Survey
di: Ather, Hammad, et al.
Pubblicazione: (2024)

CUTHERMO: Understanding GPU Memory Inefficiencies with Heat Map Profiling
di: Zhao, Yanbo, et al.
Pubblicazione: (2025)

Cache Blocking of Distributed-Memory Parallel Matrix Power Kernels
di: Lacey, Dane C., et al.
Pubblicazione: (2024)

Analytic Roofline Modeling and Energy Analysis of LULESH Proxy Application on Multi-Core Clusters
di: Afzal, Ayesha, et al.
Pubblicazione: (2024)

Collaborative Processing for Multi-Tenant Inference on Memory-Constrained Edge TPUs
di: Ng, Nathan, et al.
Pubblicazione: (2026)

Dissecting CPU-GPU Unified Physical Memory on AMD MI300A APUs
di: Wahlgren, Jacob, et al.
Pubblicazione: (2025)

Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
di: Yu, Shan, et al.
Pubblicazione: (2025)

Usability Evaluation of Cloud for HPC Applications
di: Sochat, Vanessa, et al.
Pubblicazione: (2025)

InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models
di: Chen, Hongyu, et al.
Pubblicazione: (2026)

Opt4GPTQ: Co-Optimizing Memory and Computation for 4-bit GPTQ Quantized LLM Inference on Heterogeneous Platforms
di: Zhang, Yaozheng, et al.
Pubblicazione: (2025)

Inductive Loop Analysis for Practical HPC Application Optimization
di: Schaad, Philipp, et al.
Pubblicazione: (2025)