Copertina

HarmonyBatch: Batching multi-SLO DNN Inference with Heterogeneous Serverless Functions

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chen, Jiabin, Xu, Fei, Gu, Yikun, Chen, Li, Liu, Fangming, Zhou, Zhi
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Tangram: High-resolution Video Analytics on Serverless Platform with SLO-aware Batching
di: Peng, Haosong, et al.
Pubblicazione: (2024)

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs
di: Chen, Aodong, et al.
Pubblicazione: (2023)

Scalable and Cost-Efficient ML Inference: Parallel Batch Processing with Serverless Functions
di: Barrak, Amine, et al.
Pubblicazione: (2025)

HAS-GPU: Efficient Hybrid Auto-scaling with Fine-grained GPU Allocation for SLO-aware Serverless Inferences
di: Gu, Jianfeng, et al.
Pubblicazione: (2025)

FairBatching: Fairness-Aware Batch Formation for LLM Inference
di: Lyu, Hongtao, et al.
Pubblicazione: (2025)

Joint Optimization of Offloading, Batching and DVFS for Multiuser Co-Inference
di: Xu, Yaodan, et al.
Pubblicazione: (2025)

AntBatchInfer: Elastic Batch Inference in the Kubernetes Cluster
di: Li, Siyuan, et al.
Pubblicazione: (2024)

Online Optimization of DNN Inference Network Utility in Collaborative Edge Computing
di: Li, Rui, et al.
Pubblicazione: (2024)

Symphony: Optimized DNN Model Serving using Deferred Batch Scheduling
di: Chen, Lequn, et al.
Pubblicazione: (2023)

SLO-Aware Scheduling for Large Language Model Inferences
di: Huang, Jinqi, et al.
Pubblicazione: (2025)

SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines
di: Cheng, Ke, et al.
Pubblicazione: (2024)

Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learning
di: Chen, Hongyao, et al.
Pubblicazione: (2025)

GPU-Accelerated Batch-Dynamic Subgraph Matching
di: Qiu, Linshan, et al.
Pubblicazione: (2024)

Towards Resource-Efficient Serverless LLM Inference with SLINFER
di: Xu, Chuhao, et al.
Pubblicazione: (2025)

CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control
di: Chen, Qiaoling, et al.
Pubblicazione: (2026)

Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching
di: Pang, Bowen, et al.
Pubblicazione: (2025)

It Takes Two to Tango: Serverless Workflow Serving via Bilaterally Engaged Resource Adaptation
di: Wu, Jing, et al.
Pubblicazione: (2025)

Enabling Efficient Batch Serving for LMaaS via Generation Length Prediction
di: Cheng, Ke, et al.
Pubblicazione: (2024)

The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution
di: Luan, Frank Sifei, et al.
Pubblicazione: (2025)

ESG: Pipeline-Conscious Efficient Scheduling of DNN Workflows on Serverless Platforms with Shareable GPUs
di: Hui, Xinning, et al.
Pubblicazione: (2024)

Batch Denoising for AIGC Service Provisioning in Wireless Edge Networks
di: Xu, Jinghang, et al.
Pubblicazione: (2025)

Memory Offloading for Large Language Model Inference with Latency SLO Guarantees
di: Ma, Chenxiang, et al.
Pubblicazione: (2025)

CASA: A Framework for SLO and Carbon-Aware Autoscaling and Scheduling in Serverless Cloud Computing
di: Qi, S., et al.
Pubblicazione: (2024)

BSODiag: A Global Diagnosis Framework for Batch Servers Outage in Large-scale Cloud Infrastructure Systems
di: Duan, Tao, et al.
Pubblicazione: (2025)

MaaSO: SLO-aware Orchestration of Heterogeneous Model Instances for MaaS
di: Xuan, Mo, et al.
Pubblicazione: (2025)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
di: Hu, Jianmin, et al.
Pubblicazione: (2025)

BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
di: Zheng, Zhen, et al.
Pubblicazione: (2024)

Taming Request Imbalance: SLO-Aware Scheduling for Disaggregated LLM Inference
di: Wang, Qipeng
Pubblicazione: (2026)

SLICE: SLO-Driven Scheduling for LLM Inference on Edge Computing Devices
di: Chow, Will
Pubblicazione: (2025)

Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking
di: Mo, Zizhao, et al.
Pubblicazione: (2026)

Torpor: GPU-Enabled Serverless Computing for Low-Latency, Resource-Efficient Inference
di: Yu, Minchen, et al.
Pubblicazione: (2023)

A House United Within Itself: SLO-Awareness for On-Premises Containerized ML Inference Clusters via Faro
di: Jeon, Beomyeol, et al.
Pubblicazione: (2024)

Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance
di: Piao, XinYu, et al.
Pubblicazione: (2021)

Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
di: Recasens, Pol G., et al.
Pubblicazione: (2025)

ICPS: Real-Time Resource Configuration for Cloud Serverless Functions Considering Affinity
di: Chen, Long, et al.
Pubblicazione: (2025)

Hummingbird: SLO-Oriented GPU Preemption at Microsecond-scale
di: Hu, Tiancheng, et al.
Pubblicazione: (2026)

Are Your Epochs Too Epic? Batch Free Can Be Harmful
di: Kim, Daewoo, et al.
Pubblicazione: (2024)

Batched DGEMMs for scientific codes running on long vector architectures
di: Banchelli, Fabio, et al.
Pubblicazione: (2025)

SMDP-Based Dynamic Batching for Improving Responsiveness and Energy Efficiency of Batch Services
di: Xu, Yaodan, et al.
Pubblicazione: (2025)

COPUS: Co-adaptive Parallelism and Batch Size Selection in Large Language Model Training
di: Sakip, Akhmed, et al.
Pubblicazione: (2026)