Image de couverture de livre

UELLM: A Unified and Efficient Approach for LLM Inference Serving

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	He, Yiyuan, Xu, Minxian, Wu, Jingfeng, Zheng, Wanyi, Ye, Kejiang, Xu, Chengzhong
Format:	Preprint
Publié:	2024
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Cloud Native System for LLM Inference Serving
par: Xu, Minxian, et autres
Publié: (2025)

Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
par: Wu, Jingfeng, et autres
Publié: (2025)

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
par: Zheng, Wanyi, et autres
Publié: (2025)

CloudNativeSim: a toolkit for modeling and simulation of cloud-native applications
par: Wu, Jingfeng, et autres
Publié: (2024)

DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
par: Liao, Junhan, et autres
Publié: (2025)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
par: He, Yiyuan, et autres
Publié: (2025)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
par: Hu, Jianmin, et autres
Publié: (2025)

LSRAM: A Lightweight Autoscaling and SLO Resource Allocation Framework for Microservices Based on Gradient Descent
par: Hu, Kan, et autres
Publié: (2024)

Collaborative Resource Management and Workloads Scheduling in Cloud-Assisted Mobile Edge Computing across Timescales
par: Tang, Lujie, et autres
Publié: (2024)

DRPC: Distributed Reinforcement Learning Approach for Scalable Resource Provisioning in Container-based Clusters
par: Bai, Haoyu, et autres
Publié: (2024)

Auto-scaling Approaches for Microservice Applications: A Survey and Taxonomy
par: Xu, Minxian, et autres
Publié: (2025)

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
par: Lin, Yanying, et autres
Publié: (2025)

An Interference-aware Approach for Co-located Container Orchestration with Novel Metric
par: Li, Xiang, et autres
Publié: (2024)

TempoScale: A Cloud Workloads Prediction Approach Integrating Short-Term and Long-Term Information
par: Wen, Linfeng, et autres
Publié: (2024)

MSARS: A Meta-Learning and Reinforcement Learning Framework for SLO Resource Allocation and Adaptive Scaling for Microservices
par: Hu, Kan, et autres
Publié: (2024)

SealOS+: A Sealos-based Approach for Adaptive Resource Optimization Under Dynamic Workloads for Securities Trading System
par: Jia, Haojie, et autres
Publié: (2025)

C-Koordinator: Interference-aware Management for Large-scale and Co-located Microservice Clusters
par: Song, Shengye, et autres
Publié: (2025)

StatuScale: Status-aware and Elastic Scaling Strategy for Microservice Applications
par: Wen, Linfeng, et autres
Publié: (2024)

TD3-Sched: Learning to Orchestrate Container-based Cloud-Edge Resources via Distributed Reinforcement Learning
par: Song, Shengye, et autres
Publié: (2025)

Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking
par: Mo, Zizhao, et autres
Publié: (2026)

Efficient Multi-round LLM Inference over Disaggregated Serving
par: He, Wenhao, et autres
Publié: (2026)

Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda
par: Xu, Minxian, et autres
Publié: (2026)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
par: Mo, Zizhao, et autres
Publié: (2025)

Collaborative Speculative Inference for Efficient LLM Inference Serving
par: Gao, Luyao, et autres
Publié: (2025)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
par: Ruan, Chaoyi, et autres
Publié: (2025)

eLLM: Elastic Memory Management Framework for Efficient LLM Serving
par: Xu, Jiale, et autres
Publié: (2025)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
par: Li, Suyi, et autres
Publié: (2024)

Multi-Layer Scheduling for MoE-Based LLM Reasoning
par: Sun, Yifan, et autres
Publié: (2026)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
par: Peng, You, et autres
Publié: (2026)

CALVO: Improve Serving Efficiency for LLM Inferences with Intense Network Demands
par: Wang, Weiye, et autres
Publié: (2026)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
par: Du, Boxiao, et autres
Publié: (2026)

PROSERVE: Unified Multi-Priority Request Scheduling for LLM Serving
par: Huang, Weizhe, et autres
Publié: (2025)

SIMPLE: Disaggregating Sampling from GPU Inference into a Decision Plane for Faster Distributed LLM Serving
par: Zhao, Bohan, et autres
Publié: (2025)

EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
par: Du, Jiangsu, et autres
Publié: (2025)

Communication-Efficient Collaborative LLM Inference over LEO Satellite Networks
par: Zhang, Songge, et autres
Publié: (2026)

SiPipe: Bridging the CPU-GPU Utilization Gap for Efficient Pipeline-Parallel LLM Inference
par: He, Yongchao, et autres
Publié: (2025)

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
par: He, Xuan, et autres
Publié: (2025)

DeServe: Towards Affordable Offline LLM Inference via Decentralization
par: Wu, Linyu, et autres
Publié: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
par: Hu, Cunchen, et autres
Publié: (2024)

SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
par: Xie, Jincheng, et autres
Publié: (2026)