Image de couverture de livre

Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wu, Jingfeng, He, Yiyuan, Xu, Minxian, Gao, Xitong, Ye, Kejiang, Xu, Chengzhong
Format:	Preprint
Publié:	2025
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Cloud Native System for LLM Inference Serving
par: Xu, Minxian, et autres
Publié: (2025)

UELLM: A Unified and Efficient Approach for LLM Inference Serving
par: He, Yiyuan, et autres
Publié: (2024)

CloudNativeSim: a toolkit for modeling and simulation of cloud-native applications
par: Wu, Jingfeng, et autres
Publié: (2024)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
par: He, Yiyuan, et autres
Publié: (2025)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
par: Hu, Jianmin, et autres
Publié: (2025)

DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
par: Liao, Junhan, et autres
Publié: (2025)

Collaborative Resource Management and Workloads Scheduling in Cloud-Assisted Mobile Edge Computing across Timescales
par: Tang, Lujie, et autres
Publié: (2024)

LSRAM: A Lightweight Autoscaling and SLO Resource Allocation Framework for Microservices Based on Gradient Descent
par: Hu, Kan, et autres
Publié: (2024)

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
par: Zheng, Wanyi, et autres
Publié: (2025)

DRPC: Distributed Reinforcement Learning Approach for Scalable Resource Provisioning in Container-based Clusters
par: Bai, Haoyu, et autres
Publié: (2024)

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
par: Lin, Yanying, et autres
Publié: (2025)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
par: Mo, Zizhao, et autres
Publié: (2025)

MSARS: A Meta-Learning and Reinforcement Learning Framework for SLO Resource Allocation and Adaptive Scaling for Microservices
par: Hu, Kan, et autres
Publié: (2024)

StatuScale: Status-aware and Elastic Scaling Strategy for Microservice Applications
par: Wen, Linfeng, et autres
Publié: (2024)

Auto-scaling Approaches for Microservice Applications: A Survey and Taxonomy
par: Xu, Minxian, et autres
Publié: (2025)

TempoScale: A Cloud Workloads Prediction Approach Integrating Short-Term and Long-Term Information
par: Wen, Linfeng, et autres
Publié: (2024)

C-Koordinator: Interference-aware Management for Large-scale and Co-located Microservice Clusters
par: Song, Shengye, et autres
Publié: (2025)

An Interference-aware Approach for Co-located Container Orchestration with Novel Metric
par: Li, Xiang, et autres
Publié: (2024)

SealOS+: A Sealos-based Approach for Adaptive Resource Optimization Under Dynamic Workloads for Securities Trading System
par: Jia, Haojie, et autres
Publié: (2025)

TD3-Sched: Learning to Orchestrate Container-based Cloud-Edge Resources via Distributed Reinforcement Learning
par: Song, Shengye, et autres
Publié: (2025)

Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking
par: Mo, Zizhao, et autres
Publié: (2026)

Optimizing Long-context LLM Serving via Fine-grained Sequence Parallelism
par: Li, Cong, et autres
Publié: (2025)

SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
par: Zhou, Qihui, et autres
Publié: (2025)

Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda
par: Xu, Minxian, et autres
Publié: (2026)

BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models
par: Hu, Bodun, et autres
Publié: (2024)

FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving
par: Chen, Wenyan, et autres
Publié: (2026)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
par: Ruan, Chaoyi, et autres
Publié: (2025)

Multi-Layer Scheduling for MoE-Based LLM Reasoning
par: Sun, Yifan, et autres
Publié: (2026)

ORACL: Optimized Reasoning for Autoscaling via Chain of Thought with LLMs for Microservices
par: Bai, Haoyu, et autres
Publié: (2026)

DeepServe: Serverless Large Language Model Serving at Scale
par: Hu, Junhao, et autres
Publié: (2025)

TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
par: Bian, Zhuohang, et autres
Publié: (2026)

Bridging Memory Gaps: Scaling Federated Learning for Heterogeneous Clients
par: Wu, Yebo, et autres
Publié: (2024)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
par: Peng, You, et autres
Publié: (2026)

Efficient Multi-round LLM Inference over Disaggregated Serving
par: He, Wenhao, et autres
Publié: (2026)

Fine-grained MoE Load Balancing with Linear Programming
par: Zhao, Chenqi, et autres
Publié: (2025)

SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
par: Jaiswal, Shashwat, et autres
Publié: (2025)

OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration
par: Jiang, Youhe, et autres
Publié: (2026)

Fast State Restoration in LLM Serving with HCache
par: Gao, Shiwei, et autres
Publié: (2024)

ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
par: Qiao, Yifan, et autres
Publié: (2024)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
par: Hu, Cunchen, et autres
Publié: (2024)