Cover Image

DeepServe: Serverless Large Language Model Serving at Scale

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hu, Junhao, Xu, Jiang, Liu, Zhixia, He, Yulong, Chen, Yuetao, Xu, Hao, Liu, Jiang, Meng, Jie, Zhang, Baoquan, Wan, Shining, Dan, Gengyuan, Dong, Zhiyu, Ren, Zhihao, Liu, Changhong, Xie, Tao, Lin, Dayun, Zhang, Qin, Yu, Yue, Feng, Hao, Chen, Xusheng, Shan, Yizhou
Format:	Preprint
Published:	2025
Subjects:	Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
by: Hu, Cunchen, et al.
Published: (2024)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
by: Hu, Junhao, et al.
Published: (2024)

The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving
by: Zeng, Pai, et al.
Published: (2024)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
by: Li, Suyi, et al.
Published: (2024)

Efficient Serving of LLM Applications with Probabilistic Demand Modeling
by: Liu, Yifei, et al.
Published: (2025)

HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Clouds
by: Lou, Chiheng, et al.
Published: (2025)

TridentServe: A Stage-level Serving System for Diffusion Pipelines
by: Xia, Yifei, et al.
Published: (2025)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
by: Zhong, Yinmin, et al.
Published: (2024)

It Takes Two to Tango: Serverless Workflow Serving via Bilaterally Engaged Resource Adaptation
by: Wu, Jing, et al.
Published: (2025)

RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning
by: Hu, Junhao, et al.
Published: (2025)

MoEless: Efficient MoE LLM Serving via Serverless Computing
by: Yu, Hanfei, et al.
Published: (2026)

AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
by: Li, Zikun, et al.
Published: (2025)

Accelerating Retrieval-Augmented Language Model Serving with Speculation
by: Zhang, Zhihao, et al.
Published: (2024)

ENOVA: Autoscaling towards Cost-effective and Stable Serverless LLM Serving
by: Huang, Tao, et al.
Published: (2024)

PolyServe: Efficient Multi-SLO Serving at Scale
by: Zhu, Kan, et al.
Published: (2025)

Towards High-Goodput LLM Serving with Prefill-decode Multiplexing
by: Chen, Yukang, et al.
Published: (2025)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
by: Du, Boxiao, et al.
Published: (2026)

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
by: Duan, Jiangfei, et al.
Published: (2024)

DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving
by: Huang, Heyang, et al.
Published: (2025)

MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
by: Wang, Xin, et al.
Published: (2026)

WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
by: Lou, Chiheng, et al.
Published: (2025)

EdgeServing: Deadline-Aware Multi-DNN Serving at the Edge
by: Cao, Jiahe, et al.
Published: (2026)

Fast Self‐Charging Flexible Device Based on Metal–Air Redox Reaction under Normal Humidity
by: Dan Gao, et al.
Published: (2024)

Do Not Wait: Learning Re-Ranking Model Without User Feedback At Serving Time in E-Commerce
by: Wang, Yuan, et al.
Published: (2024)

AugServe: Adaptive Request Scheduling for Augmented Large Language Model Inference Serving
by: Wang, Ying, et al.
Published: (2025)

Multi-stage Flow Scheduling for LLM Serving
by: Sun, Yijun, et al.
Published: (2026)

RelServe: Fast LLM Inference Serving on Relational Data
by: Zhang, Xin, et al.
Published: (2025)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
by: Liu, Dong, et al.
Published: (2025)

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
by: Zhang, Hang, et al.
Published: (2025)

Serv-Drishti: An Interactive Serverless Function Request Simulation Engine and Visualiser
by: Agarwal, Siddharth, et al.
Published: (2025)

PlanetServe: A Decentralized, Scalable, and Privacy-Preserving Overlay for Democratizing Large Language Model Serving
by: Fang, Fei, et al.
Published: (2025)

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
by: Lin, Yanying, et al.
Published: (2025)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
by: Peng, You, et al.
Published: (2026)

Collaborative Speculative Inference for Efficient LLM Inference Serving
by: Gao, Luyao, et al.
Published: (2025)

Improving Smart Contract Security with Contrastive Learning-based Vulnerability Detection
by: Chen, Yizhou, et al.
Published: (2024)

JITServe: SLO-aware LLM Serving with Imprecise Request Information
by: Zhang, Wei, et al.
Published: (2025)

Time Served
by: Milton, Evan, et al.
Published: (2025)

Not All Prefills Are Equal: PPD Disaggregation for Multi-turn LLM Serving
by: Li, Zongze, et al.
Published: (2026)

The Simpler the Better: One‐Pot Production of γ‐Valerolactone from Biomass Conversion through All‐In‐One Sc(OTf)3 Catalyzed Cascade Reaction
by: Rui Hao, et al.
Published: (2024)

P/D-Serve: Serving Disaggregated Large Language Model at Scale
by: Jin, Yibo, et al.
Published: (2024)