Copertina

HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Clouds

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lou, Chiheng, Qi, Sheng, Jin, Chao, Nie, Dapeng, Yang, Haoran, Ding, Yu, Liu, Xuanzhe, Jin, Xin
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
di: Lou, Chiheng, et al.
Pubblicazione: (2025)

KUBEDIRECT: Unleashing the Full Power of the Cluster Manager for Serverless Computing
di: Qi, Sheng, et al.
Pubblicazione: (2026)

DeepServe: Serverless Large Language Model Serving at Scale
di: Hu, Junhao, et al.
Pubblicazione: (2025)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
di: Zhong, Yinmin, et al.
Pubblicazione: (2024)

Cold Start Latency in Serverless Computing: A Systematic Review, Taxonomy, and Future Directions
di: Golec, Muhammed, et al.
Pubblicazione: (2023)

Torpor: GPU-Enabled Serverless Computing for Low-Latency, Resource-Efficient Inference
di: Yu, Minchen, et al.
Pubblicazione: (2023)

LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
di: Wu, Bingyang, et al.
Pubblicazione: (2024)

Enabling Efficient Serverless Inference Serving for LLM (Large Language Model) in the Cloud
di: Ghosh, Himel
Pubblicazione: (2024)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

OOCO: Latency-disaggregated Architecture for Online-Offline Co-locate LLM Serving
di: Wu, Siyu, et al.
Pubblicazione: (2025)

Cloud Native System for LLM Inference Serving
di: Xu, Minxian, et al.
Pubblicazione: (2025)

SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
di: Jaiswal, Shashwat, et al.
Pubblicazione: (2025)

StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving
di: Kumar, Satyam, et al.
Pubblicazione: (2026)

HydraInfer: Hybrid Disaggregated Scheduling for Multimodal Large Language Model Serving
di: Dong, Xianzhe, et al.
Pubblicazione: (2025)

Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
di: Liu, Xueshen, et al.
Pubblicazione: (2026)

Cold-Start Anti-Patterns and Refactorings in Serverless Systems: An Empirical Study
di: Tariq, Syed Salauddin Mohammad, et al.
Pubblicazione: (2025)

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
di: Lin, Yanying, et al.
Pubblicazione: (2025)

Aladdin: Joint Placement and Scaling for SLO-Aware LLM Serving
di: Nie, Chengyi, et al.
Pubblicazione: (2024)

ENOVA: Autoscaling towards Cost-effective and Stable Serverless LLM Serving
di: Huang, Tao, et al.
Pubblicazione: (2024)

SeaLLM: Service-Aware and Latency-Optimized Resource Sharing for Large Language Model Inference
di: Zhao, Yihao, et al.
Pubblicazione: (2025)

CascadeInfer: Length-Aware Scheduling of LLM Serving with Low Latency and Load Balancing
di: Yuan, Yitao, et al.
Pubblicazione: (2025)

OmniInfer: System-Wide Acceleration Techniques for Optimizing LLM Serving Throughput and Latency
di: Wang, Jun, et al.
Pubblicazione: (2025)

ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
di: Xiang, Yuxing, et al.
Pubblicazione: (2025)

PROSERVE: Unified Multi-Priority Request Scheduling for LLM Serving
di: Huang, Weizhe, et al.
Pubblicazione: (2025)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
di: Ruan, Chaoyi, et al.
Pubblicazione: (2025)

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
di: Duan, Jiangfei, et al.
Pubblicazione: (2024)

PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Serving
di: Wang, Wenfeng, et al.
Pubblicazione: (2026)

TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
di: Wu, Bingyang, et al.
Pubblicazione: (2025)

Dilu: Enabling GPU Resourcing-on-Demand for Serverless DL Serving via Introspective Elasticity
di: Lv, Cunchi, et al.
Pubblicazione: (2025)

It Takes Two to Tango: Serverless Workflow Serving via Bilaterally Engaged Resource Adaptation
di: Wu, Jing, et al.
Pubblicazione: (2025)

Fast Distributed Inference Serving for Large Language Models
di: Wu, Bingyang, et al.
Pubblicazione: (2023)

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
di: Agrawal, Amey, et al.
Pubblicazione: (2024)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
di: Hu, Cunchen, et al.
Pubblicazione: (2024)

EconoServe: Maximizing Multi-Resource Utilization with SLO Guarantees in LLM Serving
di: Shen, Haiying, et al.
Pubblicazione: (2024)

Taming Cold Starts: Proactive Serverless Scheduling with Model Predictive Control
di: Nguyen, Chanh, et al.
Pubblicazione: (2025)

Universal Workers: A Vision for Eliminating Cold Starts in Serverless Computing
di: Akbari, Saman, et al.
Pubblicazione: (2025)

SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
di: Zhou, Qihui, et al.
Pubblicazione: (2025)

Predictable LLM Serving on GPU Clusters
di: Darzi, Erfan, et al.
Pubblicazione: (2025)

Taming Serverless Cold Starts Through OS Co-Design
di: Holmes, Ben, et al.
Pubblicazione: (2025)

Kairos: Low-latency Multi-Agent Serving with Shared LLMs and Excessive Loads in the Public Cloud
di: Chen, Jinyuan, et al.
Pubblicazione: (2025)