Cover Image

Towards Resiliency in Large Language Model Serving with KevlarFlow

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qian, Shangshu, Liu, Kipling, Sruthi, P. C., Tan, Lin, Zhang, Yongle
Format:	Preprint
Published:	2026
Subjects:	Distributed, Parallel, and Cluster Computing Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CSnake: Detecting Self-Sustaining Cascading Failure via Causal Stitching of Fault Propagations
by: Qian, Shangshu, et al.
Published: (2025)

P/D-Serve: Serving Disaggregated Large Language Model at Scale
by: Jin, Yibo, et al.
Published: (2024)

Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow
by: Mei, Yixuan, et al.
Published: (2024)

Towards Sustainable Large Language Model Serving
by: Nguyen, Sophia, et al.
Published: (2024)

NanoFlow: Towards Optimal Large Language Model Serving Throughput
by: Zhu, Kan, et al.
Published: (2024)

SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
by: Miao, Xupeng, et al.
Published: (2023)

Stateful Large Language Model Serving with Pensieve
by: Yu, Lingfan, et al.
Published: (2023)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
by: Hu, Junhao, et al.
Published: (2024)

Fast Distributed Inference Serving for Large Language Models
by: Wu, Bingyang, et al.
Published: (2023)

DeepServe: Serverless Large Language Model Serving at Scale
by: Hu, Junhao, et al.
Published: (2025)

LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
by: Wu, Bingyang, et al.
Published: (2024)

ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
by: Xiang, Yuxing, et al.
Published: (2025)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
by: Griggs, Tyler, et al.
Published: (2024)

semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storage
by: Hong, Ke, et al.
Published: (2025)

Optimizing Cross-Client Domain Coverage for Federated Instruction Tuning of Large Language Models
by: Wang, Zezhou, et al.
Published: (2024)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
by: Zhong, Yinmin, et al.
Published: (2024)

Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
by: Qiu, Haoran, et al.
Published: (2024)

Optimizing RLHF Training for Large Language Models with Stage Fusion
by: Zhong, Yinmin, et al.
Published: (2024)

InstCache: A Predictive Cache for LLM Serving
by: Zou, Longwei, et al.
Published: (2024)

SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
by: Lin, Zheng, et al.
Published: (2024)

Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization
by: Che, Tianshi, et al.
Published: (2023)

P/D-Device: Disaggregated Large Language Model between Cloud and Devices
by: Jin, Yibo, et al.
Published: (2025)

Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution
by: Xu, Yechen, et al.
Published: (2024)

Enabling Efficient Serverless Inference Serving for LLM (Large Language Model) in the Cloud
by: Ghosh, Himel
Published: (2024)

DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving
by: Yu, Fengze, et al.
Published: (2025)

Unlocking Full Efficiency of Token Filtering in Large Language Model Training
by: Chai, Di, et al.
Published: (2025)

HydraInfer: Hybrid Disaggregated Scheduling for Multimodal Large Language Model Serving
by: Dong, Xianzhe, et al.
Published: (2025)

A Universal Load Balancing Principle and Its Application to Large Language Model Serving
by: Chen, Zixi, et al.
Published: (2026)

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
by: Oliaro, Gabriele, et al.
Published: (2024)

InferCept: Efficient Intercept Support for Augmented Large Language Model Inference
by: Abhyankar, Reyna, et al.
Published: (2024)

Energy-Efficient Split Learning for Fine-Tuning Large Language Models in Edge Networks
by: Li, Zuguang, et al.
Published: (2024)

Cascadia: An Efficient Cascade Serving System for Large Language Models
by: Jiang, Youhe, et al.
Published: (2025)

FastSwitch: Optimizing Context Switching Efficiency in Fairness-aware Large Language Model Serving
by: Shen, Ao, et al.
Published: (2024)

FailSafe: High-performance Resilient Serving
by: Xu, Ziyi, et al.
Published: (2025)

Inference Acceleration for Large Language Models on CPUs
by: PS, Ditto, et al.
Published: (2024)

Llumnix: Dynamic Scheduling for Large Language Model Serving
by: Sun, Biao, et al.
Published: (2024)

Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Models
by: Pan, Xuchen, et al.
Published: (2025)

TetriServe: Efficient DiT Serving for Heterogeneous Image Generation
by: Lu, Runyu, et al.
Published: (2025)

AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
by: Li, Zikun, et al.
Published: (2025)

FLYING SERVING: On-the-Fly Parallelism Switching for Large Language Model Serving
by: Gao, Shouwei, et al.
Published: (2026)