:: Library Catalog

Cover Image

Saved in:

Bibliographic Details
Main Authors:	Xue, Leyang, Fu, Yao, Mai, Luo, Marina, Mahesh K.
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	https://arxiv.org/abs/2505.12566
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
by: Xue, Leyang, et al.
Published: (2024)

CascadeServe: Unlocking Model Cascades for Inference Serving
by: Kossmann, Ferdi, et al.
Published: (2024)

TUBO: A Tailored ML Framework for Reliable Network Traffic Forecasting
by: Yuan, Zhihang, et al.
Published: (2026)

Towards Decentralized and Sustainable Foundation Model Training with the Edge
by: Xue, Leyang, et al.
Published: (2025)

ServerlessLLM: Low-Latency Serverless Inference for Large Language Models
by: Fu, Yao, et al.
Published: (2024)

LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
by: Wu, Bingyang, et al.
Published: (2024)

Echo: Efficient Co-Scheduling of Hybrid Online-Offline Tasks for Large Language Model Serving
by: Wang, Zhibin, et al.
Published: (2025)

On Harnessing Idle Compute at the Edge for Foundation Model Training
by: Xue, Leyang, et al.
Published: (2025)

PolyServe: Efficient Multi-SLO Serving at Scale
by: Zhu, Kan, et al.
Published: (2025)

Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving
by: Gao, Shihong, et al.
Published: (2025)

Grounded AI for Code Review: Resource-Efficient Large-Model Serving in Enterprise Pipelines
by: Mandal, Sayan, et al.
Published: (2025)

GreenServ: Energy-Efficient Context-Aware Dynamic Routing for Multi-Model LLM Inference
by: Ziller, Thomas, et al.
Published: (2026)

P/D-Serve: Serving Disaggregated Large Language Model at Scale
by: Jin, Yibo, et al.
Published: (2024)

The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving
by: Zeng, Pai, et al.
Published: (2024)

TetriServe: Efficient DiT Serving for Heterogeneous Image Generation
by: Lu, Runyu, et al.
Published: (2025)

Fairness in Serving Large Language Models
by: Sheng, Ying, et al.
Published: (2023)

Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
by: Wu, Fangzhou, et al.
Published: (2025)

MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
by: Xu, Tairan, et al.
Published: (2025)

IC-Cache: Efficient Large Language Model Serving via In-context Caching
by: Yu, Yifan, et al.
Published: (2025)

DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
by: Yao, Xiaozhe, et al.
Published: (2023)

MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Serving
by: Lee, Jungi, et al.
Published: (2025)

iServe: An Intent-based Serving System for LLMs
by: Liakopoulos, Dimitrios, et al.
Published: (2025)

Towards Sustainable Large Language Model Serving
by: Nguyen, Sophia, et al.
Published: (2024)

Stateful Large Language Model Serving with Pensieve
by: Yu, Lingfan, et al.
Published: (2023)

EdgeServe: A Streaming System for Decentralized Model Serving
by: Shaowang, Ted, et al.
Published: (2023)

CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
by: Yao, Jiayi, et al.
Published: (2024)

Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately
by: Wang, Yuhang, et al.
Published: (2025)

Scalable AI Inference: Performance Analysis and Optimization of AI Model Serving
by: Pham, Hung Cuong, et al.
Published: (2026)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
by: Griggs, Tyler, et al.
Published: (2024)

Enabling Efficient Serverless Inference Serving for LLM (Large Language Model) in the Cloud
by: Ghosh, Himel
Published: (2024)

Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
by: Gao, Bin, et al.
Published: (2024)

Sparse Prefix Caching for Hybrid and Recurrent LLM Serving
by: Shirokikh, Mikhail, et al.
Published: (2026)

Efficiently Serving Large Multimodal Models Using EPD Disaggregation
by: Singh, Gursimran, et al.
Published: (2024)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
by: Hu, Junhao, et al.
Published: (2024)

SLOs-Serve: Optimized Serving of Multi-SLO LLMs
by: Chen, Siyuan, et al.
Published: (2025)

xGR: Efficient Generative Recommendation Serving at Scale
by: Sun, Qingxiao, et al.
Published: (2025)

MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
by: Wang, Xin, et al.
Published: (2026)

Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management
by: Zheng, Haoyu, et al.
Published: (2026)

Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
by: Kim, Minsu, et al.
Published: (2025)

Fast Distributed Inference Serving for Large Language Models
by: Wu, Bingyang, et al.
Published: (2023)