Cover Image

Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Zhibin, Li, Shipeng, Zhou, Yuhang, Li, Xue, Zhang, Zhonghui, Cam-Tu, Nguyen, Gu, Rong, Tian, Chen, Chen, Guihai, Zhong, Sheng
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Echo: Efficient Co-Scheduling of Hybrid Online-Offline Tasks for Large Language Model Serving
by: Wang, Zhibin, et al.
Published: (2025)

CoDec: Prefix-Shared Decoding Kernel for LLMs
by: Wang, Zhibin, et al.
Published: (2025)

STAR: Decode-Phase Rescheduling for LLM Inference
by: Wang, Zhibin, et al.
Published: (2025)

Scaling Graph Chain-of-Thought Reasoning: A Multi-Agent Framework with Efficient LLM Serving
by: Huan, Chengying, et al.
Published: (2025)

Nova: Real-Time Agentic Vision-Language Model Serving with Adaptive Cross-Stage Parallelization
by: Xu, Yuhang, et al.
Published: (2025)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
by: Wang, Zhibin, et al.
Published: (2025)

SSV: Sparse Speculative Verification for Efficient LLM Inference
by: Wang, Zhibin, et al.
Published: (2026)

Chameleon: Taming Dynamic Operator Sequences for Memory-Intensive LLM Training
by: Wang, Zibo, et al.
Published: (2025)

MPIC: Position-Independent Multimodal Context Caching System for Efficient MLLM Serving
by: Zhao, Shiju, et al.
Published: (2025)

DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference
by: Liu, Fuliang, et al.
Published: (2026)

Towards Objective Metrics for Procedurally Generated Video Game Levels
by: Beukman, Michael, et al.
Published: (2022)

Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model
by: Gou, Qi, et al.
Published: (2024)

Slice-Level Scheduling for High Throughput and Load Balanced LLM Serving
by: Cheng, Ke, et al.
Published: (2024)

Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
by: Li, Jiaxi, et al.
Published: (2025)

Learning to Compress: Unlocking the Potential of Large Language Models for Text Representation
by: Zhang, Yeqin, et al.
Published: (2025)

Fairness in Serving Large Language Models
by: Sheng, Ying, et al.
Published: (2023)

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
by: Hu, Chenzhi, et al.
Published: (2026)

Bi-Level Decision-Focused Causal Learning for Large-Scale Marketing Optimization: Bridging Observational and Experimental Data
by: Zhang, Shuli, et al.
Published: (2025)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
by: Chen, Xing, et al.
Published: (2025)

Multi-Objective Linguistic Control of Large Language Models
by: Nguyen, Dang, et al.
Published: (2024)

An Evaluation of Large Language Models in Bioinformatics Research
by: Yin, Hengchuang, et al.
Published: (2024)

Revisiting LH Levels on Trigger Day in IVF Protocols
by: Xiushen Li, et al.
Published: (2025)

Chordless Structure: A Pathway to Simple and Expressive GNNs
by: Pan, Hongxu, et al.
Published: (2025)

ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
by: Xiang, Yuxing, et al.
Published: (2025)

Bayesian Network‐Based Uncertainty Quantification of Ship Isolation System Using Multi‐Level Data
by: Rongqiao Wang, et al.
Published: (2025)

Retrospex: Language Agent Meets Offline Reinforcement Learning Critic
by: Xiang, Yufei, et al.
Published: (2025)

AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization
by: Li, Qiyang, et al.
Published: (2026)

Objective Metrics for Evaluating Large Language Models Using External Data Sources
by: Du, Haoze, et al.
Published: (2025)

Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models
by: Sun, Haoxiang, et al.
Published: (2025)

MiMIC: Mitigating Visual Modality Collapse in Universal Multimodal Retrieval While Avoiding Semantic Misalignment
by: Li, Juan, et al.
Published: (2026)

Can a Large Language Model Assess Urban Design Quality? Evaluating Walkability Metrics Across Expertise Levels
by: Cai, Chenyi, et al.
Published: (2025)

Decentralized COVID-19 Health System Leveraging Blockchain
by: Chen, Lingsheng, et al.
Published: (2025)

Are Expert-Level Language Models Expert-Level Annotators?
by: Tseng, Yu-Min, et al.
Published: (2024)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
by: Zhong, Yinmin, et al.
Published: (2024)

Beyond Tokens: Concept-Level Training Objectives for LLMs
by: Iyer, Laya, et al.
Published: (2026)

Mitigating the Impact of False Negatives in Dense Retrieval with Contrastive Confidence Regularization
by: Wang, Shiqi, et al.
Published: (2023)

PromCopilot: Simplifying Prometheus Metric Querying in Cloud Native Online Service Systems via Large Language Models
by: Zhang, Chenxi, et al.
Published: (2025)

Are Large Language Models Capable of Generating Human-Level Narratives?
by: Tian, Yufei, et al.
Published: (2024)

Studying and Benchmarking Large Language Models For Log Level Suggestion
by: Heng, Yi Wen, et al.
Published: (2024)

From Inference Efficiency to Embodied Efficiency: Revisiting Efficiency Metrics for Vision-Language-Action Models
by: Li, Zhuofan, et al.
Published: (2026)