Image de couverture de livre

Prompt-Aware Scheduling for Low-Latency LLM Serving

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Tao, Yiheng, Zhang, Yihe, Dearing, Matthew T., Wang, Xin, Fan, Yuping, Lan, Zhiling
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing Performance
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Extracting Practical, Actionable Energy Insights from Supercomputer Telemetry and Logs
par: Cornelius, Melanie, et autres
Publié: (2025)

Leveraging LLMs to Automate Energy-Aware Refactoring of Parallel Scientific Codes
par: Dearing, Matthew T., et autres
Publié: (2025)

LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codes
par: Dearing, Matthew T., et autres
Publié: (2024)

GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving
par: Jayakody, Shakya, et autres
Publié: (2026)

Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
par: Yu, Shan, et autres
Publié: (2025)

MRSch: Multi-Resource Scheduling for HPC
par: Li, Boyang, et autres
Publié: (2024)

Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage
par: Yuan, Ziqi, et autres
Publié: (2025)

Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
par: Miao, Xupeng, et autres
Publié: (2023)

ProTrain: Efficient LLM Training via Memory-Aware Techniques
par: Yang, Hanmei, et autres
Publié: (2024)

Reliable Microservice Tail Latency Prediction via Decoupled Dual-Stream Learning and Gradient Modulation
par: Qian, Wenzhuo, et autres
Publié: (2025)

A Real-Time Digital Twin for Adaptive Scheduling
par: Zhang, Yihe, et autres
Publié: (2025)

The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures
par: Ma, Bole, et autres
Publié: (2026)

ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
par: Shen, Zixu, et autres
Publié: (2025)

CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
par: Suo, Jiashun, et autres
Publié: (2025)

A Comparative Study of OpenMP Scheduling Algorithm Selection Strategies
par: Korndörfer, Jonas H. Müller, et autres
Publié: (2025)

LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
par: Yang, Shang, et autres
Publié: (2025)

StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving
par: Kumar, Satyam, et autres
Publié: (2026)

Low-Rank GEMM: Efficient Matrix Multiplication via Low-Rank Approximation with FP8 Acceleration
par: Metere, Alfredo
Publié: (2025)

Optimizing the Deployment of Tiny Transformers on Low-Power MCUs
par: Jung, Victor J. B., et autres
Publié: (2024)

Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling
par: Li, Boyang, et autres
Publié: (2024)

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
par: Zhou, Zhongzhu, et autres
Publié: (2026)

GPU Kernel Optimization Beyond Full Builds: An LLM Framework with Minimal Executable Programs
par: Chu, Ruifan, et autres
Publié: (2025)

BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architectures
par: Hu, Xiannan, et autres
Publié: (2025)

BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems
par: Wang, Yuxin, et autres
Publié: (2024)

DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inference
par: Jeong, Bodon, et autres
Publié: (2026)

ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
par: Fan, Ruibo, et autres
Publié: (2026)

Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
par: Li, Jiaxi, et autres
Publié: (2025)

TrainMover: An Interruption-Resilient Runtime for ML Training
par: Lao, ChonLam, et autres
Publié: (2024)

Rethinking Inference Placement for Deep Learning across Edge and Cloud Platforms: A Multi-Objective Optimization Perspective and Future Directions
par: Zhang, Zongshun, et autres
Publié: (2025)

STAlloc: Enhancing Memory Efficiency in Large-Scale Model Training with Spatio-Temporal Planning
par: Huang, Zixiao, et autres
Publié: (2025)

CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe
par: Saba, Tara, et autres
Publié: (2026)

SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators
par: Odema, Mohanad, et autres
Publié: (2024)

CascadeInfer: Length-Aware Scheduling of LLM Serving with Low Latency and Load Balancing
par: Yuan, Yitao, et autres
Publié: (2025)

Compiler-First State Space Duality and Portable $O(1)$ Autoregressive Caching for Inference
par: Santoni, Cosmo
Publié: (2026)

Multi-Dimensional Autoscaling of Stream Processing Services on Edge Devices
par: Sedlak, Boris, et autres
Publié: (2025)

Record-Remix-Replay: Hierarchical GPU Kernel Optimization using Evolutionary Search
par: Nichols, Daniel, et autres
Publié: (2026)

Syno: Structured Synthesis for Neural Operators
par: Zhuo, Yongqi, et autres
Publié: (2024)

FastPersist: Accelerating Model Checkpointing in Deep Learning
par: Wang, Guanhua, et autres
Publié: (2024)

Lightweight Software Kernels and Hardware Extensions for Efficient Sparse Deep Neural Networks on Microcontrollers
par: Daghero, Francesco, et autres
Publié: (2025)

Longer Attention Span: Increasing Transformer Context Length with Sparse Graph Processing Techniques
par: Tomczak, Nathaniel, et autres
Publié: (2025)