Copertina

LatencyPrism: Online Non-intrusive Latency Sculpting for SLO-Guaranteed LLM Inference

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Du, Yin, Ren, Jiayi, Sun, Xiayu, Zhou, Tianyao, Zhou, Haizhu, Ma, Ruiyan, Zhang, Danyang
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Distributed, Parallel, and Cluster Computing Machine Learning Operating Systems
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Memory Offloading for Large Language Model Inference with Latency SLO Guarantees
di: Ma, Chenxiang, et al.
Pubblicazione: (2025)

Mitigating context switching in densely packed Linux clusters with Latency-Aware Group Scheduling
di: Isstaif, Al Amjad Tawfiq, et al.
Pubblicazione: (2025)

Nixie: Efficient, Transparent Temporal Multiplexing for Consumer GPUs
di: Xu, Yechen, et al.
Pubblicazione: (2026)

TIDAL: Recovering Temporal Phase for Cloud Block Storage Placement from LLM-Derived Semantics
di: Tan, Difan, et al.
Pubblicazione: (2026)

Towards Efficient and Practical GPU Multitasking in the Era of LLM
di: Xing, Jiarong, et al.
Pubblicazione: (2025)

LMetric: Simple is Better - Multiplication May Be All You Need for LLM Request Scheduling
di: Zhang, Dingyan, et al.
Pubblicazione: (2026)

Peformance Isolation for Inference Processes in Edge GPU Systems
di: Martín, Juan José, et al.
Pubblicazione: (2026)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
di: Wang, Zhibin, et al.
Pubblicazione: (2025)

RAGDoll: Efficient Offloading-based Online RAG System on a Single GPU
di: Yu, Weiping, et al.
Pubblicazione: (2025)

EdgeFlow: Fast Cold Starts for LLMs on Mobile Devices
di: Yan, Yongsheng, et al.
Pubblicazione: (2026)

Toward Systems Foundations for Agentic Exploration
di: Xu, Jiakai, et al.
Pubblicazione: (2025)

ContiguousKV: Accelerating LLM Prefill with Granularity-Aligned KV Cache Management
di: Zou, Jing, et al.
Pubblicazione: (2026)

EconoServe: Maximizing Multi-Resource Utilization with SLO Guarantees in LLM Serving
di: Shen, Haiying, et al.
Pubblicazione: (2024)

GPUOS: A GPU Operating System Primitive for Transparent Operation Fusion
di: Yang, Yiwei, et al.
Pubblicazione: (2026)

OOCO: Latency-disaggregated Architecture for Online-Offline Co-locate LLM Serving
di: Wu, Siyu, et al.
Pubblicazione: (2025)

Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
di: Hidayetoglu, Mert, et al.
Pubblicazione: (2025)

SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines
di: Cheng, Ke, et al.
Pubblicazione: (2024)

Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking
di: Mo, Zizhao, et al.
Pubblicazione: (2026)

Taming Request Imbalance: SLO-Aware Scheduling for Disaggregated LLM Inference
di: Wang, Qipeng
Pubblicazione: (2026)

SLICE: SLO-Driven Scheduling for LLM Inference on Edge Computing Devices
di: Chow, Will
Pubblicazione: (2025)

SeaLLM: Service-Aware and Latency-Optimized Resource Sharing for Large Language Model Inference
di: Zhao, Yihao, et al.
Pubblicazione: (2025)

Hyperion: Low-Latency Ultra-HD Video Analytics via Collaborative Vision Transformer Inference
di: Jiang, Linyi, et al.
Pubblicazione: (2025)

Areon: Latency-Friendly and Resilient Multi-Proposer Consensus
di: Castro-Castilla, Álvaro, et al.
Pubblicazione: (2025)

EROICA: Online Performance Troubleshooting for Large-scale Model Training
di: Guan, Yu, et al.
Pubblicazione: (2025)

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
di: Agrawal, Amey, et al.
Pubblicazione: (2024)

HarmonyBatch: Batching multi-SLO DNN Inference with Heterogeneous Serverless Functions
di: Chen, Jiabin, et al.
Pubblicazione: (2024)

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
di: Oliaro, Gabriele, et al.
Pubblicazione: (2024)

DPC: A Distributed Page Cache over CXL
di: Bergman, Shai, et al.
Pubblicazione: (2026)

Unlocking True Elasticity for the Cloud-Native Era with Dandelion
di: Kuchler, Tom, et al.
Pubblicazione: (2025)

A Periodic Space of Distributed Computing: Vision & Framework
di: Salehi, Mohsen Amini, et al.
Pubblicazione: (2026)

THEMIS: Time, Heterogeneity, and Energy Minded Scheduling for Fair Multi-Tenant Use in FPGAs
di: Karabulut, Emre, et al.
Pubblicazione: (2024)

Formal Definitions and Performance Comparison of Consistency Models for Parallel File Systems
di: Wang, Chen, et al.
Pubblicazione: (2024)

Mewz: Lightweight Execution Environment for WebAssembly with High Isolation and Portability using Unikernels
di: Ueda, Soichiro, et al.
Pubblicazione: (2024)

Taming Serverless Cold Starts Through OS Co-Design
di: Holmes, Ben, et al.
Pubblicazione: (2025)

Fix: externalizing network I/O in serverless computing
di: Deng, Yuhan, et al.
Pubblicazione: (2025)

Optimizing Task Scheduling in Heterogeneous Computing Environments: A Comparative Analysis of CPU, GPU, and ASIC Platforms Using E2C Simulator
di: Mohammadjafari, Ali, et al.
Pubblicazione: (2024)

Performance Isolation and Semantic Determinism in Efficient GPU Spatial Sharing
di: Yang, Zhenyuan, et al.
Pubblicazione: (2026)

Telepathic Datacenters: Fast RPCs using Shared CXL Memory
di: Mahar, Suyash, et al.
Pubblicazione: (2024)

Equilibria: Fair Multi-Tenant CXL Memory Tiering At Scale
di: Zhao, Kaiyang, et al.
Pubblicazione: (2026)

CvxCluster: Solving Large, Complex, Granular Resource Allocation Problems 100-1000x Faster
di: Nnorom Jr, Obi, et al.
Pubblicazione: (2026)