Image de couverture de livre

ML Inference Scheduling with Predictable Latency

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhao, Haidong, Georgantas, Nikolaos
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Strait: Perceiving Priority and Interference in ML Inference Serving
par: Zhao, Haidong, et autres
Publié: (2026)

Gradient Boosted Risk Scores
par: Georgantas, Costa, et autres
Publié: (2026)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
par: Li, Ruixiao, et autres
Publié: (2025)

InstMeter: An Instruction-Level Method to Predict Energy and Latency of DL Model Inference on MCUs
par: Liu, Hao, et autres
Publié: (2026)

Benchmarking Energy and Latency in TinyML: A Novel Method for Resource-Constrained AI
par: Bartoli, Pietro, et autres
Publié: (2025)

Guaranteed Dynamic Scheduling of Ultra-Reliable Low-Latency Traffic via Conformal Prediction
par: Cohen, Kfir M., et autres
Publié: (2023)

Stateful Inference for Low-Latency Multi-Agent Tool Calling
par: Norgren, Victor
Publié: (2026)

Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling
par: Winston, Caleb, et autres
Publié: (2026)

Battery-aware Cyclic Scheduling in Energy-harvesting Federated Learning
par: Jeong, Eunjeong, et autres
Publié: (2025)

Computation-aware Energy-harvesting Federated Learning: Cyclic Scheduling with Selective Participation
par: Jeong, Eunjeong, et autres
Publié: (2025)

Toward Robust and Efficient ML-Based GPU Caching for Modern Inference
par: Chen, Peng, et autres
Publié: (2025)

FluidML: Fast and Memory Efficient Inference Optimization
par: Liu, Jinjie, et autres
Publié: (2024)

SafeTail: Efficient Tail Latency Optimization in Edge Service Scheduling via Computational Redundancy Management
par: Shokhanda, Jyoti, et autres
Publié: (2024)

Optimal Inference Schedules for Masked Diffusion Models
par: Chen, Sitan, et autres
Publié: (2025)

Slim Scheduler: A Runtime-Aware RL and Scheduler System for Efficient CNN Inference
par: Harshbarger, Ian, et autres
Publié: (2025)

Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml
par: Jiang, Zhixing, et autres
Publié: (2024)

Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving
par: Dai, Yinwei, et autres
Publié: (2023)

lm-Meter: Unveiling Runtime Inference Latency for On-Device Language Models
par: Wang, Haoxin, et autres
Publié: (2025)

Prompt-Aware Scheduling for Low-Latency LLM Serving
par: Tao, Yiheng, et autres
Publié: (2025)

Chronic Diseases Prediction Using ML
par: Mulakala, Sri Varsha, et autres
Publié: (2025)

Training ML Models with Predictable Failures
par: Schwarzer, Will, et autres
Publié: (2026)

Hybrid JIT-CUDA Graph Optimization for Low-Latency Large Language Model Inference
par: Yadav, Divakar Kumar, et autres
Publié: (2026)

Accelerating TinyML Inference on Microcontrollers through Approximate Kernels
par: Armeniakos, Giorgos, et autres
Publié: (2024)

LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers
par: Yu, Enda, et autres
Publié: (2025)

Applied Causal Inference Powered by ML and AI
par: Chernozhukov, Victor, et autres
Publié: (2024)

Development and Deployment of Hybrid ML Models for Critical Heat Flux Prediction in Annulus Geometries
par: Furlong, Aidan, et autres
Publié: (2025)

Greening AI Inference with Accuracy and Latency-aware User Incentives
par: Siris, Vasilios A., et autres
Publié: (2026)

Flow-Controlled Scheduling for LLM Inference with Provable Stability Guarantees
par: Dong, Zhuolun, et autres
Publié: (2026)

Semantic Scheduling for LLM Inference
par: Hua, Wenyue, et autres
Publié: (2025)

LatencyPrism: Online Non-intrusive Latency Sculpting for SLO-Guaranteed LLM Inference
par: Du, Yin, et autres
Publié: (2026)

Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines
par: Chang, Chaokun, et autres
Publié: (2024)

TIP-Search: Time-Predictable Inference Scheduling for Market Prediction under Uncertain Load
par: Wang, Xibai
Publié: (2025)

Prediction-Powered Adaptive Shrinkage Estimation
par: Li, Sida, et autres
Publié: (2025)

Computing Within Limits: An Empirical Study of Energy Consumption in ML Training and Inference
par: Mavromatis, Ioannis, et autres
Publié: (2024)

3D Optimization for AI Inference Scaling: Balancing Accuracy, Cost, and Latency
par: Jung, Minseok, et autres
Publié: (2025)

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
par: Agrawal, Amey, et autres
Publié: (2024)

DVFS-Aware DNN Inference on GPUs: Latency Modeling and Performance Analysis
par: Han, Yunchu, et autres
Publié: (2025)

InferF: Declarative Factorization of AI/ML Inferences over Joins
par: Chowdhury, Kanchan, et autres
Publié: (2025)

Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents
par: Dai, J. G., et autres
Publié: (2025)

Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption
par: Biswas, Sayan, et autres
Publié: (2025)