Imagen de Portada

ML Inference Scheduling with Predictable Latency

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhao, Haidong, Georgantas, Nikolaos
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Strait: Perceiving Priority and Interference in ML Inference Serving
por: Zhao, Haidong, et al.
Publicado: (2026)

Gradient Boosted Risk Scores
por: Georgantas, Costa, et al.
Publicado: (2026)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
por: Li, Ruixiao, et al.
Publicado: (2025)

InstMeter: An Instruction-Level Method to Predict Energy and Latency of DL Model Inference on MCUs
por: Liu, Hao, et al.
Publicado: (2026)

Benchmarking Energy and Latency in TinyML: A Novel Method for Resource-Constrained AI
por: Bartoli, Pietro, et al.
Publicado: (2025)

Guaranteed Dynamic Scheduling of Ultra-Reliable Low-Latency Traffic via Conformal Prediction
por: Cohen, Kfir M., et al.
Publicado: (2023)

Stateful Inference for Low-Latency Multi-Agent Tool Calling
por: Norgren, Victor
Publicado: (2026)

Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling
por: Winston, Caleb, et al.
Publicado: (2026)

Battery-aware Cyclic Scheduling in Energy-harvesting Federated Learning
por: Jeong, Eunjeong, et al.
Publicado: (2025)

Computation-aware Energy-harvesting Federated Learning: Cyclic Scheduling with Selective Participation
por: Jeong, Eunjeong, et al.
Publicado: (2025)

Toward Robust and Efficient ML-Based GPU Caching for Modern Inference
por: Chen, Peng, et al.
Publicado: (2025)

SafeTail: Efficient Tail Latency Optimization in Edge Service Scheduling via Computational Redundancy Management
por: Shokhanda, Jyoti, et al.
Publicado: (2024)

FluidML: Fast and Memory Efficient Inference Optimization
por: Liu, Jinjie, et al.
Publicado: (2024)

Optimal Inference Schedules for Masked Diffusion Models
por: Chen, Sitan, et al.
Publicado: (2025)

Slim Scheduler: A Runtime-Aware RL and Scheduler System for Efficient CNN Inference
por: Harshbarger, Ian, et al.
Publicado: (2025)

Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml
por: Jiang, Zhixing, et al.
Publicado: (2024)

Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving
por: Dai, Yinwei, et al.
Publicado: (2023)

lm-Meter: Unveiling Runtime Inference Latency for On-Device Language Models
por: Wang, Haoxin, et al.
Publicado: (2025)

Prompt-Aware Scheduling for Low-Latency LLM Serving
por: Tao, Yiheng, et al.
Publicado: (2025)

Chronic Diseases Prediction Using ML
por: Mulakala, Sri Varsha, et al.
Publicado: (2025)

Training ML Models with Predictable Failures
por: Schwarzer, Will, et al.
Publicado: (2026)

Hybrid JIT-CUDA Graph Optimization for Low-Latency Large Language Model Inference
por: Yadav, Divakar Kumar, et al.
Publicado: (2026)

Accelerating TinyML Inference on Microcontrollers through Approximate Kernels
por: Armeniakos, Giorgos, et al.
Publicado: (2024)

LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers
por: Yu, Enda, et al.
Publicado: (2025)

Applied Causal Inference Powered by ML and AI
por: Chernozhukov, Victor, et al.
Publicado: (2024)

Development and Deployment of Hybrid ML Models for Critical Heat Flux Prediction in Annulus Geometries
por: Furlong, Aidan, et al.
Publicado: (2025)

Greening AI Inference with Accuracy and Latency-aware User Incentives
por: Siris, Vasilios A., et al.
Publicado: (2026)

Flow-Controlled Scheduling for LLM Inference with Provable Stability Guarantees
por: Dong, Zhuolun, et al.
Publicado: (2026)

Semantic Scheduling for LLM Inference
por: Hua, Wenyue, et al.
Publicado: (2025)

LatencyPrism: Online Non-intrusive Latency Sculpting for SLO-Guaranteed LLM Inference
por: Du, Yin, et al.
Publicado: (2026)

Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines
por: Chang, Chaokun, et al.
Publicado: (2024)

TIP-Search: Time-Predictable Inference Scheduling for Market Prediction under Uncertain Load
por: Wang, Xibai
Publicado: (2025)

Prediction-Powered Adaptive Shrinkage Estimation
por: Li, Sida, et al.
Publicado: (2025)

Computing Within Limits: An Empirical Study of Energy Consumption in ML Training and Inference
por: Mavromatis, Ioannis, et al.
Publicado: (2024)

3D Optimization for AI Inference Scaling: Balancing Accuracy, Cost, and Latency
por: Jung, Minseok, et al.
Publicado: (2025)

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
por: Agrawal, Amey, et al.
Publicado: (2024)

DVFS-Aware DNN Inference on GPUs: Latency Modeling and Performance Analysis
por: Han, Yunchu, et al.
Publicado: (2025)

InferF: Declarative Factorization of AI/ML Inferences over Joins
por: Chowdhury, Kanchan, et al.
Publicado: (2025)

Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents
por: Dai, J. G., et al.
Publicado: (2025)

Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption
por: Biswas, Sayan, et al.
Publicado: (2025)