Cover Image

Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yildiz, Mert, Spadaccino, Pietro, Rolich, Alexey, Cuomo, Francesca, Baiocchi, Andrea
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Merit of Simple Policies: Buying Performance With Parallelism and System Architecture
by: Yildiz, Mert, et al.
Published: (2025)

Dispatching Odyssey: Exploring Performance in Computing Clusters under Real-world Workloads
by: Yildiz, Mert, et al.
Published: (2025)

"Two-Stagification": Job Dispatching in Large-Scale Clusters via a Two-Stage Architecture
by: Yildiz, Mert, et al.
Published: (2025)

FlowPrefill: Decoupling Preemption from Prefill Scheduling Granularity to Mitigate Head-of-Line Blocking in LLM Serving
by: Hsieh, Chia-chi, et al.
Published: (2026)

Preemption Aware Task Scheduling for Priority and Deadline Constrained DNN Inference Task Offloading in Homogeneous Mobile-Edge Networks
by: Cotter, Jamie, et al.
Published: (2025)

Duration-Informed Workload Scheduler
by: Loreti, Daniela, et al.
Published: (2026)

Joint Resource Optimization, Computation Offloading and Resource Slicing for Multi-Edge Traffic-Cognitive Networks
by: Xiaoyang, Ting, et al.
Published: (2024)

A Model Aware AIGC Task Offloading Algorithm in IIoT Edge Computing
by: Wang, Xin, et al.
Published: (2025)

Topology-aware Preemptive Scheduling for Co-located LLM Workloads
by: Zhang, Ping, et al.
Published: (2024)

HiveMind: OS-Inspired Scheduling for Concurrent LLM Agent Workloads
by: Agyemang, Justice Owusu, et al.
Published: (2026)

Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
by: Da, Wei, et al.
Published: (2025)

TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
by: Stojkovic, Jovan, et al.
Published: (2025)

Evaluating the Efficacy of LLM-Based Reasoning for Multiobjective HPC Job Scheduling
by: Jadhav, Prachi, et al.
Published: (2025)

DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inference
by: Jeong, Bodon, et al.
Published: (2026)

KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference
by: Zhang, Huawei, et al.
Published: (2025)

Compass: A Decentralized Scheduler for Latency-Sensitive ML Workflows
by: Yang, Yuting, et al.
Published: (2024)

Reducing Fragmentation and Starvation in GPU Clusters through Dynamic Multi-Objective Scheduling
by: Mamirov, Akhmadillo
Published: (2025)

CoRaiS: Lightweight Real-Time Scheduler for Multi-Edge Cooperative Computing
by: Hu, Yujiao, et al.
Published: (2024)

LeMix: Unified Scheduling for LLM Training and Inference on Multi-GPU Systems
by: Li, Yufei, et al.
Published: (2025)

Equinox: Holistic Fair Scheduling in Serving Large Language Models
by: Wei, Zhixiang, et al.
Published: (2025)

GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management
by: Duan, Jiaang, et al.
Published: (2025)

Mixture-of-Schedulers: An Adaptive Scheduling Agent as a Learned Router for Expert Policies
by: Wang, Xinbo, et al.
Published: (2025)

DataCenterGym: A Physics-Grounded Simulator for Multi-Objective Data Center Scheduling
by: Pathak, Nilavra, et al.
Published: (2026)

D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
by: Wang, Haodong, et al.
Published: (2025)

Towards One-shot Federated Learning: Advances, Challenges, and Future Directions
by: Amato, Flora, et al.
Published: (2025)

TCM-Serve: Modality-aware Scheduling for Multimodal Large Language Model Inference
by: Papaioannou, Konstantinos, et al.
Published: (2026)

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems
by: Sgambati, Matthew, et al.
Published: (2025)

Workload Schedulers -- Genesis, Algorithms and Differences
by: Sliwko, Leszek, et al.
Published: (2025)

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
by: Jiang, Xuanlin, et al.
Published: (2024)

Accuracy-Delay Trade-Off in LLM Offloading via Token-Level Uncertainty
by: Kim, Yumin, et al.
Published: (2026)

DeServe: Towards Affordable Offline LLM Inference via Decentralization
by: Wu, Linyu, et al.
Published: (2025)

Power- and Fragmentation-aware Online Scheduling for GPU Datacenters
by: Lettich, Francesco, et al.
Published: (2024)

Dynamic Scheduling Strategies for Resource Optimization in Computing Environments
by: Wang, Xiaoye
Published: (2024)

Capacity Planning and Scheduling for Jobs with Uncertainty in Resource Usage and Duration
by: Patra, Sunandita, et al.
Published: (2025)

Xe-Forge: Multi-Stage LLM-Powered Kernel Optimization for Intel GPU
by: Spoczynski, Marcin, et al.
Published: (2026)

Reconstruction-Based Adaptive Scheduling Using AI Inferences in Safety-Critical Systems
by: Alshaer, Samer, et al.
Published: (2025)

Reinforcement Learning-driven Data-intensive Workflow Scheduling for Volunteer Edge-Cloud
by: Mounesan, Motahare, et al.
Published: (2024)

Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
by: Pan, Xinglin, et al.
Published: (2025)

WORKSWORLD: A Domain for Integrated Numeric Planning and Scheduling of Distributed Pipelined Workflows
by: Paul, Taylor, et al.
Published: (2026)

SparOA: Sparse and Operator-aware Hybrid Scheduling for Edge DNN Inference
by: Zhang, Ziyang, et al.
Published: (2025)