Copertina

Enabling Disaggregated Multi-Stage MLLM Inference via GPU-Internal Scheduling and Resource Sharing

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhao, Lingxiao, Zhou, Haoran, Che, Yuezhi, Cheng, Dazhao
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

RAPID-Serve: Resource-efficient and Accelerated P/D Intra-GPU Disaggregation
di: Masood, Amna, et al.
Pubblicazione: (2026)

PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
di: Woo, Sunghyeon, et al.
Pubblicazione: (2026)

Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation
di: Liang, Yunkai, et al.
Pubblicazione: (2025)

Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
di: Shi, Xiaoxiang, et al.
Pubblicazione: (2025)

Parallel Track Transformers: Enabling Fast GPU Inference with Reduced Synchronization
di: Wang, Chong, et al.
Pubblicazione: (2026)

HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
di: Zhang, Zeyu, et al.
Pubblicazione: (2025)

ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference
di: Oh, Hyungjun, et al.
Pubblicazione: (2024)

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing
di: Luo, Yizhou, et al.
Pubblicazione: (2024)

KVDirect: Distributed Disaggregated LLM Inference
di: Chen, Shiyang, et al.
Pubblicazione: (2024)

HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
di: Zhong, Shuzhang, et al.
Pubblicazione: (2025)

Timeliness-Oriented Scheduling and Resource Allocation in Multi-Region Collaborative Perception
di: Zhu, Mengmeng, et al.
Pubblicazione: (2026)

FIKIT: Priority-Based Real-time GPU Multi-tasking Scheduling with Kernel Identification
di: Wu, Wenqing
Pubblicazione: (2023)

A Practical Two-Stage Framework for GPU Resource and Power Prediction in Heterogeneous HPC Systems
di: Oztop, Beste, et al.
Pubblicazione: (2026)

SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference
di: Li, Luchang, et al.
Pubblicazione: (2026)

Semantic-Aware Scheduling for GPU Clusters with Large Language Models
di: Wang, Zerui, et al.
Pubblicazione: (2025)

MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
di: Zhu, Ruidong, et al.
Pubblicazione: (2025)

WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
di: Lou, Chiheng, et al.
Pubblicazione: (2025)

MultiTASC++: A Continuously Adaptive Scheduler for Edge-Based Multi-Device Cascade Inference
di: Nikolaidis, Sokratis, et al.
Pubblicazione: (2024)

ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache
di: Wang, Shao, et al.
Pubblicazione: (2026)

Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving
di: Ding, Jianru, et al.
Pubblicazione: (2026)

Prediction-Assisted Online Distributed Deep Learning Workload Scheduling in GPU Clusters
di: Luo, Ziyue, et al.
Pubblicazione: (2025)

Torpor: GPU-Enabled Serverless Computing for Low-Latency, Resource-Efficient Inference
di: Yu, Minchen, et al.
Pubblicazione: (2023)

StraightLine: An End-to-End Resource-Aware Scheduler for Machine Learning Application Requests
di: Ching, Cheng-Wei, et al.
Pubblicazione: (2024)

Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecture
di: Wu, Yu, et al.
Pubblicazione: (2025)

GPU Cluster Scheduling for Network-Sensitive Deep Learning
di: Sharma, Aakash, et al.
Pubblicazione: (2024)

StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation
di: Zhong, Yinmin, et al.
Pubblicazione: (2025)

DHP: Efficient Scaling of MLLM Training with Dynamic Hybrid Parallelism
di: Niu, Yifan, et al.
Pubblicazione: (2026)

Optimal Scheduling Algorithms for LLM Inference: Theory and Practice
di: Bari, Agrim, et al.
Pubblicazione: (2025)

Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference
di: Zhang, Haolin, et al.
Pubblicazione: (2025)

SIMPLE: Disaggregating Sampling from GPU Inference into a Decision Plane for Faster Distributed LLM Serving
di: Zhao, Bohan, et al.
Pubblicazione: (2025)

Taming Request Imbalance: SLO-Aware Scheduling for Disaggregated LLM Inference
di: Wang, Qipeng
Pubblicazione: (2026)

PecSched: Preemptive and Efficient Cluster Scheduling for LLM Inference
di: Zhang, Zeyu, et al.
Pubblicazione: (2024)

Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
di: Recasens, Pol G., et al.
Pubblicazione: (2025)

GPU-Accelerated Optimization of Transformer-Based Neural Networks for Real-Time Inference
di: Mukherjee, Soutrik, et al.
Pubblicazione: (2026)

Reinforcement Learning for Adaptive Resource Scheduling in Complex System Environments
di: Li, Pochun, et al.
Pubblicazione: (2024)

SAIR: Cost-Efficient Multi-Stage ML Pipeline Autoscaling via In-Context Reinforcement Learning
di: Su, Jianchang, et al.
Pubblicazione: (2026)

SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
di: Zhang, Hengrui, et al.
Pubblicazione: (2025)

Tessera: Unlocking Heterogeneous GPUs through Kernel-Granularity Disaggregation
di: Hu, Tiancheng, et al.
Pubblicazione: (2026)

70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)
di: Zhang, Tianyi, et al.
Pubblicazione: (2025)

Metadata-Guided Adaptable Frequency Scaling across Heterogeneous Applications and Devices
di: Yan, Jinqi, et al.
Pubblicazione: (2025)