Image de couverture de livre

Llumnix: Dynamic Scheduling for Large Language Model Serving

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Sun, Biao, Huang, Ziming, Zhao, Hanyu, Xiao, Wencong, Zhang, Xinyi, Li, Yong, Lin, Wei
Format:	Preprint
Publié:	2024
Sujets:	Hardware Architecture Distributed, Parallel, and Cluster Computing Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
par: Lin, Bin, et autres
Publié: (2024)

Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
par: Tian, Yuyang, et autres
Publié: (2025)

Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving
par: Ding, Jianru, et autres
Publié: (2026)

GreenLLM: Disaggregating Large Language Model Serving on Heterogeneous GPUs for Lower Carbon Emissions
par: Shi, Tianyao, et autres
Publié: (2024)

Serving Large Language Models on Huawei CloudMatrix384
par: Zuo, Pengfei, et autres
Publié: (2025)

Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens
par: Yu, Yanpeng, et autres
Publié: (2025)

PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System
par: Liu, Lian, et autres
Publié: (2026)

Pooling Engram Conditional Memory in Large Language Models using CXL
par: Ma, Ruiyang, et autres
Publié: (2026)

Understanding Bottlenecks for Efficiently Serving LLM Inference With KV Offloading
par: Meng, William, et autres
Publié: (2025)

Performance Modeling and Workload Analysis of Distributed Large Language Model Training and Inference
par: Kundu, Joyjit, et autres
Publié: (2024)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
par: li, Fei, et autres
Publié: (2026)

Scheduling Techniques of AI Models on Modern Heterogeneous Edge GPU -- A Critical Review
par: Majeed, Ashiyana Abdul, et autres
Publié: (2025)

Hybrid Offline-online Scheduling Method for Large Language Model Inference Optimization
par: Pang, Bowen, et autres
Publié: (2025)

Optimizing Task Scheduling in Fog Computing with Deadline Awareness
par: Sirjani, Mohammad Sadegh, et autres
Publié: (2025)

DeFiNES: Enabling Fast Exploration of the Depth-first Scheduling Space for DNN Accelerators through Analytical Modeling
par: Mei, Linyan, et autres
Publié: (2022)

SLIM: A Heterogeneous Accelerator for Edge Inference of Sparse Large Language Model via Adaptive Thresholding
par: Xu, Weihong, et autres
Publié: (2025)

CELLO: Co-designing Schedule and Hybrid Implicit/Explicit Buffer for Complex Tensor Reuse
par: Garg, Raveesh, et autres
Publié: (2023)

A Survey of Real-time Scheduling on Accelerator-based Heterogeneous Architecture for Time Critical Applications
par: Zou, An, et autres
Publié: (2025)

ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
par: Fan, Ruibo, et autres
Publié: (2026)

MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems
par: Hsia, Samuel, et autres
Publié: (2023)

Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
par: Li, Jiaxi, et autres
Publié: (2025)

Deep Reinforcement Learning based Online Scheduling Policy for Deep Neural Network Multi-Tenant Multi-Accelerator Systems
par: Blanco, Francesco G., et autres
Publié: (2024)

Towards Fair and Firm Real-Time Scheduling in DNN Multi-Tenant Multi-Accelerator Systems via Reinforcement Learning
par: Russo, Enrico, et autres
Publié: (2024)

ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs
par: Yang, Jinwu, et autres
Publié: (2026)

A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardware
par: Zhang, Shichang, et autres
Publié: (2023)

Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs
par: Zhang, Qijun, et autres
Publié: (2026)

BlockAMC: Scalable In-Memory Analog Matrix Computing for Solving Linear Systems
par: Pan, Lunshuai, et autres
Publié: (2024)

Dynamic Simultaneous Multithreaded Architecture
par: Ortiz-Arroyo, Daniel, et autres
Publié: (2024)

Affinity Tailor: Dynamic Locality-Aware Scheduling at Scale
par: Ng, Jin Xin, et autres
Publié: (2026)

Leveraging SIMD for Accelerating Large-number Arithmetic
par: Das, Subhrajit, et autres
Publié: (2026)

The Feasibility of Implementing Large-Scale Transformers on Multi-FPGA Platforms
par: Gao, Yu, et autres
Publié: (2024)

Data-aware Dynamic Execution of Irregular Workloads on Heterogeneous Systems
par: Bai, Zhenyu, et autres
Publié: (2025)

Switchboard: An Open-Source Framework for Modular Simulation of Large Hardware Systems
par: Herbst, Steven, et autres
Publié: (2024)

WaferLLM: Large Language Model Inference at Wafer Scale
par: He, Congjie, et autres
Publié: (2025)

PRESERVE: Prefetching Model Weights and KV-Cache in Distributed LLM Serving
par: Yüzügüler, Ahmet Caner, et autres
Publié: (2025)

LLaMCAT: Optimizing Large Language Model Inference with Cache Arbitration and Throttling
par: Zhou, Zhongchun, et autres
Publié: (2025)

FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
par: Zhu, Zeyu, et autres
Publié: (2024)

A High Energy-Efficiency Multi-core Neuromorphic Architecture for Deep SNN Training
par: Li, Mingjing, et autres
Publié: (2024)

DP-HLS: A High-Level Synthesis Framework for Accelerating Dynamic Programming Algorithms in Bioinformatics
par: Cao, Yingqi, et autres
Publié: (2024)

A Lightweight High-Throughput Collective-Capable NoC for Large-Scale ML Accelerators
par: Colagrande, Luca, et autres
Publié: (2026)