Imagen de Portada

LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yu, Enda, Dong, Dezun, Zhang, Zhaoning, Bai, Zhe, Yang, Weiling, Wang, Haojie, Li, Dongsheng, Wu, Yongwei, Liao, Xiangke
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Nezha: Breaking Multi-Rail Network Barriers for Distributed DNN Training
por: Yu, Enda, et al.
Publicado: (2024)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
por: Liu, Baihui, et al.
Publicado: (2026)

Multi-Layer Scheduling for MoE-Based LLM Reasoning
por: Sun, Yifan, et al.
Publicado: (2026)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
por: Qian, Yulei, et al.
Publicado: (2024)

From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
por: Lee, Gunjun, et al.
Publicado: (2025)

LExI: Layer-Adaptive Active Experts for Efficient MoE Model Inference
por: Chitty-Venkata, Krishna Teja, et al.
Publicado: (2025)

Demystifying ARM SME to Optimize General Matrix Multiplications
por: Deng, Chencheng, et al.
Publicado: (2025)

A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systems
por: Wu, Qi, et al.
Publicado: (2026)

Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling
por: Ma, Songchen, et al.
Publicado: (2026)

MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
por: Xu, Tairan, et al.
Publicado: (2025)

Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
por: Pan, Xinglin, et al.
Publicado: (2025)

XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference
por: Vankov, Daniil, et al.
Publicado: (2026)

Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
por: Gupta, Vima, et al.
Publicado: (2024)

Static Batching of Irregular Workloads on GPUs: Framework and Application to Efficient MoE Model Inference
por: Li, Yinghan, et al.
Publicado: (2025)

Synergistic Intra- and Cross-Layer Regularization Losses for MoE Expert Specialization
por: Hu, Rizhen, et al.
Publicado: (2026)

HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
por: Zhong, Shuzhang, et al.
Publicado: (2025)

ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
por: Shen, Zixu, et al.
Publicado: (2025)

Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling
por: Li, Yan, et al.
Publicado: (2025)

Priority-Aware Preemptive Scheduling for Mixed-Priority Workloads in MoE Inference
por: Siavashi, Mohammad, et al.
Publicado: (2025)

D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
por: Wang, Haodong, et al.
Publicado: (2025)

GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
por: Han, Yu, et al.
Publicado: (2025)

MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
por: Xue, Leyang, et al.
Publicado: (2024)

DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference
por: Zhang, Yujie, et al.
Publicado: (2024)

HarMoEny: Efficient Multi-GPU Inference of MoE Models
por: Doucet, Zachary, et al.
Publicado: (2025)

SERE: Similarity-based Expert Re-routing for Efficient Batch Decoding in MoE Models
por: Wu, Juntong, et al.
Publicado: (2026)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
por: Li, Shuhuai, et al.
Publicado: (2026)

Strategic Joining and Optimal Pricing in a Single‐Server Batch Arrival Queue With Different Information of Batch Size
por: Kaili Li, et al.
Publicado: (2025)

MoETTA: Test-Time Adaptation Under Mixed Distribution Shifts with MoE-LayerNorm
por: Fan, Xiao, et al.
Publicado: (2025)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
por: Cao, Shiyi, et al.
Publicado: (2024)

Fast MoE Inference via Predictive Prefetching and Expert Replication
por: Jyothish, Ankit, et al.
Publicado: (2026)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
por: Cao, Mingyu, et al.
Publicado: (2024)

Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers
por: Liao, Zhu, et al.
Publicado: (2024)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
por: Chen, Liangkun, et al.
Publicado: (2025)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
por: Wang, Liujianfu, et al.
Publicado: (2025)

ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Scheduling
por: Yang, Yuchen, et al.
Publicado: (2026)

SecMoE: Communication-Efficient Secure MoE Inference via Select-Then-Compute
por: Shen, Bowen, et al.
Publicado: (2026)

Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
por: Sun, Xun, et al.
Publicado: (2026)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
por: Ma, Songkai, et al.
Publicado: (2025)

Telemetry-Based Server Selection in the Quantum Internet via Cross-Layer Runtime Estimation
por: Nagai, Masaki, et al.
Publicado: (2026)

Remoe: Towards Efficient and Low-Cost MoE Inference in Serverless Computing
por: Liu, Wentao, et al.
Publicado: (2025)