Copertina

Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Fang, Zhiyuan, Huang, Yuegui, Hong, Zicong, Lyu, Yufeng, Chen, Wuhui, Yu, Yue, Yu, Fan, Zheng, Zibin
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate
di: Fang, Zhiyuan, et al.
Pubblicazione: (2025)

DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge
di: Huang, Yuegui, et al.
Pubblicazione: (2026)

Krul: Efficient State Restoration for Multi-turn Conversations with Dynamic Cross-layer KV Sharing
di: Wen, Junyi, et al.
Pubblicazione: (2025)

DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
di: Bai, Sikai, et al.
Pubblicazione: (2025)

Training and Serving System of Foundation Models: A Comprehensive Survey
di: Zhou, Jiahang, et al.
Pubblicazione: (2024)

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
di: Zhao, Adrian, et al.
Pubblicazione: (2026)

ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
di: He, Xin, et al.
Pubblicazione: (2024)

UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
di: Huang, Minbin, et al.
Pubblicazione: (2026)

Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
di: Bambhaniya, Abhimanyu, et al.
Pubblicazione: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
di: Liu, Baihui, et al.
Pubblicazione: (2026)

PreMoE: Proactive Inference for Efficient Mixture-of-Experts
di: Pei, Zehua, et al.
Pubblicazione: (2025)

Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Experts
di: He, Shwai, et al.
Pubblicazione: (2025)

Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling
di: Ma, Songchen, et al.
Pubblicazione: (2026)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
di: Yan, Jiaming, et al.
Pubblicazione: (2025)

GriDB: Scaling Blockchain Database via Sharding and Off-Chain Cross-Shard Mechanism
di: Hong, Zicong, et al.
Pubblicazione: (2024)

Speculating Experts Accelerates Inference for Mixture-of-Experts
di: Madan, Vivan, et al.
Pubblicazione: (2026)

HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference
di: Lin, Haoran, et al.
Pubblicazione: (2025)

Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
di: Gupta, Vima, et al.
Pubblicazione: (2024)

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
di: Liu, Enshu, et al.
Pubblicazione: (2024)

MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
di: Zhao, Yushu, et al.
Pubblicazione: (2025)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
di: Chu, Kexin, et al.
Pubblicazione: (2025)

CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert Routing
di: Zhou, Yifan, et al.
Pubblicazione: (2025)

RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs
di: Xu, Zhiyuan, et al.
Pubblicazione: (2026)

SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inference
di: Chen, Qian, et al.
Pubblicazione: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
di: He, Yifei, et al.
Pubblicazione: (2025)

Multi-Type Context-Aware Conversational Recommender Systems via Mixture-of-Experts
di: Zou, Jie, et al.
Pubblicazione: (2025)

Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts
di: Zhang, Xue, et al.
Pubblicazione: (2025)

Multi-Head Mixture-of-Experts
di: Wu, Xun, et al.
Pubblicazione: (2024)

Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
di: Fan, Zehao, et al.
Pubblicazione: (2025)

Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
di: Jiang, Fan, et al.
Pubblicazione: (2026)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
di: Dwivedi, Chaitanya, et al.
Pubblicazione: (2026)

Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction
di: Liu, Shuai, et al.
Pubblicazione: (2025)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
di: Gu, Naibin, et al.
Pubblicazione: (2025)

DeRS: Towards Extremely Efficient Upcycled Mixture-of-Experts Models
di: Huang, Yongqi, et al.
Pubblicazione: (2025)

Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts
di: Bai, Li, et al.
Pubblicazione: (2025)

FairBatching: Fairness-Aware Batch Formation for LLM Inference
di: Lyu, Hongtao, et al.
Pubblicazione: (2025)

BuddyMoE: Exploiting Expert Redundancy to Accelerate Memory-Constrained Mixture-of-Experts Inference
di: Wang, Yun, et al.
Pubblicazione: (2025)

Mixture of Cache-Conditional Experts for Efficient Mobile Device Inference
di: Skliar, Andrii, et al.
Pubblicazione: (2024)

Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUs
di: Benazir, Afsara, et al.
Pubblicazione: (2026)

Prediction-powered Inference by Mixture of Experts
di: Gu, Yanwu, et al.
Pubblicazione: (2026)