Image de couverture de livre

Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yao, Jinghan, Anthony, Quentin, Shafi, Aamir, Subramoni, Hari, K., Dhabaleswar, Panda
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning
par: Xu, Lang, et autres
Publié: (2025)

Accelerating Large Language Model Training with Hybrid GPU-based Compression
par: Xu, Lang, et autres
Publié: (2024)

Demystifying the Communication Characteristics for Distributed Transformer Models
par: Anthony, Quentin, et autres
Publié: (2024)

The Case for Co-Designing Model Architectures with Hardware
par: Anthony, Quentin, et autres
Publié: (2024)

Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer
par: Yao, Jinghan, et autres
Publié: (2024)

From Skew to Symmetry: Node-Interconnect Multi-Path Balancing with Execution-time Planning for Modern GPU Clusters
par: Yao, Jinghan, et autres
Publié: (2026)

MAC-Attention: a Match-Amend-Complete Scheme for Fast and Accurate Attention Computation
par: Yao, Jinghan, et autres
Publié: (2026)

Characterizing Communication Patterns in Distributed Large Language Model Inference
par: Xu, Lang, et autres
Publié: (2025)

BlackMamba: Mixture of Experts for State-Space Models
par: Anthony, Quentin, et autres
Publié: (2024)

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
par: Hankendi, Can, et autres
Publié: (2026)

A Survey on Inference Optimization Techniques for Mixture of Experts Models
par: Liu, Jiacheng, et autres
Publié: (2024)

Accelerating MoE Model Inference with Expert Sharding
par: Balmau, Oana, et autres
Publié: (2025)

MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services
par: Yu, Dianhai, et autres
Publié: (2022)

Keep Your Friends Close: Leveraging Affinity Groups to Accelerate AI Inference Workflows
par: Garrett, Thiago, et autres
Publié: (2023)

Mixture-of-Schedulers: An Adaptive Scheduling Agent as a Learned Router for Expert Policies
par: Wang, Xinbo, et autres
Publié: (2025)

ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
par: Shen, Zixu, et autres
Publié: (2025)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
par: Wang, Zhibin, et autres
Publié: (2025)

Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
par: Pan, Xinglin, et autres
Publié: (2025)

Optimal Transport Aggregation for Distributed Mixture-of-Experts
par: Chamroukhi, Faïcel, et autres
Publié: (2023)

Utility-Driven Speculative Decoding for Mixture-of-Experts
par: Saxena, Anish, et autres
Publié: (2025)

Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models
par: Kamahori, Keisuke, et autres
Publié: (2024)

CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
par: Suo, Jiashun, et autres
Publié: (2025)

Mixture of Experts with Mixture of Precisions for Tuning Quality of Service
par: Imani, HamidReza, et autres
Publié: (2024)

Scalable Pretraining of Large Mixture of Experts Language Models on Aurora Super Computer
par: Vooturi, Dharma Teja, et autres
Publié: (2026)

SpaceMoE: Realizing Distributed Mixture-of-Experts Inference over Space Networks
par: Wang, Zhanwei, et autres
Publié: (2026)

PiKV: KV Cache Management System for Mixture of Experts
par: Liu, Dong, et autres
Publié: (2025)

Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts
par: Zhang, Shulai, et autres
Publié: (2025)

RaMP: Runtime-Aware Megakernel Polymorphism for Mixture-of-Experts
par: Sharma, Vyom, et autres
Publié: (2026)

Scattered Mixture-of-Experts Implementation
par: Tan, Shawn, et autres
Publié: (2024)

From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing
par: Shahout, Rana, et autres
Publié: (2025)

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs
par: Chen, Aodong, et autres
Publié: (2023)

Verify Distributed Deep Learning Model Implementation Refinement with Iterative Relation Inference
par: Wang, Zhanghan, et autres
Publié: (2025)

LLM as HPC Expert: Extending RAG Architecture for HPC Data
par: Miyashita, Yusuke, et autres
Publié: (2024)

HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference
par: Lin, Haoran, et autres
Publié: (2025)

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
par: Xue, Fuzhao, et autres
Publié: (2024)

Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments
par: Liu, Junming, et autres
Publié: (2025)

Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference
par: He, Zifan, et autres
Publié: (2026)

SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting
par: Xu, Jiaming, et autres
Publié: (2025)

Accelerating LLM Inference with Precomputed Query Storage
par: Park, Jay H., et autres
Publié: (2025)

Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks
par: Li, Haiyuan, et autres
Publié: (2026)