Image de couverture de livre

MoNTA: Accelerating Mixture-of-Experts Training with Network-Traffc-Aware Parallel Optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Guo, Jingming, Liu, Yan, Meng, Yu, Tao, Zhiwei, Liu, Banglan, Chen, Gang, Li, Xiang
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
par: Jin, Peng, et autres
Publié: (2024)

Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
par: Hua, Yongxiang, et autres
Publié: (2025)

MixGCN: Scalable GCN Training by Mixture of Parallelism and Mixture of Accelerators
par: Wan, Cheng, et autres
Publié: (2025)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
par: Hao, Jiawei, et autres
Publié: (2026)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
par: Yan, Jiaming, et autres
Publié: (2025)

PWC-MoE: Privacy-Aware Wireless Collaborative Mixture of Experts
par: Su, Yang, et autres
Publié: (2025)

Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
par: Liu, Zhili, et autres
Publié: (2024)

MoSEs: Uncertainty-Aware AI-Generated Text Detection via Mixture of Stylistics Experts with Conditional Thresholds
par: Wu, Junxi, et autres
Publié: (2025)

BuddyMoE: Exploiting Expert Redundancy to Accelerate Memory-Constrained Mixture-of-Experts Inference
par: Wang, Yun, et autres
Publié: (2025)

QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts
par: Li, Pingzhi, et autres
Publié: (2024)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
par: Liu, Baihui, et autres
Publié: (2026)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
par: Chen, Yuanteng, et autres
Publié: (2025)

TAG-MoE: Task-Aware Gating for Unified Generative Mixture-of-Experts
par: Xu, Yu, et autres
Publié: (2026)

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
par: Feng, Jiarui, et autres
Publié: (2026)

Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
par: Liu, Boan, et autres
Publié: (2023)

MobileMoE: Scaling On-Device Mixture of Experts
par: Chen, Yanbei, et autres
Publié: (2026)

HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies
par: Du, Zhiying, et autres
Publié: (2025)

PoseMoE: Mixture-of-Experts Network for Monocular 3D Human Pose Estimation
par: Liu, Mengyuan, et autres
Publié: (2025)

Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
par: Luo, Shuqing, et autres
Publié: (2025)

Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction
par: Liu, Shuai, et autres
Publié: (2025)

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
par: Xie, Zhitian, et autres
Publié: (2024)

Speculating Experts Accelerates Inference for Mixture-of-Experts
par: Madan, Vivan, et autres
Publié: (2026)

MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization
par: Zhao, Zhixiong, et autres
Publié: (2026)

MoST: Mixing Speech and Text with Modality-Aware Mixture of Experts
par: Lou, Yuxuan, et autres
Publié: (2026)

BrainNet-MoE: Brain-Inspired Mixture-of-Experts Learning for Neurological Disease Identification
par: Zhang, Jing, et autres
Publié: (2025)

HiMoE: Heterogeneity-Informed Mixture-of-Experts for Fair Spatial-Temporal Forecasting
par: Yu, Shaohan, et autres
Publié: (2024)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
par: Huang, Wei, et autres
Publié: (2025)

DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts
par: Yao, Zelin, et autres
Publié: (2024)

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
par: Wei, Tianwen, et autres
Publié: (2024)

OrdMoE: Preference Alignment via Hierarchical Expert Group Ranking in Multimodal Mixture-of-Experts LLMs
par: Gao, Yuting, et autres
Publié: (2025)

TradExpert: Revolutionizing Trading with Mixture of Expert LLMs
par: Ding, Qianggang, et autres
Publié: (2024)

MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusion
par: Jiang, Ruixiang, et autres
Publié: (2024)

Mixture of Experts (MoE): A Big Data Perspective
par: Gan, Wensheng, et autres
Publié: (2025)

MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services
par: Yu, Dianhai, et autres
Publié: (2022)

MoE-DisCo:Low Economy Cost Training Mixture-of-Experts Models
par: Ye, Xin, et autres
Publié: (2026)

Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts
par: Nguyen, Xuan-Phi, et autres
Publié: (2026)

MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts
par: Lin, Xi Victoria, et autres
Publié: (2024)

Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
par: Fang, Zhiyuan, et autres
Publié: (2025)

MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
par: Jiang, Weisen, et autres
Publié: (2026)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
par: Chu, Kexin, et autres
Publié: (2025)