Copertina

MoFa: A Unified Performance Modeling Framework for LLM Pretraining

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhao, Lu, Shi, Rong, Zhang, Shaoqing, Su, Shangchao, Yin, Ziqing, Cui, Zhiyan, Sun, Hongfeng, He, Baoguo, Chen, Yueqiang, Dong, Liang, Li, Xiyuan, Wang, Lingbin, Ma, Lijun, Huang, Qiang, Liu, Ting, Wang, Chong, Wei, Can
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MemFine: Memory-Aware Fine-Grained Scheduling for MoE Training
di: Zhao, Lu, et al.
Pubblicazione: (2025)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
di: Chen, Xing, et al.
Pubblicazione: (2025)

Decentralized Proactive Model Offloading and Resource Allocation for Split and Federated Learning
di: Huang, Binbin, et al.
Pubblicazione: (2024)

ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
di: Li, Haley, et al.
Pubblicazione: (2026)

ReaLB: Real-Time Load Balancing for Multimodal MoE Inference
di: Wang, Yingping, et al.
Pubblicazione: (2026)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
di: Zheng, Size, et al.
Pubblicazione: (2026)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
di: Nie, Xiaonan, et al.
Pubblicazione: (2024)

FedRA: A Random Allocation Strategy for Federated Tuning to Unleash the Power of Heterogeneous Clients
di: Su, Shangchao, et al.
Pubblicazione: (2023)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
di: Wang, Liujianfu, et al.
Pubblicazione: (2025)

Accelerating Distributed MoE Training and Inference with Lina
di: Li, Jiamin, et al.
Pubblicazione: (2022)

HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments
di: He, Yongjun, et al.
Pubblicazione: (2025)

When MoE Meets Blockchain: A Trustworthy Distributed Framework of Large Models
di: Zhu, Weihao, et al.
Pubblicazione: (2025)

Janus: Disaggregating Attention and Experts for Scalable MoE Inference
di: Zhang, Zhexiang, et al.
Pubblicazione: (2025)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
di: Luo, Shuqing, et al.
Pubblicazione: (2024)

Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
di: Luo, Jiajun, et al.
Pubblicazione: (2024)

HarMoEny: Efficient Multi-GPU Inference of MoE Models
di: Doucet, Zachary, et al.
Pubblicazione: (2025)

MOSS: A Large-scale Open Microscopic Traffic Simulation System
di: Zhang, Jun, et al.
Pubblicazione: (2024)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
di: He, Yiyuan, et al.
Pubblicazione: (2025)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
di: Chen, Liangkun, et al.
Pubblicazione: (2025)

Automatic BLAS Offloading on Unified Memory Architecture: A Study on NVIDIA Grace-Hopper
di: Li, Junjie, et al.
Pubblicazione: (2024)

Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems
di: Liu, Guowei, et al.
Pubblicazione: (2026)

Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement
di: Wu, Tian, et al.
Pubblicazione: (2025)

ProMoE: Fast MoE-based LLM Serving using Proactive Caching
di: Song, Xiaoniu, et al.
Pubblicazione: (2024)

Radio Labeling of Strong Prismatic Network With Star
di: Wang, Liming, et al.
Pubblicazione: (2026)

GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
di: Han, Yu, et al.
Pubblicazione: (2025)

MPipeMoE: Memory Efficient MoE for Pre-trained Models with Adaptive Pipeline Parallelism
di: Zhang, Zheng, et al.
Pubblicazione: (2025)

Joint Temporal-Structural Representation Learning for Distributed Fault Discrimination in Microservice Architectures
di: Xue, Yihan, et al.
Pubblicazione: (2026)

Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
di: Shi, Long, et al.
Pubblicazione: (2025)

Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
di: Wang, Chao, et al.
Pubblicazione: (2025)

Argus: Token Aware Distributed LLM Inference Optimization
di: Wu, Panlong, et al.
Pubblicazione: (2025)

Relay Buffer Independent Communication over Pooled HBM for Efficient MoE Inference on Ascend
di: Hu, Tianlun, et al.
Pubblicazione: (2026)

Unifying Partial Synchrony
di: Constantinescu, Andrei, et al.
Pubblicazione: (2024)

MoEntwine: Unleashing the Potential of Wafer-scale Chips for Large-scale Expert Parallel Inference
di: Tang, Xinru, et al.
Pubblicazione: (2025)

Pro-Prophet: A Systematic Load Balancing Method for Efficient Parallel Training of Large-scale MoE Models
di: Wang, Wei, et al.
Pubblicazione: (2024)

Sparse Checkpointing for Fast and Reliable MoE Training
di: Gandhi, Swapnil, et al.
Pubblicazione: (2024)

DiFache: Efficient and Scalable Caching on Disaggregated Memory using Decentralized Coherence
di: Zhang, Hanze, et al.
Pubblicazione: (2025)

Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
di: Liu, Ziming, et al.
Pubblicazione: (2025)

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing
di: Luo, Yizhou, et al.
Pubblicazione: (2024)

Fine-grained MoE Load Balancing with Linear Programming
di: Zhao, Chenqi, et al.
Pubblicazione: (2025)

MoLink: Distributed and Efficient Serving Framework for Large Models
di: Jin, Lewei, et al.
Pubblicazione: (2025)