Copertina

Echo: Simulating Distributed Training At Scale

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Feng, Yicheng, Chen, Yuetao, Chen, Kaiwen, Li, Jingzong, Wu, Tianyuan, Cheng, Peng, Wu, Chuan, Wang, Wei, Ho, Tsung-Yi, Xu, Hong
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Heta: Distributed Training of Heterogeneous Graph Neural Networks
di: Zhong, Yuchen, et al.
Pubblicazione: (2024)

MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
di: Xue, Chunyu, et al.
Pubblicazione: (2026)

ACE-Sync: An Adaptive Cloud-Edge Synchronization Framework for Communication-Efficient Large-Scale Distributed Model Training
di: Yang, Yi, et al.
Pubblicazione: (2025)

PRISM: Probabilistic Runtime Insights and Scalable Performance Modeling for Large-Scale Distributed Training
di: Golden, Alicia, et al.
Pubblicazione: (2025)

BurstEngine: an Efficient Distributed Framework for Training Transformers on Extremely Long Sequences of over 1M Tokens
di: Sun, Ao, et al.
Pubblicazione: (2025)

FALCON: Pinpointing and Mitigating Stragglers for Large-Scale Hybrid-Parallel Training
di: Wu, Tianyuan, et al.
Pubblicazione: (2024)

DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Training
di: Tan, Xin, et al.
Pubblicazione: (2025)

On-the-fly Communication-and-Computing to Enable Representation Learning for Distributed Point Clouds
di: Chen, Xu, et al.
Pubblicazione: (2024)

MegatronApp: Efficient and Comprehensive Management on Distributed LLM Training
di: Zhao, Bohan, et al.
Pubblicazione: (2025)

HexiSeq: Accommodating Long Context Training of LLMs over Heterogeneous Hardware
di: Liang, Yan, et al.
Pubblicazione: (2026)

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
di: Zhang, Shiwei, et al.
Pubblicazione: (2024)

QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
di: Zhao, Juntao, et al.
Pubblicazione: (2024)

MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training
di: Zhao, Juntao, et al.
Pubblicazione: (2025)

Accelerating Distributed MoE Training and Inference with Lina
di: Li, Jiamin, et al.
Pubblicazione: (2022)

Adaptra: Straggler-Resilient Hybrid-Parallel Training with Pipeline Adaptation
di: Wu, Tianyuan, et al.
Pubblicazione: (2025)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
di: Chen, Liangkun, et al.
Pubblicazione: (2025)

SWIFT: Expedited Failure Recovery for Large-scale DNN Training
di: Zhong, Yuchen, et al.
Pubblicazione: (2023)

DiffusionPipe: Training Large Diffusion Models with Efficient Pipelines
di: Tian, Ye, et al.
Pubblicazione: (2024)

RollMux: Phase-Level Multiplexing for Disaggregated RL Post-Training
di: Wu, Tianyuan, et al.
Pubblicazione: (2025)

Optimizing Distributed Training Approaches for Scaling Neural Networks
di: Baligodugula, Vishnu Vardhan, et al.
Pubblicazione: (2025)

DeepServe: Serverless Large Language Model Serving at Scale
di: Hu, Junhao, et al.
Pubblicazione: (2025)

Mosaic: Towards Efficient Training of Multimodal Models with Spatial Resource Multiplexing
di: Wang, Yanbo, et al.
Pubblicazione: (2026)

AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
di: Yan, Ran, et al.
Pubblicazione: (2025)

Optimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing
di: Liu, Mengfan, et al.
Pubblicazione: (2025)

DiT-HC: Enabling Efficient Training of Visual Generation Model DiT on HPC-oriented CPU Cluster
di: Zhang, Jinxiao, et al.
Pubblicazione: (2026)

Efficient Distributed MLLM Training with Cornstarch
di: Jang, Insu, et al.
Pubblicazione: (2025)

Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training
di: Xu, Guanbin, et al.
Pubblicazione: (2026)

A Study on the Performance of Distributed Training of Data-driven CFD Simulations
di: Iserte, Sergio, et al.
Pubblicazione: (2026)

MTGenRec: An Efficient Distributed Training System for Generative Recommendation Models in Meituan
di: Wang, Yuxiang, et al.
Pubblicazione: (2025)

LuWu: An End-to-End In-Network Out-of-Core Optimizer for 100B-Scale Model-in-Network Data-Parallel Training on Distributed GPUs
di: Sun, Mo, et al.
Pubblicazione: (2024)

Scaling Large-scale GNN Training to Thousands of Processors on CPU-based Supercomputers
di: Zhuang, Chen, et al.
Pubblicazione: (2024)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
di: Li, Suyi, et al.
Pubblicazione: (2024)

Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
di: Chen, Fahao, et al.
Pubblicazione: (2024)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
di: Zhang, WenZheng, et al.
Pubblicazione: (2024)

Will LLMs Scaling Hit the Wall? Breaking Barriers via Distributed Resources on Massive Edge Devices
di: Shen, Tao, et al.
Pubblicazione: (2025)

ModTrans: Translating Real-world Models for Distributed Training Simulator
di: Lyu, Yi
Pubblicazione: (2026)

Spatiotemporal Traffic Prediction in Distributed Backend Systems via Graph Neural Networks
di: Qiu, Zhimin, et al.
Pubblicazione: (2025)

Hybrid Dual-Batch and Cyclic Progressive Learning for Efficient Distributed Training
di: Lu, Kuan-Wei, et al.
Pubblicazione: (2025)

TurboGR: An Accelerated Training System for Large-Scale Generative Recommendation
di: Chai, Huichao, et al.
Pubblicazione: (2026)

A Survey of End-to-End Modeling for Distributed DNN Training: Workloads, Simulators, and TCO
di: Svedas, Jonas, et al.
Pubblicazione: (2025)