Image de couverture de livre

veScale-FSDP: Flexible and High-Performance FSDP at Scale

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Zezhou, Li, Youjie, Lin, Zhiqi, Yang, Jiacheng, Xie, Cong, Feng, Guanyu, Zhong, Zheng, Huang, Ziyue, Zhu, Hongyu, Zhang, Zhi, Peng, Yanghua, Liu, Xin
Format:	Preprint
Publié:	2026
Sujets:	Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

veScale: Consistent and Efficient Tensor Programming with Eager-Mode SPMD
par: Li, Youjie, et autres
Publié: (2025)

SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile
par: Zhang, Ruisi, et autres
Publié: (2024)

Performance Characterization of Distributed Deep Learning Strategies: A Quantitative Evaluation of DDP, FSDP, and Parameter Server Architectures on GPU Clusters
par: Ovi, Md Sultanul Islam
Publié: (2025)

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
par: Ma, Qianli, et autres
Publié: (2025)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
par: Jin, Chao, et autres
Publié: (2025)

MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
par: Xue, Chunyu, et autres
Publié: (2026)

Exploring Uncore Frequency Scaling for Heterogeneous Computing
par: Zheng, Zhong, et autres
Publié: (2025)

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
par: Jiang, Ziheng, et autres
Publié: (2024)

PRISM: Dynamic Primitive-Based Forecasting for Large-Scale GPU Cluster Workloads
par: Wu, Xin, et autres
Publié: (2026)

MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training
par: Zhao, Juntao, et autres
Publié: (2025)

FlexKV: Flexible Index Offloading for Memory-Disaggregated Key-Value Store
par: Hu, Zhisheng, et autres
Publié: (2025)

Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
par: Feng, Weiqi, et autres
Publié: (2024)

Benchmarking the Performance of Large Language Models on the Cerebras Wafer Scale Engine
par: Zhang, Zuoning, et autres
Publié: (2024)

Comparing Cross-Platform Performance via Node-to-Node Scaling Studies
par: Weiss, Kenneth, et autres
Publié: (2025)

madupite: A High-Performance Distributed Solver for Large-Scale Markov Decision Processes
par: Gargiani, Matilde, et autres
Publié: (2025)

PRISM: Probabilistic Runtime Insights and Scalable Performance Modeling for Large-Scale Distributed Training
par: Golden, Alicia, et autres
Publié: (2025)

StatuScale: Status-aware and Elastic Scaling Strategy for Microservice Applications
par: Wen, Linfeng, et autres
Publié: (2024)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
par: Zhang, WenZheng, et autres
Publié: (2024)

DeepServe: Serverless Large Language Model Serving at Scale
par: Hu, Junhao, et autres
Publié: (2025)

EcoShift: Performance-Aware Power Management for Power-Constrained Heterogeneous Systems
par: Zheng, Zhong, et autres
Publié: (2026)

Data Caching for Enterprise-Grade Petabyte-Scale OLAP
par: Tang, Chunxu, et autres
Publié: (2024)

λScale: Enabling Fast Scaling for Serverless Large Language Model Inference
par: Yu, Minchen, et autres
Publié: (2025)

ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
par: Li, Haley, et autres
Publié: (2026)

M$^2$-MFP: A Multi-Scale and Multi-Level Memory Failure Prediction Framework for Reliable Cloud Infrastructure
par: Xie, Hongyi, et autres
Publié: (2025)

A Tale of Two Scales: Reconciling Horizontal and Vertical Scaling for Inference Serving Systems
par: Razavi, Kamran, et autres
Publié: (2024)

Optimizing High-Throughput Distributed Data Pipelines for Reproducible Deep Learning at Scale
par: Mittal, Kashish, et autres
Publié: (2026)

PolarStore: High-Performance Data Compression for Large-Scale Cloud-Native Databases
par: Hu, Qingda, et autres
Publié: (2025)

Case Study: Performance Analysis of a Virtualized XRootD Frontend in Large-Scale WAN Transfers
par: da Silva, J M, et autres
Publié: (2026)

FAIR Ecosystems for Science at Scale
par: Wilkinson, Sean R., et autres
Publié: (2025)

Scaling MPI Applications on Aurora
par: Ibeid, Huda, et autres
Publié: (2025)

Scaling Real-Time Traffic Analytics on Edge-Cloud Fabrics for City-Scale Camera Networks
par: Sharma, Akash, et autres
Publié: (2026)

Steering a Fleet: Adaptation for Large-Scale, Workflow-Based Experiments
par: Pruyne, Jim, et autres
Publié: (2024)

Deep Learning-Enabled Supercritical Flame Simulation at Detailed Chemistry and Real-Fluid Accuracy Towards Trillion-Cell Scale
par: Guo, Zhuoqiang, et autres
Publié: (2025)

SDSL-Solver: Scalable Distributed Sparse Linear Solvers for Large-Scale Interior Point Methods
par: Yang, Shaofeng, et autres
Publié: (2026)

MPI-Q: A Message Communication Library for Large-Scale Classical-Quantum Heterogeneous Hybrid Distributed Computing
par: Wang, Feng, et autres
Publié: (2026)

ScalePool: Hybrid XLink-CXL Fabric for Composable Resource Disaggregation in Unified Scale-up Domains
par: Woo, Hyein, et autres
Publié: (2025)

Scaling atomic ordering in shared memory
par: Martignetti, Lorenzo, et autres
Publié: (2025)

Barycentric Coded Distributed Computing with Flexible Recovery Threshold for Collaborative Mobile Edge Computing
par: Qiu, Houming, et autres
Publié: (2025)

EvoSort: A Genetic-Algorithm-Based Adaptive Parallel Sorting Framework for Large-Scale High Performance Computing
par: Raj, Shashank, et autres
Publié: (2025)

Understanding Inference Scaling for LLMs: Bottlenecks, Trade-offs, and Performance Principles
par: Arif, Moiz, et autres
Publié: (2026)