Image de couverture de livre

veScale: Consistent and Efficient Tensor Programming with Eager-Mode SPMD

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Youjie, Wan, Cheng, Lin, Zhiqi, Zhu, Hongyu, Yang, Jiacheng, Song, Ziang, Di, Xinyi, Wu, Jiawei, Shu, Huiyao, Bao, Wenlei, Peng, Yanghua, Lin, Haibin, Chang, Li-Wen
Format:	Preprint
Publié:	2025
Sujets:	Programming Languages Distributed, Parallel, and Cluster Computing Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

veScale-FSDP: Flexible and High-Performance FSDP at Scale
par: Wang, Zezhou, et autres
Publié: (2026)

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
par: Zhang, Shiwei, et autres
Publié: (2024)

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
par: Ma, Qianli, et autres
Publié: (2025)

LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
par: Zhao, Juntao, et autres
Publié: (2024)

QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
par: Zhao, Juntao, et autres
Publié: (2024)

Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
par: Zheng, Size, et autres
Publié: (2025)

PartIR: Composing SPMD Partitioning Strategies for Machine Learning
par: Alabed, Sami, et autres
Publié: (2024)

TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives
par: Zheng, Size, et autres
Publié: (2025)

Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
par: Feng, Weiqi, et autres
Publié: (2024)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
par: Jin, Chao, et autres
Publié: (2025)

Balancing Pipeline Parallelism with Vocabulary Parallelism
par: Yeung, Man Tsung, et autres
Publié: (2024)

FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
par: Chang, Li-Wen, et autres
Publié: (2024)

Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts
par: Zhang, Shulai, et autres
Publié: (2025)

PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
par: Wan, Xinyi, et autres
Publié: (2025)

Revisiting Parameter Server in LLM Post-Training
par: Wan, Xinyi, et autres
Publié: (2026)

Laminar: A Scalable Asynchronous RL Post-Training Framework
par: Sheng, Guangming, et autres
Publié: (2025)

HybridFlow: A Flexible and Efficient RLHF Framework
par: Sheng, Guangming, et autres
Publié: (2024)

Zero Bubble Pipeline Parallelism
par: Qi, Penghui, et autres
Publié: (2023)

Pipeline Parallelism with Controllable Memory
par: Qi, Penghui, et autres
Publié: (2024)

MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
par: Xue, Chunyu, et autres
Publié: (2026)

SW-TNC : Reaching the Most Complex Random Quantum Circuit via Tensor Network Contraction
par: Chen, Yaojian, et autres
Publié: (2025)

Paralleling and Accelerating Arc Consistency Enforcement with Recurrent Tensor Computations
par: Yang, Mingqi
Publié: (2024)

Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching
par: Ruan, Chaoyi, et autres
Publié: (2025)

MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training
par: Zhao, Juntao, et autres
Publié: (2025)

Synergistic Tensor and Pipeline Parallelism
par: Qi, Mengshi, et autres
Publié: (2025)

NineToothed: A Triton-Based High-Level Domain-Specific Language for Machine Learning
par: Huang, Jiacheng, et autres
Publié: (2025)

Verify Distributed Deep Learning Model Implementation Refinement with Iterative Relation Inference
par: Wang, Zhanghan, et autres
Publié: (2025)

A Dynamic Mode Decomposition Approach for Decentralized Spectral Clustering of Graphs
par: Zhu, Hongyu, et autres
Publié: (2022)

Strongly-Consistent Distributed Discrete-event Systems
par: Donovan, Peter, et autres
Publié: (2024)

Algorithmics and Complexity of Cost-Driven Task Offloading with Submodular Optimization in Edge-Cloud Environments
par: Guo, Longkun, et autres
Publié: (2024)

Federated Self-supervised Domain Generalization for Label-efficient Polyp Segmentation
par: Tan, Xinyi, et autres
Publié: (2025)

DIAL: Decentralized I/O AutoTuning via Learned Client-side Local Metrics for Parallel File System
par: Rashid, Md Hasanur, et autres
Publié: (2026)

Delta Tensor: Efficient Vector and Tensor Storage in Delta Lake
par: Bao, Zhiwei, et autres
Publié: (2024)

Dual UAV Cluster-Assisted Maritime Physical Layer Secure Communications via Collaborative Beamforming
par: Huang, Jiawei, et autres
Publié: (2024)

Enabling Dynamic Sparsity in Quantized LLM Inference
par: Wang, Rongxiang, et autres
Publié: (2025)

Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances
par: Duan, Jiangfei, et autres
Publié: (2024)

Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generator
par: Luo, Kangyang, et autres
Publié: (2024)

Rubick: Exploiting Job Reconfigurability for Deep Learning Cluster Scheduling
par: Zhang, Xinyi, et autres
Publié: (2024)

Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
par: Srivatsa, Vikranth, et autres
Publié: (2026)

Vortex: Efficient Sample-Free Dynamic Tensor Program Optimization via Hardware-aware Strategy Space Hierarchization
par: Zhou, Yangjie, et autres
Publié: (2024)