Image de couverture de livre

Robust LLM Training Infrastructure at ByteDance

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wan, Borui, Liu, Gaohong, Song, Zuquan, Wang, Jun, Zhang, Yun, Sheng, Guangming, Wang, Shuguang, Wei, Houmin, Wang, Chenyuan, Lou, Weiqiang, Yang, Xi, Zhang, Mofan, Jiang, Kaihua, Ren, Cheng, Zhi, Xiaoyun, Yu, Menghan, Nan, Zhe, Zheng, Zhuolin, Zhong, Baoquan, Wang, Qinlong, Yu, Huan, Chi, Jinxin, Zhang, Wang, Li, Yuhan, Du, Zixian, Zhao, Sida, Zhang, Yongqiang, Tang, Jingzhe, Liu, Zherui, Wu, Chuan, Peng, Yanghua, Lin, Haibin, Xiao, Wencong, Liu, Xin, Xiang, Liang
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development
par: Wan, Borui, et autres
Publié: (2024)

Mycroft: Tracing Dependencies in Collective Communication Towards Reliable LLM Training
par: Deng, Yangtao, et autres
Publié: (2025)

StreamShield: A Production-Proven Resiliency Solution for Apache Flink at ByteDance
par: Fang, Yong, et autres
Publié: (2026)

Understanding Stragglers in Large Model Training Using What-if Analysis
par: Lin, Jinkun, et autres
Publié: (2025)

ByteCard: Enhancing ByteDance's Data Warehouse with Learned Cardinality Estimation
par: Han, Yuxing, et autres
Publié: (2024)

ByteHouse: ByteDance's Cloud-Native Data Warehouse for Real-Time Multimodal Data Analytics
par: Han, Yuxing, et autres
Publié: (2026)

Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
par: He, Guoliang, et autres
Publié: (2025)

BootSeer: Analyzing and Mitigating Initialization Bottlenecks in Large-Scale LLM Training
par: Li, Rui, et autres
Publié: (2025)

Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets
par: Feng, Jiashi, et autres
Publié: (2025)

Minder: Faulty Machine Detection for Large-scale Distributed Model Training
par: Deng, Yangtao, et autres
Publié: (2024)

Applications and Progress of Machine Learning Techniques in the Ladle Furnace Refining Process: A Review
par: Zixian Wang, et autres
Publié: (2024)

Dual Analysis of Continuous-time Economic Dispatch and Its Price Implications
par: Zhang, Menghan, et autres
Publié: (2026)

Dance Any Beat: Blending Beats with Visuals in Dance Video Generation
par: Wang, Xuanchen, et autres
Publié: (2024)

Defect‐Engineered Ni‐MOF‐74 Catalyst for Boosting the Dielectric Barrier Discharge Plasma‐Assisted Ammonia Synthesis
par: Qinlong Song, et autres
Publié: (2025)

FPI-Det: a face--phone Interaction Dataset for phone-use detection and understanding
par: Gao, Jianqin, et autres
Publié: (2025)

Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models
par: Li, Chaozhuo, et autres
Publié: (2025)

WoW: A Window-to-Window Incremental Index for Range-Filtering Approximate Nearest Neighbor Search
par: Wang, Ziqi, et autres
Publié: (2025)

Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives
par: Li, Ronghui, et autres
Publié: (2024)

FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generation
par: Yang, Kaixing, et autres
Publié: (2025)

A Glitch Detection and Removal Method for Three‐Component Seismic Data From Mars Based On Deep Learning
par: Jiangjie Zhang, et autres
Publié: (2025)

Orthogonalized Policy Optimization:Policy Optimization as Orthogonal Projection in Hilbert Space
par: Zixian, Wang
Publié: (2026)

ADPO: Anchored Direct Preference Optimization
par: Zixian, Wang
Publié: (2025)

Group Orthogonalized Policy Optimization:Group Policy Optimization as Orthogonal Projection in Hilbert Space
par: Zixian, Wang
Publié: (2026)

APO: Alpha-Divergence Preference Optimization
par: Zixian, Wang
Publié: (2025)

Attention Saturation and Gradient Suppression at Inflection Layers: Diagnosing and Mitigating Bottlenecks in Transformer Adaptation
par: Zixian, Wang
Publié: (2025)

Orthogonal Quadratic Complements for Vision Transformer Feed-Forward Networks
par: Zixian, Wang
Publié: (2026)

OpenDance: Multimodal Controllable 3D Dance Generation with Large-scale Internet Data
par: Zhang, Jinlu, et autres
Publié: (2025)

Influence of pore water pressure on concrete creep and a creep model considering the effect of cohesion and internal friction angle
par: Wenbo Liu, et autres
Publié: (2024)

A Novel Clinical Score Integrating Low‐Voltage Zones and Biomarkers Predicts Atrial Fibrillation Recurrence Post‐Ablation
par: Ying Han, et autres
Publié: (2025)

Measuring all the noises of LLM Evals
par: Wang, Sida
Publié: (2025)

An analysis on efficacy of applying β‐elemene intervention on chemically ‐induced tongue lesions using SAM algorithm
par: Feng Liu, et autres
Publié: (2024)

InterDance:Reactive 3D Dance Generation with Realistic Duet Interactions
par: Li, Ronghui, et autres
Publié: (2024)

A Multiscale Transformer Model for Long Time Series Forecasting Based on Discrete Wavelet Transform and Residual Learning Modules
par: Menghan Li, et autres
Publié: (2025)

Preparation of Gradient Nanostructured Cu–Al Alloy Matrix and Evolution Microstructure and Properties
par: Xiaoxian Li, et autres
Publié: (2024)

First-principles study of photovoltaic and thermoelectric properties of AgBiSCl2
par: Wang, Sihang, et autres
Publié: (2026)

Quantum forgery attacks against OTR structures based on Simon's algorithm
par: Liu, Wenjie, et autres
Publié: (2023)

Quantum All-Subkeys-Recovery Attacks on 6-round Feistel-2* Structure Based on Multi-Equations Quantum Claw Finding
par: Liu, Wenjie, et autres
Publié: (2023)

Conditional Generative Modeling for Enhanced Credit Risk Management in Supply Chain Finance
par: Zhang, Qingkai, et autres
Publié: (2025)

Conditional Generative Modeling for Enhanced Credit Risk Management in Supply Chain Finance
par: Qingkai Zhang, et autres
Publié: (2025)

Performance Forecast and Analytic for Sports Dance Athletes Using a Hybrid Method
par: Qiaohui Wang, et autres
Publié: (2025)