Copertina

BurstEngine: an Efficient Distributed Framework for Training Transformers on Extremely Long Sequences of over 1M Tokens

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Sun, Ao, Zhao, Weilin, Han, Xu, Yang, Cheng, Liu, Zhiyuan, Shi, Chuan, sun, Maosong
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
di: Sun, Ao, et al.
Pubblicazione: (2024)

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training
di: Sun, Ao, et al.
Pubblicazione: (2024)

HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
di: Zhang, Geng, et al.
Pubblicazione: (2025)

LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
di: Gu, Diandian, et al.
Pubblicazione: (2024)

Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
di: Chen, Fahao, et al.
Pubblicazione: (2024)

Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
di: Shi, Long, et al.
Pubblicazione: (2025)

HexiSeq: Accommodating Long Context Training of LLMs over Heterogeneous Hardware
di: Liang, Yan, et al.
Pubblicazione: (2026)

StarTrail: Concentric Ring Sequence Parallelism for Efficient Near-Infinite-Context Transformer Model Training
di: Liu, Ziming, et al.
Pubblicazione: (2024)

SPPO:Efficient Long-sequence LLM Training via Adaptive Sequence Pipeline Parallel Offloading
di: Chen, Qiaoling, et al.
Pubblicazione: (2025)

MTGenRec: An Efficient Distributed Training System for Generative Recommendation Models in Meituan
di: Wang, Yuxiang, et al.
Pubblicazione: (2025)

DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training
di: Wang, Zhixin, et al.
Pubblicazione: (2025)

Heta: Distributed Training of Heterogeneous Graph Neural Networks
di: Zhong, Yuchen, et al.
Pubblicazione: (2024)

FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training
di: Qi, Shuyao, et al.
Pubblicazione: (2026)

AME: An Efficient Heterogeneous Agentic Memory Engine for Smartphones
di: Zhao, Xinkui, et al.
Pubblicazione: (2025)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
di: Zhang, WenZheng, et al.
Pubblicazione: (2024)

Bandwidth-Aware and Cost-Efficient Pipeline Parallel Scheduling in Geo-Distributed LLM Training
di: Zhang, Han, et al.
Pubblicazione: (2026)

Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
di: Strati, Foteini, et al.
Pubblicazione: (2025)

Efficient Distributed MLLM Training with Cornstarch
di: Jang, Insu, et al.
Pubblicazione: (2025)

Galvatron: Automatic Distributed Training for Large Transformer Models
di: Gumaan, Esmail
Pubblicazione: (2025)

DiffusionPipe: Training Large Diffusion Models with Efficient Pipelines
di: Tian, Ye, et al.
Pubblicazione: (2024)

Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference
di: Shyam, Vasu, et al.
Pubblicazione: (2026)

On the Performance and Memory Footprint of Distributed Training: An Empirical Study on Transformers
di: Lu, Zhengxian, et al.
Pubblicazione: (2024)

MoC-System: Efficient Fault Tolerance for Sparse Mixture-of-Experts Model Training
di: Cai, Weilin, et al.
Pubblicazione: (2024)

ACE-Sync: An Adaptive Cloud-Edge Synchronization Framework for Communication-Efficient Large-Scale Distributed Model Training
di: Yang, Yi, et al.
Pubblicazione: (2025)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
di: Wang, Shiju, et al.
Pubblicazione: (2025)

AGAThA: Fast and Efficient GPU Acceleration of Guided Sequence Alignment for Long Read Mapping
di: Park, Seongyeon, et al.
Pubblicazione: (2024)

MegatronApp: Efficient and Comprehensive Management on Distributed LLM Training
di: Zhao, Bohan, et al.
Pubblicazione: (2025)

Distributed Continuous Range-Skyline Query Monitoring over the Internet of Mobile Things
di: Lai, Chuan-Chi, et al.
Pubblicazione: (2019)

Efficient Training of Large Language Models on Distributed Infrastructures: A Survey
di: Duan, Jiangfei, et al.
Pubblicazione: (2024)

CoCoDiff: Optimizing Collective Communications for Distributed Diffusion Transformer Inference Under Ulysses Sequence Parallelism
di: Ma, Bin, et al.
Pubblicazione: (2026)

MoLink: Distributed and Efficient Serving Framework for Large Models
di: Jin, Lewei, et al.
Pubblicazione: (2025)

Survey on Token-Based Distributed MutualExclusion Algorithms
di: Tohidi, Elahe, et al.
Pubblicazione: (2025)

Argus: Token Aware Distributed LLM Inference Optimization
di: Wu, Panlong, et al.
Pubblicazione: (2025)

A Flexible Programmable Pipeline Parallelism Framework for Efficient DNN Training
di: Jiang, Lijuan, et al.
Pubblicazione: (2025)

Towards Affordable, Adaptive and Automatic GNN Training on CPU-GPU Heterogeneous Platforms
di: Qiao, Tong, et al.
Pubblicazione: (2025)

RapidGNN: Communication Efficient Large-Scale Distributed Training of Graph Neural Networks
di: Niam, Arefin, et al.
Pubblicazione: (2025)

CondenseGraph: Communication-Efficient Distributed GNN Training via On-the-Fly Graph Condensation
di: Zhang, Zizhao, et al.
Pubblicazione: (2026)

GreenDyGNN: Runtime-Adaptive Energy-Efficient Communication for Distributed GNN Training
di: Niam, Arefin, et al.
Pubblicazione: (2026)

SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines
di: Cheng, Ke, et al.
Pubblicazione: (2024)

Cloudless-Training: A Framework to Improve Efficiency of Geo-Distributed ML Training
di: Tan, Wenting, et al.
Pubblicazione: (2023)