Cover Image

BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Zhengyang, Liu, Ziyue, Zhang, Ruijie, Maurya, Avinash, Hovland, Paul, Nicolae, Bogdan, Cappello, Franck, Zhang, Zheng
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload
by: Liu, Ziyue, et al.
Published: (2026)

DataStates-LLM: Scalable Checkpointing for Transformer Models Using Composable State Providers
by: Maurya, Avinash, et al.
Published: (2026)

Breaking the Memory Wall: A Study of I/O Patterns and GPU Memory Utilization for Hybrid CPU-GPU Offloaded Optimizers
by: Maurya, Avinash, et al.
Published: (2024)

DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models
by: Maurya, Avinash, et al.
Published: (2024)

Deep Optimizer States: Towards Scalable Training of Transformer Models Using Interleaved Offloading
by: Maurya, Avinash, et al.
Published: (2024)

Understanding Inference Scaling for LLMs: Bottlenecks, Trade-offs, and Performance Principles
by: Arif, Moiz, et al.
Published: (2026)

Understanding LLM Checkpoint/Restore I/O Strategies and Patterns
by: Gossman, Mikaila J., et al.
Published: (2025)

MLP-Offload: Multi-Level, Multi-Path Offloading for LLM Pre-training to Break the GPU Memory Wall
by: Maurya, Avinash, et al.
Published: (2025)

SPARe: Stacked Parallelism with Adaptive Reordering for Fault-Tolerant LLM Pretraining Systems with 100k+ GPUs
by: Lee, Jin, et al.
Published: (2026)

DGRO: Diameter-Guided Ring Optimization for Integrated Research Infrastructure Membership
by: Wu, Shixun, et al.
Published: (2024)

An Optimized Error-controlled MPI Collective Framework Integrated with Lossy Compression
by: Huang, Jiajun, et al.
Published: (2023)

Wilkins: HPC In Situ Workflows Made Easy
by: Yildiz, Orcun, et al.
Published: (2024)

To Compress or Not To Compress: Energy Trade-Offs and Benefits of Lossy Compressed I/O
by: Wilkins, Grant, et al.
Published: (2024)

DiffusionPipe: Training Large Diffusion Models with Efficient Pipelines
by: Tian, Ye, et al.
Published: (2024)

Efficient Data-Parallel Continual Learning with Asynchronous Distributed Rehearsal Buffers
by: Bouvier, Thomas, et al.
Published: (2024)

Mitigating Artifacts in Pre-quantization Based Scientific Data Compressors with Quantization-aware Interpolation
by: Jiao, Pu, et al.
Published: (2026)

DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training
by: Wang, Zhixin, et al.
Published: (2025)

FlowMoE: A Scalable Pipeline Scheduling Framework for Distributed Mixture-of-Experts Training
by: Gao, Yunqi, et al.
Published: (2025)

cuSZ-$i$: High-Ratio Scientific Lossy Compression on GPUs with Optimized Multi-Level Interpolation
by: Liu, Jinyang, et al.
Published: (2023)

IPComp: Interpolation Based Progressive Lossy Compression for Scientific Applications
by: Yang, Zhuoxun, et al.
Published: (2025)

Boosting Scientific Error-Bounded Lossy Compression through Optimized Synergistic Lossy-Lossless Orchestration
by: Wu, Shixun, et al.
Published: (2025)

TopoSZp: Lightweight Topology-Aware Error-controlled Compression for Scientific Data
by: Agarwal, Tripti, et al.
Published: (2026)

Lattica: A Decentralized Cross-NAT Communication Framework for Scalable AI Inference and Training
by: Yang, Ween, et al.
Published: (2025)

CFP: Efficient Optimization of Intra-Operator Parallelism Plans for Large Model Training
by: Hu, Weifang, et al.
Published: (2025)

Optimizing Federated Learning in the Era of LLMs: Message Quantization and Streaming
by: Xu, Ziyue, et al.
Published: (2025)

PRISM: Probabilistic Runtime Insights and Scalable Performance Modeling for Large-Scale Distributed Training
by: Golden, Alicia, et al.
Published: (2025)

ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training
by: Kang, Xueze, et al.
Published: (2025)

Runtime-Orchestrated Second-Order Optimization for Scalable LLM Training
by: Lu, Yishun, et al.
Published: (2026)

FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications
by: Wilkins, Grant, et al.
Published: (2023)

D-Rex: Heterogeneity-Aware Reliability Framework and Adaptive Algorithms for Distributed Storage
by: Gonthier, Maxime, et al.
Published: (2025)

gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters
by: Huang, Jiajun, et al.
Published: (2023)

DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
by: Zhang, Zili, et al.
Published: (2024)

TurboFFT: A High-Performance Fast Fourier Transform with Fault Tolerance on GPU
by: Wu, Shixun, et al.
Published: (2024)

TurboFFT: Co-Designed High-Performance and Fault-Tolerant Fast Fourier Transform on GPUs
by: Wu, Shixun, et al.
Published: (2024)

Advancing Blockchain Scalability: A Linear Optimization Framework for Diversified Node Allocation in Shards
by: Assmann, Björn, et al.
Published: (2024)

Optimizing Frequent Checkpointing via Low-Cost Differential for Distributed Training Systems
by: Yao, Chenxuan, et al.
Published: (2025)

Preserving Clusters in Error-Bounded Lossy Compression of Particle Data
by: Ren, Congrong, et al.
Published: (2026)

A Scalable Recipe on SuperMUC-NG Phase 2: Efficient Large-Scale Training of Language Models
by: Rajgopal, Ajay Navilarekal, et al.
Published: (2026)

HoSZp: An Efficient Homomorphic Error-bounded Lossy Compressor for Scientific Data
by: Agarwal, Tripti, et al.
Published: (2024)

pMSz: A Distributed Parallel Algorithm for Correcting Extrema and Morse Smale Segmentations in Lossy Compression
by: Li, Yuxiao, et al.
Published: (2026)