Copertina

Optimizing Frequent Checkpointing via Low-Cost Differential for Distributed Training Systems

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yao, Chenxuan, Hu, Yuchong, Liu, Feifan, Liu, Zhengyu, Wang, Lin, Li, Mingqi, Feng, Dan
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching
di: Pang, Bowen, et al.
Pubblicazione: (2025)

Scrutinizing Variables for Checkpoint Using Automatic Differentiation
di: Huang, Xin, et al.
Pubblicazione: (2026)

Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelis
di: Lian, Xinyu, et al.
Pubblicazione: (2024)

Bandwidth-Aware and Cost-Efficient Pipeline Parallel Scheduling in Geo-Distributed LLM Training
di: Zhang, Han, et al.
Pubblicazione: (2026)

Hiding Communication Cost in Distributed LLM Training via Micro-batch Co-execution
di: Wang, Haiquan, et al.
Pubblicazione: (2024)

Sparse Checkpointing for Fast and Reliable MoE Training
di: Gandhi, Swapnil, et al.
Pubblicazione: (2024)

Spatiotemporal Traffic Prediction in Distributed Backend Systems via Graph Neural Networks
di: Qiu, Zhimin, et al.
Pubblicazione: (2025)

QPOPSS: Query and Parallelism Optimized Space-Saving for Finding Frequent Stream Elements
di: Jarlow, Victor, et al.
Pubblicazione: (2024)

MTGenRec: An Efficient Distributed Training System for Generative Recommendation Models in Meituan
di: Wang, Yuxiang, et al.
Pubblicazione: (2025)

Orbax: Distributed Checkpointing with JAX
di: Gaffney, Colin, et al.
Pubblicazione: (2026)

Low-Communication Resilient Distributed Estimation Algorithm Based on Memory Mechanism
di: Li, Wei, et al.
Pubblicazione: (2025)

Harpagon: Minimizing DNN Serving Cost via Efficient Dispatching, Scheduling and Splitting
di: Zhao, Zhixin, et al.
Pubblicazione: (2024)

Checkmate: Zero-Overhead Model Checkpointing via Network Gradient Replication
di: Bhardwaj, Ankit, et al.
Pubblicazione: (2025)

Optimizing Distributed Training Approaches for Scaling Neural Networks
di: Baligodugula, Vishnu Vardhan, et al.
Pubblicazione: (2025)

Asynchronous Checkpoint for Eventually Consistent Databases
di: Ravishankar, Raaghav, et al.
Pubblicazione: (2025)

DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training
di: Tanaka, Masahiro, et al.
Pubblicazione: (2025)

Scalable and Adaptively Secure Any-Trust Distributed Key Generation and All-hands Checkpointing
di: Feng, Hanwen, et al.
Pubblicazione: (2023)

CRIUgpu: Transparent Checkpointing of GPU-Accelerated Workloads
di: Stoyanov, Radostin, et al.
Pubblicazione: (2025)

Optimal Checkpoint Interval with Availability as an Objective Function
di: Saxena, Nirmal Raj, et al.
Pubblicazione: (2024)

Checkpoint and Restart: An Energy Consumption Characterization in Clusters
di: Moran, Marina, et al.
Pubblicazione: (2024)

DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training
di: Wang, Zhixin, et al.
Pubblicazione: (2025)

Fault-Tolerant Hybrid-Parallel Training at Scale with Reliable and Efficient In-memory Checkpointing
di: Wang, Yuxin, et al.
Pubblicazione: (2023)

Magneton: Optimizing Energy Efficiency of ML Systems via Differential Energy Debugging
di: Pan, Yi, et al.
Pubblicazione: (2025)

Approximate Byzantine Fault-Tolerance in Distributed Optimization
di: Liu, Shuo, et al.
Pubblicazione: (2021)

MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization
di: Hu, Rizhen, et al.
Pubblicazione: (2025)

Revisiting the Time Cost Model of AllReduce
di: Xiong, Dian, et al.
Pubblicazione: (2024)

Efficient LLM Inference with Activation Checkpointing and Hybrid Caching
di: Lee, Sanghyeon, et al.
Pubblicazione: (2025)

TierCheck: Tiered Checkpointing for Fault Tolerance in Large Language Model Training
di: Han, Shujie, et al.
Pubblicazione: (2026)

FedCostAware: Enabling Cost-Aware Federated Learning on the Cloud
di: Sinha, Aditya, et al.
Pubblicazione: (2025)

Chameleon: Adaptive Fault Tolerance for Distributed Training via Real-time Policy Selection
di: Zhou, Yuhang, et al.
Pubblicazione: (2025)

Remoe: Towards Efficient and Low-Cost MoE Inference in Serverless Computing
di: Liu, Wentao, et al.
Pubblicazione: (2025)

DeFT: Mitigating Data Dependencies for Flexible Communication Scheduling in Distributed Training
di: Meng, Lin, et al.
Pubblicazione: (2025)

CondenseGraph: Communication-Efficient Distributed GNN Training via On-the-Fly Graph Condensation
di: Zhang, Zizhao, et al.
Pubblicazione: (2026)

Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching
di: Ruan, Chaoyi, et al.
Pubblicazione: (2025)

MegatronApp: Efficient and Comprehensive Management on Distributed LLM Training
di: Zhao, Bohan, et al.
Pubblicazione: (2025)

FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training
di: Qi, Shuyao, et al.
Pubblicazione: (2026)

CRIU -- Checkpoint Restore in Userspace for computational simulations and scientific applications
di: Andrijauskas, Fabio, et al.
Pubblicazione: (2024)

Understanding LLM Checkpoint/Restore I/O Strategies and Patterns
di: Gossman, Mikaila J., et al.
Pubblicazione: (2025)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
di: Zhang, WenZheng, et al.
Pubblicazione: (2024)

LiveR: Fine-Grained Elasticity via Live Reconfiguration for Model Training
di: Liu, Haoyuan, et al.
Pubblicazione: (2026)