Image de couverture de livre

A Comparative Analysis of Distributed Training Strategies for GPT-2

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Patwardhan, Ishan, Gandhi, Shubham, Khare, Om, Joshi, Amit, Sawant, Suraj
Format:	Preprint
Publié:	2024
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Hybrid Quantum-HPC Solutions for Max-Cut: Bridging Classical and Quantum Algorithms
par: Patwardhan, Ishan, et autres
Publié: (2024)

PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
par: Patel, Ishan, et autres
Publié: (2026)

Sparse Checkpointing for Fast and Reliable MoE Training
par: Gandhi, Swapnil, et autres
Publié: (2024)

Distributed Locking: Performance Analysis and Optimization Strategies
par: Rodriguez, Andre, et autres
Publié: (2025)

Comparative Analysis of Distributed Caching Algorithms: Performance Metrics and Implementation Considerations
par: Mayer, Helen, et autres
Publié: (2025)

Comparative Analysis of Lightweight Kubernetes Distributions for Edge Computing: Performance and Resource Efficiency
par: Yakubov, Diyaz, et autres
Publié: (2025)

Cost-Performance Analysis: A Comparative Study of CPU-Based Serverless and GPU-Based Training Architectures
par: Barrak, Amine, et autres
Publié: (2025)

KnapsackLB: Enabling Performance-Aware Layer-4 Load Balancing
par: Gandhi, Rohan, et autres
Publié: (2024)

ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks
par: Davis, Joshua H., et autres
Publié: (2025)

Efficient Distributed MLLM Training with Cornstarch
par: Jang, Insu, et autres
Publié: (2025)

Efficient Training of Large Language Models on Distributed Infrastructures: A Survey
par: Duan, Jiangfei, et autres
Publié: (2024)

A Study on the Performance of Distributed Training of Data-driven CFD Simulations
par: Iserte, Sergio, et autres
Publié: (2026)

A flexible FPGA accelerator for convolutional neural networks
par: Majumder, Kingshuk, et autres
Publié: (2019)

Heta: Distributed Training of Heterogeneous Graph Neural Networks
par: Zhong, Yuchen, et autres
Publié: (2024)

Galvatron: Automatic Distributed Training for Large Transformer Models
par: Gumaan, Esmail
Publié: (2025)

Addressing Variable Heterogeneity in Distributed Multimodal Training with Entrain
par: Jang, Insu, et autres
Publié: (2026)

Accelerating Distributed MoE Training and Inference with Lina
par: Li, Jiamin, et autres
Publié: (2022)

Optimizing Distributed Training Approaches for Scaling Neural Networks
par: Baligodugula, Vishnu Vardhan, et autres
Publié: (2025)

Fault-Tolerant Decentralized Distributed Asynchronous Federated Learning with Adaptive Termination Detection
par: Akkinepally, Phani Sahasra, et autres
Publié: (2025)

On the Performance and Memory Footprint of Distributed Training: An Empirical Study on Transformers
par: Lu, Zhengxian, et autres
Publié: (2024)

Characterizing FaaS Workflows on Public Clouds: The Good, the Bad and the Ugly
par: Kulkarni, Varad, et autres
Publié: (2025)

MegatronApp: Efficient and Comprehensive Management on Distributed LLM Training
par: Zhao, Bohan, et autres
Publié: (2025)

Leveraging Hardware Performance Counters for Predicting Workload Interference in Vector Supercomputers
par: Shubham, et autres
Publié: (2024)

DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training
par: Tanaka, Masahiro, et autres
Publié: (2025)

A Survey of End-to-End Modeling for Distributed DNN Training: Workloads, Simulators, and TCO
par: Svedas, Jonas, et autres
Publié: (2025)

FailSafe: High-performance Resilient Serving
par: Xu, Ziyi, et autres
Publié: (2025)

Tetris: Efficient Intra-Datacenter Calls Packing for Large Conferencing Services
par: Gandhi, Rohan, et autres
Publié: (2025)

A Comparative Analysis of Identifier Schemes: UUIDv4, UUIDv7, and ULID for Distributed Systems
par: Kakolaki, Nima Karimian
Publié: (2025)

A Comparative Evaluation of Automated Analysis Tools for Solidity Smart Contracts
par: Wei, Zhiyuan, et autres
Publié: (2023)

Nezha: Breaking Multi-Rail Network Barriers for Distributed DNN Training
par: Yu, Enda, et autres
Publié: (2024)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
par: Zhang, WenZheng, et autres
Publié: (2024)

Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training
par: Xu, Guanbin, et autres
Publié: (2026)

DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training
par: Wang, Zhixin, et autres
Publié: (2025)

PruneX: A Hierarchical Communication-Efficient System for Distributed CNN Training with Structured Pruning
par: Olama, Alireza, et autres
Publié: (2025)

FlowMoE: A Scalable Pipeline Scheduling Framework for Distributed Mixture-of-Experts Training
par: Gao, Yunqi, et autres
Publié: (2025)

Analysis of Distributed Algorithms for Big-data
par: Purohit, Rajendra, et autres
Publié: (2024)

How to Evaluate Distributed Coordination Systems? -- A Survey and Analysis
par: Turkkan, Bekir, et autres
Publié: (2024)

PowerTrip: Exploiting Federated Heterogeneous Datacenter Power for Distributed ML Training
par: Mehboob, Talha, et autres
Publié: (2025)

MTGenRec: An Efficient Distributed Training System for Generative Recommendation Models in Meituan
par: Wang, Yuxiang, et autres
Publié: (2025)

Optimizing Frequent Checkpointing via Low-Cost Differential for Distributed Training Systems
par: Yao, Chenxuan, et autres
Publié: (2025)