Optimizing Distributed Training Approaches for Scaling Neural Networks
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915218686214144 |
|---|---|
| author | Baligodugula, Vishnu Vardhan Amsaad, Fathi |
| author_facet | Baligodugula, Vishnu Vardhan Amsaad, Fathi |
| contents | This paper presents a comparative analysis of distributed training strategies for large-scale neural networks, focusing on data parallelism, model parallelism, and hybrid approaches. We evaluate these strategies on image classification tasks using the CIFAR-100 dataset, measuring training time, convergence rate, and model accuracy. Our experimental results demonstrate that hybrid parallelism achieves a 3.2x speedup compared to single-device training while maintaining comparable accuracy. We propose an adaptive scheduling algorithm that dynamically switches between parallelism strategies based on network characteristics and available computational resources, resulting in an additional 18% improvement in training efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_23186 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimizing Distributed Training Approaches for Scaling Neural Networks Baligodugula, Vishnu Vardhan Amsaad, Fathi Distributed, Parallel, and Cluster Computing This paper presents a comparative analysis of distributed training strategies for large-scale neural networks, focusing on data parallelism, model parallelism, and hybrid approaches. We evaluate these strategies on image classification tasks using the CIFAR-100 dataset, measuring training time, convergence rate, and model accuracy. Our experimental results demonstrate that hybrid parallelism achieves a 3.2x speedup compared to single-device training while maintaining comparable accuracy. We propose an adaptive scheduling algorithm that dynamically switches between parallelism strategies based on network characteristics and available computational resources, resulting in an additional 18% improvement in training efficiency. |
| title | Optimizing Distributed Training Approaches for Scaling Neural Networks |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2503.23186 |