Training Time Prediction for Mixed Precision-based Distributed Training
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915942233014272 |
|---|---|
| author | Kang, Minchul Shin, Changyong Jeong, Jinwoo Lee, Hyunho Go, Younghun Kim, Gyeongmin Yang, Gyeongsik Yoo, Chuck |
| author_facet | Kang, Minchul Shin, Changyong Jeong, Jinwoo Lee, Hyunho Go, Younghun Kim, Gyeongmin Yang, Gyeongsik Yoo, Chuck |
| contents | Accurate prediction of training time in distributed deep learning is crucial for resource allocation, cost estimation, and job scheduling. We observe that the floating-point precision setting is a key determinant of training time, leading to training time variations of ~2.4x over its minimum. However, existing studies on distributed training time prediction rely on static model computation graphs that do not capture precision variations, including mixed precision. According to our experiments, training time prediction without considering precision results in significant prediction errors - reaching up to 147.85% in mean absolute percentage error (MAPE). To address this issue, we propose a precision-aware distributed training time predictor that achieves robust accuracy across diverse precision settings, including mixed precision, with 9.8% MAPE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_16145 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Training Time Prediction for Mixed Precision-based Distributed Training Kang, Minchul Shin, Changyong Jeong, Jinwoo Lee, Hyunho Go, Younghun Kim, Gyeongmin Yang, Gyeongsik Yoo, Chuck Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing Performance Accurate prediction of training time in distributed deep learning is crucial for resource allocation, cost estimation, and job scheduling. We observe that the floating-point precision setting is a key determinant of training time, leading to training time variations of ~2.4x over its minimum. However, existing studies on distributed training time prediction rely on static model computation graphs that do not capture precision variations, including mixed precision. According to our experiments, training time prediction without considering precision results in significant prediction errors - reaching up to 147.85% in mean absolute percentage error (MAPE). To address this issue, we propose a precision-aware distributed training time predictor that achieves robust accuracy across diverse precision settings, including mixed precision, with 9.8% MAPE. |
| title | Training Time Prediction for Mixed Precision-based Distributed Training |
| topic | Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing Performance |
| url | https://arxiv.org/abs/2604.16145 |