Multi-Resolution Model Fusion for Accelerating the Convolutional Neural Network Training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911239145259008 |
|---|---|
| author | Wang, Kewei Lee, Claire Songhyun Lee, Sunwoo Gupta, Vishu Balewski, Jan Sim, Alex Nugent, Peter Agrawal, Ankit Choudhary, Alok Wu, Kesheng Liao, Wei-keng |
| author_facet | Wang, Kewei Lee, Claire Songhyun Lee, Sunwoo Gupta, Vishu Balewski, Jan Sim, Alex Nugent, Peter Agrawal, Ankit Choudhary, Alok Wu, Kesheng Liao, Wei-keng |
| contents | Neural networks are rapidly gaining popularity in scientific research, but training the models is often very time-consuming. Particularly when the training data samples are large high-dimensional arrays, efficient training methodologies that can reduce the computational costs are crucial. To reduce the training cost, we propose a Multi-Resolution Model Fusion (MRMF) method that combines models trained on reduced-resolution data and then refined with data in the original resolution. We demonstrate that these reduced-resolution models and datasets could be generated quickly. More importantly, the proposed approach reduces the training time by speeding up the model convergence in each fusion stage before switching to the final stage of finetuning with data in its original resolution. This strategy ensures the final model retains high-resolution insights while benefiting from the computational efficiency of lower-resolution training. Our experiment results demonstrate that the multi-resolution model fusion method can significantly reduce end-to-end training time while maintaining the same model accuracy. Evaluated using two real-world scientific applications, CosmoFlow and Neuron Inverter, the proposed method improves the training time by up to 47% and 44%, respectively, as compared to the original resolution training, while the model accuracy is not affected. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_25170 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multi-Resolution Model Fusion for Accelerating the Convolutional Neural Network Training Wang, Kewei Lee, Claire Songhyun Lee, Sunwoo Gupta, Vishu Balewski, Jan Sim, Alex Nugent, Peter Agrawal, Ankit Choudhary, Alok Wu, Kesheng Liao, Wei-keng Distributed, Parallel, and Cluster Computing Neural networks are rapidly gaining popularity in scientific research, but training the models is often very time-consuming. Particularly when the training data samples are large high-dimensional arrays, efficient training methodologies that can reduce the computational costs are crucial. To reduce the training cost, we propose a Multi-Resolution Model Fusion (MRMF) method that combines models trained on reduced-resolution data and then refined with data in the original resolution. We demonstrate that these reduced-resolution models and datasets could be generated quickly. More importantly, the proposed approach reduces the training time by speeding up the model convergence in each fusion stage before switching to the final stage of finetuning with data in its original resolution. This strategy ensures the final model retains high-resolution insights while benefiting from the computational efficiency of lower-resolution training. Our experiment results demonstrate that the multi-resolution model fusion method can significantly reduce end-to-end training time while maintaining the same model accuracy. Evaluated using two real-world scientific applications, CosmoFlow and Neuron Inverter, the proposed method improves the training time by up to 47% and 44%, respectively, as compared to the original resolution training, while the model accuracy is not affected. |
| title | Multi-Resolution Model Fusion for Accelerating the Convolutional Neural Network Training |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2510.25170 |