Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913573636145152 |
|---|---|
| author | Gao, Jianhua Shen, Jiayuan Zhang, Yuxiang Ji, Weixing Huang, Hua |
| author_facet | Gao, Jianhua Shen, Jiayuan Zhang, Yuxiang Ji, Weixing Huang, Hua |
| contents | Iterative solvers are frequently used in scientific applications and engineering computations. However, the memory-bound Sparse Matrix-Vector (SpMV) kernel computation hinders the efficiency of iterative algorithms. As modern hardware increasingly supports low-precision computation, the mixed-precision optimization of iterative algorithms has garnered widespread attention. Nevertheless, existing mixed-precision methods pose challenges, including format conversion overhead, tight coupling between storage and computation representation, and the need to store multiple precision copies of data. This paper proposes a floating-point representation based on the group-shared exponent and segmented storage of the mantissa, enabling higher bit utilization of the representation vector and fast switches between different precisions without needing multiple data copies. Furthermore, a stepped mixed-precision iterative algorithm is proposed. Our experimental results demonstrate that, compared with existing floating-point formats, our approach significantly improves iterative algorithms' performance and convergence residuals. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_04686 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers Gao, Jianhua Shen, Jiayuan Zhang, Yuxiang Ji, Weixing Huang, Hua Distributed, Parallel, and Cluster Computing Numerical Analysis 68-02, 68W10, 65F50 A.1; D.1.3; G.1.3 Iterative solvers are frequently used in scientific applications and engineering computations. However, the memory-bound Sparse Matrix-Vector (SpMV) kernel computation hinders the efficiency of iterative algorithms. As modern hardware increasingly supports low-precision computation, the mixed-precision optimization of iterative algorithms has garnered widespread attention. Nevertheless, existing mixed-precision methods pose challenges, including format conversion overhead, tight coupling between storage and computation representation, and the need to store multiple precision copies of data. This paper proposes a floating-point representation based on the group-shared exponent and segmented storage of the mantissa, enabling higher bit utilization of the representation vector and fast switches between different precisions without needing multiple data copies. Furthermore, a stepped mixed-precision iterative algorithm is proposed. Our experimental results demonstrate that, compared with existing floating-point formats, our approach significantly improves iterative algorithms' performance and convergence residuals. |
| title | Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers |
| topic | Distributed, Parallel, and Cluster Computing Numerical Analysis 68-02, 68W10, 65F50 A.1; D.1.3; G.1.3 |
| url | https://arxiv.org/abs/2411.04686 |