Accelerating AI Performance using Anderson Extrapolation on GPUs
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913617809506304 |
|---|---|
| author | Dajani, Saleem Abdul Fattah Ahmed Al Keyes, David E. |
| author_facet | Dajani, Saleem Abdul Fattah Ahmed Al Keyes, David E. |
| contents | We present a novel approach for accelerating AI performance by leveraging Anderson extrapolation, a vector-to-vector mapping technique based on a window of historical iterations. By identifying the crossover point (Fig. 1) where a mixing penalty is incurred, the method focuses on reducing iterations to convergence, with fewer more compute-intensive but generally cacheable iterations, balancing speed and memory usage with accuracy and algorithmic stability, respectively. We demonstrate significant improvements, in both training and inference, motivated by scalability and efficiency extensions to the realm of high-performance computing (HPC). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_19460 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Accelerating AI Performance using Anderson Extrapolation on GPUs Dajani, Saleem Abdul Fattah Ahmed Al Keyes, David E. Machine Learning Artificial Intelligence Numerical Analysis Performance We present a novel approach for accelerating AI performance by leveraging Anderson extrapolation, a vector-to-vector mapping technique based on a window of historical iterations. By identifying the crossover point (Fig. 1) where a mixing penalty is incurred, the method focuses on reducing iterations to convergence, with fewer more compute-intensive but generally cacheable iterations, balancing speed and memory usage with accuracy and algorithmic stability, respectively. We demonstrate significant improvements, in both training and inference, motivated by scalability and efficiency extensions to the realm of high-performance computing (HPC). |
| title | Accelerating AI Performance using Anderson Extrapolation on GPUs |
| topic | Machine Learning Artificial Intelligence Numerical Analysis Performance |
| url | https://arxiv.org/abs/2410.19460 |