Introducing Milabench: Benchmarking Accelerators for AI
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916492960858112 |
|---|---|
| author | Delaunay, Pierre Bouthillier, Xavier Breuleux, Olivier Ortiz-Gagné, Satya Bilaniuk, Olexa Normandin, Fabrice Bergeron, Arnaud Carrez, Bruno Alain, Guillaume Blanc, Soline Osterrath, Frédéric Viviano, Joseph Patil, Roger Creus-Castanyer Darshan Awal, Rabiul Zhang, Le |
| author_facet | Delaunay, Pierre Bouthillier, Xavier Breuleux, Olivier Ortiz-Gagné, Satya Bilaniuk, Olexa Normandin, Fabrice Bergeron, Arnaud Carrez, Bruno Alain, Guillaume Blanc, Soline Osterrath, Frédéric Viviano, Joseph Patil, Roger Creus-Castanyer Darshan Awal, Rabiul Zhang, Le |
| contents | AI workloads, particularly those driven by deep learning, are introducing novel usage patterns to high-performance computing (HPC) systems that are not comprehensively captured by standard HPC benchmarks. As one of the largest academic research centers dedicated to deep learning, Mila identified the need to develop a custom benchmarking suite to address the diverse requirements of its community, which consists of over 1,000 researchers. This report introduces Milabench, the resulting benchmarking suite. Its design was informed by an extensive literature review encompassing 867 papers, as well as surveys conducted with Mila researchers. This rigorous process led to the selection of 26 primary benchmarks tailored for procurement evaluations, alongside 16 optional benchmarks for in-depth analysis. We detail the design methodology, the structure of the benchmarking suite, and provide performance evaluations using GPUs from NVIDIA, AMD, and Intel. The Milabench suite is open source and can be accessed at github.com/mila-iqia/milabench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11940 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Introducing Milabench: Benchmarking Accelerators for AI Delaunay, Pierre Bouthillier, Xavier Breuleux, Olivier Ortiz-Gagné, Satya Bilaniuk, Olexa Normandin, Fabrice Bergeron, Arnaud Carrez, Bruno Alain, Guillaume Blanc, Soline Osterrath, Frédéric Viviano, Joseph Patil, Roger Creus-Castanyer Darshan Awal, Rabiul Zhang, Le Machine Learning AI workloads, particularly those driven by deep learning, are introducing novel usage patterns to high-performance computing (HPC) systems that are not comprehensively captured by standard HPC benchmarks. As one of the largest academic research centers dedicated to deep learning, Mila identified the need to develop a custom benchmarking suite to address the diverse requirements of its community, which consists of over 1,000 researchers. This report introduces Milabench, the resulting benchmarking suite. Its design was informed by an extensive literature review encompassing 867 papers, as well as surveys conducted with Mila researchers. This rigorous process led to the selection of 26 primary benchmarks tailored for procurement evaluations, alongside 16 optional benchmarks for in-depth analysis. We detail the design methodology, the structure of the benchmarking suite, and provide performance evaluations using GPUs from NVIDIA, AMD, and Intel. The Milabench suite is open source and can be accessed at github.com/mila-iqia/milabench. |
| title | Introducing Milabench: Benchmarking Accelerators for AI |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2411.11940 |