Debunking the CUDA Myth Towards GPU-based AI Systems
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909547674730496 |
|---|---|
| author | Lee, Yunjae Lim, Juntaek Bang, Jehyeon Cho, Eunyeong Jeong, Huijong Kim, Taesu Kim, Hyungjun Lee, Joonhyung Im, Jinseop Hwang, Ranggi Kwon, Se Jung Lee, Dongsoo Rhu, Minsoo |
| author_facet | Lee, Yunjae Lim, Juntaek Bang, Jehyeon Cho, Eunyeong Jeong, Huijong Kim, Taesu Kim, Hyungjun Lee, Joonhyung Im, Jinseop Hwang, Ranggi Kwon, Se Jung Lee, Dongsoo Rhu, Minsoo |
| contents | This paper presents a comprehensive evaluation of Intel Gaudi NPUs as an alternative to NVIDIA GPUs, which is currently the de facto standard in AI system design. First, we create a suite of microbenchmarks to compare Intel Gaudi-2 with NVIDIA A100, showing that Gaudi-2 achieves competitive performance not only in primitive AI compute, memory, and communication operations but also in executing several important AI workloads end-to-end. We then assess Gaudi NPU's programmability by discussing several software-level optimization strategies to employ for implementing critical FBGEMM operators and vLLM, evaluating their efficiency against GPU-optimized counterparts. Results indicate that Gaudi-2 achieves energy efficiency comparable to A100, though there are notable areas for improvement in terms of software maturity. Overall, we conclude that, with effective integration into high-level AI frameworks, Gaudi NPUs could challenge NVIDIA GPU's dominance in the AI server market, though further improvements are necessary to fully compete with NVIDIA's robust software ecosystem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_00210 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Debunking the CUDA Myth Towards GPU-based AI Systems Lee, Yunjae Lim, Juntaek Bang, Jehyeon Cho, Eunyeong Jeong, Huijong Kim, Taesu Kim, Hyungjun Lee, Joonhyung Im, Jinseop Hwang, Ranggi Kwon, Se Jung Lee, Dongsoo Rhu, Minsoo Distributed, Parallel, and Cluster Computing Artificial Intelligence Hardware Architecture This paper presents a comprehensive evaluation of Intel Gaudi NPUs as an alternative to NVIDIA GPUs, which is currently the de facto standard in AI system design. First, we create a suite of microbenchmarks to compare Intel Gaudi-2 with NVIDIA A100, showing that Gaudi-2 achieves competitive performance not only in primitive AI compute, memory, and communication operations but also in executing several important AI workloads end-to-end. We then assess Gaudi NPU's programmability by discussing several software-level optimization strategies to employ for implementing critical FBGEMM operators and vLLM, evaluating their efficiency against GPU-optimized counterparts. Results indicate that Gaudi-2 achieves energy efficiency comparable to A100, though there are notable areas for improvement in terms of software maturity. Overall, we conclude that, with effective integration into high-level AI frameworks, Gaudi NPUs could challenge NVIDIA GPU's dominance in the AI server market, though further improvements are necessary to fully compete with NVIDIA's robust software ecosystem. |
| title | Debunking the CUDA Myth Towards GPU-based AI Systems |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence Hardware Architecture |
| url | https://arxiv.org/abs/2501.00210 |