Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912105228140544 |
|---|---|
| author | Zhu, Jianwei Yin, Hang Deng, Peng Almeida, Aline Zhou, Shunfan |
| author_facet | Zhu, Jianwei Yin, Hang Deng, Peng Almeida, Aline Zhou, Shunfan |
| contents | This report evaluates the performance impact of enabling Trusted Execution Environments (TEE) on NVIDIA Hopper GPUs for large language model (LLM) inference tasks. We benchmark the overhead introduced by TEE mode across various LLMs and token lengths, with a particular focus on the bottleneck caused by CPU-GPU data transfers via PCIe. Our results indicate that while there is minimal computational overhead within the GPU, the overall performance penalty is primarily attributable to data transfer. For the majority of typical LLM queries, the overhead remains below 7%, with larger models and longer sequences experiencing nearly zero overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_03992 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study Zhu, Jianwei Yin, Hang Deng, Peng Almeida, Aline Zhou, Shunfan Distributed, Parallel, and Cluster Computing Artificial Intelligence Performance This report evaluates the performance impact of enabling Trusted Execution Environments (TEE) on NVIDIA Hopper GPUs for large language model (LLM) inference tasks. We benchmark the overhead introduced by TEE mode across various LLMs and token lengths, with a particular focus on the bottleneck caused by CPU-GPU data transfers via PCIe. Our results indicate that while there is minimal computational overhead within the GPU, the overall performance penalty is primarily attributable to data transfer. For the majority of typical LLM queries, the overhead remains below 7%, with larger models and longer sequences experiencing nearly zero overhead. |
| title | Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence Performance |
| url | https://arxiv.org/abs/2409.03992 |