TURNIP: A "Nondeterministic" GPU Runtime with CPU RAM Offload
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916421166956544 |
|---|---|
| author | Ding, Zhimin Yao, Jiawen Barrow, Brianna Botran, Tania Lorido Jermaine, Christopher Tang, Yuxin Li, Jiehui Yao, Xinyu Abdelghafar, Sleem Mahmoud Bourgeois, Daniel |
| author_facet | Ding, Zhimin Yao, Jiawen Barrow, Brianna Botran, Tania Lorido Jermaine, Christopher Tang, Yuxin Li, Jiehui Yao, Xinyu Abdelghafar, Sleem Mahmoud Bourgeois, Daniel |
| contents | An obvious way to alleviate memory difficulties in GPU-based AI computing is via CPU offload, where data are moved between GPU and CPU RAM, so inexpensive CPU RAM is used to increase the amount of storage available. While CPU offload is an obvious idea, it can greatly slow down a computation, due to the relatively slow transfer rate between CPU RAM and GPU RAM. Thus, any system for CPU offload needs to ensure that when such a transfer needs to happen, no computation is blocked waiting for the transfer to finish. One of the key challenges when using CPU offload is that memory transfers introduce nondeterminacy into the system: it is not possible to know before runtime when the transfers will finish, and hence what is the best order of operations to run to ensure there is no blocking. In this paper, we describe TURNIP, which is a system for running AI computations using CPU offload. The key innovation in TURNIP is the compilation of the AI computation into a dependency graph that gives the TURNIP runtime freedom to run operations such as GPU kernel calls in many different orders; at runtime, TURNIP chooses the best order in response to real-time events. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16283 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TURNIP: A "Nondeterministic" GPU Runtime with CPU RAM Offload Ding, Zhimin Yao, Jiawen Barrow, Brianna Botran, Tania Lorido Jermaine, Christopher Tang, Yuxin Li, Jiehui Yao, Xinyu Abdelghafar, Sleem Mahmoud Bourgeois, Daniel Distributed, Parallel, and Cluster Computing An obvious way to alleviate memory difficulties in GPU-based AI computing is via CPU offload, where data are moved between GPU and CPU RAM, so inexpensive CPU RAM is used to increase the amount of storage available. While CPU offload is an obvious idea, it can greatly slow down a computation, due to the relatively slow transfer rate between CPU RAM and GPU RAM. Thus, any system for CPU offload needs to ensure that when such a transfer needs to happen, no computation is blocked waiting for the transfer to finish. One of the key challenges when using CPU offload is that memory transfers introduce nondeterminacy into the system: it is not possible to know before runtime when the transfers will finish, and hence what is the best order of operations to run to ensure there is no blocking. In this paper, we describe TURNIP, which is a system for running AI computations using CPU offload. The key innovation in TURNIP is the compilation of the AI computation into a dependency graph that gives the TURNIP runtime freedom to run operations such as GPU kernel calls in many different orders; at runtime, TURNIP chooses the best order in response to real-time events. |
| title | TURNIP: A "Nondeterministic" GPU Runtime with CPU RAM Offload |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2405.16283 |