Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2305.10611 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917668093689856 |
|---|---|
| author | Fegade, Pratik Chen, Tianqi Gibbons, Phillip B. Mowry, Todd C. |
| author_facet | Fegade, Pratik Chen, Tianqi Gibbons, Phillip B. Mowry, Todd C. |
| contents | Dynamic control flow is an important technique often used to design expressive and efficient deep learning computations for applications such as text parsing, machine translation, exiting early out of deep models and so on. The control flow divergence resulting from dynamic control flow makes batching, an important optimization enabling high throughput and hardware utilization, difficult to perform manually. In this paper, we present ACRoBat, a framework that enables efficient automatic batching for dynamic deep learning computations by performing hybrid static+dynamic compiler optimizations and end-to-end tensor code generation. ACRoBat performs up to 8.5X better than DyNet, a state-of-the-art framework for automatic batching, on an Nvidia GeForce GPU. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_10611 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | ACRoBat: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time Fegade, Pratik Chen, Tianqi Gibbons, Phillip B. Mowry, Todd C. Machine Learning Dynamic control flow is an important technique often used to design expressive and efficient deep learning computations for applications such as text parsing, machine translation, exiting early out of deep models and so on. The control flow divergence resulting from dynamic control flow makes batching, an important optimization enabling high throughput and hardware utilization, difficult to perform manually. In this paper, we present ACRoBat, a framework that enables efficient automatic batching for dynamic deep learning computations by performing hybrid static+dynamic compiler optimizations and end-to-end tensor code generation. ACRoBat performs up to 8.5X better than DyNet, a state-of-the-art framework for automatic batching, on an Nvidia GeForce GPU. |
| title | ACRoBat: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2305.10611 |