Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.20067 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912958856036352 |
|---|---|
| author | Hira, Moto Puhrsch, Christian Andrei, Valentin Malinovskyy, Roman Lan, Gael Le Krishnan, Abhinandan Cummings, Joseph Bourgin, Victor Gerasimova, Olga Martin, Miguel Gunasekaran, Gokul Inoue, Yuta Turner, Alex J Krishnamoorthi, Raghuraman |
| author_facet | Hira, Moto Puhrsch, Christian Andrei, Valentin Malinovskyy, Roman Lan, Gael Le Krishnan, Abhinandan Cummings, Joseph Bourgin, Victor Gerasimova, Olga Martin, Miguel Gunasekaran, Gokul Inoue, Yuta Turner, Alex J Krishnamoorthi, Raghuraman |
| contents | We present SPDL (Scalable and Performant Data Loading), an open-source, framework-agnostic library designed for efficiently loading array data to GPU. Data loading is often a bottleneck in AI applications, and is challenging to optimize because it requires coordination of network calls, CPU-bound tasks, and GPU device transfer. On top of that, Python's GIL (Global Interpreter Lock) makes it difficult to gain performance improvement from multi-threading. We found that when data preprocessing functions release the GIL entirely, it is possible to execute them concurrently in a thread pool, thereby improving the workflow performance. Our benchmark shows that compared to the PyTorch DataLoader, SPDL can iterate through the ImageNet dataset 74% faster while using 38% less CPU and 50GB less memory. When training ViT-B/16 model, SPDL can send data to the GPU at a speed that does not starve the training. Additionally, when using SPDL on Python 3.13t, without changing any code, the throughput is further by improved by 33%, thanks to the disabled GIL. SPDL can improve the performance of current AI model training, and receives further performance improvements when Free-Threaded Python is adopted in production systems. SPDL is available at https://github.com/facebookresearch/spdl. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_20067 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scalable and Performant Data Loading Hira, Moto Puhrsch, Christian Andrei, Valentin Malinovskyy, Roman Lan, Gael Le Krishnan, Abhinandan Cummings, Joseph Bourgin, Victor Gerasimova, Olga Martin, Miguel Gunasekaran, Gokul Inoue, Yuta Turner, Alex J Krishnamoorthi, Raghuraman Distributed, Parallel, and Cluster Computing We present SPDL (Scalable and Performant Data Loading), an open-source, framework-agnostic library designed for efficiently loading array data to GPU. Data loading is often a bottleneck in AI applications, and is challenging to optimize because it requires coordination of network calls, CPU-bound tasks, and GPU device transfer. On top of that, Python's GIL (Global Interpreter Lock) makes it difficult to gain performance improvement from multi-threading. We found that when data preprocessing functions release the GIL entirely, it is possible to execute them concurrently in a thread pool, thereby improving the workflow performance. Our benchmark shows that compared to the PyTorch DataLoader, SPDL can iterate through the ImageNet dataset 74% faster while using 38% less CPU and 50GB less memory. When training ViT-B/16 model, SPDL can send data to the GPU at a speed that does not starve the training. Additionally, when using SPDL on Python 3.13t, without changing any code, the throughput is further by improved by 33%, thanks to the disabled GIL. SPDL can improve the performance of current AI model training, and receives further performance improvements when Free-Threaded Python is adopted in production systems. SPDL is available at https://github.com/facebookresearch/spdl. |
| title | Scalable and Performant Data Loading |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2504.20067 |