Globus Service Enhancements for Exascale Applications and Facilities
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917971634421760 |
|---|---|
| author | Zheng, Weijian Kordas, Jack Skluzacek, Tyler J. Kettimuthu, Raj Foster, Ian |
| author_facet | Zheng, Weijian Kordas, Jack Skluzacek, Tyler J. Kettimuthu, Raj Foster, Ian |
| contents | Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the home institutions of facility users. These applications, particularly when leadership computing facilities are involved, can touch upon edge cases (e.g., terabyte files) that had not been a focus of previous Globus optimization work, which had emphasized rather the movement of many smaller (megabyte to gigabyte) files. We report here on how automated client-driven chunking can be used to accelerate both the movement of large files and the integrity checking operations that have proven to be essential for large data transfers. We present detailed performance studies that provide insights into the benefits of these modifications in a range of file transfer scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_22981 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Globus Service Enhancements for Exascale Applications and Facilities Zheng, Weijian Kordas, Jack Skluzacek, Tyler J. Kettimuthu, Raj Foster, Ian Networking and Internet Architecture Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the home institutions of facility users. These applications, particularly when leadership computing facilities are involved, can touch upon edge cases (e.g., terabyte files) that had not been a focus of previous Globus optimization work, which had emphasized rather the movement of many smaller (megabyte to gigabyte) files. We report here on how automated client-driven chunking can be used to accelerate both the movement of large files and the integrity checking operations that have proven to be essential for large data transfers. We present detailed performance studies that provide insights into the benefits of these modifications in a range of file transfer scenarios. |
| title | Globus Service Enhancements for Exascale Applications and Facilities |
| topic | Networking and Internet Architecture |
| url | https://arxiv.org/abs/2503.22981 |