Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.03030 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911827435192320 |
|---|---|
| author | Groet, Philip Hoozemans, Joost Grapentin, Andreas Eberhardt, Felix Al-Ars, Zaid Hofstee, H. Peter |
| author_facet | Groet, Philip Hoozemans, Joost Grapentin, Andreas Eberhardt, Felix Al-Ars, Zaid Hofstee, H. Peter |
| contents | This paper describes a distributed implementation of Apache Arrow that can leverage cluster-shared load-store addressable memory that is hardware-coherent only within each node. The implementation is built on the ThymesisFlow prototype that leverages the OpenCAPI interface to create a shared address space across a cluster. While Apache Arrow structures are immutable, simplifying their use in a cluster shared memory, this paper creates distributed Apache Arrow tables and makes them accessible in each node. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_03030 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Leveraging Apache Arrow for Zero-copy, Zero-serialization Cluster Shared Memory Groet, Philip Hoozemans, Joost Grapentin, Andreas Eberhardt, Felix Al-Ars, Zaid Hofstee, H. Peter Emerging Technologies This paper describes a distributed implementation of Apache Arrow that can leverage cluster-shared load-store addressable memory that is hardware-coherent only within each node. The implementation is built on the ThymesisFlow prototype that leverages the OpenCAPI interface to create a shared address space across a cluster. While Apache Arrow structures are immutable, simplifying their use in a cluster shared memory, this paper creates distributed Apache Arrow tables and makes them accessible in each node. |
| title | Leveraging Apache Arrow for Zero-copy, Zero-serialization Cluster Shared Memory |
| topic | Emerging Technologies |
| url | https://arxiv.org/abs/2404.03030 |