DynoStore: A wide-area distribution system for the management of data over heterogeneous storage
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913920963313664 |
|---|---|
| author | Sanchez-Gallegos, Dante D. Gonzalez-Compean, J. L. Gonthier, Maxime Hayot-Sasson, Valerie Pauloski, J. Gregory Pan, Haochen Chard, Kyle Carretero, Jesus Foster, Ian |
| author_facet | Sanchez-Gallegos, Dante D. Gonzalez-Compean, J. L. Gonthier, Maxime Hayot-Sasson, Valerie Pauloski, J. Gregory Pan, Haochen Chard, Kyle Carretero, Jesus Foster, Ian |
| contents | Data distribution across different facilities offers benefits such as enhanced resource utilization, increased resilience through replication, and improved performance by processing data near its source. However, managing such data is challenging due to heterogeneous access protocols, disparate authentication models, and the lack of a unified coordination framework. This paper presents DynoStore, a system that manages data across heterogeneous storage systems. At the core of DynoStore are data containers, an abstraction that provides standardized interfaces for seamless data management, irrespective of the underlying storage systems. Multiple data container connections create a cohesive wide-area storage network, ensuring resilience using erasure coding policies. Furthermore, a load-balancing algorithm ensures equitable and efficient utilization of storage resources. We evaluate DynoStore using benchmarks and real-world case studies, including the management of medical and satellite data across geographically distributed environments. Our results demonstrate a 10\% performance improvement compared to centralized cloud-hosted systems while maintaining competitive performance with state-of-the-art solutions such as Redis and IPFS. DynoStore also exhibits superior fault tolerance, withstanding more failures than traditional systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_00576 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DynoStore: A wide-area distribution system for the management of data over heterogeneous storage Sanchez-Gallegos, Dante D. Gonzalez-Compean, J. L. Gonthier, Maxime Hayot-Sasson, Valerie Pauloski, J. Gregory Pan, Haochen Chard, Kyle Carretero, Jesus Foster, Ian Distributed, Parallel, and Cluster Computing Data distribution across different facilities offers benefits such as enhanced resource utilization, increased resilience through replication, and improved performance by processing data near its source. However, managing such data is challenging due to heterogeneous access protocols, disparate authentication models, and the lack of a unified coordination framework. This paper presents DynoStore, a system that manages data across heterogeneous storage systems. At the core of DynoStore are data containers, an abstraction that provides standardized interfaces for seamless data management, irrespective of the underlying storage systems. Multiple data container connections create a cohesive wide-area storage network, ensuring resilience using erasure coding policies. Furthermore, a load-balancing algorithm ensures equitable and efficient utilization of storage resources. We evaluate DynoStore using benchmarks and real-world case studies, including the management of medical and satellite data across geographically distributed environments. Our results demonstrate a 10\% performance improvement compared to centralized cloud-hosted systems while maintaining competitive performance with state-of-the-art solutions such as Redis and IPFS. DynoStore also exhibits superior fault tolerance, withstanding more failures than traditional systems. |
| title | DynoStore: A wide-area distribution system for the management of data over heterogeneous storage |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2507.00576 |