Performance of Distributed File Systems on Cloud Computing Environment: An Evaluation for Small-File Problem
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909057001979904 |
|---|---|
| author | Duong, Thanh Luu, Quoc Nguyen, Hung |
| author_facet | Duong, Thanh Luu, Quoc Nguyen, Hung |
| contents | Various performance characteristics of distributed file systems have been well studied. However, the performance efficiency of distributed file systems on small-file problems with complex machine learning algorithms scenarios is not well addressed. In addition, demands for unified storage of big data processing and high-performance computing have been crucial. Hence, developing a solution combining high-performance computing and big data with shared storage is very important. This paper focuses on the performance efficiency of distributed file systems with small-file datasets. We propose an architecture combining both high-performance computing and big data with shared storage and perform a series of experiments to investigate the performance of these distributed file systems. The result of the experiments confirms the applicability of the proposed architecture in terms of complex machine learning algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_17524 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Performance of Distributed File Systems on Cloud Computing Environment: An Evaluation for Small-File Problem Duong, Thanh Luu, Quoc Nguyen, Hung Distributed, Parallel, and Cluster Computing Various performance characteristics of distributed file systems have been well studied. However, the performance efficiency of distributed file systems on small-file problems with complex machine learning algorithms scenarios is not well addressed. In addition, demands for unified storage of big data processing and high-performance computing have been crucial. Hence, developing a solution combining high-performance computing and big data with shared storage is very important. This paper focuses on the performance efficiency of distributed file systems with small-file datasets. We propose an architecture combining both high-performance computing and big data with shared storage and perform a series of experiments to investigate the performance of these distributed file systems. The result of the experiments confirms the applicability of the proposed architecture in terms of complex machine learning algorithms. |
| title | Performance of Distributed File Systems on Cloud Computing Environment: An Evaluation for Small-File Problem |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2312.17524 |