Dynamic Adaptation in Data Storage: Real-Time Machine Learning for Enhanced Prefetching
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929690361462784 |
|---|---|
| author | Cheng, Chiyu Zhou, Chang Zhao, Yang Cao, Jin |
| author_facet | Cheng, Chiyu Zhou, Chang Zhao, Yang Cao, Jin |
| contents | The exponential growth of data storage demands has necessitated the evolution of hierarchical storage management strategies [1]. This study explores the application of streaming machine learning [3] to revolutionize data prefetching within multi-tiered storage systems. Unlike traditional batch-trained models, streaming machine learning [5] offers adaptability, real-time insights, and computational efficiency, responding dynamically to workload variations. This work designs and validates an innovative framework that integrates streaming classification models for predicting file access patterns, specifically the next file offset. Leveraging comprehensive feature engineering and real-time evaluation over extensive production traces, the proposed methodology achieves substantial improvements in prediction accuracy, memory efficiency, and system adaptability. The results underscore the potential of streaming models in real-time storage management, setting a precedent for advanced caching and tiering strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_14771 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dynamic Adaptation in Data Storage: Real-Time Machine Learning for Enhanced Prefetching Cheng, Chiyu Zhou, Chang Zhao, Yang Cao, Jin Distributed, Parallel, and Cluster Computing Machine Learning Operating Systems The exponential growth of data storage demands has necessitated the evolution of hierarchical storage management strategies [1]. This study explores the application of streaming machine learning [3] to revolutionize data prefetching within multi-tiered storage systems. Unlike traditional batch-trained models, streaming machine learning [5] offers adaptability, real-time insights, and computational efficiency, responding dynamically to workload variations. This work designs and validates an innovative framework that integrates streaming classification models for predicting file access patterns, specifically the next file offset. Leveraging comprehensive feature engineering and real-time evaluation over extensive production traces, the proposed methodology achieves substantial improvements in prediction accuracy, memory efficiency, and system adaptability. The results underscore the potential of streaming models in real-time storage management, setting a precedent for advanced caching and tiering strategies. |
| title | Dynamic Adaptation in Data Storage: Real-Time Machine Learning for Enhanced Prefetching |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning Operating Systems |
| url | https://arxiv.org/abs/2501.14771 |